AWS re:Invent 2022 - Manage your infrastructure with Terraform and GitHub Actions (BOA328)
Aug 16, 2023
AWS re:Invent 2022 - Manage your infrastructure with Terraform and GitHub Actions (BOA328)
Automating your infrastructure changes to deploy the same way as your code deploys reduces the risk of it breaking and spoiling your weekend. Being able to reliably, predictably, and consistently roll out infrastructure changes across multiple environments sounds like a challenge, but it can be achieved by applying DevOps practices to your existing CI/CD pipelines. In this session, learn how to use Terraform and GitHub Actions to manage and automate your infrastructure creation and updates via git. See how testing and deploying to a preproduction environment can help you ensure any changes can be deployed safely without service interruptions. Learn more about AWS re:Invent at https://go.aws/3ikK4dD . Subscribe: More AWS videos http://bit.ly/2O3zS75 More AWS events videos http://bit.ly/316g9t4 ABOUT AWS Amazon Web Services (AWS) hosts events, both online and in-person, bringing the cloud computing community together to connect, collaborate, and learn from AWS experts. AWS is the world’s most comprehensive and broadly adopted cloud platform, offering over 200 fully featured services from data centers globally. Millions of customers—including the fastest-growing startups, largest enterprises, and leading government agencies—are using AWS to lower costs, become more agile, and innovate faster. #reInvent2022 #AWSreInvent2022 #AWSEvents
Content
0.18 -> - Hello, I'm Emily Freeman.
1.71 -> I'm the head of community
engagement at AWS
3.618 -> and I'm so excited to
be here with you all.
5.73 -> We're like kicking off re:Invent today.
7.644 -> - [Audience Member] Whoo!
8.645 -> (laughing)Cobus
9.478 -> - Thank you.
11.061 -> - Awesome.
12.605 -> And I'm Cobus Bernard and I'm
a senior developer advocate
15.143 -> with AWS, and as you can hear,
I'm not from around here.
17.893 -> I moved to the States end of
last year with all the drama.
21.609 -> - Where are you from though?
22.95 -> - South Africa.
23.97 -> - South, is that your American accent?
26.01 -> - No, that's a funny American accent
27.937 -> (speaking faintly off mic)
30.187 -> - Well, he did a bad American accent
32.94 -> so we had to cut his mic.
34.511 -> - South African.
36.33 -> - South African.
37.26 -> - No, south with a F.
38.461 -> - Oh, I like it.
40.057 -> - Cool.
41.34 -> - We were supposed to be
joined by Julie Gunderson.
43.77 -> She is ill, unfortunately
won't be able to join us,
46.5 -> but we wanted to say
hello and wish her well.
48.72 -> So if you'll join me in
wishing her well to the video.
51.3 -> I really appreciate it.
52.53 -> 1, 2, 3, feel better, Julie.
54.976 -> Thank you.
57.573 -> Just to quickly go over what
we're gonna talk about today,
60.847 -> we're going to kind of review simplifying
63.72 -> the software delivery process,
65.979 -> DevOps practices,
infrastructures, code, tooling,
69.525 -> a balanced infrastructure
approach to DevOps
73.32 -> if there is such a thing.
75.166 -> - Failure.
76.234 -> - And yes, dealing with failure.
78.512 -> We embrace failure.
80.336 -> And then finally, and I'm
really excited about this,
82.092 -> Cobus is gonna walk us
through a hands on demo
84.076 -> using GitHub Actions with
Terraform and I'm going
86.16 -> to supervise his work and
ask him questions throughout
88.65 -> to make him nervous.
90.46 -> (indistinct)
93.48 -> - This is us.
94.74 -> We have, oh, you can't even see it.
96.66 -> You would see some really
cool 8-bit animations
99.78 -> of us with our, you know,
101.707 -> doing a little dance but
we're gonna work through this.
105.66 -> - Yes.
107.867 -> - Okay, so I think we know, intrinsically,
111.373 -> that what we do is hard, but
I don't think we actually take
114.427 -> the time to fully appreciate
just how different
117.915 -> our work is today versus
5 or 10 years ago.
122.472 -> We have the most capable tooling anyone
127.479 -> in software has ever had.
129.849 -> We have access to
compute power and storage
132.33 -> that previous generations
could not even dream of.
135.656 -> But here we are, we're still
dealing with daily frustrations
139.334 -> and I think a lot of this
has to do with the fact
142.17 -> that we have to decide on
which tools and processes
145.934 -> and new methodologies to
adopt and then we have
149.97 -> to make all those tools
integrate together.
153 -> It's a lot of work on top of
already complicated systems.
157.74 -> On top of all of that
there is endless pressure
160.17 -> to move faster and to never...
164.52 -> Those pain points can
often look like this.
168.118 -> Teams are siloed, they're
blocked at almost all stages.
172.35 -> I mean, raise your hand if
you've had to deal with this kind
175.613 -> of chaos and confusion
in your day to day work.
180.27 -> I certainly have.
181.103 -> - [Man] Only three of four people.
182.28 -> - Only three or four! (laughing)
183.36 -> - Someone's not being honest.
184.975 -> - They're still asleep.
186.57 -> It's the first, two hands, I like that.
188.274 -> Thank you for your commitment.
190.711 -> When you, friction,
you can miss deadlines.
194.435 -> You feel like you have tons of pressure
197.91 -> and this is especially important right now
200.34 -> as we move faster and faster.
202.83 -> Being able to operate
reliably, consistently,
205.725 -> and provide value to your end
customer is deeply valuable.
209.171 -> CICD pipeline, DevOps practices,
211.351 -> all of this gives us agility.
213.215 -> By moving at this higher velocity,
215.625 -> we can build more effective teams
218.35 -> under a DevOps cultural
model which emphasizes values
222.227 -> such as ownership, and accountability,
225.251 -> and a willingness to fail.
230.212 -> There's really six key DevOps practices
234.146 -> that we want to talk about today.
236.225 -> The first is continuous integration.
239.592 -> This is really the software
development practice where
242.73 -> developers regularly merge
their code changes back
245.79 -> into a central code repository.
248.31 -> Why are you causing problems?
250.35 -> - You keep coming closer to me and then...
251.7 -> - Sorry.
253.389 -> - Or I smell.
255.642 -> - Don't run away, Cobus.
258.624 -> So you all, this is very common.
262.004 -> The CI part, I think a lot
of us actually operate on.
265.52 -> The CD aspect, this continuous delivery
268.38 -> or continuous deployment is where things
270.09 -> get a little bit more dicey.
271.41 -> This is where you have
an automated test suite.
273.72 -> Your code automatically
either gets merged in,
276.494 -> or if you're very bold,
deployed late to production.
280.24 -> Microservices are key, though apparently,
283.377 -> there's become debate around that.
285.595 -> That's fine.
287.19 -> I can't look.
288.023 -> I can't wait for next year's talk
289.23 -> on Make Your Microservices a Monolith.
291.51 -> That'll be exciting.
293.307 -> Okay, so now you have
infrastructure as code.
295.521 -> We all kind of understand this,
297.99 -> sometimes, we call it GitOps.
299.91 -> It's really about taking
infrastructure and making sure
302.496 -> that it's provisioned and
managed like we would with code.
306.57 -> We actually store it in
our source repositories
308.853 -> like you would application code.
311.101 -> Monitoring and logging,
super, super important.
315.24 -> I think sometimes we forget
'cause it's kind of old school.
317.76 -> I feel like we've always
had monitoring and logging.
319.89 -> - You just SSH the server.
321.39 -> - Yeah, there you go.
322.223 -> See it's fine, DevOps practices!
324.723 -> But really, capturing and categorizing
327.48 -> and being able to go back
and review what worked,
330.3 -> what didn't work, and why I
think is key to a lot of this,
333.27 -> especially if you're handling an incident.
335.727 -> Finally, my favorite thing,
337.92 -> communication and collaboration
is absolutely required.
343.17 -> That we are capable of communicating
345.81 -> with our team honestly and openly.
347.543 -> That we aren't hiding things,
349.89 -> that we have this failure
first kind of culture.
352.445 -> - Emily, you broke the build, again.
354.711 -> - I know, yeah, I wanted to...
356.481 -> (speaking faintly off mic)
358.508 -> Like, if you broke a
build and you got a sash.
360.84 -> You're like, I broke it today!
361.673 -> - Sash of shame.
363.45 -> - Yeah, sash of shame.
365.1 -> No, that's not the point.
367.054 -> But yeah, it's very, very important
369.06 -> that we have this kind of culture.
371.19 -> Looping back quickly to
infrastructures, codes, since
374.347 -> that's core to this session.
375.48 -> When we talk about IAC,
377.25 -> we're really talking about
managing the IT infrastructure
379.77 -> in an automated way.
381.78 -> Using configuration files versus GitOps
384.81 -> which builds upon
infrastructure's code by further
387.27 -> automating with Git repositories.
389.663 -> We could spend truly all day on IAC.
392.383 -> Not everyone builds stuff
on top of Kubernetes though
395.28 -> and so that's where you really
see GitOps kind of shine.
400.443 -> - Yeah, and we quickly
threw this one together
403.83 -> just to give an idea of what,
406.544 -> infrastructure as code,
and/or GitOps can look like.
410.31 -> And we were talking about this earlier,
412.432 -> and when you get down
to what's the difference
413.31 -> between DevOps and infrastructure as code,
417.024 -> and GitOps, it's probably
along the lines of messy,
418.988 -> but I can sell you five units
of it if you want a discount,
420.9 -> which is starting to happen
with GitOps now as well.
423.162 -> Tends to happen, it's like all of a sudden
425.1 -> this idea culture turns
into a, Here's a product,
428.31 -> come buy it.
429.36 -> But yeah, fine.
431.43 -> Okay, so we quickly touched
on infrastructure as code.
435.12 -> Basically, it's that whole idea
436.59 -> of treating it exactly the same.
438.635 -> Literally, you've got a repo
for your infrastructure,
440.01 -> you create pull requests to
review it by someone, hopefully,
442.877 -> before it goes to production,
444.39 -> it goes through some kind of (indistinct),
446.898 -> unlike this mic.
448.178 -> And then, ultimately, when you merge
450.78 -> to your main branch, it
goes out and is automated
453.018 -> or will be deployed in some fashion.
455.893 -> So, we spoke about a couple
of things we're gonna be doing
460.155 -> and we've got them in two
buckets at the moment,
462.51 -> what we're gonna be looking at today.
464.04 -> You can see a long list of things here.
465.42 -> I've played with everything
except SaltStack and Spinnaker,
470.28 -> which is why they were at the bottom.
472.369 -> I was looking for more names.
473.55 -> But basically, today we are
just gonna focus on Terraform
477.288 -> and GitHub Actions.
479.07 -> This is a lot of fun.
480.435 -> - (indistinct) This
organizational nightmare.
483.765 -> After implementing the
practices we just talked about,
487.65 -> we can now shift to
something much more simple.
490.86 -> And we do this by actually
firing five people
493.446 -> from every team.
494.529 -> (audience laughing)
496.14 -> - Bye.
497.974 -> - I'm kidding!
502.11 -> I think, where we are as an industry,
504.53 -> we get a little hand wavy with DevOps
507.057 -> and I might have contributed
to this, so I'm sorry.
509.435 -> I think somewhere along
the line we lost our way.
519.72 -> Put more and more load onto individuals,
523.12 -> specifically individual...
528.912 -> (indistinct) Of DevOps
has been this pressure
532.53 -> or belief that everyone
has to do everything.
535.956 -> That a JavaScript
individual has to know just
538.98 -> as much about infrastructure
and the CIC pipelines
542.31 -> as your operations engineer.
544.96 -> That's not exactly the best
way to find specialties
548.45 -> and to work as a team.
550.98 -> On the other side of that coin,
552.93 -> there's been this extreme
focus on this developer
556.17 -> and it's left operations engineers
558.91 -> feeling a little left out (indistinct),
561.434 -> somewhat ignored, unsure of their role.
565.59 -> I don't like that.
567.45 -> DevOps is not a methodology
that I encourages everyone to,
570.125 -> nor is it (indistinct) stack rings.
575.134 -> So that's not, we should be able,
581.08 -> (indistinct) our weaknesses.
586.84 -> We had something a little like this.
601.754 -> - That's your problem, not my problem.
604.38 -> - We would type into our
machines and then when it,
607.17 -> the program actually compiled,
608.61 -> it was someone else's problem.
610.397 -> DevOps really focused on principles
613.552 -> on the developer community
into, (indistinct)
627.938 -> I believe, DevOps, like this.
632.79 -> This is overlap, this is communication.
643.063 -> (speaking faintly off mic)
647.13 -> Collaboration, it's teamwork.
655.267 -> (speaking faintly off mic)
660.387 -> I am sorry because,
663.57 -> the lack of nuance in messaging,
665.73 -> this is a compression of
ideas and methodologies
670.08 -> and it all led us here.
672.09 -> I think it's important for
people to have ownership
675.03 -> and accountability over
the services they build.
692.91 -> We need to reverse out (indistinct).
700.609 -> This overlap.
702.45 -> This isn't to say that
operations isn't changing.
705.24 -> It is.
706.073 -> Infrastructure as code or GitOps is,
710.227 -> (speaking faintly) and repeatable.
713.13 -> This is awesome.
715.501 -> People who do that work have to identify.
719.126 -> For me a balanced approach means
722.193 -> that there are fundamental
opinionated approaches
726 -> to infrastructure at
the organizational level
728.538 -> and that every individual
service team can benefit
732.399 -> from the existing accounts, environments,
736.557 -> pipelines and more.
739.86 -> You can call it platform a
team or (speaking faintly).
742.83 -> it doesn't really (indistinct).
751.884 -> - So the names don't matter.
755.043 -> (speaking faintly) work looked like.
760.151 -> You can see specifically like if you work
763.02 -> with any kind CICD system,
764.82 -> you've got multiple steps
inside your pipeline
766.8 -> where you can see in the bottom,
768.952 -> and then where the, (indistinct)
773.202 -> is that we're gonna use GitOps
Actions with our AWS account
776.999 -> and we're not gonna be sharing
any credentials with it.
780.12 -> We're gonna be using
one of the new features.
781.89 -> Amazon allows you to just--
784.874 -> (overlapping discussion)
786.681 -> - I think we missed it.
788.474 -> - Amazon OpenID Connect.
789.33 -> - Thank you.
790.163 -> - Yes, cool.
791.043 -> Permissions and I'll share the demo,
792.84 -> how we're gonna get all of that going.
794.58 -> But this is roughly where
I initially aimed for.
797.82 -> The name is gonna be slightly different.
798.93 -> We won't get anything running
up and running because DNS
801.24 -> takes time and I'm setting up,
803.912 -> completely new account that I set up
806.276 -> with nothing in it.
809.34 -> So this is just me, image for construction
815.09 -> and for something to talk
about infrastructure as codes.
817.68 -> I thought, you know, nice
sexy beards, and you know,
820.651 -> this is what you do when you plan it.
822.81 -> Infrastructure everywhere.
827.88 -> Chatting to people, can I
actually just switch off my mic?
830.73 -> I'll do projection, loud voice.
833.69 -> (speaking faintly off mic)
836.37 -> Sorry.
837.203 -> (speaking faintly)
843.523 -> - [Man] Project?
844.617 -> - Project, can we mute my mic?
848.511 -> - [Man] I think the mic--
850.29 -> - Cool, let's see, let's see, let's see.
852.18 -> Okay, I'm still there, I'm still there.
853.74 -> I'm not there, is the mic still on?
855.99 -> Can we completely, can I just--
858.125 -> - Boom.
859.588 -> - It is one better than
the time I had to project
861.96 -> with the mic on my beard?
863.79 -> Yes, I've done that.
865.659 -> So getting back to the
important questions,
867.12 -> should you use Terraform, CDK,
868.74 -> CloudFormation, or one
of the other mini tools?
871.8 -> Now my friend and twin
from Serbia has an answer
874.671 -> to that if you haven't seen this before
876.93 -> on one of our streams.
878.712 -> Yes.
880.971 -> But I went through CloudFormation,
883.687 -> done a bit of CDK, I've
used Terraform a lot
887.01 -> since about 2014 I think.
889.453 -> And what that's taught me
is that the different tools
892.994 -> are comfortable with the mindset.
895.831 -> Not every mindset is the same as we know.
898.11 -> So in other words, play with at least two.
899.76 -> That's the advice.
900.593 -> Take one that is declarative, (indistinct)
903.395 -> like for example,
careful oral confirmation
904.44 -> and then try something else
that might be programmatic
906.78 -> like GLOO-MEE or CDK.
908.222 -> Just give two of them
a go, see what happens.
910.65 -> You learn things along the way.
912.33 -> Younger Cobus thought
that the tools were crap.
914.16 -> That's why I kept on
switching and then later
915.54 -> Cobus realized that you
know how to use them.
917.37 -> So yeah, play with a couple to figure out
919.17 -> which ones work best for you.
922.08 -> Then infrastructure
automation always has...
928.89 -> Guess we cut off.
930.09 -> So I'll show you now we're
gonna cheat a little bit.
932.703 -> (indistinct)
936.381 -> (speaking faintly off mic)
953.7 -> Literally brand new, nothing in that,
955.23 -> and where we want end up is a setup
957.18 -> where we've five total accounts.
961.151 -> (speaking faintly off mic)
970.786 -> It's got the information that's in there.
973.38 -> So then, right, and not only things like,
979.352 -> thing that's common across
all the different environments
982.32 -> that you're gonna be spinning up,
983.821 -> so let's say a bulk job
or a container registry
985.8 -> where you wanna store
your image 'cause you...
991.491 -> And then finally at the bottom,
992.64 -> we've got three different
environments, as I call them.
996.621 -> (speaking faintly)
998.613 -> And what the image shows
you is how we are actually
1001.37 -> going to a feature called (indistinct).
1008.84 -> Between different accounts.
1013.592 -> As we will from the GitOps action side,
1016.787 -> reach into the AWS account
using OpenID Connect,
1019.207 -> so no credentials,
1020.57 -> share those just based
on the configuration...
1026.975 -> (indistinct)
1054.563 -> To the other account...
1057.95 -> 'Cause like I said,
fooling around in that.
1066.667 -> There we go.
1068.513 -> Okay, we have a mic that's, whoa,
1071.196 -> (audience applauding)
1074.31 -> nice, which karaoke song first?
1078.8 -> I believe Final Countdown
would be appropriate
1080.63 -> so we can get you the demo.
1082.332 -> Cool.
1084.26 -> Alrighty.
1085.43 -> So like I was saying,
1086.99 -> initially we're gonna do
that chicken/egg situation.
1089.36 -> We're gonna resolve that with
a couple of shell scripts
1091.64 -> and then we're gonna
head over to Terraform
1093.826 -> and GitOps Actions to provision
everything else we do.
1095.87 -> So it's effectively
setting up the permissions
1097.727 -> for GitOps Actions to get
into that main account,
1100.28 -> is gonna be done by
Terraform with GitOps Actions
1103.16 -> on the billing accounts.
1104.748 -> It's gonna reach all the
way into that one as well,
1106.688 -> environment accounts.
1107.521 -> And then once that is in place
we kick off a second repo,
1109.352 -> which is then gonna set up
everything in our main account
1111.886 -> that then from there GitOps
Actions comes directly into Main
1114.924 -> and then from there reaches
1116.33 -> into the different accounts there.
1119.36 -> The reason we do this
is that it means that,
1121.1 -> in your main account
you set the IAM policies
1123.62 -> for who can do what.
1124.453 -> So this is where you'll
have a different policy
1126.08 -> for your engineers that
need to work on dev testing
1128.96 -> and staging and they have
different permissions
1130.73 -> based on which environment
they're dealing with.
1132.809 -> So what we're gonna try and do is keep
1135.463 -> the shared infrastructure
in the accounts example,
1138.86 -> like I mentioned, the container image,
1140.63 -> the main account, and then
each of the different accounts
1142.79 -> can reach into it to actually grab a copy
1144.53 -> when it wants to do a
deployment over there.
1147.47 -> And what this gives you,
and the reason I like
1149.48 -> approaching Terraform
this way is that because
1151.284 -> we're gonna be using variable files,
1153.5 -> and I'll show all of this with Terraform.
1155.39 -> There's no copy and pasting
of your infrastructure code
1158.03 -> between your dev test and
production environment.
1161 -> When you define infrastructure,
1162.17 -> it's in one repo for let's
say your base environment
1164.823 -> or for a specific service.
1166.759 -> And then what you do is
you've got variable files
1169.01 -> to change the quantities,
1169.843 -> which means if you've got dev and testing,
1171.92 -> mainly look the same.
1173.932 -> But then when you start
getting into production,
1175.73 -> you want to have maybe
a couple more instances
1177.71 -> of the container up and running.
1178.73 -> So you just change the
variable that defines
1180.2 -> the quantity of that or the size of that.
1182.69 -> So there are some you can pull
if you don't wanna spin up,
1185.373 -> you can do that with a bit of testing.
1190.87 -> - I think your mic might be failing again,
1193.4 -> which is--
1194.233 -> - Again.
1195.066 -> - Killing me.
1195.899 -> (laughing)
1203.33 -> - That's incredible though.
1206.137 -> - Cool.
1207.26 -> Okay, so the reason for
this is we want to get
1210.073 -> to infrastructure parity where basically
1211.19 -> we don't have to deal, oh,
1213.53 -> remember piece of paper, checklist,
1216.046 -> create that queue or
remember to copy and paste
1218.88 -> because an issue where
worked well on my laptop
1222.609 -> and Dave and then didn't
work in production...
1228.64 -> (laughing)
1231.044 -> Cool, okay.
1233.33 -> And with that we are
gonna start having fun.
1235.4 -> We have got about 30 minutes.
1237.35 -> I'm hoping that's enough
to go from nothing
1239.51 -> to multiple accounts and things running.
1242 -> - I'm excited.
1243.41 -> - Well let's see if this works.
1245.3 -> I'm just checking in.
1246.133 -> Can you hear me okay through this?
1248.06 -> Thank you.
1250.975 -> Would it be better for
you all if Cobus projected
1254.932 -> or if we passed the mic around?
1257.775 -> Pass the mic.
1259.28 -> I like it.
1260.6 -> All yours.
1261.95 -> - One-handed typing, no
challenge whatsoever.
1263.463 -> Okay cool.
1264.462 -> So do I need--
1266.222 -> - I can hold it for you.
1268.34 -> - Why, oh nice.
1270.222 -> (audience laughing)
1271.43 -> - I think this makes me like
1272.39 -> the world's most expensive
microphone stand.
1274.355 -> (laughing)
1276.609 -> - So many inappropriate jokes I can make.
1280.94 -> - All right, how to get started.
1282.86 -> - First I need to figure out
how to get my screen up there.
1284.72 -> I remember pushing buttons.
1286.19 -> Do I push them?
1289.34 -> - Jill, or how do we
get this screen to show?
1293.12 -> - I remember red button
and the blue, green button.
1295.25 -> Do I do it?
1296.651 -> - I don't know.
1297.542 -> I feel like this is diffusing something.
1298.906 -> (overlapping discussion)
1300.154 -> - Okay cool.
1300.987 -> You're amazing.
1301.844 -> Let's check, make a little bit bigger.
1304.34 -> That's good. Cool.
1305.45 -> We've got that.
1306.44 -> We've got this one.
1308.15 -> Let's just refresh.
1309.65 -> - [Emily] Can you make
it just a little bigger?
1313.9 -> (laughing)
1316.986 -> (indistinct)
1325.535 -> Perfect.
1329.821 -> It's too early.
1331.52 -> - Yes, I know it's too early.
1332.99 -> So two things I forgot to mention before
1334.52 -> we kick off the demo.
1335.353 -> Firstly is this session is being recorded
1337.31 -> which is great news for you
and terrible news for me.
1339.5 -> Which means if you don't
see me here next year,
1340.97 -> you know why.
1342.74 -> And secondly, I will be taking everything
1344.78 -> that I've done here
with the different repos
1346.76 -> and creating a series of
tutorials that we'll publish
1348.687 -> probably end of Jan or
during the course of Jan
1351.313 -> 'cause December's re:Invent
Recovery Month and Planning.
1353.755 -> But everything we'll go,
there will be recording,
1356.791 -> so if I do go a bit quickly,
1358.96 -> you'll have everything afterwards,
1360.35 -> so don't worry about that.
1362.295 -> Okay, so remember when I said we need
1364.981 -> to get things going first?
1366.77 -> So what I've got here,
the colors are okay,
1369.53 -> sorry I should have gone high contrast.
1372.02 -> But what I've done is I've
created a little shell script
1374.57 -> that is very advanced.
1376.97 -> It's got five commands I think.
1379.16 -> Yeah five.
1380.18 -> So effectively all that
this does is it calls
1383.166 -> some AWS endpoints.
1384.23 -> It creates a S3 bucket for us.
1386.747 -> It creates, enables versioning on it.
1390.139 -> 'Cause with Terraform
we're gonna be using S3
1392 -> as our backend to store our state file.
1394.82 -> Quick show of hands,
1396.93 -> who has worked with Terraform
and state files before?
1398.84 -> - [Emily] That's incredible.
1400.16 -> - [Cobus] Nice.
1401.446 -> - [Emily] It's very popular.
1402.279 -> - [Cobus] Okay, for those that
haven't, quick version here,
1404.354 -> what Terraform does is you define
1405.23 -> what infrastructure you want,
1407.777 -> you then tell it to go
figure out what's going on.
1408.83 -> It goes and looks at your AWS
account or other resources
1411.32 -> 'cause it has many other providers,
1413.03 -> and then it figures out what it has sought
1415.19 -> from the last run in the state
file, what's the difference
1418.04 -> it needs to apply.
1420.416 -> Then it'll go out and
when you say, okay, apply,
1421.46 -> apply those changes to infrastructure,
1422.64 -> and then write that to its own state file.
1424.477 -> So it keeps effectively, when
it saw the rule was lost,
1426.861 -> what you want the rule to look like
1429.14 -> and what it actually looks like
1430.16 -> and then figures out that mess.
1431.87 -> And it would solve so many things.
1433.703 -> We could use this in other
areas of life as well.
1435.764 -> So that is why we need an S3 bucket.
1438.501 -> So what I'm doing here
is just creating it,
1440.36 -> enabling versioning on it.
1441.8 -> And then what I'm doing is creating
1444.128 -> that Amazon OpenID Connect provider.
1445.46 -> And what you can see
over here is literally,
1447.47 -> is provided by the OS,
1449.46 -> so that's where the
request is gonna come from.
1450.53 -> There's a little fingerprint
for the certificate
1452.57 -> that GitOps supplies.
1453.83 -> It's public, you can
get it off those sites.
1455.66 -> So nothing special there.
1457.07 -> And then what we're doing is
we're creating that IAM role
1459.791 -> that gives the GitOps Actions permission
1461.3 -> to actually access our account
and do things in our account.
1464.54 -> And then also, that's just
the trust policy portion,
1467.15 -> and the second part is, now
what can I do in the account,
1469.28 -> is the actual IAM policy,
1470.51 -> and this is a demo so obviously I'm going
1472.94 -> with YOLO admin rights.
1474.933 -> You can lock this down,
1476.18 -> you can have multiple different roles,
1478.312 -> you can really get very fine
grained here if you want to.
1480.713 -> I just wanted to work at the moment.
1483.77 -> So we've got this script.
1485.75 -> So what I've done is I have preemptively
1488.387 -> copied it over here.
1489.22 -> Oh, I was waiting for that.
1492.23 -> There we go.
1501.356 -> Do I remember it?
1502.61 -> Yes.
1503.443 -> Okay, also don't be like me, put MFA on.
1505.761 -> - Yeah that was a short password, Cobus.
1508.485 -> - Can we stop with that?
1510.89 -> I'm getting self-conscious now.
1512.69 -> (speaking faintly off mic)
1515.12 -> Cool.
1517.032 -> All I've done in this new
account, you can see the fresh,
1519.318 -> literally last night, I've created the,
1522.497 -> it's on the screen, yeah, the
script, so the bootstrap one.
1523.73 -> So this is the script
I just copied across,
1525.68 -> and in here you can see
there's a brand new name
1529.165 -> and just the name for the IAM role,
1531.14 -> the braces all standard in here.
1533.45 -> And then in the actual trust policy.
1536.965 -> In here, what we can see
it says is the principle,
1541.775 -> the federated principle,
so that's the identity
1544.67 -> that we are allowing in,
1545.93 -> is via this specific OpenID provider.
1549.951 -> And the one thing you
do have to pace in here
1552.92 -> is the account ID.
1554.27 -> So I pre-pasted that last
night when I created it.
1556.79 -> And other than that in here,
1558.14 -> the way the magic really
happens off screen
1560.42 -> because it's too big is
here's where you limit
1563.21 -> who can run this.
1564.043 -> And this part is extremely
important because if you,
1568.144 -> sorry, do not, if you set this
just to a wild card string,
1571.28 -> it means that any GitOps repo can do this.
1573.446 -> So if I see your public
repo and I fork it,
1575.81 -> I can go and, yeah, order you some shoes.
1578.96 -> I don't know.
1580.22 -> - [Emily] I would love shoes.
1581.669 -> - [Cobus] That's how that works.
1582.824 -> - [Emily] We didn't tell
them about our shoes.
1584.552 -> - [Cobus] Oh yes, sorry.
1585.93 -> - [Emily] Did you all notice
that we have AWS shoes.
1588.084 -> (audience laughing)
1589.37 -> I think they're cool.
1593.162 -> They are.
1595.46 -> - [Cobus] Right, cool, so this is the part
1597.35 -> where you wanna focus.
1599.15 -> See Emily, focus?
1600.41 -> - [Emily] No.
1602.13 -> - [Cobus] Okay, so you
can see here what I did
1603.912 -> is that I specifically limited to a repo
1607.437 -> in my account called BOA328-billing-fresh.
1612.829 -> And the colon asterisk is the wild card,
1616.19 -> all the different branches in git,
1618.404 -> when GitHub to allow,
if I do wanna do a PR
1620.93 -> versus main branch consider
that you can limit this
1623.36 -> once again to different things,
1624.32 -> which gives you
interesting options to say,
1626.42 -> well, when I'm doing a PR
I've got these permissions,
1629 -> but when I'm doing a main merge,
1631.01 -> I've got different permissions and you
1632.09 -> can actually manage it here and set it.
1633.59 -> Obviously if someone knows
how the things are set up
1636.193 -> and they have access to your source repos,
1637.873 -> you are kind of already
slightly in trouble.
1640.579 -> But yeah, so that's what we've got here.
1642.74 -> So with that we are
going to quickly run that
1646.28 -> and this is the first part where I hope
1648.08 -> I didn't break anything.
1648.98 -> So we've got our buckets,
we've got our OpenID Connect,
1651.657 -> we've got the role, is this now done?
1656.207 -> Oh yes, the quit.
1658.542 -> There we go, it seemed to work perfectly.
1661.88 -> Okay, so far so good.
1663.233 -> Nothing has really happened.
1665.3 -> But what I can show you now
is if we go to the console,
1669.525 -> oh nice, that is very big.
1674.18 -> - [Emily] It's massive because--
1675.83 -> - [Cobus] Okay, we're gonna be
jumping between these a lot.
1678.74 -> Okay I am, I am here.
1683.423 -> Cool.
1685.31 -> And we go to identity providers.
1687.74 -> We should see our GitOps
Actions identity providers
1690.8 -> set up here.
1691.67 -> Ooh okay, that's horrible.
1693.974 -> - [Emily] But that was just
from what we set up, correct.
1696.68 -> That didn't preexist.
1697.7 -> - [Cobus] We only created
a bucket, we versioned it,
1699.35 -> we set up the OpenID
Connect, and then we created
1702.074 -> an IAM role with a policy attached to it
1704.33 -> and a trust policy.
1705.32 -> Cool.
1706.674 -> So nothing too far now.
1707.507 -> So now the refund part comes
where things can go wrong
1709.715 -> because I need to start copying things.
1711.74 -> So what I've got here is I have a repo
1714.8 -> and that was still early in the morning.
1717.71 -> I did a nice cleanup.
1719.755 -> So we're starting off
with nice empty repos
1721.16 -> as well 'cause we're
gonna do this properly.
1722.84 -> We now have our
infrastructures code bootstrap
1725.06 -> so we can start creating a PR
1726.68 -> and then start setting everything up.
1729.351 -> Obviously a couple of steps
there with dependencies,
1731.219 -> but let's get this going.
1732.845 -> Okay, so is that, yep, that's big enough.
1734.84 -> Cool.
1736.73 -> I'm not laughing.
1739.286 -> So firstly what have we got?
1741.954 -> We have got some Terraform over here.
1744.8 -> The important part is the providers one
1746.69 -> which we set up first.
1748.304 -> So what we did is, that's
already the first thing
1751.28 -> I have to fix because I
remember my notes now,
1753.71 -> which I didn't make is,
what did I call my bucket?
1758.54 -> There we go.
1760.048 -> Cause this is the first
thing that'll break
1761.3 -> if I can't find that bucket.
1763.34 -> So there we go.
1764.6 -> That's my new bucket.
1765.83 -> And then all I'm saying over here
1766.97 -> is when don't you configure Terraform,
1769.001 -> it uses the what's known as HCL,
1769.834 -> HashiCorp configuration language.
1772.22 -> It's JSON-like syntax
except you don't have
1774.2 -> to deal with a lot of
the quotes around things.
1776.51 -> You can put commas at the end of lists
1778.28 -> and not have to worry about it.
1779.84 -> It's very nice.
1781.79 -> Here we say, okay, the backend is S3
1783.829 -> and what we're doing is we are saying
1786.469 -> the required providers here is just AWS
1788.205 -> and you can do some
version management here
1789.038 -> if you wanna pin the specific versions.
1791.06 -> Then what we're doing
over here is you can say,
1792.56 -> we're saying we're gonna use US East One.
1795.35 -> And the reason this is
different from that region
1797.33 -> is the top region is
where is my configuration
1799.76 -> state files soared.
1800.593 -> So we say bucket, region.
1802.43 -> This is where am I creating
infrastructure by default
1804.784 -> when using the AWS provider.
1807.29 -> Okay then what we have over here is,
1810.77 -> this is the part I still
need to comment out
1813.53 -> because we don't have accounts yet.
1816.2 -> So do that, and that's
where it's gonna break.
1818.69 -> But we're gonna get to that part back.
1821.177 -> But so for now, we've got our,
1822.567 -> we can talk to our billing
account and we can use things.
1823.7 -> Then in terms of the actual
content we're gonna be creating
1826.25 -> is we'll see in the main account,
1828.65 -> everything is commented out.
1829.94 -> Lovely, can't break.
1831.742 -> In the environment accounts,
everything is commented out,
1833.03 -> can't break.
1833.99 -> DBS, which is always fun,
is also commented out.
1836.662 -> So what we effectively
have here is we've tested,
1838.942 -> set up the basics for Terraform
to be able to do things
1842.6 -> but we don't specify anything yet.
1844.19 -> So first step is let's quickly
put that on a PR branch
1846.83 -> to see if our stuff is working.
1854.69 -> - [Emily] It's commented out as a default.
1856.82 -> That's useful.
1858.471 -> - [Cobus] No, no I edited that.
1859.956 -> (speaking faintly off mic)
1861.802 -> It's artisanal.
1862.793 -> - [Emily] Artisanal Terraform, I like it.
1869 -> Is that just like Terraform
that's twice as expensive.
1871.7 -> Got it.
1872.533 -> - [Cobus] But you get
discount on five units.
1877.61 -> Cool.
1878.443 -> Well I need to do this.
1879.89 -> I'm lazy, I always forget this.
1881.39 -> Come on, so I just copy and paste, YOLO.
1883.343 -> Okay cool.
1884.18 -> Well that is pushing the one
part I didn't touch on here
1886.25 -> is this little friendly file over here,
1887.87 -> which is the thing that tells GitHub
1889.51 -> how to actually run workflows.
1892.032 -> So there's a YAML file over
here with a bunch of steps.
1895.152 -> So effectively give it a nice name,
1897.38 -> some info and then you
specify when it runs.
1899.985 -> So we've got it running
on the branches of Maine
1903.29 -> and also on all requests.
1904.742 -> And then it needs some permissions here.
1906.935 -> The ID token is required to
do the JWT token exchange
1910.292 -> with OpenID so you need to give it
1912.26 -> a write permissions there.
1913.85 -> And then for the content itself,
and this one is read only,
1919.22 -> it doesn't need write
'cause it's not writing
1921.26 -> through the repo itself.
1922.34 -> What it does do but further down,
1923.75 -> which I'll show you is
actually writes a comment
1925.58 -> on the PR itself.
1927.02 -> So what it does then is does the checkout,
1929.03 -> which is just normal and this
is where the magic happens
1932 -> with the OpenID Connect,
which is I provide it
1934.58 -> with the IAM role to assume
including the account ID
1938.42 -> and I give it some kind of name,
1939.47 -> in this case the session
images, GH Actions billing,
1943.304 -> 'cause I can track it in my
logs and see what's happening,
1946.301 -> and that's it.
1949.16 -> As you can see I have not copy
and pasted a single API key
1951.605 -> or secret in here and that's
where the fun part comes in.
1954.2 -> So that's there.
1955.49 -> Then we just go, we set up Terraform.
1957.05 -> I love version pinning because
I've had issues in the past.
1959.462 -> Then we do a format check,
1961.523 -> which is just Terraform's
got a built in command FMT
1964.323 -> that tells you whether or not something's
1966.83 -> formatted correctly and
you'll see that in nice use
1969.11 -> at the end.
1969.943 -> - [Emily] That's great.
1971.39 -> - [Cobus] Let me show you the end then.
1973.474 -> There is my favorite part.
1975.26 -> If you don't format your
Terraform code correctly,
1977.45 -> I will not allow your build to pause.
1979.43 -> It will fail.
1981.08 -> I've had too many
religious arguments about
1983.438 -> what formatting to use.
1985.19 -> Terraform ships with a default one,
1986.93 -> so how about we use it and stop arguing,
1988.812 -> build some infrastructure.
1989.99 -> Cool.
1992.538 -> So getting back to what we have up here,
1994.79 -> when you start with Terraform,
1995.797 -> you've got the backend configured
1997.31 -> and now you, on a new
machine, there might be
1999.14 -> some state files you need to initialize.
2000.55 -> It also pulls down the providers,
pulls down the plugins,
2003.305 -> stores in a little hidden
directory on the actual directory
2006.293 -> that you're running it from,
2007.93 -> and then what we have
is we first go plans.
2010.39 -> Terraform's got this lovely
feature where it says,
2013.12 -> this is what I want the
infrastructure to look like.
2015.04 -> Go plan it and show me
what the changes are,
2016.373 -> which is one of the main
things that swayed me way back,
2018.509 -> I think, it's 2014 when I started,
2020.24 -> it is like the ability to see
what it wants to do before,
2022.498 -> you know, enter yellow, I
hope this works on production,
2024.78 -> the scenario was like, that
sold me so I kept using it.
2027.513 -> So this'll tell us,
you know, does it work?
2030.218 -> What you can see over here is
I am saying continue on error,
2032.828 -> which sounds odd at first.
2034.87 -> But the reason for that is that
in the pull request itself,
2037.6 -> we actually publish the output
of the different stages,
2040.105 -> did the initialization
work, formatting work,
2042.828 -> the plan work, and the
validation steps work.
2045.61 -> That's just a bunch of
steps that makes super easy
2048.22 -> to make sure that the
Terraform is actually accurate.
2051.507 -> And then what it does, it
actually publishes the details
2053.71 -> inside the pull request of what happens.
2056.136 -> So you can see the actual output
2057.46 -> that you would have in
the GitOps Action runner
2059.92 -> is basically here.
2060.753 -> So you can see what Terraform outputted
2061.92 -> on the shell while it goes long.
2063.16 -> Okay cool.
2064 -> Then it does my little formatting check,
2066.333 -> and then it does here,
is a step that says,
2069.85 -> if the plan stage failed, we
are gonna exit this workflow.
2073.793 -> Because when we go to
production or merge the main,
2077.38 -> we definitely don't want to
deploy anything that's broken,
2080.62 -> 'cause guess what, you're
not gonna have a fun thing.
2083.125 -> So what you'll see there is that
2084.728 -> that's where it says
it'll exit out for that.
2085.72 -> And you'll see in some of the other steps,
2087.94 -> for example, the plan one,
there's a little if in here.
2090.13 -> So this is where we only
do the planning phase
2093.04 -> when we are on the branch.
2095.11 -> Now I know that you can
do the plan and output it
2096.417 -> to the file and then execute that file,
2099.189 -> which is a bit safer.
2100.022 -> But with multiple branches
and things especially
2101.35 -> we go very quickly with
infrastructure changes
2104.472 -> that sometimes gets a bit messy to handle
2106.09 -> or you need to do sequential builds.
2108.52 -> So in this case we're
getting a little YOLO
2109.93 -> and saying as soon as you merge master
2111.67 -> we are just going apply auto-approved.
2114.34 -> Thank you, let's hope it works.
2115.81 -> Cool.
2117.38 -> So while that was running and
it's actually fairly quick,
2120.426 -> so I don't, oh, it didn't run because
2121.48 -> I don't click pull request.
2122.53 -> Let's do that quick.
2123.94 -> This does run fairly quickly.
2125.44 -> So Actions, let's go over here.
2128.807 -> Is this big enough?
2130.99 -> Yeah, let's make it a little bit bigger.
2133.87 -> Okay, so it hasn't
picked up that file yet.
2136.468 -> Let's give it a second.
2139.36 -> Did I push? Yes I pushed.
2140.964 -> No this just takes a while
for it to pick it up. Come on.
2147.1 -> Did I?
2148.81 -> Sorry?
2150.243 -> (speaking faintly off mic)
2151.076 -> No, no I'm not intending to merge.
2152.245 -> I want this to run on the pull request.
2154.341 -> So let's see, let's hope it works.
2158.749 -> Did I actually include,
wait, wait, you know what?
2162.809 -> Let's double check if I included
the GitOps file that is...
2165.885 -> (speaking faintly off mic)
2169.848 -> - [Emily] Do you have it on a line?
2173.103 -> - [Cobus] Okay, interesting.
2177.28 -> I was sitting in this work last night.
2179.836 -> Okay, let's then do the
yellow part and merge
2184.841 -> 'cause that's, yep, come
on, merge, confirm merge.
2190.979 -> Thanks for that.
2192.85 -> I actually forgot about, I
didn't know about that part.
2196.87 -> There we go, thank you.
2200.17 -> - [Emily] You're awesome.
2202.005 -> - [Cobus] Yes, canna come take over?
2204.238 -> (audience laughing)
2205.47 -> Cool.
2206.5 -> So now what we've got going
here is GitOps Actions
2209.71 -> is starting, oh sorry, not that one,
2211.66 -> this one over here.
2213.414 -> We can see it running
through the different steps.
2214.42 -> It's hanging on this one,
2215.993 -> and this is the one where
I always get nervous
2217.9 -> 'cause when it hangs here it means
2219.07 -> that I messed up something
with the IAM policy.
2221.518 -> I get the feeling I'm gonna
go back to my backup account.
2226.137 -> Let's see quickly, okay,
this is taking too long.
2230.981 -> That takes too long, it
means that my, in here,
2237.265 -> let's get my account ID,
please don't steal this.
2241.857 -> Actually you can, shouldn't do anything.
2244.619 -> Let's double check actually in here, IAM,
2246.758 -> roles, GitOps Actions.
2254.464 -> Where's my, sorry I just have
to go a little bit smaller.
2258.64 -> Trust relationship,
we've got that in there.
2262.3 -> Is that this edit?
2266.92 -> Ooh this is rough.
2269.95 -> Let's double check, I got that right.
2273.35 -> Yep, and I have got the,
oh sorry, good example.
2277.68 -> The IAM policy was set
up with what I thought
2280.894 -> was going to be the new
repo and I decided not
2283.27 -> to change that, so that
is why it's breaking.
2285.512 -> So now you can see is if you
don't have the right IAM role,
2288.1 -> it actually will not
allow you to do anything
2290.618 -> in the account, which means
this should have failed by now.
2293.02 -> Sweet.
2294.49 -> Let's just cancel.
2298.033 -> Sorry.
2304.75 -> Any good jokes for us, Emily?
2306.76 -> - [Emily] I do.
2308.031 -> There was a snail and the snail
2309.973 -> wanted to buy a sports car.
2313.965 -> So he goes to the the
dealership and he asks
2317.08 -> for a sports car, he asks
for a red sports car.
2319.42 -> So the dealer's like no
problem, we've got that.
2321.175 -> And the snail's like just one more thing.
2322.602 -> Dealer's like, Sure what do you want?
2324.933 -> He's like, I want an S
on the back of the car.
2327.516 -> The dealer's like okay we can do that.
2329.62 -> But I mean why?
2330.674 -> So that when people see
me drive by they say look
2333.516 -> at that S-cargo.
2335.212 -> (audience laughing)
2339.55 -> No, thank you, thank you.
2343 -> - Okay sweet, so what has
happened now in the meantime
2345.79 -> is that that was actually
the issue, so it's working,
2348.28 -> and I forgot to open up the one in here.
2352.24 -> This one actually had code in it.
2353.92 -> So this is our starting point
2355.983 -> where we're gonna set up
some new database accounts.
2356.83 -> So I'm using Terraform to actually build
2358.18 -> all those other child accounts,
2360.25 -> the main one, the dev
test, and production one
2362.95 -> as well that I find in here.
2364.63 -> A fun trick for those that
are not familiar with this
2367.33 -> is that in this instance
you can see I just
2369.7 -> use different emails on my personal domain
2371.74 -> just to get it in there.
2373.44 -> But if you do want to
use the same mailbox,
2374.273 -> you can literally just put a plus there
2376.404 -> and put whatever you want in there
2377.5 -> and then reuse the same mail address that,
2379.27 -> plus it'll just still be in the mail,
2381.544 -> the two mail, but it'll
come to the same email box,
2384.37 -> so lots of fun things you
can do then not to have
2386.08 -> to set up like a million mailboxes
2388.24 -> just to set up more AWS accounts.
2389.53 -> But any case, what we could
see here in the background now
2393.22 -> was that this, is it still running?
2397.06 -> - [Emily] Make it a little bigger.
2398.5 -> - [Cobus] Oh yes, there we go.
2400.774 -> It actually is completed successfully.
2402.52 -> And now if we go in here and
look at Edward's organizations,
2409.224 -> we should see a lot of fun over here.
2413.92 -> So come on, sorry, I hit the touchpad.
2416.729 -> What we can see is we've
got our main account,
2418.93 -> we've got our dev
production and test account,
2421.42 -> and then also I've set up
the organizational units
2424.012 -> to actually do a tier
defect in terms of just
2426.49 -> to keep an nice and neat in here.
2427.81 -> So if you want attach like other systems
2430.48 -> 'cause you can connect for
example domain controllers
2433.36 -> to provide permission
to certain environments
2435.61 -> and things here you can do a
lot of interesting integrations
2437.77 -> if you want.
2438.603 -> But the important part for
us is that I need to get
2440.53 -> some account IDs.
2442.15 -> So we're gonna pop these across
and we're gonna go over here
2445.57 -> into this variable.
2446.68 -> So I need the main one, I
need, and this is hopefully
2452.651 -> the last of the copying
and pasting we have to do.
2456.432 -> This is not gonna go well,
let's not switch test--
2459.061 -> - [Emily] And this has
switch test and prod.
2460.3 -> Why not?
2463.75 -> This has to be manual.
2464.77 -> There's no way to automate this part.
2467.89 -> - [Cobus] No way to
automate it an easy way.
2469.6 -> You can go funky and
say split the Terraform
2472.57 -> into multiple like units where as soon
2475.24 -> as the first one complete,
2476.14 -> you output the account ID into a file,
2478.87 -> then kick off the second one.
2480.52 -> I just feel when it gets that complicated
2482.53 -> for something you're gonna
do once, hopefully twice,
2484.679 -> maybe don't go that far.
2487.15 -> It gets a bit rough.
2489.026 -> There's a very nice XKCD on it, cartoon,
2490.81 -> so xkcd.com search for, I
think it's worth automation,
2494.722 -> gives a national graph
about the time of automation
2497.8 -> spent on something versus how
much you get back out of it.
2500.146 -> Definitely go look at that.
2501.73 -> Okay cool, so we're getting along nicely.
2504.479 -> We have got about seven minutes left.
2506.349 -> Let's see if we can get a container built
2508.639 -> and an environment set up.
2510.138 -> So we've got our accounts.
2511.453 -> Now what we're going do is
we are going to go provision
2513.585 -> that main account so we can
actually get GitOps Actions
2516.07 -> to work directly with it.
2516.97 -> So this is just the same as
we had in that bash script,
2520.54 -> which is creating the different,
2522.28 -> the bucket with the rules
around it and versioning,
2524.17 -> setting up the IAM policy.
2525.76 -> Little bit different in here
where it allows different IDs
2529.06 -> or to go to the different
accounts there as well.
2532.07 -> And then here's where we set
up the OpenID Connect provider
2535.406 -> again for GitOps and yeah
a lot more I policies
2538.805 -> just to deal with all the missions
2540.464 -> from going from GitOps Actions into main,
2542.23 -> into dev, prod and testing.
2544.091 -> So that is that part.
2546.147 -> Then what we also wanna
do is we wanna reach
2548.189 -> into the development, test,
and production accounts
2552.911 -> to set up the required roles there.
2555.016 -> Because when we're coming in from billing.
2557.53 -> Mainly at the moment, can't
get into those three accounts,
2559.96 -> we set that up as well to allow main in
2561.55 -> and then GitOps Actions from that side.
2563.29 -> Cool.
2564.559 -> And then the last part,
2566.2 -> which is why I won't be
doing a live, live demo,
2568.69 -> is we always have to deal
with DNS, which is fun.
2571.254 -> So this sets up three DNS zones,
2573.4 -> one in in dev, test, and production,
2575.05 -> and how it works is devs
got the sub domain dev,
2578.67 -> testing is test.whatever,
and then production
2580.897 -> is the main production one.
2582.392 -> And then what you'll see
is we do zone delegation
2584.552 -> with the name servers
for from the prod one
2586.63 -> to the test one as well as the dev one.
2589.673 -> And the way this works
is you tell the DNS,
2591.91 -> use that subdomain,
there's the name server,
2593.74 -> that name server, that account then deals
2595.18 -> with any queries to it.
2596.65 -> Is that a warning bell?
2599.17 -> Okay, let's go quickly.
2600.76 -> So we've got DNS in here as well.
2602.05 -> So now what we're gonna do is
we need to go quickly get add.
2610.78 -> Okay.
2612.238 -> (speaking faintly off mic)
2615.37 -> Did I?
2616.6 -> Yes, thank you.
2618.31 -> Someone is listening.
2619.18 -> I'm so happy.
2621.01 -> Cool.
2622.427 -> The reason for this is now
just quickly on that one
2623.71 -> is that to be able for a Terraform to go
2625.6 -> into the different accounts,
2626.433 -> you need to tell it like how do I do it?
2627.88 -> What roles I assume,
2628.99 -> which is what we've
got over here and using
2630.64 -> the alias function of providers.
2632.08 -> So I'm saying there's
an alias called test,
2634.57 -> so I'll reference it as
AWS.test as the provider,
2637.27 -> also in the same region.
2638.53 -> And how it's gonna get
in there is gonna assume
2641.08 -> this specific role and
here I can interpolate
2643.049 -> the account ID from a variable
since I just pasted it out.
2645.31 -> That's available at
runtime for the provider.
2647.2 -> And then once again, just
give it a nice session name
2649.814 -> so your auditing tools
know what's happening.
2651.76 -> So now with all of this,
it should be able to get
2653.65 -> into those test accounts
once I remember to merge.
2665.41 -> Okay, let's push that.
2666.64 -> That should be over here.
2668.83 -> Let's quickly, is this the main one?
2673.27 -> Yeah, Compare and pull.
2676.45 -> Oh well, that doesn't
look like the right repo.
2680.02 -> Oh okay, no, come on.
2684.537 -> Branches.
2690.223 -> Create pull requests.
2696.34 -> That doesn't sound right, but okay.
2698.62 -> Let's see, yellow.
2704.53 -> Okay, code.
2710.02 -> Wait, am I on the wrong tab?
2714.032 -> There was no, this is the right one.
2715.12 -> Sorry, I'm just eyeballing my--
2717.817 -> - [Emily] Five tabs is
better than my typical 560.
2721.692 -> - [Cobus] Cool.
2723.79 -> So now what we've got here is,
okay, we've got this again.
2726.97 -> What did I miss?
2728.41 -> Sorry.
2730.092 -> Oh, this one from the main, okay,
2734.66 -> so what this means is I didn't
update the IAM permissions
2742.081 -> here in the main account,
oh wait, variables possibly.
2745.396 -> Nope, they're there, which will all work.
2754 -> Let's just double check
if I'm just too hasty.
2756.1 -> No, that's definitely not gonna work.
2757.66 -> So, oh, repo names.
2759.79 -> Let's double check in this one.
2762.79 -> Did I have a mistake again?
2764.32 -> Those are fine.
2767.02 -> (speaking faintly off mic)
2768.544 -> (overlapping discussion)
2777.31 -> But that shouldn't be a problem.
2779.74 -> Okay, get status.
2782.74 -> Yeah, no, it's in.
2786.52 -> Main call.
2789.1 -> So what I'm concerned about
here is the fact that it's
2791.23 -> getting stuck on the
credentials side on there,
2793.78 -> which means if my, wait, wait, wait, wait.
2797.89 -> This over here is the one
part that, yeah, there we go.
2802.313 -> So what happened here now is,
because there's one more part,
2806.53 -> we have to do things by hand,
2807.82 -> which is I need to get the main account
2810.19 -> because this GitOps action is not allowed
2812.23 -> into the billing account.
2813.79 -> So once again, security
actually doing its job.
2821.44 -> We're getting creative with that.
2823.238 -> (speaking faintly)
2824.14 -> Yeah, Yellow.
2825.897 -> Wait, wait, what did I do?
2827.38 -> Oh, pull, pull.
2829.686 -> Now we're gonna go to merge.
2830.74 -> Okay, cool.
2836.11 -> Huh?
2837.374 -> Am I being very, very confused here?
2840.46 -> Sorry, let me check.
2841.81 -> - [Emily] I have another joke.
2844.045 -> - [Cobus] Okay.
2845.44 -> - [Emily] It actually
came from you earlier.
2847.758 -> Oh I know, it's pretty good.
2849.265 -> Why can't you mate an eel and an eagle?
2853.06 -> It's EEL-EE-GUHL.
2855.68 -> - Thank you.
2858.04 -> - So bad, it's good.
2858.873 -> I'll be here all week.
2860.59 -> Thank you.
2867.629 -> - [Cobus] Come on, we're so close.
2870.52 -> Get status.
2872.8 -> Why do I not see the difference here?
2875.134 -> Let's just put a place space in here.
2882.85 -> I am in the right folder.
2884.38 -> Oh, okay, there we go.
2885.46 -> Nope, Cobus is making mistakes again.
2888.317 -> There we go.
2890.32 -> Maybe we should actually
be in the right repo.
2893.53 -> This is IoT production by the way.
2896.5 -> Okay, there we go.
2897.7 -> Get status, get that.
2904.75 -> That's also on main.
2905.65 -> That should now solve our
little conundrum over there.
2909.543 -> Eh, we are gonna start having
to go really quick now.
2919.84 -> Okay, let's just see
this should go past that.
2922.283 -> No, okay, I ran out of time.
2927.711 -> Yeah 'cause the thing is
it's 10 minutes before
2930.07 -> the next session and we
want to kind of stop now.
2932.2 -> So what I'll do is I'll stand,
2933.49 -> be outside for any questions other
2935.23 -> than why didn't this work?
2936.754 -> (audience laughing)
2938.844 -> - You don't have like
finished infrastructure
2942.46 -> like at a cooking show, they
like pull it out of the oven.
2945.289 -> - I do.
2946.213 -> That would be the 3:00 AM version,
2948.64 -> 'cause I had an idea to
change this last night,
2950.65 -> so I wouldn't trust that Cobus.
2952.51 -> - In two minutes, what would
you have seen had this run?
2955.641 -> - So what would've happened now
is that this would've set up
2958.359 -> the main account and
then we would've pushed
2960.61 -> the environment account,
2962.504 -> which would have created a VPC,
2964.865 -> basically in each of the
environment accounts,
2969.354 -> and then what we would've done in there is
2971.774 -> as a last little step
is we would've created
2974.244 -> a demo service using a module.
2975.853 -> So if you look at that and
what this actually does
2978.84 -> is it creates the ECR
repo sets up a bull job
2981.479 -> using Terraform to actually
create Terraform templates.
2986.504 -> And the GitOps provider would've created
2989.41 -> a new GitOps repository
with a PR pre-populated,
2993.302 -> with all the Terraform infrastructure,
2995.11 -> with the GitOps Actions workflow.
2997.24 -> And that would've then
kicked off your whole,
2999.19 -> let's get the container
built and then you can add
3000.96 -> in your deployment in
there if it had worked.
3002.958 -> - Thank you.
3004.747 -> Thank you all so much for coming.
3006.75 -> We really appreciate it,
3007.8 -> and your patience with the mic issues.
3010.286 -> (audience applauding)
Source: https://www.youtube.com/watch?v=P6Ngme9KBqs