AWS re:Invent 2022 - Building next-gen applications with event-driven architectures (API311-R)
AWS re:Invent 2022 - Building next-gen applications with event-driven architectures (API311-R)
Event-driven architectures can solve many difficult problems in modern enterprise workloads. For example, it can be challenging to work with large amounts of data in different data stores and locations. Teams building microservices architecture often find that integration with other applications and external services can make their workloads more monolithic and tightly coupled. In this chalk talk, learn how to use event-based architecture to decouple and decentralize application components. Discover how you can use AWS messaging services to connect microservices and coordinate data flow using minimal custom code.
Learn more about AWS re:Invent at https://go.aws/3ikK4dD.
Subscribe:
More AWS videos http://bit.ly/2O3zS75
More AWS events videos http://bit.ly/316g9t4
ABOUT AWS
Amazon Web Services (AWS) hosts events, both online and in-person, bringing the cloud computing community together to connect, collaborate, and learn from AWS experts.
AWS is the world’s most comprehensive and broadly adopted cloud platform, offering over 200 fully featured services from data centers globally. Millions of customers—including the fastest-growing startups, largest enterprises, and leading government agencies—are using AWS to lower costs, become more agile, and innovate faster.
#reInvent2022 #AWSreInvent2022 #AWSEvents
Content
0 -> all right good morning how y'all doing
3.659 -> all right to the five of you that would
5.88 -> I appreciate it I've been wooed hey I'm
9.599 -> glad y'all are here this is how many for
11.94 -> let's try it again they pay me to speak
13.74 -> here at reinvent isn't that crazy how
15.719 -> many of y'all are here this is your
17.4 -> first session re invent
18.96 -> okay yes all right very cool that's
22.5 -> exciting uh let me tell you where we are
24.779 -> uh we are here to talk about containers
27.18 -> and that's not true
29.58 -> you're like oh I gotta get out so we're
31.199 -> going to be talking about building
32.399 -> next-gen applications with event driven
35.399 -> architectures if you're in the wrong
37.739 -> place stay anyway we're going to have
39.66 -> fun
40.8 -> okay I can't see you just so you know so
43.92 -> if I ask you to raise your hand oh there
46.26 -> there you're still out there so uh my
48.66 -> name is Eric Johnson
50.7 -> tell you a little bit about who I am uh
53.1 -> why I should even be up here I probably
54.96 -> shouldn't but they let me anyway uh I am
57.42 -> a principal developer advocate for AWS
60.48 -> I am a father of five that is not a typo
63.739 -> our house is crazy now I'm gonna
67.32 -> interrupt there for a moment
69.119 -> uh has anybody ever heard me speak
71.159 -> before
72.18 -> okay a couple of you okay well we're
74.28 -> gonna go over the rules anyway okay when
76.979 -> I speak There Are Rules okay and here
79.74 -> they are and you think of yourself how
81.54 -> do you get to make rules I have the mic
83.88 -> so there we go so rule number one is
87.659 -> this is any number I want it to be
91.74 -> okay I'm gonna hold this up at some
94.08 -> point and I'm gonna say I have five kids
96.299 -> and there's always somebody in the back
98.46 -> says that's not a five that's a one
100.92 -> okay I can get to four if I take my
103.799 -> shoes off but it gets real awkward for
105.54 -> everybody
106.68 -> okay so that's any number I want it to
109.02 -> be rule number two
111.96 -> which one
113.46 -> all right good for those listening
115.56 -> someone coming in late saying I don't
116.88 -> get it these are quotes
119.159 -> not apostrophes
121.619 -> and I know that because this looks silly
125.759 -> Never Never Fails is he doing a bunny
129.539 -> I'm not doing a bunny unless I am and
133.44 -> it's contextual you have to figure it
135 -> out okay and the last rule is these are
137.34 -> thumbs
138.48 -> okay I do this all the time I'm like hey
140.7 -> you good
141.9 -> and they're like I'm good but I don't
143.34 -> know what you're pointing at yourself
144.239 -> but I'm not putting out myself unless I
146.7 -> am so I I was born this way I didn't
149.16 -> wake up this way for the first time
150.78 -> today if I did I would not be here with
153.239 -> you okay I like to make lots of jokes uh
157.56 -> I'm comfortable with that if that makes
160.379 -> you uncomfortable I'm also comfortable
163.14 -> with that so
165.36 -> either way I'm good all right so let's
168.239 -> get started so continue on I'm a brand
170.64 -> new grandpa I took everything in me not
172.62 -> to put a bunch of slides of my grandson
174.66 -> up there thank you uh I'm a drummer that
177.66 -> used to say musicians but people are
179.34 -> like really so I am a drummer I went
181.92 -> through college playing drums and here's
184.08 -> what I learned I'm a phenomenal drummer
188.28 -> for one finger
190.68 -> past that I'm pretty average and
193.019 -> nobody's going to pay me all right it's
195.239 -> always fun to go in I played in a lot of
197.04 -> churches in college I would go in you
198.84 -> could always see because I was hired out
200.4 -> they would call me and when I would get
201.72 -> there you could always see them going oh
202.92 -> no
204.18 -> what have we gotten ourselves into
206.64 -> that was always fun because they're a
208.019 -> church they can't tell you never mind
209.04 -> we're not paying you so I like to sit
210.659 -> down and take one stick of my hand and
211.98 -> go okay let's rock
214.5 -> so
215.64 -> lots of fun I am also foodie obviously I
218.879 -> do like to eat I often people ask me how
222.12 -> do you feed yourself oh I feed myself
224.879 -> I figured it out so all right enough
226.68 -> about me about me let's jump into this
228.54 -> today we're going to be talking like I
229.739 -> said about uh
232.2 -> occasions with
234.42 -> event right so we're going to talk about
236.519 -> some of the challenges some of the some
238.92 -> of the things that we want to be aware
240.599 -> of uh and so let's jump in
243.18 -> so we're going to cover three things
244.44 -> kind of break it down uh first is we're
247.019 -> going to talk about Enterprise
247.68 -> Integration patterns what does that even
249.48 -> mean we're going to be uh okay we hear
251.64 -> this word all the time but what does it
253.62 -> mean we're going to talk about
254.76 -> event-driven architecture right we we've
257.4 -> been throwing this word around for quite
258.9 -> a while Eda is not new we didn't invent
261.9 -> it but we're really doubling down on it
264 -> we think it's the it's really the best
265.62 -> way to build architectures uh so we'll
268.08 -> kind of talk about that idea and finally
269.88 -> we'll talk about handling event
271.56 -> duplication using item potency and and
274.5 -> this one's always a challenge when we
276.78 -> talk about that and a lot of times when
278.04 -> we talk and I've done it myself when we
279.419 -> talked about uh Eda or any kind of
282.419 -> architecture we talk about building
283.86 -> large distributed systems we say and
286.919 -> make sure they're they're built for item
288.54 -> potency
289.5 -> and then we leave it at that and you see
291.24 -> everybody going okay that makes sense
294.18 -> I don't know what you're saying but I'll
295.5 -> do that so we're going to dive into a
297.66 -> little bit what does that mean today
301.8 -> I've already been asked this is not that
303.419 -> Dr Pepper
304.62 -> because my wife made me promise to drink
306.3 -> water while I'm talking so it'll be diet
308.88 -> Dr Pepper later all right so we're gonna
311.639 -> jump in here so the first thing
313.56 -> Enterprise Integration patterns
316.979 -> when we talk about integration patterns
318.66 -> one of the things we talk about a lot is
320.22 -> coupling we throw this word around you
322.32 -> decouple your apps be careful you're not
324 -> tightly coupled all these kinds of
325.68 -> things and coupling is integration's
327.96 -> magic word right and it's this A to B A
331.199 -> requires B it's this this this when we
333.96 -> say couple it's this idea of I'm reliant
337.44 -> on you and coupling is is the measure I
340.259 -> love this it's the measure of
341.759 -> independent variability between
343.44 -> connected systems because coupling can
346.139 -> be variables sometimes entirely if
348.84 -> you're down I'm down sometimes it's
351.06 -> kinda if you're down I could survive but
353.1 -> I need you to come back up pretty soon
354.419 -> okay
355.919 -> decoupling has costs when we build
359 -> distributed architectures
361.8 -> decoupling has cost both at a design
364.68 -> time and run time
366.479 -> coupling isn't binary okay sometimes
369.419 -> it's more than one thing it goes is a
371.1 -> lot of things and finally coupling isn't
373.02 -> one-dimensional so there's a lot of
375.06 -> different things that that can relate to
376.979 -> that and when it was starting to put
378.12 -> this together we're starting to think
379.199 -> about this it's like how many different
380.58 -> ways can we couple so we put down a list
382.56 -> here this probably isn't all of them but
384.96 -> when you think about coupling you may
386.34 -> not have thought about some of these we
387.9 -> have technology dependencies like what
389.699 -> language well we can only build in Java
392.34 -> and before we start building
393.72 -> applications with something like a
395.759 -> Lambda function that's kind of what we
397.8 -> did when we're going to build this is
399.84 -> the language we build in you're tied to
402.6 -> that there's no changing now Lambda kind
405.24 -> of change that Paradigm because we said
406.56 -> you know what you can have a function
407.639 -> that's in Python you could have a
408.72 -> function that's a function that's in
410.039 -> Java whatever is best for that
412.74 -> particular
414.06 -> function that you need to do
416.88 -> okay you can have location dependency
418.68 -> only this IP address
420.539 -> data format dependency data type
423.3 -> dependency
424.56 -> semantic dependency one we're going to
426.3 -> talk about quite a bit today is temporal
427.74 -> dependencies sync versus async
429.78 -> interaction style dependency how are we
432.24 -> talking and and conversation dependency
434.759 -> pay adjacent things like that
436.8 -> so different ideas and coupling that and
440.46 -> and coupling we say this because you
442.62 -> hear it's like you should always
443.4 -> decouple decoupled a couple decouple
445.38 -> coupling is not an evil word but you
448.139 -> want to understand your coupling okay
450.72 -> and I love this quote
452.34 -> this is Gregor hope who actually works
454.44 -> for AWS he says the appropriate level of
457.02 -> coupling depends on the level of control
458.88 -> you have over the endpoints
461.699 -> okay how much control you have over the
464.4 -> endpoints is going to decide how much
466.44 -> you're reliant on that or should and
468.66 -> sometimes like look I need I can't help
470.52 -> it we've got to have it we're going to
472.319 -> use it so whatever they dictate we're
474.06 -> going to do
475.259 -> but sometimes like look we have no
477.06 -> control over that if they go down we go
479.4 -> down so we need to pull back our
482.34 -> coupling make that less reliant on those
485.52 -> systems
487.979 -> so let's talk about some of those things
489.84 -> as we're looking at building
492.36 -> applications we're looking at building
494.16 -> distributed applications let's talk
496.199 -> about uh the the two the types of of
498.479 -> patterns that we would use so the first
500.819 -> and this is I when I speak to developers
503.039 -> a lot of times they really struggle with
504.599 -> this idea of synchronous versus
506.52 -> asynchronous so let's kind of talk about
508.44 -> this a little bit and and point this out
510.24 -> so the first we're going to talk about
511.08 -> is the synchronous request response
513.06 -> model so the synchronous response
515.88 -> request response model that's a that's a
517.979 -> big those are big words
519.899 -> um this is basically what this looks
522.3 -> like okay so you have this you have a
523.979 -> sender you have a receiver you make the
526.68 -> request it comes back and this is often
528.6 -> what we call also a blocking call
530.279 -> because what it means is when I've made
532.5 -> that call I can't do anything else until
534.48 -> you respond to me I am completely
536.58 -> reliant on you know there's some
538.019 -> advantages of this there's very low
539.88 -> latency right it's a quick call it's
542.399 -> simple yes I'm not having to do a lot of
544.32 -> distributed stuff and finally it's a
546.72 -> fast fail okay how many of y'all look at
549.48 -> fast fail as an advantage
551.82 -> okay good good that's right fast fails
555.12 -> are good for us because we want them to
556.32 -> fail when we're developing
557.88 -> okay it's always beautiful to get a
560.04 -> fully distributed system in and four
561.899 -> steps down is where it's failing and I
563.64 -> don't know until it's in production
565.8 -> right I've been there anybody been there
568.08 -> yeah exactly if you didn't raise your
570.839 -> hand lucky dog or liar
573.779 -> so okay
575.88 -> but disadvantages
578.1 -> you know what if your receiver fails so
580.5 -> we talked about this if my receiver
582.48 -> fails my sender's dead in the water well
585.839 -> okay I'd love to take your payment but I
588.42 -> gotta wait okay that's something because
591 -> the client what client in the world is
592.8 -> going to go oh oh you can't make payment
594.24 -> processing right now I'll wait
596.519 -> no they're gonna you know what I can go
598.26 -> over to a little startup called
600.54 -> amazon.com and buy that
602.58 -> okay so they they you you want you don't
605.279 -> want your client down you want them up
607.14 -> and running okay so that's first thing
608.88 -> the second thing is throttling okay so
612.72 -> as you start building more and more
614.94 -> systems talking to this this is your
617.04 -> central point of failure okay or a
619.92 -> single point of failure would be a
621.06 -> better way to say I can do that one
622.2 -> single
623.7 -> single point of failure okay so that's
626.94 -> what that could cause problems right so
628.92 -> you have receiver failure you have rehab
630.6 -> you have throttle failure or receiver
632.399 -> throttle it may not be down
634.38 -> but it's definitely slowing down right
636.899 -> and and to today's culture slow is
640.14 -> broken right slow is the new broken
642.66 -> there you go I'm gonna put that on a
644.7 -> shirt somewhere
646.92 -> all right so let's jump the other one
649.019 -> and I'll tell you here's a pattern we
651.839 -> always recommend always go asynchronous
654.839 -> when you can there are times for
656.7 -> synchronous I am not here to tell you
658.26 -> use synchronous and you're not a 10x
660.959 -> developer whatever that means right
662.94 -> there are some times when you need
664.5 -> synchronous but try asynchronous first
667.2 -> when you can use asynchronous do it but
669.839 -> there are some advantages and
670.8 -> disadvantages on that so let's look at
672.24 -> these so there's a couple of ways of
674.22 -> doing asynchronous the first we're going
675.839 -> to talk about is the point-to-point
677.04 -> model or a q
679.26 -> so what does that look like
680.88 -> so you have you have a sender and you
682.8 -> have a receiver much like before but now
684.899 -> you have a Q in the middle of it right
686.339 -> so you could be sending things into the
688.5 -> queue and then as as the receiver is
691.44 -> ready they'll handle that but this
692.82 -> becomes an asynchronous model what the
695.459 -> sender gets if you're not familiar with
696.959 -> this model what the sender is going to
699.12 -> get a sink or synchronously I know we're
701.519 -> talking about async but synchronously is
703.26 -> a I got your request thank you very much
706.44 -> we'll get back to you okay it's an
708.839 -> acknowledgment of this of the data the
712.14 -> event whatever being sent in right then
715.079 -> later they'll need to either pull or or
717.54 -> websocks or somehow and get the response
719.64 -> back if we need to this is what we often
721.68 -> call fire and forget
723.839 -> excuse me
727.2 -> sorry you have to hear me slurp on the
728.76 -> mic there apologize for that okay
731.519 -> again you have it
732.839 -> it's a temporal coupling we've now our
735.12 -> Cinder is not broken okay if if the
738.12 -> receiver breaks down the sender is still
740.7 -> working uh if they if you know if they
743.519 -> need to they're just it's just going
744.839 -> into the queue and when the receiver is
747.18 -> ready you can grab more so if it goes
748.74 -> down and we bring it back up we haven't
750.42 -> lost those requests
752.1 -> it's resilience receiver failures we
754.019 -> just talked about and the receiver
755.459 -> controls consumption rate right so
758.22 -> that's important okay this is a little
760.56 -> statement down there if you think about
762.44 -> Downstream applications a lot of times
765.54 -> I've been there and I won't ask you to
767.339 -> raise your hand but I will mine when we
769.2 -> come in we work for a company it's like
770.76 -> hey we have this Legacy system that can
774.12 -> handle three requests per day
777.6 -> right that's probably an exaggeration a
779.639 -> little bit but it can do three requests
781.8 -> per second but unfortunately we're
783.899 -> getting a billion requests per second we
786.779 -> need you to make that work
788.519 -> all right well let's rebuild it no no no
790.26 -> no
791.16 -> no we want you to make that work
793.92 -> okay but let's add more to it no no no
796.98 -> so what happens is you you need to
799.139 -> maintain these Legacy systems that can't
801.3 -> handle that traffic so this is where a
803.459 -> queue becomes very handy because one
805.92 -> you're not invoking that system you're
807.42 -> not running that system for each and
808.98 -> every event coming in you're now able to
811.2 -> batch those events okay so if they're
813.3 -> small and you can run 100 at a time this
816.42 -> gives you a lot of control so this
818.16 -> receiver controls consumption rate helps
821.04 -> you protect your Downstream system okay
823.38 -> we talk about that when people build
825 -> with serverless all the time is yeah
827.16 -> serverless scales like crazy but a lot
829.139 -> of times it's talking to our Legacy
830.76 -> systems that can't handle that so this
833.579 -> is one way to maintain that and control
835.32 -> that
837.24 -> okay so what happens here is what if
839.459 -> what if one of the events goes bad what
840.899 -> if something breaks
842.279 -> well in our in our sin receive in our
844.92 -> synchronous model we lost the data which
847.68 -> is always great when someone's making a
849.54 -> payment and you take the payment you
852.18 -> process it and it breaks and then you go
854.399 -> back to the customer go hey
856.92 -> we lost your card number
859.38 -> go ahead and give it to us again you can
861.24 -> trust us
862.2 -> okay your credibility is being shot
864.06 -> there right so this time now it's
866.1 -> persisted it's in that queue uh because
868.8 -> we say okay do it now if something
871.019 -> happens the queue can be programmed to
873.24 -> retry or configure to retry and to
876.42 -> handle errors and worst case scenario
879.12 -> drop to a dead letter Q so we don't lose
881.699 -> the data it just needs some manual
883.56 -> intervention or maybe some other
884.579 -> automation that you have okay so you
887.22 -> have this resiliency built in
889.639 -> automatically by using a queue or
893.1 -> point-to-point model
896.04 -> all right so another another advantage
899.519 -> to this here
900.779 -> is only one receiver can can consume
904.38 -> each message and this kind of people
905.94 -> look at the queue and the way it works
907.8 -> is when and I'll get more into this
910.26 -> later when a queue is working what will
912.959 -> happen we'll have several copies of it
914.279 -> for redundancy and then when you grab
917.339 -> that one of your systems grabs those uh
920.519 -> items off the queue it actually doesn't
922.68 -> remove it from the queue it just hides
924.66 -> it okay it hides it until you say I've
927.36 -> processed it okay and then it removes it
930.18 -> and so what happens is it keeps other
932.22 -> other receivers from grabbing it so you
934.92 -> can you can make sure that it's
937.019 -> for the most part we're going to get
938.459 -> into that at the end of this that is
939.72 -> only once or a read at least once so
943.56 -> okay
945.899 -> a couple disadvantages response
948.06 -> correlation
949.44 -> what happens if there is a problem and I
951.959 -> and I need to figure out where that goes
953.459 -> you want to be sure and again we'll talk
954.899 -> about the item potency a little bit
956.339 -> later backlog recovery time if it goes
959.22 -> down sometimes you when you've got to
961.139 -> recover those but
962.88 -> we say this Advantage maybe it's like
965.04 -> just a known thing because that's better
967.199 -> than losing all the data right agreed
970.26 -> anybody in the room God I'd just rather
972 -> lose it let's start fresh okay because I
975.54 -> want to hire you so okay and finally uh
979.26 -> fairness and multi-tenant systems so uh
981.839 -> this kid if you have really hot items
983.699 -> grabbing and stuff you can you can run
985.199 -> in some making sure that's balanced out
988.5 -> all right
990.24 -> a clicker you know it's like there we go
993.12 -> all right so a service we use for this
996.12 -> if you're not familiar with this I've
997.199 -> been talking about it is Amazon sqs I
999.959 -> highly encourage you to look at this
1001.339 -> this is one of the models we'll talk
1002.6 -> about a couple other ones but if you're
1004.1 -> looking to deal with that with there's
1005.54 -> there's fifo uh keys where you can keep
1008.06 -> things in order uh and so this is the
1010.94 -> service that we use this it's fully
1012.259 -> managed message queue it scales almost
1014.48 -> infinitely I love that number how do you
1016.82 -> scale
1017.6 -> almost infinitely
1019.22 -> so close to infinite I can't even tell
1021.5 -> you the numbers it handles mass amount
1024.38 -> of traffic simple easy to use has dlq as
1027.5 -> we pointed out and again the fifo like I
1029.419 -> said
1030.98 -> all right so the next pattern we have is
1032.54 -> a point-to-point model here is a router
1035.179 -> model okay
1036.439 -> so in this we see this a lot okay let's
1039.439 -> say you need to route between uh some
1042.02 -> sometimes receiver green has to get some
1044.54 -> stuff sometimes receiver blue has to get
1046.339 -> some stuff
1048.26 -> what happens we see a lot of people
1049.58 -> build this in
1051.559 -> the disadvantage is first of all I mean
1053.84 -> we're talking coupling we want to reduce
1055.52 -> coupling right this is increases
1057.26 -> coupling only blue can get blue only
1059.6 -> green can get green right and it also
1062.539 -> and this is a big one the sender
1064.039 -> maintains the logic now my title I think
1067.94 -> I told you is developer Advocate but the
1070.46 -> truth is I'm a hack developer okay the
1073.52 -> most brittle part of your application is
1075.2 -> going to be Eric Johnson's code there's
1076.58 -> no lie about that it doesn't say hack
1078.2 -> developer on my card because that's
1079.88 -> pretty low but it should
1082.1 -> the thing about this is my Mantra is if
1085.64 -> all else fails
1087.679 -> then code
1089.539 -> right only code when you have to use
1092.48 -> configuration let the services do their
1094.82 -> work and this is one place where in this
1097.4 -> the the sender is maintaining all your
1100.34 -> logic it's having to evaluate and Route
1102.74 -> Based on data and that's a lot of
1105.32 -> unnecessary code that you really don't
1107.059 -> need
1110.539 -> let me show you it gets more complex
1113.96 -> when we start adding more no application
1116.66 -> or very few applications is two
1118.46 -> functions two functions we're up and
1120.02 -> running we're making a million dollars
1121.039 -> if you did that that's correct but the
1123.44 -> reality is you start building in more
1124.7 -> features more functions uh things like
1126.679 -> that and you end up now you got a purple
1128.78 -> and a dark blue and a light blue and a
1130.46 -> green and a yellow and you have a lot of
1132.02 -> these things and your Cinder is now a
1135.919 -> big long list of if thens
1138.5 -> right and if the ends are the worst or
1141.559 -> switches right or whatever smart code
1144.919 -> thing I'm a hack you know I'm an
1146.66 -> architect so so that's that's where that
1148.82 -> becomes a problem so how do we fix that
1151.76 -> okay so the ace the asynchronous message
1154.64 -> router model using a bus let me show you
1157.4 -> how that works
1159.64 -> instead of
1161.66 -> the sender doing it the sender now only
1163.94 -> needs to know of one thing and that's an
1166.039 -> event bus okay and so what we do is we
1168.679 -> drop all this stuff onto the event we
1170.96 -> just say hey here you go here's another
1172.16 -> one here's another one here's another
1173.24 -> one and we just pump it in and that
1174.919 -> takes all that logic out of that sender
1177.5 -> and it happened now now it's at the bus
1180.38 -> so how do we divide we still need the
1182.66 -> blue to get blue and the green to get
1184.039 -> green so what do we do well with our bus
1187.34 -> we're able to assign rules and say if
1189.98 -> you're green go to this endpoint if
1193.16 -> you're blue go to this endpoint and they
1195.74 -> can be much more complex we do a lot of
1197.84 -> different things we can Target a lot of
1199.52 -> different you know uh apis and Lambda
1201.98 -> functions and whatever and a lot of that
1204.32 -> happens just simply using rules so this
1206.6 -> logic is pulled out of your Lambda
1208.52 -> function
1209.48 -> or your sender you may not be a Lambda
1212.059 -> function maybe something else and it's
1213.26 -> put into configuration
1216.14 -> the service we use for that is
1217.82 -> eventbridge
1219.32 -> okay now if you're not familiar with
1220.76 -> eventbridge and I hope you are we talked
1222.799 -> about it a lot but it's always funny
1223.88 -> when I say eventbridge I've not heard of
1225.799 -> that one let me tell you a little bit
1227.059 -> about vimbridge and how it works
1229.16 -> so it comes with multiple parts right so
1231.74 -> the first is you can have all kinds of
1233.96 -> producers for event Bridge you can have
1236.059 -> you know you can have SAS you can have
1238.7 -> your own application you can have AWS
1241.1 -> Services there's all type of things that
1243.62 -> can dump events on a bus right so then
1246.86 -> in your bus you can have a custom bus
1248.84 -> you can have your own your default did
1251.539 -> you know you already have a default bus
1253.22 -> just so you know if you have an AWS
1254.84 -> account you already have a default bus
1258.26 -> you can make your own you can use that
1260.24 -> those all go into that then what we do
1262.82 -> is we use these rules and these are
1265.22 -> pattern matching where you could say if
1267.74 -> it matches this pattern do this do this
1270.14 -> Target and those targets again a Lambda
1273.08 -> function
1274.12 -> SNS but my eyes are going here all kinds
1277.46 -> of things there's all kinds of targets
1278.72 -> that you can do
1280.1 -> therefore now your logic is in
1282.5 -> eventbridge and not at your Lambda
1284.9 -> function or your sender
1287.12 -> and this gets important because you're
1289.039 -> you're centralizing that logic because
1291.44 -> you're because reality is you're
1292.7 -> probably going to have more than one
1293.72 -> sender more than one client maybe you
1295.82 -> have a different mobile client maybe you
1297.02 -> have I mean all kinds of different
1298.039 -> clients that are doing this so rather
1299.72 -> than having to repeat that logic
1302.36 -> you just do it at the event Bridge
1305.84 -> so what does that look like so here's
1307.4 -> here's an example of a simple event
1308.96 -> right we have a simple event it has a
1311.419 -> currency in it of a u right so we can
1314.6 -> write a rule that says hey if the
1316.7 -> currency equals a U or NZ then do
1320.539 -> something in this particular instance it
1322.52 -> is a u so we have a match that's how the
1325.76 -> rules work nothing terribly complex you
1327.62 -> don't have to write you know pages and
1329.6 -> pages of python to match the rule you
1331.7 -> just simply write a Json object that
1333.44 -> says does it match and we have a lot of
1335.36 -> granularity on that that you can build
1337.039 -> out in fact we've got uh we've got teams
1339.98 -> that are using this all over companies
1341.72 -> that are using this all over to just
1343.46 -> ship events around and when we talk next
1345.799 -> about next-gen applications event driven
1349.34 -> is what we're talking about so we want
1351.38 -> to be able to handle these events and do
1352.88 -> the right thing with the event
1355.22 -> so with this
1356.9 -> there's multiple Targets this is 20 plus
1359.24 -> uh examples here you see ec2 sqs glue I
1363.799 -> won't read them all but you can see them
1365.059 -> see those there uh and then in the one I
1367.7 -> do want to point out down the bottom
1368.9 -> right hand corner is the API
1369.919 -> destinations so if you've got to make a
1371.6 -> call somewhere else that API destination
1373.64 -> is very helpful
1376.76 -> all right so that's that's
1378.559 -> some of the pattern integration patterns
1380.179 -> patterns we talked about decoupling what
1381.559 -> does that look like how do we do that so
1383.659 -> the second thing I want to talk about is
1385.28 -> actually event driven architectures and
1387.62 -> it's fun
1388.76 -> because people ask me to Define what is
1390.86 -> event-driven architecture and like I
1393.44 -> said we didn't invent event driven
1394.88 -> architecture but the interesting thing
1396.08 -> is
1396.86 -> while not all event-driven architecture
1399.559 -> is serverless
1401 -> all serverless is a vendor driven
1403.22 -> architecture
1404.36 -> okay so when you use serverless and
1406.4 -> that's you know it's this where this
1407.84 -> falls under you're using a venture of an
1410.059 -> architecture so here it is this if you
1412.22 -> get anything this is the most technical
1413.96 -> part of the talks to try to hang with me
1415.4 -> I'll probably do it twice okay so here's
1417.62 -> event driven architecture you ready
1419.78 -> all right
1422.659 -> something happens
1425.299 -> and we respond
1429.5 -> Jimmy do it again
1431.24 -> all right I got you it's hard
1434.84 -> something happens
1437.059 -> and we respond
1439.46 -> that's even driven architecture thank
1441.2 -> you
1441.919 -> thank you
1444.32 -> that's a major architecture obviously at
1447.08 -> its simplest form right but this idea is
1449.72 -> rather than these functional do this now
1451.28 -> do this now do this now do this it is
1453.799 -> I'm going to generate an event and if
1455.72 -> you need to do something do something
1457.7 -> react to it subscribe to the to a topic
1460.58 -> you get get a rule on an eventbridge
1462.5 -> we're passing events around so let's
1464.84 -> climb into that a bit
1467.96 -> all right
1470.48 -> okay I'm not talking to you sir all
1473 -> right
1474.02 -> so he's like it happens all the time so
1476.539 -> uh
1477.74 -> so they have a property an event what
1479.84 -> does an event look like there's several
1481.34 -> things and you can see this and not all
1482.72 -> events are exactly alike but it's a good
1484.34 -> example you have metadata and you have
1486.62 -> data okay the metadata is about the
1488.6 -> event the data is the data of the event
1490.88 -> right so there's differences here events
1493.1 -> are signals that a systems state has
1496.039 -> changed
1497.24 -> something happens so what does the event
1498.799 -> look like I've written data to a
1500.419 -> dynamodb table I've dropped something
1502.34 -> into an S3 bucket I've pushed an iot
1505.58 -> button
1506.659 -> when I first started doing this this
1508.46 -> talk but talk similar to this years ago
1510.86 -> I got one of those iot buttons it was
1512.36 -> all about iot and I got an iot button
1514.159 -> and I wrote a Lambda function and the
1516.14 -> program or the program was every time I
1518.72 -> would click it it would send my wife a
1520.82 -> text message that says I love you
1523.4 -> which I thought was cute
1526.1 -> until I was demonstrating it
1529.34 -> yep yep yep yep finally I got a text
1531.62 -> back first says stop loving me
1534.799 -> right now
1536.36 -> so I changed it
1538.34 -> an event something happens metadata how
1541.34 -> long did I push the button how hard did
1542.96 -> I push the button when did I push the
1544.82 -> button data which button did I push
1548.24 -> all kinds of things about it so you have
1549.86 -> information so it's a change in state
1551.799 -> events occur in the past okay these are
1554.659 -> events that something happened we
1556.279 -> collected the data here's the event
1558.08 -> it cannot be changed or immutable this
1560.48 -> is what happened
1561.98 -> and they decreased coupling by
1563.659 -> restricting information to key data okay
1565.82 -> so we're saying here it is here's your
1569.24 -> key data if a b and c need to do
1572.059 -> something they're not reliant on each
1574.1 -> other they're not reliant on this
1575.059 -> they're just reliant on the information
1576.02 -> in the data so let's look at how that
1577.94 -> how that looks here so let's say we've
1580.82 -> got uh so we're going to talk sparse
1583.7 -> versus full State descriptions and
1585.559 -> here's what I mean when I was saying
1586.7 -> decoupling by by giving key data
1589.159 -> sometimes it's hard to decide what is
1591.799 -> the key data I've seen folks like their
1594.74 -> their events have 300 lines
1597.98 -> okay color blue description well not
1601.1 -> blue but kind of blue maybe purple
1603.38 -> you know all kinds of different things
1605.539 -> all right so then there's other ones who
1608 -> just put this is the order ID you want
1610.58 -> to know more talk to me so how do you
1612.32 -> find that balance well some things to
1614.24 -> consider so in a sparse event you have
1615.919 -> this on the left here it's got the order
1617.539 -> 123 was created at 10 47 am by customer
1620.26 -> 456. that's it okay whereas we have more
1623.96 -> information the current status is open
1625.46 -> and the total was 23
1627.22 -> 237.51 and more and more and more data
1630.32 -> so what happens when you use this okay
1632.659 -> so when we send it so we might have
1635.12 -> people say okay there's not enough
1637.34 -> detail here what what are the details
1640.279 -> where do I get those right so you want
1642.62 -> to think about how much can I put in
1644.48 -> this event but there is a risk when you
1647.12 -> get too much so let's say
1649.1 -> when you do a full state so you got here
1651.08 -> Order ID you got status you got total uh
1653.779 -> different things what happens is the
1655.46 -> event schemas should be backwards
1657.44 -> compatible all right so when we think
1659.179 -> about that and why does this matter with
1660.86 -> the full state
1662.059 -> when I'm making something backwards
1663.799 -> compatible
1666.32 -> let's say I've got 10 Services consuming
1668.659 -> this event if they expect a certain
1670.58 -> record or name or something in camel
1674.24 -> case that needs to stay there but if you
1676.64 -> need to change it and we see this is
1679.159 -> this is a living example okay we had
1681.74 -> record ID camel case large r large eye
1685.4 -> record ID they needed snake case low R
1688.7 -> gas ID
1690.86 -> so now what happens is you can't remove
1692.9 -> record ID camel case now you have to add
1695.9 -> record ID snake case right under and now
1698.72 -> you're populating two fields with the
1701.36 -> same number
1702.559 -> because you need to maintain backwards
1704.419 -> compatibility okay so these are things
1706.58 -> to think about and that may be a case
1708.08 -> look I have to do it we had no choice
1710 -> okay that may there may be a case in
1712.46 -> that but these are things to think about
1713.659 -> as you're as you're building that in
1715.82 -> okay but what happens on that is as you
1718.52 -> get more it becomes more costly to
1720.44 -> compute this okay I'm passing bigger
1722.779 -> data around I'm having to work through
1724.76 -> different things just keep that in mind
1726.5 -> just as you're scaling larger
1728.779 -> okay
1730.7 -> all right so we often talk about well
1732.74 -> we're building event driven
1733.88 -> architectures and I'm sorry it's just
1735.98 -> really dry up here or here in Vegas
1741.98 -> and I'm doing all the Talking obviously
1743.6 -> so
1746 -> we're talking about building
1747.86 -> distributed architectures event-driven
1749.84 -> architectures
1751.46 -> there's a couple of words we're going to
1752.539 -> learn here that's choreography is the
1755.059 -> first one and choreography or
1756.74 -> choreograph is events between domains
1759.02 -> using subscriptions okay so domains how
1762.02 -> do we break our domains out
1763.64 -> we have retail we have fulfillment a lot
1766.399 -> of times I encourage you to look at uh
1768.86 -> some of the parameters that you need
1770.059 -> when you're breaking that out uh but
1772.1 -> let's let's use these for our examples
1773.48 -> so the idea here is hey instead of send
1777.32 -> something out to everybody who needs
1778.94 -> that I want you to notify me when it's
1781.76 -> when an order is created so I'm going to
1783.64 -> subscribe to you
1786.5 -> okay now what happens is when an order
1789.32 -> is created then they get notified orders
1792.62 -> created I'm going to send notifications
1794.36 -> so that choreography goes between the
1797.059 -> domains okay so orchestration is we're
1800.12 -> going to orchestrate a business process
1801.679 -> with the end the domain how do we
1803.36 -> orchestrate how the events move through
1806 -> the domain and get used so here's our
1808.82 -> order so we have an order up retail
1810.86 -> sends it out we have an inventory
1812.48 -> manager they check is it in stock yes it
1815.48 -> is in stock if it is then create an
1817.82 -> order all right I've created an order
1819.98 -> and now what I'm going to do is I'm
1821.96 -> going to send out I'm going to generate
1823.76 -> another event and the service that we're
1826.58 -> using in this example is Step functions
1828.98 -> because it's made to orchestrate it's
1831.679 -> made to do just that how many are
1833.24 -> familiar with step functions
1834.98 -> okay part of the room but not all the
1837.2 -> room okay
1838.88 -> I encourage you to go out of the room
1841.94 -> right now and look up step folks no I'm
1843.62 -> just kidding don't do that but I do
1845 -> encourage you as soon as you can to
1846.86 -> learn about step functions because if
1848.899 -> you find yourself doing a lot of
1850.279 -> choreography or I'm sorry orchestration
1852.679 -> in your function writing a lot of code
1855.2 -> to go if you're this then this then this
1856.88 -> then this then this
1858.44 -> that's probably not the best way to do
1860.12 -> it
1860.96 -> what you want is a step function
1862.7 -> especially with when you can just drag
1864.2 -> and drop and build a step function uses
1866.779 -> SDK to to integrate we'll jump we'll
1869.059 -> jump into that a little bit
1870.559 -> so the sub functions it's a workflow you
1873.02 -> can build the step functions they're
1874.1 -> called they're called State machines uh
1876.26 -> you know and and each step of your
1877.76 -> workflow is called a state so how what's
1879.679 -> the data look like as you're moving
1881.179 -> through when you execute your state
1882.86 -> machine each move from one state to the
1884.659 -> next is called a state transition and
1886.7 -> that's exactly what you're doing so
1887.899 -> you're saying hey this is my state and I
1890.779 -> need to process some data reach out to a
1892.94 -> Lambda function to process it great now
1895.159 -> I need to process some more data Maybe
1896.84 -> I'm going to do a translation I'll talk
1898.399 -> directly to translate come back and so
1901.22 -> I'm changing the data in the state so as
1903.26 -> it's moving through and how you build
1905.419 -> those is you you have a definition you
1907.64 -> define it in Json you can cdk is just
1909.86 -> one example you can use Sam as well
1911.559 -> there's other IEC choices you can use uh
1915.74 -> and then we've got like the data science
1917.299 -> SDK you can use in there and then you'd
1919.82 -> visualize it and I'm not showing in here
1922.46 -> and I'll probably add it later but the
1924.86 -> visual workflow Studio where you
1926.36 -> literally just drag and drop boxes and
1928.52 -> configure okay it's just phenomenal
1930.5 -> nominal right you visualize it and then
1932.539 -> you execute and monitor and you can
1934.46 -> monitor each Loop okay State sub
1936.559 -> functions help you iterate it helps you
1939.14 -> do parallel processing so and you can
1942.2 -> monitor and and watch or and evaluate to
1945.74 -> that as it's going through
1947.72 -> all right so step function integration
1949.94 -> types you have optimized optimized
1951.62 -> Integrations uh you can customize to
1953.6 -> simplify the uses of 17 AWS services so
1956 -> these are direct Integrations but we
1957.74 -> also have SDK Integrations this is this
1960.2 -> was announced last year and there's a
1962.6 -> game changer okay what this means is
1965.96 -> Step functions has the AWS SDK built
1969.02 -> right into it so rather than you calling
1971.419 -> a Lambda function that then uses the SDK
1973.82 -> to talk to a service step function talks
1976.159 -> directly to a service so this takes a
1979.039 -> lot of code out when people hear me talk
1980.779 -> they're like do you not like Lambda I
1983 -> love Lambda don't don't go out here
1984.86 -> going Eric's against Lambda functions
1986.48 -> here it's going to get fired when you
1987.86 -> say that okay I love Lambda functions
1990.98 -> when they're appropriate I'm a big
1993.98 -> believer in the right tool for the right
1996.44 -> job and so if you find yourself and even
2000.279 -> a Jane Eyre who's who's charged Lambda
2003.46 -> big and Lambda is known who's been doing
2005.32 -> Lambda longer than any of us
2007.179 -> he said a Lambda function should be used
2009.7 -> to transform and not transport if you're
2013.419 -> just passing data around get it out of
2015.46 -> there
2016.24 -> okay
2017.559 -> all right so let's take a look at how
2019.96 -> these work so if you think about the
2022.419 -> request response in a in a step
2024.7 -> functions let's say we get a signal says
2026.98 -> hey something's in a bucket when you do
2028.72 -> something we can do a call a synchronous
2031.12 -> request and get the response back
2033.1 -> we can then the next phase will then do
2035.62 -> a synchronous request to a Lambda
2037.12 -> function and get it back again there are
2038.799 -> times in synchronous makes sense and
2041.32 -> then finally
2042.399 -> you're a jewel I love you thank you very
2045.82 -> much all right
2047.559 -> all right and then finally what happens
2049.419 -> is we can do an async request to
2052.359 -> eventbridge saying we've done this work
2055.119 -> we're done okay so synchronous
2058.599 -> synchronous async I didn't write any
2061 -> code I didn't have to do anything I just
2062.8 -> simply did the only code I did was maybe
2064.419 -> some custom transformation in the Lambda
2066.639 -> function and the nice thing is it works
2068.619 -> in all these services
2070.659 -> including the AWS SDK so I can talk to
2073.659 -> so many of these different ones
2075.46 -> another pattern you can use is the wait
2077.619 -> for task token have it have any of you
2080.679 -> all ever been to the serval espresso
2082.06 -> booth
2083.32 -> oh good anybody planning on going to the
2085.419 -> several espresso Booth all right okay
2087.28 -> yeah there's a couple of you I know out
2089.08 -> there had better okay so serverless
2091.419 -> espresso Booth we use this all the time
2093.22 -> a step function runs when you order
2095.98 -> coffee and what you'll do is you go up
2097.96 -> you'll scan a barcode you do phone
2099.58 -> number get a token you'll enter the menu
2101.26 -> and then we'll make it all and it's all
2103.3 -> done just the way I'm showing you with
2105.52 -> with these different patterns
2110.26 -> foreign
2112.38 -> that's not gonna work all right so the
2115.18 -> Wave 4 call uh the wait for task token
2117.64 -> how this works is we're going to get
2119.5 -> we're going to make a call and we're
2121.06 -> going to pass a task token okay
2123.46 -> step functions will do that for you it
2125.02 -> sends it out it says okay here it is
2127.18 -> now we're going to do something that
2129.76 -> we're going to pass that to the receiver
2131.079 -> but what if it fails well then we pass
2133.839 -> that task token back to Lambda function
2136.06 -> that maybe handles the failure and does
2138.22 -> some code and then the system ends it
2140.8 -> but if it's if it works then we send it
2143.5 -> back and the step function moves on now
2145.96 -> here's an interesting thing a standard
2148.48 -> step function or state machine can wait
2151.24 -> up to one year for interaction
2155.5 -> okay well one year why would I use that
2158.32 -> well let's say you have to order some
2159.76 -> things you have to manual some things
2160.9 -> and you're checking things okay this is
2162.7 -> done okay now once that's done I need to
2164.8 -> do this so you see this orchestration
2167.02 -> power of saying this has to happen first
2169.599 -> then this has to happen then these five
2172.119 -> things can happen and now this has to
2174.16 -> happen once all five are done so you
2176.2 -> have this ability to wait and say is it
2178.66 -> done and that's how that's how it works
2181.78 -> the other pattern we have I can get my
2184 -> clicker to work there there we go uh is
2186.099 -> the is the runner runner job sync does
2188.26 -> this work with all our services but some
2189.82 -> of our services and how this works is
2191.98 -> you're actually going to send out uh
2193.72 -> let's let's say Athena you're going to
2194.98 -> send it you're going to say to Athena
2196.359 -> when you're done sync with me let me
2200.2 -> know so it lets them know and then what
2202.72 -> we can do is we can then do a second one
2204.28 -> that says get the results so we start a
2206.56 -> job when you're done let me know I'll
2208.72 -> sync that up okay now I'm going to grab
2210.64 -> the results I'm going to do something
2212.14 -> with them and then I'm going to drop an
2214.24 -> event into eventbridge to say this is
2217 -> done okay so it's these different
2218.92 -> patterns that you can look at
2221.079 -> this particular one does not work with
2223.119 -> the SDK so just keep that in mind it
2225.16 -> only works with those particular
2226.24 -> services
2229.9 -> all right so when you're building
2232 -> applications when you're building
2233.38 -> distributed it's it's important to
2235.54 -> understand orchestration and
2237.579 -> choreography how to use them together
2239.579 -> choreograph between your domains moving
2242.38 -> events around I'm passing around to the
2244.3 -> event bridge I'm passing around with you
2246.339 -> know I'm using keys things like that
2247.66 -> internally how do I orchestrate step
2250.3 -> functions is is is your go-to on that
2252.88 -> right so let step functions be that
2254.92 -> Central orchestrator and then call your
2256.72 -> Lambda function or whatever to do that
2258.94 -> you can call a step function
2260.38 -> synchronously or asynchronously by
2262.599 -> default but you can also call it
2263.8 -> synchronously as well and don't do that
2265.839 -> if you don't have to but it's possible
2268.599 -> all right so
2270.339 -> last part of this we're going to talk
2271.9 -> about our last section is handling event
2274.3 -> duplication using item potency has
2276.64 -> anybody struggled with that impotency
2278.859 -> okay good we have a couple anybody go I
2281.14 -> don't know what that is Eric
2282.7 -> okay good don't be a shy to raise your
2284.44 -> hand when I went to work for AWS I
2286.599 -> remember coming into AWS and thinking
2289.24 -> man
2290.079 -> I'm the big dummy in the room and that
2292.24 -> was true that was right but I sat in the
2294.82 -> rooms and said it was all these acronyms
2296.38 -> being thrown around these big words well
2299.079 -> I don't know what that means item
2300.46 -> potency was one of them finally about
2302.02 -> six months in I went okay whatever I
2304.3 -> raised my hand yes sir
2306.76 -> so I don't know what that means and the
2308.859 -> funniest thing is about six other people
2310.72 -> in the room and yeah I don't know what
2311.68 -> that means either
2313.3 -> I've been sitting in here for three
2314.68 -> years have no clue
2317.079 -> so if you don't know ask so item potency
2319.42 -> all right so we're gonna we're gonna
2320.8 -> tell you what is item potent's item
2323.8 -> potency item potent they're said
2325.72 -> different things
2326.98 -> the basic idea is mathematical
2329.68 -> operations is that can be applied
2331.359 -> multiple times without changing the
2333.88 -> results
2334.9 -> okay let's let's jump in the idea is no
2339.52 -> matter how many times I send the same
2341.14 -> request
2342.22 -> my system should only handle it once and
2344.56 -> return the right response each time
2346.96 -> okay so it's basically watching for
2349.06 -> duplicates duplicates can be tough and
2351.52 -> and in distributed systems this is going
2353.619 -> to happen
2354.7 -> things are going to break Werner Vogel
2356.92 -> says everything breaks all the time
2359.619 -> right it happens and AWS we build for it
2364.06 -> not to break
2365.859 -> or designed for it not to break but
2367.839 -> build for expecting it to break things
2370 -> happen right and so you need to be able
2372.46 -> to account for that in your system so
2373.96 -> let's talk about how that works so uh uh
2378.16 -> hope and wolf in their book they said a
2380.02 -> message that has the same effect whether
2381.76 -> it's received once or multiple times
2383.26 -> that's what I said but they say better
2385.18 -> and more eloquent
2386.8 -> all right and when it really comes down
2388.599 -> to did that and this goes back to our
2390.46 -> story earlier did that internet charge
2392.619 -> my card or not or is it charged twice
2395.26 -> how many gotten a call from their mom I
2398.619 -> have I think I've been charged four
2400.54 -> times
2401.44 -> like man I understand twice but a third
2403.3 -> and a fourth time mom come on all right
2405.7 -> so how do we deal with that
2408.099 -> all right so
2409.54 -> multiple things when we deal with with
2411.7 -> duplication so the first thing is
2413.38 -> problem sending acknowledgment so let's
2415.599 -> say we have a sender okay and we've got
2418.42 -> a receiver
2419.619 -> we send the request the receiver gets it
2422.76 -> processes the request and sends the
2425.5 -> response ah didn't work okay something
2429.22 -> happened internet went down kids are
2431.92 -> watching too many movies something
2433.3 -> happened right
2434.8 -> okay you have a timeout so the sender
2437.5 -> says well did it work or not
2440.02 -> so we send another
2442.18 -> one okay so this is our duplicate
2445.06 -> message it's the same request but we've
2448 -> had to manage that at the client and the
2450.22 -> acknowledgment it happens and now the
2452.74 -> sender says great it worked I'm good but
2455.26 -> the receiver now has two records okay
2459.28 -> we've got a duplicate
2462.22 -> here's another way that can happen
2463.66 -> problems processing the message
2467.56 -> okay so you have let's let's use sqs for
2469.78 -> example in sqs we have a sender they
2472.3 -> send it in and remember I said what we
2474.52 -> do is we keep multiple copies right in
2476.38 -> different places for reliability in case
2478.359 -> one something goes down something
2479.92 -> happens so then you get your receiver
2481.9 -> and your receiver grabs a copy okay so
2484.9 -> we lock that in a system we say hey this
2487.119 -> is hidden don't do anything all right so
2490.06 -> then
2490.96 -> when the receiver can't send it back
2494.14 -> it gets unlocked right
2496.66 -> but nothing happens right so another
2498.82 -> receiver comes in but the thing about
2500.92 -> this is let's go back here real quick
2502.3 -> this receiver was able to maybe not
2505.54 -> process it we don't know that's that's
2507.94 -> the how far in the process did it get
2510.94 -> did it charge the card did it read the
2513.52 -> card did it what did it do right so then
2516.099 -> then it comes back then another one
2517.72 -> comes in and grabs it says okay I'll
2519.579 -> handle it and
2521.92 -> then it goes away
2523.599 -> so whether it's like well what happened
2526.3 -> did it get processed or not okay
2529.3 -> so the third thing we're going to look
2530.5 -> at is is the at least once delivery
2533.8 -> real quick
2538.66 -> so in this one we have our extender
2541.3 -> they send it in we got all our copies
2543.52 -> right the receiver grabs it
2546.52 -> does its work sends it back but it
2550.78 -> doesn't get marked complete okay things
2553.78 -> break it happens right so what happens
2556.24 -> is another receiver grabs it and grabs
2559.3 -> another he grabs a copy of it okay so
2561.82 -> either we had a problem with it not
2563.32 -> getting Martin or we just had a problem
2566.079 -> getting a copy so now again we've got a
2569.26 -> duplicate
2570.579 -> all right so these are some of the
2571.78 -> issues so so what the answer to this
2574.54 -> okay is I don't tokens all right so
2577.66 -> we're going to show you how to use item
2578.92 -> potent tokens if you use Java or python
2582.839 -> uh or I think there's net there's a
2585.46 -> there's some power tools it's something
2587.92 -> I would encourage you to look and I'll
2589.18 -> be I'll be using some of their example
2591.16 -> from Power Tools in Python
2593.56 -> but this is how it works okay a
2595.72 -> well-behaved item potent token is
2597.52 -> generated by the client the client needs
2599.74 -> to be aware of the token don't have a
2602.44 -> generator on your backside something
2603.88 -> like that the client needs to be aware
2605.74 -> okay they like I said they're generated
2607.96 -> by the client and they're generated by
2609.339 -> the client on retry so the client should
2611.56 -> know oh that didn't work let's send the
2613.18 -> same token again okay there should be
2615.339 -> unique per message doesn't do us any
2616.96 -> good if they're not unique right and
2618.94 -> finally uh use a dedicated field
2621.16 -> separate from message content we'll talk
2623.079 -> through that in just a second
2624.46 -> so what does that look like well this is
2627.4 -> how we handle it in power tools okay so
2630.46 -> before you do this you need to create a
2632.02 -> dynamodb table and that'll that'll
2633.88 -> handle it so what we do is the first
2636.28 -> thing we do is we actually send the
2638.14 -> request okay and in the Lambda function
2641.14 -> we will actually immediately grab the ID
2644.44 -> from the client okay so we grab the ID
2647.74 -> and we set it in in dynamodb and we
2650.14 -> write to the dynamodb table and it's
2653.859 -> locked hey this is being worked on don't
2655.839 -> do anything else with that ID okay then
2658.42 -> what we do is we process the Lambda
2660.46 -> function this is where the code runs
2662.2 -> this is your code right once that code
2664.78 -> is run we then update that record in the
2669.099 -> Lambda function
2670.42 -> okay so now now we've got so this is the
2672.94 -> response that we're saving in the land
2674.619 -> I'm sorry in dynamodb we're saving that
2677.14 -> response then we update the record with
2680.14 -> the result and we respond to the client
2682.24 -> all goes well right okay so let's see
2686.8 -> how this works
2688.3 -> so when your client comes in
2690.579 -> we again we do this we do everything
2692.98 -> that I just explained there but what
2695.26 -> happens when the response doesn't get
2697.119 -> back to the client
2698.859 -> Well what we'll then do is we'll say oh
2701.74 -> the client says you know what I didn't
2703.72 -> get that back so I'm going to try again
2705.54 -> with the same token that's important
2709.18 -> so I send it and we read and go that
2711.819 -> already exists we've got already
2713.56 -> processed that so here's your response
2716.14 -> and we just send the same response that
2718 -> was stored in the dynamodb table
2720.04 -> okay so remember we're getting the same
2722.319 -> response no matter how how many times we
2724.54 -> send this this is very important it's a
2727.119 -> good way to handle item potent and I
2728.74 -> really encourage you to grab the power
2730.54 -> tools uh if you want to take a picture
2732.28 -> of this screen grab that link uh but I
2735.22 -> will tell you I have one QR code at the
2737.44 -> end has all the resources on a page so
2740.2 -> actually you don't even have to you
2741.339 -> don't have to grab this just grab the QR
2742.66 -> code at the end
2744.16 -> all right
2745.48 -> so when we looked at item potency
2747.64 -> duration how long is that going to last
2749.14 -> we've got some defaults the Lambda power
2750.88 -> tools by default you can change it is
2753.28 -> one hour Amazon sqs does five minutes
2755.98 -> SNS does five minutes step functions
2758.319 -> does 90 days why 90 days because we can
2760.54 -> wait up to a year remember that so
2763.48 -> let's look at some examples so in sqs
2766.599 -> this was with a fifo queue that's a
2768.22 -> first in first out cue that holds order
2770.5 -> this is how it's going to work so we
2772.78 -> generate a token so it's just a bash
2776.02 -> script right or this is in command line
2778.359 -> so we generate a token there's our token
2781.24 -> all right then we're going to sin this
2782.98 -> is you just using the CLI as an example
2784.72 -> we're going to send a request and you
2786.28 -> can see the message deduplication ID is
2789.4 -> the token the response we get you can
2792.64 -> see the message ID in there
2794.44 -> and we get a response and everything
2796 -> worked fine so then we're going to test
2797.44 -> it and we send the same
2799.9 -> token again
2802.119 -> and we get the same response
2805.06 -> right so it's going to handle that
2806.56 -> automatically for you if you look at how
2808.48 -> we do that in Step functions and step
2810.579 -> functions is the same way we have a
2812.92 -> state machine on I'm sorry how you doing
2815.26 -> we have yeah we have a name token there
2817.78 -> this is It's called the name that's
2819.579 -> actually how we How We Do individually
2821.56 -> with State machine runs so that's the
2823.24 -> name that it's going to run so we pass
2825.04 -> that we get a response back uh and then
2827.98 -> you run it again
2829.72 -> and you get the same response back so we
2833.319 -> we already know what the results from
2834.88 -> that run is we're not going to dupe that
2836.68 -> if you want to run that process again
2838.66 -> then change the name
2843.52 -> so for eventbridge
2845.44 -> with eventbridge we take and you can see
2847.96 -> here this is just an event that we're
2849.64 -> going to drop in
2851.26 -> when we send it we get a response back
2853.3 -> it says here's your event ID
2855.64 -> we send it again
2857.859 -> oh
2860.2 -> not that's right event bridge is not
2863.5 -> item potent right because you're sending
2865.18 -> events all over the place it could be
2866.74 -> that you're doing multiple things right
2869.079 -> so it's not that important so what do
2870.94 -> you do
2871.839 -> so to get around this we're going to
2874.06 -> create so the event ID is not at
2875.8 -> important like we said in the metadata
2877.839 -> we're going to create a key call Item
2879.22 -> potency key or really anything we'll
2881.68 -> talk about that in a minute so we're
2882.7 -> going to create a key okay and in that
2885.339 -> when we send this so let's let's walk
2887.56 -> through the process again we send this
2889.18 -> we're going to send that as our token
2891.099 -> and the item potency key we're going to
2893.2 -> get an event back
2894.94 -> if we send it then what now you notice I
2897.339 -> didn't send it again because it's still
2898.48 -> going to happen Okay adding an item
2900.4 -> potency key is not going to make event
2901.78 -> bridge that impotent what it's going to
2903.64 -> do is allow you on your back end to look
2906.4 -> at that key and say I've already done
2908.14 -> something like that okay so you're aware
2910.54 -> of that
2911.859 -> Okay and like I said I had importancy by
2914.14 -> any other name would smell as sweet the
2916.119 -> item potency ID
2917.56 -> these are some examples of what folks
2919.359 -> use you can you can call it Bob if you
2921.46 -> want I would encourage you to call it
2923.14 -> something unique uh but name it
2926.079 -> something because when you think about
2927.7 -> an item potency token flows through
2929.619 -> intermediaries it's going to go through
2931.06 -> a lot of things and if you use message
2932.5 -> ID going in is one two three the next
2935.44 -> one changes it to four five six the
2937.78 -> third one changes it to 789 that doesn't
2939.88 -> do you any good when you get it back
2941.56 -> your comparison falls down doesn't it
2944.319 -> okay so when you're working with that
2946.48 -> that's important to understand I know
2949.599 -> what my item potency token is I know how
2952.3 -> that's going to work and so I may name
2953.8 -> it you know bob123 loves me all right so
2957.339 -> that's the name people go what does that
2958.66 -> mean I know that's all that matters
2960.4 -> right so when it flows through you get
2962.14 -> it back
2963.819 -> I promised you at the end I would give
2966.579 -> you a QR code this is the one you want
2968.26 -> to grab I encourage you to take a look
2971.14 -> at this
2972.22 -> I hope this session's been helpful in
2974.5 -> covering when we build next-gen
2976.38 -> applications we're using event driven
2978.88 -> architectures you want to be thinking
2981.28 -> ahead right you want to think how do I
2983.56 -> decouple how do I get my system
2985.74 -> resilient right it's up to us as
2988 -> Architects to say when this fails I'm
2991.3 -> going to make it you know I'm going to
2992.56 -> make it work and we build resilience
2994.359 -> again I hope you understand event driven
2997 -> how that works choreography and
2998.74 -> orchestration and finally item potency
3001.26 -> which one of the biggest challenges
3002.52 -> folks deal with on how to work with that
3003.96 -> I encourage you to grab this QR code if
3006.359 -> you've not been to serverless land check
3007.859 -> out serverless land we have a lot going
3009.96 -> on there it was just updated it looks
3011.7 -> fresh and new we've got some more stuff
3013.26 -> uh going on there
3015.3 -> if you want to continue serverless
3017.04 -> learning we have s12d.com serverless
3019.92 -> learning QR code here and finally thank
3023.28 -> you very much that's my Twitter handle I
3025.56 -> would love for you to follow me I I warn
3027.839 -> you you're going to hear about
3029.48 -> serverless Eda you're going to hear
3031.859 -> about pizza and why pineapple shouldn't
3033.839 -> be on it and uh diet Dr Pepper and all
3036.48 -> kinds of things with that thank you very
3038.579 -> much y'all have a great day enjoy
3040.56 -> reinvent
3044.22 -> thank you
Source: https://www.youtube.com/watch?v=SbL3a9YOW7s