AWS Service Spotlight: AWS Step Functions
AWS Service Spotlight: AWS Step Functions
In this episode, we put the spotlight on AWS Step Functions a serverless, function orchestration service. Using Step Functions, you can configure AWS Lambda functions in a series of steps, and combine use of multiple AWS services into business-critical applications through the creation of state machines to manage application parallelization sequencing and error handling. Join Errol Hooper as he gives us an overview of this service, the costs involved, and see Step Functions in action in a hands-on demo technical demo. Errol then wraps up with the pros and cons, and his final verdict.
Timestamps:
Introduction (0:00)
AWS Step Functions Overview (2:31)
An Introduction to AWS Step Functions Pricing (6:18)
Standard Workflow Pricing (7:13)
Express Workflow Pricing (7:59)
A hands-on demo of AWS Step Functions (8:40)
Our Verdict on AWS Step Functions (16:13)
Subscribe to A Cloud Guru!
/ @acloudguru
To see more AWS service reviews visit: https://bit.ly/39wmioh
What is AWS Step Functions?: https://aws.amazon.com/quicksight/
How AWS Step Functions Works: https://docs.aws.amazon.com/step-func…
AWS Step Functions Pricing: https://aws.amazon.com/step-functions…
Creating a Serverless Workflow with AWS Step Functions and Lambda: https://aws.amazon.com/getting-starte…
Like us on Facebook: https://www.facebook.com/acloudguru/
Follow us on Twitter: https://twitter.com/acloudguru
#aws #amazonwebservices
Content
6.77 -> Hello, Cloud Gurus and welcome to AWS service spotlight. In this series,
11.541 -> we put the spotlight on specific AWS services,
14.9 -> walking through detailed technical demos to explore their features, benefits,
19.49 -> and costs I'm Errol.
21.291 -> And today we're going to talk about AWS step functions.
25.07 -> As our business needs evolve.
26.78 -> There's growing demand for us to creatively approach those needs in a way that's
31.521 -> both cost effective for our businesses while still effective for our consumers.
36.291 -> We serve for this case.
38.271 -> Many engineers rely on the power of serverless architectures and
42.891 -> functions to provide resilience in their applications And microservice
47.391 -> solutions.
48.35 -> More specifically engineering teams are turning away from the monolith
53.03 -> architectures, which are difficult to manage, support and upgrade.
57.92 -> And they're moving towards breaking the application into smaller,
61.341 -> more manageable pieces,
62.99 -> which can be easily changed without large impact to the overall process.
67.52 -> A great example of this use case is AWS Lambda,
71.031 -> which is a great option for small repeatable tasks,
74.69 -> which we can complete within a given run time and memory constraints. However,
79.641 -> the challenge introduced with Lambdas is that very much like containers,
84.47 -> they don't maintain state persistence or error handling once its
89.181 -> runtime has completed. Additionally,
92.45 -> once the application requires more than a couple Lambdas,
96.53 -> its complexity grows exponentially and supporting the application becomes a
100.911 -> daunting task. This is where AWS step functions comes into play.
105.71 -> Initially announced at reinvent 2016,
108.92 -> AWS step functions makes it easy to build complex applications by
113.661 -> connecting and orchestrating multiple smaller microservices
118.82 -> through step functions.
120.14 -> You can configure multiple serverless functions in a series of steps,
124.73 -> as well as combine the use of multiple AWS services into business critical
129.651 -> applications through the creation of state machines.
133.31 -> During this episode,
134.72 -> I'll give a rundown of AWS step functions with an overview of the service.
139.94 -> We'll take a look at its features and costs,
142.31 -> and I'll run you through a demo showing you how step function works before
146.331 -> giving my final verdict of the service itself. So if you're ready,
149.93 -> let's get started.
155.27 -> Now let's talk about the service AWS step functions is a serverless function
160.011 -> orchestration service,
161.241 -> which easily allows you to sequence multiple AWS Lambda functions
166.19 -> or other services.
167.69 -> And so larger business applications through the creation of workflows.
172.43 -> Using this service,
173.9 -> you can join loosely coupled application components into larger business
178.401 -> solutions, creating what's called state machines.
182.65 -> AWS step functions allows you to easily configure state
187.481 -> machines or workflows using Amazon's state language.
191.92 -> And I know what you're thinking. I'll have to learn another language.
195.4 -> No worries about that.
197.05 -> Amazon state language is a proprietary AWS language,
201.43 -> which is very close to JSON or JavaScript object notation
206.29 -> within your state machine. You can define various States.
210.58 -> Now States refer to the status of a task or a service.
215.08 -> The States are used as triggers for your workflow and can be used
220.031 -> to begin, pause, complete or completely terminated task.
224.56 -> Now step functions also natively integrates with multiple AWS
229.511 -> services, including compute services like AWS Lambda,
234.52 -> ECS, EKS, Kubernetes service, and AWS far gate,
239.41 -> database services, including dynamo DB messaging,
243.19 -> including simple notification as well as Queues,
246.37 -> API gateway and more step functions makes it fast and easy
251.5 -> to introduce these services to your state machine by providing Amazon state
256.481 -> language code snippets,
258.28 -> which will help you reduce your development time and get your micro service off
262.871 -> the ground. In no time.
264.34 -> Step functions provide a great visual interface,
268.3 -> which outlines checkpoints as well as event driven workflow within the
273.281 -> application.
274.54 -> This makes monitoring and troubleshooting the workflow easy.
278.5 -> Since you can see the progress of each step of the application.
282.37 -> Step function also provides two kinds of workflows to meet your
286.751 -> application needs - a standard workflow as well as express workflows
292.06 -> standard or the default workflow allows for a minimum duration of
297.001 -> one year.
298.11 -> And are ideal for long running durable and auditable workflows while
303.01 -> express workflows expects the workflow to be completed in five
307.951 -> minutes and possibly include synchronous as well as asynchronous
312.451 -> workflows.
313.47 -> Express workflows are ideal for high volume event
317.971 -> processing workloads, such as internet of things, data ingestion,
322.77 -> streaming, data processing,
324.72 -> and transformation and mobile application backends.
328.56 -> Also through the configuration of workflows.
331.8 -> Step functions allows you to manage application parallelization
336.3 -> sequencing, error handling, and retry logic as well as state.
341.19 -> The parallelization feature is great for reducing the right time of long running
345.631 -> processes.
347.07 -> This functionality also makes it easy for States to pass data to
351.931 -> other States and microservices,
354.6 -> to handle exceptions at time,
357.74 -> make decisions as well as execute multiple paths in parallel and more
363.05 -> and last but not least AWS step functions has a budget friendly pricing
367.941 -> model,
368.63 -> which is pay-per-use with a low cost price for state transitions and
373.91 -> workflow duration, which brings us to our next point cost.
377.81 -> Now let's talk about AWS step functions's pricing model.
381.74 -> Like most of the AWS services offered pricing can be a little bit
386.331 -> complicated for AWS step functions.
389.331 -> There are multiple factors that plays in to how you are charged for your use of
393.471 -> the service.
394.34 -> First you're charged based on the type of workflow you use either standard or
399.261 -> express as well as the number of state transitions and the duration of the
403.791 -> workflow. Luckily with this service,
406.58 -> you're charged minimally for each state transition and the duration of the
410.661 -> workflow.
411.62 -> So you'll have to run thousands of workflows for several hours before you
416.69 -> notice any impact to your bill. However, auxiliary services such as compute,
421.94 -> database, messaging and more will incur their own charges.
426.65 -> Here's what you need to know about AWS Step functions Pricing. First,
431.36 -> let's talk about standard workflows.
434.12 -> With standard workflows You're only charged for the number of state transitions.
438.77 -> Step functions,
439.701 -> count state transitions Each time a step of your workflow is
443.931 -> executed.
445.07 -> You are charged for the total number of state transitions across all of your
449.421 -> machines, including retries. For example,
453.11 -> from one Lambda execution to another will count as one state transition,
457.88 -> a retry would count as another.
460.25 -> Luckily for smaller applications and newcomers desiring to get their hands dirty
465.081 -> with AWS step functions The free tier includes 4,000 state
469.461 -> transitions per month. After 4,000 state transitions,
473.6 -> AWS charges you two and a half cent per thousand state
478.071 -> transitions thereafter. Now let's talk about express workflows.
482.21 -> With express workflows You're charged for both the number of state transitions,
487.25 -> as well as the duration of the workflow.
490.04 -> You basically are charged $1 per million request.
494 -> Duration is calculated from the time your workflow begins
498.56 -> executing until it completes or otherwise terminates,
502.52 -> rounded up to the nearest hundred milliseconds.
505.52 -> And the amount of memory used in the execution of your workflow billed
510.41 -> in 64 megabyte chunks.
512.81 -> Duration is currently charged at 6 cent per gigabyte hours.
517.34 -> Now let's head to the demo and check out step functions.
525.01 -> Hello and welcome to this demo on AWS step functions in today's
529.871 -> demo, we'll build a small step function workflow,
533.29 -> which checks the price of Bitcoin.
536.1 -> And then that buys us on whether we should buy more or sell based on its
541.081 -> current price.
542.64 -> To get started We'll head over to the service by typing step functions here on
547.081 -> the AWS management console search bar,
551.25 -> we'll click the service. And now we're here on the main splash page,
554.881 -> which gives us some more information on how the service works.
558.18 -> Some of its benefits and features as well as its use cases.
562.89 -> So I'll expand the pane to the left and I'll head over to state machines.
567.6 -> And here on the state machine page is where I'll define my workflow by creating
572.491 -> a state machine. So I'll click create state machine.
576.24 -> And here we can see that there are several ways which AWS gives us to define our
580.381 -> state machine. One way,
582.301 -> will be to author our workflow using code snippets of the Amazon state language,
587.041 -> the proprietary language here use for step functions.
590.61 -> And this option we can define whether we would like a standard workflow or an
595.111 -> express workflow.
596.76 -> We also have options to modify the code definition as well as see how
601.531 -> those options change our visual workflow here. Another way,
606.18 -> which is really useful for those who may be interested in using step functions
610.141 -> for the first time is to run a sample project by selecting any of the
615.001 -> sample projects. For example, make a call to API gateway.
619.59 -> We can see here that we're given a definition and that we can also see how that
624.001 -> definition works as well.
626.13 -> And the final option would be to start with the template.
629.7 -> By starting with the template We're given some simple suggestions from AWS,
634.261 -> which we can later modify our state language based on the template
639.241 -> provided. For this example,
641.071 -> let's select the hello world template and we'll simply leave our standard
645.661 -> workflow here. And then we'll click next.
649.95 -> Let's name our state machine here, Bitcoin pricing.
654 -> And for this particular example,
655.561 -> I'd like to create a new role and then I'll click create state
660.091 -> machine. And here we see that our state machine has been successfully created.
665.37 -> So if I go over to the definition, scroll down,
668.791 -> I can see my Amazon state language here with the hello,
673.83 -> starting at hello. There's my hello.
676.98 -> I can see the next is up as the world function,
680.551 -> which is also a pass and I can see the world function there.
684.72 -> So now I'd like to substitute this hello portion of the example with the
689.581 -> Lambda that I've already created,
691.74 -> that calls out to an API and checks the price of Bitcoin.
696.06 -> So I'll do that by going to the top of the page here and clicking edit,
699.63 -> and here under definition, I'll click the drop down to generate a code snippet.
705.48 -> And the first election I have here is the AWS lambda invoke a function which
709.71 -> I'll select.
711.18 -> And this option opens up the Lambda invoke task date,
715.81 -> where I can select the options for my Lambda.
718.75 -> The first thing I'd like to know here is that I'm selecting the function from a
722.141 -> list and then the next dropdown I'd select my Bitcoin pricing
727.06 -> prod Lambda.
728.59 -> I'll keep the payload the same as there's no payload required for this function.
732.79 -> And then I'll just click copy to clipboard.
735.96 -> Now our code snippet has been copied to the clipboard.
739.32 -> I need to rearrange my definition to include the Lambda in which we just
744.061 -> created a task state for.
747 -> So there's a couple of steps that I need to do here.
749.7 -> As we look at the state language, we see that the state starts hello,
753.93 -> and after hello, we pass on to world.
757.53 -> So what I need to do is highlight this particular section,
761.4 -> hello and paste What I have from my clipboard here.
765.06 -> Now there's a couple more steps. First.
767.431 -> I'll need to start by telling my state machine where to start now,
771.27 -> which is now our Bitcoin pricing Lambda.
774.24 -> And it's a really good idea to name the step based on the action that it will be
778.711 -> taking.
779.61 -> So I'll just name this Check Bitcoin
784.47 -> pricing. And I'll copy this.
788.91 -> So rename my state here, which is initially invoke Lambda function.
794.58 -> And after this is completed,
795.931 -> I need to check line 14 and tell the state machine what to do
800.491 -> next. And in this case, while we'll change this in just a moment,
804.99 -> to fix our error here, we need to change this to world. And as you can see,
809.281 -> almost instantaneously, our visualization has updated.
814.05 -> And now we see our start.
815.97 -> We see we go into our check Bitcoin pricing lambda followed by world
821.04 -> and end. So I'll click save. I'll ignore my
824.91 -> IAM role permissions as I'll fix that on the back end in just a moment,
828.93 -> click save anyway, and we've updated our state machine.
833.01 -> Now that we've added one step to our hello world state machine.
837.42 -> I've went ahead and added multiple steps to the Amazon state language,
842.46 -> which I'll paste here in the definition box.
845.16 -> So we could get a better understanding of how a longer process would work with
849.391 -> step functions. So as you can see,
851.851 -> I've added multiple steps using another integration that exists within my
856.621 -> current step functions within the AWS account.
860.25 -> So I'll go ahead and click save. I'll save any way.
863.4 -> And now I'll go back to my Bitcoin pricing step function.
867.09 -> Now let's execute our state machine so we can see how the process
872.011 -> works step by step. So I'll click start execution,
875.37 -> no need to change any of the input here.
878.73 -> And now we can see that our state machine is underway.
882.12 -> And as we can notice that after every successful step,
886.02 -> we'll see that the step it turns green. So here on our check Bitcoin pricing,
891.44 -> if we go over to step output, we can see that currently the price of Bitcoin is
896.181 -> 57,754.
899.39 -> Let's click the next step to see what's our buy sell recommendation.
903.92 -> We can see that at that particular price,
906.86 -> our Lambda recommended that we sell. I'll scroll down a bit
913.79 -> and now we can see that the process is continuing.
917.51 -> So we're currently waiting on the step, which requests human approval,
922.13 -> which received the stock price, as well as the recommendation type.
926.72 -> We'll go to the next step,
928.13 -> which generates the buy or sell in which this case,
931.761 -> the state machine has decided to sell the stock.
935.36 -> So here we see the id of the run, the price, the type,
939.681 -> which is sell at a quantity of one and a timestamp. And then at the end here,
944.661 -> we are reporting out more results.
947.81 -> So our process has successfully completed.
951.47 -> And the really cool thing here about step functions here at the bottom is that
955.25 -> you have an execution event history where you can both access
959.99 -> easily, the Lambda, as well as the CloudWatch logs,
963.38 -> as well as view the history of where the step may have failed or successfully
968.211 -> completed. So that concludes this demo on AWS step functions.
975.13 -> So what's the verdict.
976.69 -> Let's briefly discuss the pros and cons of AWS step functions.
981.25 -> We'll start with the pros.
983.02 -> First off AWS step functions is all things serverless,
987.79 -> meaning we don't have to manage infrastructure to create these workflows,
991.99 -> managing the running of our source code.
994.48 -> AWS step functions also allows you to easily create complex
999.221 -> workflows across multiple AWS services.
1002.46 -> Due to its statefullness It easily manages States without the use of
1006.901 -> messaging queues and databases.
1009.87 -> Also another plus of using this service is it's parallelization,
1014.4 -> which allows us to shorten the run time of our long running processes.
1019.26 -> Lastly, step functions provide a great visualization interface,
1024.18 -> which is easy to follow for the technical and untechnical alike.
1028.17 -> Now let's talk about the cons.
1030.33 -> One of the few drawbacks of learning AWS step functions is that you
1035.071 -> must come to a pretty good understanding of Amazon state language,
1039.3 -> which is the proprietary AWS language used for step functions.
1043.98 -> Also, there is a learning curve for learning to programmatically integrate other
1048.721 -> services within the workflow. Lastly,
1051.721 -> one con of using this service is that defining too much business logic within
1056.431 -> the service or your state machine creates vendor lock-in so should you
1061.261 -> invest, evaluate or revisit.
1069.78 -> [Inaudible]. My recommendation.
1071.39 -> Is to invest. AWS Step function is a great orchestration tool,
1075.741 -> which allows us to seamlessly couple our microservice and loosely coupled
1080.721 -> application components into a single unified workflow.
1084.95 -> If you're currently supporting a monolith application,
1088.16 -> which is difficult to support and upgrade using a service like step functions
1093.051 -> would be great in reducing complexity through simplifying your
1097.401 -> architecture into smaller microservices and creating a visual
1101.961 -> workflow to monitor the smaller steps of the process as they are occurring.
1106.58 -> Additionally, AWS step functions are useful for data processing,
1111.11 -> ensuring that long running jobs execute an order and complete successfully.
1116.12 -> It's also a great service for automating recurring tasks,
1119.57 -> such as patch management, infrastructure selection,
1123.14 -> and data synchronization,
1124.94 -> and step functions will automatically scale and respond to timeouts
1129.74 -> as well as retry fail tasks.
1132.17 -> This service is also great for those applications,
1135.5 -> which require human interaction within a larger workflow.
1139.61 -> From a programmatic perspective because AWS step functions, native languages,
1144.5 -> Amazon statement, which, which is very similar to JSON,
1148.91 -> it makes use of this service very easy and the technical barrier of
1153.591 -> entry for this service, somewhat low.
1156.35 -> Step functions also make kicking off workflows very easy for those
1161.331 -> less technical business users who support processes and needs to kick them
1166.311 -> off at any time.
1167.57 -> So that's it for this episode of AWS service spotlight keep being awesome
1172.431 -> cloud gurus, and I hope to see you next time on AWS service spotlight.
Source: https://www.youtube.com/watch?v=VaWXk-Hy9bo