AWS re:Inforce 2022 - Uplifting AWS service API data protection to TLS 1.2+ (DPP202)

Aug 16, 2023

AWS re:Inforce 2022 - Uplifting AWS service API data protection to TLS 1.2+ (DPP202)

AWS is constantly raising the bar to help customers use the most modern Transport Layer Security (TLS) encryption protocols, which meet regulatory and security standards. In this session, learn how AWS can help you easily identify if you have any clients or applications using older TLS versions. Hear guidance and insight for detecting the use of outdated TLS protocols, and see a demo of how to update your clients and applications to use only modern versions.

Learn more about AWS re:Inforce at https://bit.ly/3baitIT.

Subscribe:
More AWS videos http://bit.ly/2O3zS75
More AWS events videos http://bit.ly/316g9t4

ABOUT AWS
Amazon Web Services (AWS) hosts events, both online and in-person, bringing the cloud computing community together to connect, collaborate, and learn from AWS experts.

AWS is the world’s most comprehensive and broadly adopted cloud platform, offering over 200 fully featured services from data centers globally. Millions of customers—including the fastest-growing startups, largest enterprises, and leading government agencies—are using AWS to lower costs, become more agile, and innovate faster.

#reInforce2022 #CloudSecurity #AWS #AmazonWebServices #CloudComputing

Content

0.57 -> - Hello and welcome, everyone.

3.45 -> If you are not 100% certain that your AWS API applications

10.38 -> are TLS 1.2 ready, then this is the right session for you,

14.97 -> because 99.9% sure is probably not how you want to be

19.65 -> responsible for keeping your applications up.

24.03 -> We will talk about how to know if you're good to go,

27.78 -> meaning you'll be able to continue to connect to AWS.

32.61 -> Or if you're not so good,

34.32 -> we will also cover some simple remediation steps

37.47 -> you can take to fix your application.

40.32 -> I'm Janelle, and over to the right is my partner, Daniel.

44.4 -> And we were going to go over next

47.64 -> what we will cover with you today.

50.4 -> Looking at our agenda.

52.41 -> For today's session, we will be talking to you about

56.01 -> our current TLS deprecation effort,

59.7 -> meaning we're going to modernize the TLS implementation

64.56 -> on all of our AWS API endpoints.

68.7 -> Now this work has already started

71.7 -> and will finalize in 11 months.

74.4 -> So the clock is ticking.

76.56 -> We are splitting this talk into two parts.

79.53 -> First, I'm going to be covering

82.05 -> what is this current work we are doing?

84.54 -> Why we are doing it now.

86.55 -> I will talk a little bit about TLS

89.97 -> and the Shared Responsibility Model.

92.85 -> And I will also go over the TLS connection workflow

96.96 -> and a bit about TLS negotiation

99.57 -> so we can fully demonstrate how our current work

103.47 -> can pose a risk of availability impact to you.

108.51 -> I will then hand off to Daniel who will go through

112.08 -> how to know if you are impacted, what to do if so.

116.31 -> He will cover a few advanced scenarios

118.47 -> that you wanna be aware of just in case

121.56 -> and provide a demo on how to find and verify fixes

125.34 -> in your applications to prevent any impact.

128.88 -> So let's get started.

133.17 -> Like this dad changing their adorable baby,

136.08 -> let's talk about what AWS is also changing in regards

139.59 -> to our TLS implementation.

142.14 -> Now in response to evolving technology

144.81 -> and regulatory standards,

146.67 -> we have announced a TLS deprecation effort,

150.15 -> which means we'll be ensuring only modern TLS,

153.69 -> meaning TLS versions 1.2 and upwards,

157.08 -> can be used when connecting to our APIs.

161.46 -> This is a global change, affecting all AWS APIs,

165.78 -> all AWS services.

167.64 -> And we did recently announce this just about a month ago

171.11 -> in an AWS Security blog post that we posted on June 28th.

176.4 -> Now we will be making these changes gradually

178.92 -> over the next month...

180.66 -> Or next 11 months, pardon me.

183.06 -> So we will be doing these changes

184.71 -> on a service-by-service basis.

187.29 -> But once we hit June 28th, 2023,

192.18 -> this will be the last date that you can use TLS 1.0

196.35 -> or TLS 1.1 to make a successful connection to our APIs.

202.35 -> Now, fortunately, this only affects a small subset of you.

205.29 -> I will go into this in more detail on the later slides,

208.32 -> but just important to know.

211.11 -> Now we've talked about what is changing.

213.27 -> Let's cover a little bit about what is not changing.

216.87 -> I also wanna highlight that we are not changing

220.32 -> the endpoints that you, our customers, own.

223.83 -> If you are running your own APIs

227.1 -> built using our AWS services,

229.74 -> you may have built them using API Gateway,

232.23 -> any of our load balancing services

234.36 -> or you have your own CloudFront distribution,

237.12 -> you fully own the TLS configurations

240.3 -> for these endpoints that you own.

243.87 -> The only thing we are changing is the TLS configured

248.28 -> on the endpoints for our AWS services.

251.85 -> Now we encourage you to go ahead and update those,

254.55 -> but this is not required.

255.87 -> This is fully up to you.

258.75 -> The other thing that we are also not changing

262.08 -> is we are not touching certificates.

264.21 -> So there are no changes to the X.509 certificates

267.27 -> that you are using for your TLS connections.

270.45 -> And there are no changes to the chain of trust.

273.39 -> Again, we are only changing the TLS versions

275.79 -> that you can negotiate.

278.7 -> Next, I wanna go over a bit more of

282.84 -> on the context of these planned changes

284.85 -> by looking at our TLS history.

289.59 -> Now, as you can see, originally TLS was known

294 -> as Secure Socket Layer or SSL,

297.45 -> and it's still fairly common to hear someone say

301.08 -> or use SSL just out of habit,

304.41 -> even when the underlying connection

306.27 -> is technically using TLS.

309.6 -> Now you can also see that not only has the name changed,

314.88 -> also the versions have evolved over time.

318.3 -> So just taking it from the beginning here.

321.45 -> Looking at SSL v2 and SSL v3,

325.89 -> developed in 1995 and 1996, respectively, by Netscape.

331.05 -> So, Netscape originally created these versions

333.24 -> and they were the first widely used network protocols

337.29 -> to enable cryptographically secured communications

341.25 -> on the internet.

343.41 -> Netscape then handed this over to the IETF,

347.28 -> what it is now publicly owned,

349.65 -> and because SSL was trademarked, the name became TLS.

355.44 -> So starting with the first IETF version,

357.66 -> which is surprise, TLS 1.0.

360.27 -> so this was published back in 1999,

362.97 -> and then came TLS 1.1 in 2006.

367.35 -> Now these have both been officially deprecated

369.75 -> as of March of last year.

372.42 -> However, AWS has historically continued to support these

376.83 -> for backward compatibility purposes for our customers.

380.04 -> But now this is changing.

381.6 -> We are giving you 11 months.

384.27 -> We also want to note we have deprecated support

386.79 -> for SSL v2 and v3 many years ago,

390.36 -> with v3 getting deprecated back in 2016.

395.7 -> Now, when we refer to modern TLS,

398.43 -> what we mean is TLS version 1.2 published in 2008,

404.16 -> and TLS version 1.3 published in 2018.

408.57 -> These are the only two versions that we are going to support

412.56 -> on our service endpoints when we complete our work.

420.15 -> Let's talk a little bit about why

421.47 -> we're making these changes now.

423.99 -> Why we think now is the right time for two primary reasons.

428.19 -> First, many of you, more and more of you frankly,

432.72 -> are asking us to make this change.

435.3 -> So, we recently had a fairly large customer

438.8 -> in a call not too long ago say,

441.607 -> "We don't even want you to figure out

442.98 -> what applications might break.

444.66 -> Just make the change, go ahead and break us.

448.14 -> Just do it."

449.07 -> Now, these are global settings,

450.57 -> so we are not going to take that drastic measure.

453.42 -> And we are giving you 11 months to make this change,

456.39 -> 'cause it would impact all customers.

458.43 -> But just to show the passion that some of our customers

460.56 -> do have about us completing this work.

463.59 -> Secondly, and I mentioned this earlier,

465.75 -> that another reason why we are making this change now

469.56 -> is that less and less of you are impacted.

472.44 -> We have determined that less than 5% of you

475.56 -> are still using TLS 1.0 and 1.1

478.05 -> when you're making connections to us.

480.99 -> So most clients, as long as you have built or updated

484.65 -> your software since 2014,

488.163 -> then most likely you are already defaulting to use TLS 1.2.

493.02 -> So that is 95% of you are using TLS 1.2.

497.61 -> But another kind of word of caution here

499.56 -> is we also did another small pilot

501.93 -> before we started this effort,

503.7 -> reached out to about 30 of our larger customers again

506.67 -> that we detected using TLS 1.0 to try to understand

510.99 -> what was their use case; why were they using this?

513.63 -> Could they update?

515.61 -> And what we found actually was the overwhelming majority

519.51 -> of them were shocked, really surprised that they even had

523.47 -> anything running in their environments that used TLS 1.0.

527.52 -> So oftentimes this can be like forgotten test jobs,

531.06 -> backup jobs, I think one case was one host got missed.

534.99 -> So, just things to be aware of.

538.02 -> Even if you think you're good, it is good to verify, right?

542.91 -> The other point here is that all of them were very happy

545.4 -> we reached out, happy we're doing this work.

548.1 -> So, again, there is overwhelming support,

550.17 -> we hear from your customers for us to go ahead

552.48 -> and make these changes.

557.19 -> So let's talk a little bit about

558.87 -> the AWS Shared Responsibility Model

561.06 -> and how that relates to TLS.

564.24 -> You're probably familiar with this model.

567.15 -> As you know, you, our customers,

570.27 -> are responsible for security in the cloud,

573.66 -> meaning how you use the cloud,

576.36 -> how you configure and secure your resources in the cloud.

580.23 -> In context of TLS, what this means is that right now

585.06 -> you already have control to set the TLS versions

588.54 -> you're going to use to connect.

590.52 -> I will go into this in more detail in the later slides,

593.55 -> but it's just important to know that you do have

595.89 -> this control now to set the TLS version

598.11 -> that your client applications will use.

601.32 -> Now, we, AWS as a service provider,

604.08 -> are responsible for security of the cloud,

607.62 -> meaning how we build and maintain the entire infrastructure

611.55 -> for the benefit of all of you.

614.49 -> Putting this into context of our TLS deprecation work,

618.3 -> that means we control the TLS configuration

621.42 -> on our service endpoints and we will be updating those

624.84 -> to remove TLS 1.0 and 1.1.

628.89 -> If you are using TLS 1.0 and 1.1, to repeat,

632.34 -> that you will need to take action, update your application,

636.24 -> to be able to continue to connect to our service endpoints.

642.03 -> Now, when we say you need to take action and update,

644.49 -> I wanna be very specific about what we mean there.

647.7 -> So we're using terms like API client,

649.62 -> so let's just talk a little bit about what that is.

652.23 -> To clarify what we mean,

653.85 -> API client is your software application, program,

657.84 -> service or tool that is making programmatic calls

661.77 -> to our API endpoints.

664.02 -> Standard ways that you can create an API client

668.01 -> that will call us is using any of our AWS developer tools,

671.55 -> for example.

672.78 -> So, common ways are our AWS Command Line Interface tool,

676.95 -> our CLI tool, any of our AWS software developer kits,

681.09 -> which we have available in many programming languages.

684.72 -> You could also use any library, such as Boto3 for Python,

690.12 -> a library that basically wraps API calls

692.7 -> in a Python friendly format.

694.71 -> And then, lastly, another common example of an API client

698.46 -> is any browser or compute resource with internet access

703.05 -> that can make authenticated https calls.

706.35 -> For example, PUT or POST calls using cURL.

709.44 -> So all of these are examples of API clients.

716.91 -> Here, I wanna walk you through a TLS connection workflow

720.78 -> so that we can fully understand the full picture

723.87 -> and what part of this we are changing,

725.49 -> what parts we are not.

727.47 -> So, in this case, we have a client that wants to make

730.98 -> an API call, in this case to Amazon EC2.

735.75 -> They're doing a simple describe instance call.

738.93 -> The client makes the API call,

741.78 -> and first thing that happens behind the scenes

744.9 -> is there is a DNS lookup performed

747.69 -> and the full https URL is discovered.

754.74 -> Now, the fun part happens; the TLS negotiation begins,

759.96 -> meaning the TLS configuration on the client

762.9 -> will talk TLS to the endpoint in the cloud.

767.28 -> In this case, a successful TLS negotiation is established.

770.94 -> A connection is made.

773.43 -> Now, this is ensuring that there will be

775.62 -> securely encrypted data passing back and forth

779.01 -> between the client and the service, in this case, EC2.

784.17 -> The API call is processed by the service.

786.99 -> It gets written to CloudTrail,

788.82 -> and using that same secure TLS connection,

791.55 -> the service returns the results back to the client.

795.15 -> Now, the only thing here that we are changing

798.09 -> is in step two; is the TLS negotiation that happens

802.02 -> during step two.

803.07 -> The rest of the calls, the commands,

804.93 -> everything else on the service side is staying the same.

812.55 -> Again, I've mentioned this before,

813.99 -> but just to make sure we're clear.

816.15 -> If your software does not support TLS version 1.2,

820.44 -> then you will be impacted and not be able

823.05 -> to successfully connect once we do the deprecation.

828.15 -> An important call out we also want to make

830.04 -> about our scope here is that when we're talking about

833.7 -> API calls, it's quite common to think about

837.03 -> your control plane calls,

838.44 -> describe instance as I noted before,

840.99 -> but it's important to also remember that this does include

843.93 -> any data transfer calls or data plane calls.

846.99 -> So four of the most common services

848.403 -> that you may be making data transfer calls to

851.49 -> that we have here are Amazon S3, Amazon DynamoDB,

855.81 -> Amazon SQS, and Amazon SNS.

858.99 -> So as these are all in scope,

861.63 -> includes your control plane and data plane calls.

863.7 -> These also will be part of the deprecation

865.98 -> that we'll be doing.

870.18 -> Next, we're gonna go over a bit more of the impact risk

873.51 -> by walking through the TLS negotiation in more detail.

876.63 -> So, going back to that workflow,

878.85 -> if remember I talked about in step two,

880.83 -> the TLS negotiation was the only thing that we're changing.

883.8 -> So this is specifically focused on

885.39 -> how that negotiation occurs.

888.03 -> So in this case, we have...

890.31 -> Or you have, I should say, a modern API client

893.25 -> that supports TLS 1.0, 1.1 and 1.2.

896.64 -> And then the service, pre-deprecation,

898.74 -> also supports 1.0, 1.1 and 1.2.

903.6 -> So, by default, our services will select the highest

907.41 -> mutually supported version.

909.45 -> And this is where I go back to saying,

911.13 -> now you can control the TLS.

912.9 -> As long as your client supports TLS 1.2,

915.69 -> TLS 1.2 will be used for that connection.

919.62 -> After deprecation, you have not changed anything

922.56 -> in your client here, we have removed 1.0 and 1.1,

925.89 -> we've got a modern client.

928.86 -> So, great, successful TLS 1.2 connection happens again.

934.77 -> Couple points here is this all happens

937.86 -> transparently for you,

939.24 -> nothing you need to do with a modern client,

941.46 -> you're able to connect using 1.2.

944.73 -> You also may notice that you did not

946.32 -> have to remove 1.0 and 1.1.

949.47 -> So you don't technically need to remove that,

952.14 -> and also wanna call out that AWS does have

955.95 -> anti-downgrade protection, meaning that a malicious actor...

960.39 -> We have measures in place to prevent a malicious actor

964.02 -> from downgrading your TLS connection to a lower version.

969.96 -> Many of you may be subject to compliance regulators

974.01 -> or authorities that don't often deal well in nuance,

978.27 -> as you're aware.

979.32 -> So while you do not have to remove TLS 1.0 or 1.1

983.73 -> from your clients, this could be a good opportunity

985.77 -> to simplify and just make it easier

988.05 -> to get past those audits, right?

991.29 -> So taking the same case with an obsolete API client.

995.34 -> This time, the client only supports TLS 1.0,

999.3 -> the server supports 1.0, 1.1, 1.2,

1002.45 -> and a successful connection happens.

1004.76 -> But here's where the problem lies.

1007.07 -> We've now completed deprecation.

1008.81 -> Our endpoint now only supports 1.2

1011.81 -> and this connection is going to fail.

1013.82 -> This will be a hard fail.

1016.43 -> This is a situation we don't want any of you to be in,

1019.64 -> one of the primary reasons we're doing

1021.29 -> this presentation today.

1023.12 -> And a little bit later, Daniel will go in his slides

1025.7 -> on how you can verify this and also update your clients.

1032.99 -> Many of you may be wondering about TLS 1.3.

1036.68 -> So the good news is yes, we are adding 1.3.

1041.45 -> We've already started.

1042.41 -> In fact, we have some services already supporting TLS 1.3.

1046.94 -> KMS, Pinpoint are two that come to mind.

1050.39 -> We'll be adding more in the coming months.

1053.57 -> And the also good news is there's nothing

1055.22 -> you really need to do, again, if you have a modern client.

1057.2 -> It will just happen automatically.

1058.88 -> So in this case, this was before we add 1.3,

1061.52 -> you have a modern client that will support up to 1.3.

1064.728 -> 1.2 connection happens.

1067.01 -> Once we add 1.3 in the endpoint,

1069.02 -> just automatically you will make a 1.3 connection,

1071.93 -> have those performance and security benefits.

1074.06 -> Again, nothing you need to do.

1079.4 -> Going back a little bit just to tie this back

1081.38 -> to that workflow that we saw before,

1084.17 -> focusing just on the negotiation.

1086.63 -> This is just the ClientHello, part of the TLS negotiation.

1090.38 -> Modern client supports all the way up to 1.2.

1093.14 -> The server responds, I also support 1.2.

1097.1 -> Let's use 1.2.

1098.18 -> A successful TLS 1.2 connection.

1104.18 -> Again, with an obsolete client this time.

1107.42 -> The poor old client only supports TLS 1.0,

1110.78 -> the server supports TLS 1.2,

1113.81 -> and this is going to fail.

1121.55 -> So, at this time, I'm going to hand over

1123.68 -> to my partner, Daniel.

1125.6 -> He's going to cover with you how to know if you're impacted

1128.57 -> and what to do if so, some advanced scenarios,

1132.11 -> and he'll walk you through the demo.

1133.46 -> Thank you, Daniel. - Thank you, Janelle.

1136.61 -> All right, great to meet everybody here.

1138.5 -> I'm Daniel Salzedo, I'm a Security Specialist

1141.26 -> Technical Account Manager

1142.61 -> in our Worldwide Enterprise Support Team.

1144.29 -> And I'm very excited to talk to you about this project

1146.36 -> because once it's complete,

1147.62 -> it will ensure that everybody gets the benefits

1149.69 -> of modern TLS in their AWS environment.

1153.71 -> So, as Janelle has explained,

1155.93 -> if you currently have API clients that are making

1158.42 -> TLS 1.0 or 1.1 calls today,

1160.43 -> or if you're not sure if you have them,

1163.16 -> there's a danger of impact.

1164.42 -> So you need to go and make sure that you know

1167.06 -> if you're impacted and if you are,

1168.35 -> you're gonna need to take some action to resolve that.

1170.69 -> So we've got a very simple three stage plan

1173.03 -> for you to do that.

1174.08 -> So the first thing to do is to monitor

1176.12 -> for your AWS notifications.

1178.16 -> Now you should be doing this anyway.

1179.24 -> It's an operational best practice,

1180.62 -> but we're gonna be sending out some notifications

1183.02 -> specific to this project.

1185.27 -> The second thing to do is to analyze your API calls.

1188.45 -> So those calls you're making to your services,

1191.75 -> even if you don't get a notification from us,

1193.82 -> I'll show you how to do that.

1195.05 -> We've got some information there

1196.16 -> that's gonna be very useful to you.

1197.51 -> And then, lastly, but probably most importantly,

1199.67 -> check your clients 'cause the client is

1201.19 -> at the end of the day, the thing that's making the call,

1203.21 -> and that may need to be updated.

1205.28 -> So I'm gonna go through those in a bit more detail.

1207.47 -> And then we'll talk about the actual actions

1208.88 -> you may need to take.

1210.29 -> So I mentioned notifications.

1211.67 -> This is what one of the notifications we will look like.

1213.62 -> We actually sent the first batch of these out

1215.36 -> about a week ago and we're gonna be doing this continually

1218.09 -> on a rolling basis over the next 11 months

1220.52 -> as we get through this project.

1221.93 -> So if you currently are not sure who is monitoring

1225.26 -> your Personal Health Dashboard for notifications,

1227.06 -> you need to know and you need to get that done.

1228.77 -> It will also get sent to an email address,

1230.81 -> the registered email address,

1232.13 -> just like any other service notification.

1235.49 -> So if you get one of these, you need to take action on it.

1238.34 -> If you don't get any of these,

1240.02 -> it's a good indicator that you don't have this issue,

1241.58 -> but it's not the only thing you have to do, unfortunately.

1243.98 -> Because this is the best effort,

1245.39 -> we are using our backend logs to find your connections,

1247.97 -> but we can't guarantee getting each and every single one,

1251 -> especially if your calls are very infrequent

1253.55 -> or if you have created, let's say, a public S3 bucket

1256.22 -> with unauthenticated access.

1257.63 -> We can't always track those.

1259.28 -> So that's the first thing to do to be looking out for this.

1263.45 -> The second thing to do, and you can do this in parallel

1265.79 -> or you can do this right now,

1266.72 -> is to go into CloudTrail and look in your calls

1270.71 -> for TLS details.

1271.55 -> So if you haven't seen this before,

1273.29 -> you haven't been into CloudTrail lately,

1274.76 -> we added this section, I've blown it up here,

1277.16 -> this TLS detail section.

1278.75 -> Now this output is supported by about 50%

1281.96 -> of our services right now.

1283.28 -> There's a link on the screen,

1284.33 -> we've got a public documentation page.

1286.01 -> We're rolling out support trying to get it to every single

1288.47 -> service as quickly as we can.

1290.06 -> And this gives you the exact version of TLS that was used,

1292.82 -> as well as the cipher suite,

1293.84 -> which may be useful to your compliance teams.

1296.48 -> So in this particular screenshot here,

1297.98 -> you can see TLS v1 was used.

1299.63 -> Now, I'm gonna emphasize this,

1300.92 -> I'm gonna say it several times.

1302 -> Any time you see TLS v1 in a technical document,

1305.45 -> that means 1.0, not any version of one,

1308.99 -> not 1.0 through to 1.3.

1311.18 -> It means only 1.0.

1312.86 -> Very important.

1315.14 -> The other information on here, when we did this project,

1317.66 -> the first time as Janelle mentioned

1319.67 -> for our FIPs designated endpoints,

1321.35 -> when we worked with our customers and they had to track down

1324.65 -> the couple of clients that were still making TLS 1.0 calls.

1328.22 -> What we found is these two pieces of information

1330.38 -> in the CloudTrail entry were the most useful,

1332.3 -> the source IP address.

1333.47 -> Now, that'll be the public IP address

1334.94 -> the connection came from, typically a NAT,

1336.89 -> a proxy gateway, a firewall.

1338.42 -> And the second one, the user agent,

1340.16 -> that's provided by the client actually making the call.

1342.74 -> The actual piece of software making that call.

1345.53 -> If you've never seen a user agent string,

1347.81 -> it's in a key value pair format.

1350 -> So it's a set of data divided up,

1352.25 -> key value divided by a slash

1353.42 -> and there should be a space in between.

1354.92 -> That is under the control of the client.

1356.81 -> It can be very detailed or it can be very simple.

1359.36 -> This example here is from the AWS CLI.

1362.03 -> So it has three main sections;

1363.53 -> the name of the client, CLI, the version of it,

1366.35 -> then it runs on Python so it's got the name and the version

1369.02 -> of the Python library it uses,

1370.64 -> and then it has the operating system as well.

1372.26 -> And even the fact that it's a 64-bit version of Windows.

1375.26 -> There's actually even some extra data down past it,

1377.02 -> as you can see just about on that screenshot.

1378.8 -> That's an extension, not particularly relevant in this case,

1382.82 -> but some clients put that in there.

1385.19 -> However, it can also be much simpler.

1386.84 -> It can be a simple, a single key value pair

1389.27 -> at the absolute most basic.

1390.53 -> So, that is typically what you're gonna go looking for

1394.55 -> when you find that client.

1395.42 -> When we did this last time,

1396.8 -> the vast majority of our customers had basically one or two

1400.22 -> user agent strings for either an old build of a library,

1403.16 -> like an old build of Java or an old build of .NET

1405.92 -> or a specific piece of software, like a user agent

1411.29 -> for some, like a piece of backup software or something

1413.42 -> that had just never been updated for years.

1414.89 -> So that was the most common use cases we found.

1417.98 -> Now most of our customers at AWS, we got a lot of people

1421.61 -> running large, complicated environments.

1423.35 -> One of the things I'm not gonna advise you to do

1425.39 -> is to try and do this one by one,

1426.92 -> by looking at every single CloudTrail entry

1430.338 -> in an environment where we have hundreds of counts.

1431.99 -> So we've got a bunch of tools to do this at scale.

1435.14 -> Highly recommended if you've never looked at these tools,

1437.6 -> CloudTrail Lake, we just launched that this year.

1439.85 -> It's a great tool for going in and analyzing CloudTrail

1443.06 -> at scale across accounts and regions.

1446.21 -> Super easy to set it up, no infrastructure.

1448.43 -> It's like two button clicks and you've got a SQL queryable

1452.06 -> data lake ready to go.

1454.1 -> CloudWatch Log Insights is also a great feature.

1457.19 -> CloudWatch, what you do is you go on a CloudTrail,

1459.59 -> you go to your trail and say, push this to CloudWatch.

1461.75 -> And then you can do a bunch of stuff with CloudWatch;

1463.88 -> you can create metrics, you can create alarms,

1465.29 -> you can create dashboards.

1467.27 -> The Health Events are those Personal Health Dashboards,

1469.55 -> as well as other notifications from us.

1471.62 -> Hopefully you've got an idea those are important

1474.2 -> and you can aggregate those.

1475.46 -> Using the AWS Organizations feature,

1477.8 -> you can aggregate those across your fleet of account,

1480.32 -> so you can see those all in one place.

1482.57 -> And the Health Service itself,

1484.88 -> as well as CloudTrail and CloudWatch, they all have APIs.

1487.43 -> So if you are the kind of person who likes to build your own

1490.49 -> tooling or you are working with other companies

1492.71 -> or third party tools, there's APIs,

1494.84 -> you can go and extract what you need.

1498.44 -> So, I'll just show a couple of those in details.

1501.65 -> This is CloudTrail Lake if you've never seen this.

1503.6 -> You turn it on and we have published a sample query.

1506.54 -> This is on our blog where you can go in there.

1508.033 -> This is a query specifically to look for

1509.99 -> old versions of TLS.

1511.46 -> In Log Insights, this is the same idea.

1514.16 -> Once you've got this turned on, you can go in there...

1516.14 -> It uses a slightly different, it's not SQL,

1518.437 -> it's a different query language, but again,

1520.55 -> you go in here and we've got a published sample query

1523.73 -> you can use.

1526.91 -> So, the next step, once you've gone,

1529.58 -> analyzed your API calls, is to check your clients.

1533.15 -> A simple rule of thumb here is if you have clients

1535.85 -> that haven't been updated since 2014,

1538.43 -> they're probably gonna need to be touched.

1540.14 -> You're gonna need to go in and find those.

1541.37 -> If you've got that fleet of Windows 2008 servers

1544.85 -> or really old builds of Linux somewhere,

1546.98 -> that's probably what you're gonna need

1548.24 -> to focus your effort on first.

1550.04 -> Now, the good news is the inverse of that

1552.08 -> is if you have anything on this list here,

1554.06 -> then you should most likely be okay.

1558.53 -> If you are using our tooling,

1559.76 -> if you're using our CLI tool,

1561.38 -> any version of version one from 1.20 upwards,

1564.08 -> any version of version two is gonna be using TLS 1.2

1567.41 -> by default, no problems.

1568.82 -> If you're using our SDKs, we've got those

1570.83 -> in a lot of different languages.

1573.86 -> Just make sure you've got a current version

1575.45 -> and you'll be fine.

1577.28 -> If you're a Linux shop,

1578.48 -> you are probably using OpenSSL for TLS.

1580.7 -> It's not the only library,

1581.69 -> but it's definitely the most popular one.

1583.64 -> 1.0.1 is the minimum version you wanna have.

1586.007 -> So, TLS 1.2 will be enabled by default.

1589.07 -> If you are a Windows shop,

1591.44 -> Microsoft Windows Workstation version eight

1594.47 -> or Server 2012 R2 and upwards.

1597.44 -> Now, there is some limited TLS 1.2 support

1600.41 -> in older versions of Windows,

1602.06 -> but those versions are not supported

1603.715 -> currently by Microsoft anymore.

1605.39 -> You're not getting support, you're not getting patching.

1607.04 -> So I wouldn't recommend using those even if you can

1609.86 -> get TLS 1.2 to work.

1613.94 -> There are two other options you may need to be aware of.

1616.46 -> Hopefully you won't get into this,

1617.75 -> but we did have a few customers last time

1619.61 -> who had to do this.

1620.443 -> So the first is getting low level,

1622.7 -> and I will show you this in my demo

1624.02 -> because it is a useful thing to do if you're just going in

1626.45 -> and actually wanna see the traffic

1628.04 -> and troubleshoot directly.

1629.27 -> If you want to see exactly what version of TLS

1631.85 -> any outbound connection is making from your network,

1634.61 -> you can do that with a packet capture.

1636.47 -> If you're a security person and you're not a network person,

1639.65 -> ask your network team to help you.

1641.54 -> If you have a corporate network with a traditional outbound

1644.81 -> NAT gateway, you can see all the traffic in one place,

1647 -> or you can just do it locally,

1648.26 -> literally on a single system by capturing locally,

1651.32 -> which I'll show you later.

1652.67 -> The other thing you may want to do is analyze

1654.65 -> your infrastructure code deployment systems.

1657.29 -> So one thing you gotta watch out for is regression.

1660.11 -> So let's say you go home, you check all your environment,

1662.54 -> you check your API calls, you check your clients.

1664.4 -> Everything's good; everything's supporting TLS 1.2 today,

1666.95 -> but somewhere there's a little piece of automation

1669.23 -> that's gonna deploy an old client with an old image

1672.53 -> in a certain circumstance.

1673.795 -> Something that if something goes down and it replaces it.

1676.01 -> So you wanna go look through that and just make sure

1677.72 -> you don't have anything that might accidentally

1679.97 -> deploy something that has been patched in place

1682.82 -> and reconfigured over time that you've forgotten about.

1686.99 -> So that's the plan to find out if you're impacted.

1690.53 -> If you are impacted, the most important thing

1693.05 -> you've gotta take away from this

1693.883 -> is you've gotta take some action.

1695.09 -> You can't just wait.

1695.93 -> You've got 11 months.

1697.07 -> It's a lot of time,

1697.903 -> but you don't wanna be waiting till June next year,

1701.09 -> to the day that we announce that we're cutting this off

1703.13 -> to take action because this can take some time.

1704.96 -> So, again, three basic steps you need to do here.

1708.11 -> Update the clients.

1709.28 -> The clients are making the calls.

1710.57 -> That's what has to change.

1712.79 -> You may need to update underlying TLS libraries.

1716.06 -> Now, if you're using any kind of package management tool

1719.18 -> when you do your update,

1720.26 -> that will usually happen automatically.

1722 -> If you are not using package management,

1723.8 -> if you're doing this manually,

1725.03 -> if you're hand building your systems,

1727.04 -> then you may have to do this manually as well.

1730.04 -> And check configurations and code.

1733.97 -> So I mentioned the automation codes,

1735.74 -> but also the actual configuration of your applications,

1738.53 -> because they may be making a modification

1740.54 -> to how your TLS library actually works.

1742.67 -> And I'll show you an example of that in just a minute,

1745.79 -> 'cause that is something which is awkward.

1748.73 -> But the one other thing I wanna tell you

1750.56 -> about this is not technical,

1751.97 -> is you have to have conversations.

1753.95 -> We found this last time.

1755.87 -> TLS is one of those plumbing things that everybody

1757.91 -> takes for granted and it impacts potentially

1759.98 -> lots of different teams.

1760.85 -> So the security people, if you're security people here,

1763.25 -> you really care that you have the right version of TLS,

1765.38 -> but the actual people deploying the code that uses the API,

1768.53 -> that uses that TLS, maybe a development team,

1770.36 -> it may be the operations team,

1771.68 -> you may have to talk to product teams

1773.12 -> if it means impacting your applications out in the world.

1775.7 -> So, again, do this early,

1778.76 -> do it often, start the process now.

1780.89 -> Get everybody who is a stakeholder onboard

1783.86 -> with this project.

1784.693 -> So everybody knows exactly what's happening

1786.11 -> and when it's gonna happen and what needs to happen

1788.39 -> or who needs to do it.

1790.91 -> Obviously at scale, you don't wanna be doing this stuff

1794.18 -> by hand if you don't have to,

1795.44 -> so use appropriate tools to do this.

1798.08 -> If you are doing immutable infrastructure,

1800.66 -> and if you know what that means,

1801.8 -> then you should understand CI/CD pipelines,

1803.87 -> and you should be pushing the changes out that way.

1806 -> However, if you've got a more traditional infrastructure

1808.16 -> where you're patching in place and you keep your systems

1810.74 -> alive for a long time,

1811.79 -> then you can use tools like Systems Manager.

1813.217 -> That's a great tool that we have.

1815.18 -> Systems Manager, if you're not aware of it,

1816.68 -> is not just for AWS Compute resources.

1818.99 -> You can use it on-premise;

1820.4 -> just install our free open source agent

1822.8 -> on supported operating systems

1823.97 -> and you can do remote run commands,

1825.38 -> remote push, remote inventory.

1827.36 -> You can see the exact version of every executable

1829.76 -> and library on every one of those Compute resources

1832.31 -> in one place.

1833.3 -> You can even aggregate it as a great tool

1836 -> for doing this kind of project at scale.

1839.51 -> And I'm gonna show you now,

1840.5 -> I mentioned that you can have specific TLS configuration.

1843.86 -> So this is kind of an awkward thing.

1845.87 -> If you're not a developer,

1846.95 -> you never thought about this before.

1848.24 -> We've talked a lot about the client,

1850.28 -> but you can override the settings of the client.

1852.38 -> So this is an example in Java.

1853.94 -> So let's say you've got a completely up to date version

1856.34 -> of Java, the most recent builder Java.

1858.2 -> You've got the most recent SDK

1859.547 -> for your application environment,

1861.71 -> and a developer goes in and does this,

1864.02 -> they set this system property.

1865.73 -> This will hard code every https connection

1869.84 -> to use only TLS v1.0, remember it says TLS v1,

1873.47 -> that means 1.0 or v1.1.

1875.96 -> It will never be able to do 1.2 or 1.3

1878.333 -> with this configuration,

1880.07 -> even though your system supports it.

1881.93 -> So, unfortunately there are not just multiple languages,

1887.36 -> in most programming languages I've looked at,

1889.16 -> they can have two or three different ways

1890.51 -> to achieve this same thing, unfortunately.

1892.31 -> So this is something where I said, you gotta talk to people,

1894.71 -> you gotta talk to your developers.

1895.97 -> You gotta have 'em search their code bases

1897.77 -> and make sure you don't have configurations like this.

1899.6 -> And if they do, those configurations need to change.

1902.81 -> Most typically again, if you've got an old environment,

1905.24 -> an environment that's like more than six, seven-years-old,

1908.54 -> somebody in the past may have put a configuration in

1910.73 -> because they thought, oh, there was a compatibility issue

1912.92 -> with TLS 1.2 and they wanted to make sure

1914.66 -> they weren't affected by that.

1915.89 -> That is not the case anymore.

1918.5 -> You should be able to eliminate anything like that.

1922.49 -> All right, so I'm gonna talk about a couple

1924.08 -> of advanced scenarios that you may come across.

1926.51 -> And it's also a great way to kind of crystallize

1929.78 -> exactly what we're doing here technically,

1931.19 -> and then I'm gonna show you a quick demo of troubleshooting

1933.8 -> this on a real world system.

1936.53 -> So, I want to emphasize the difference

1939.11 -> between a direct and an indirect API call,

1942.35 -> so you understand exactly what we're talking about.

1943.91 -> So, in this scenario here, this architecture,

1946.58 -> you may have built an application.

1947.84 -> So, a mobile application, an IoT device,

1950 -> a smart device that's out in the world.

1951.59 -> And what you can do if you choose to

1953.36 -> is you could have that device get some credentials

1956.06 -> and make an API call to an S3 bucket and say,

1959.39 -> push a bunch of log data once a day,

1961.61 -> once every couple of hours, whatever your application is.

1963.68 -> In this first scenario here,

1965.81 -> the API client is that mobile device,

1968.93 -> that IoT device, that smart device.

1970.94 -> So that device has to support TLS 1.2

1974.72 -> once we complete the deprecation project

1976.85 -> or these connections will start to fail.

1979.19 -> So, that is very important.

1980.99 -> If you are doing this kind of thing,

1982.13 -> if you built this kind of application,

1983.96 -> you need to make sure that you know

1985.25 -> that those mobile devices have that TLS. 1.2 support.

1988.94 -> If those mobile devices have one of our SDKs on there,

1992.18 -> you can go and make sure it's up to date,

1993.35 -> but you have to make sure the underlying TLS library

1995.72 -> does support 1.2.

1997.4 -> Now there's a second kind of architecture

1999.56 -> where you've built a custom API.

2001.3 -> So, you've done this, you've still got those same

2003.16 -> mobile applications, devices out in the field,

2005.2 -> but now you've gone and you've built yourself a front end

2008.53 -> using one of our tools like an application load balancer.

2011.65 -> So, as Janelle mentioned, we are not changing that.

2014.29 -> Application load balancers,

2015.49 -> CloudFront distributions, API gateways,

2018.04 -> those services have user configurable TLS settings.

2021.04 -> If you've never noticed it,

2021.94 -> you may just be using the defaults,

2023.44 -> but you can go in there and choose from a set of predefined

2026.98 -> TLS configurations with the version of TLS 1.0 supported,

2029.95 -> if you want, and the cipher suites,

2032.29 -> you can choose from a list of cipher suites.

2034.36 -> So if you are using this right now,

2036.73 -> and then you pass that connection through

2038.95 -> to a backend EC2 instance,

2040.72 -> and then the EC2 instance then stores that data

2043.27 -> in the S3 bucket.

2044.29 -> Maybe it does a bit of manipulation on the way,

2045.82 -> it doesn't really matter.

2046.653 -> In this scenario for TLS, your mobile clients,

2050.02 -> they're completely under your control.

2051.52 -> They are not gonna be subject to the deprecation

2053.71 -> because the connection they make

2055.2 -> to that application load balancer

2057.04 -> is also under your control.

2058.51 -> We are not touching that.

2059.53 -> So if you're doing this right now, today,

2061.24 -> this will continue working exactly as it does.

2063.82 -> The ALB will typically make another TLS connection

2066.49 -> to the backend to the EC2 instance.

2068.92 -> And then that EC2 instance will have some code on it.

2071.56 -> And that is the only thing that will be subject

2075.1 -> to this deprecation project.

2076.66 -> So only the code running on that EC2 instance

2078.94 -> that's making the API call to the S3 bucket,

2081.46 -> you'll need to check supports 1.2.

2084.7 -> Now, the other reason I'm showing this architecture

2087.25 -> is this second architecture is a potential fix.

2091.12 -> If you have that first architecture

2093.04 -> and you know you've got a bunch of smart devices

2094.87 -> out in the field for your product

2096.25 -> and they cannot be updated to support 1.2,

2098.83 -> the underlying libraries, the underlying firmware,

2101.41 -> even the underlying hardware can't support it,

2104.164 -> but you can at least push a software update to them.

2108.19 -> You could potentially migrate from scenario one

2110.68 -> to scenario two.

2111.61 -> You could stand up your own custom API gateway,

2113.89 -> change the software to point to that gateway,

2115.9 -> and then make sure that the backend of that environment

2120.13 -> is supporting 1.2 and you'll be able to fix that.

2124.57 -> So, two more scenarios I'm gonna show you here.

2127.055 -> This is for cross-account access.

2129.64 -> So we mentioned early on that unauthenticated calls

2134.74 -> can be used, but this scenario here,

2136.48 -> this is an authenticated call you may be doing right now.

2139.15 -> So the scenario here is you have some secrets

2141.94 -> in an Account B, you're the account owner of Account B,

2144.64 -> and you're sharing them with Account A,

2146.29 -> and there's two ways to do this.

2147.55 -> One, is through a resource policy

2149.47 -> for services that support that.

2150.64 -> So that's what I've got in this scenario here.

2152.41 -> I've created a policy for my secrets and I've said,

2154.81 -> allow Account A to call that.

2156.04 -> So the API client in this case is Account A.

2159.94 -> That is who is actually making the call to Secrets Manager.

2162.64 -> And when it makes that call,

2163.93 -> it will be recorded in CloudTrail of Account A,

2166.75 -> and if you've never gone looking for this,

2168.69 -> if you've never done cross-account sharing,

2170.56 -> there's actually two entries in CloudTrail

2171.73 -> you need to care about: an Account ID

2173.98 -> and what's called a Recipient Account ID.

2177.13 -> So when you make that call, if Account A's client

2180.82 -> is using TLS 1.0, then it's gonna be subject to deprecation.

2184.3 -> That could potentially fail.

2185.59 -> Now Account B will see the Secrets Manager API call as well.

2189.25 -> The Recipient Account ID will change.

2191.17 -> So the important thing here,

2192.79 -> the risk is if Account A is using 1.0 after deprecation,

2196.72 -> these calls will start to fail.

2198.61 -> Now, Account B, you're the owner of Account B,

2201.04 -> you cannot fix that.

2202.87 -> You cannot fix it for Account A.

2204.67 -> You can just know it's happened.

2205.93 -> You might get the notification, you'll see it in CloudTrail,

2208.03 -> but you cannot reach out and change Account A's client.

2211.12 -> So you are gonna need to talk to the Account A's owner.

2214.69 -> If you shared a resource with them,

2216.49 -> you must have some sort of communication, hopefully.

2218.68 -> And you're gonna say, "Hey, we are detecting,

2220.33 -> we've been notified by Amazon that you are using TLS 1.0

2223.02 -> in your client when you get to Secret,

2225.64 -> and you need to fix that or it's gonna stop working.

2229.9 -> So, a second scenario, this is the exact same risk,

2233.47 -> the exact same issue.

2234.46 -> The only difference is,

2235.39 -> and I'm just including this for completeness,

2237.25 -> is we're gonna use an IAM role instead of a resource share.

2240.7 -> So this is a very flexible system.

2242.86 -> This is generally considered the best practices

2245.62 -> to get a temporary STS token

2247.33 -> by sharing the role with Account A,

2249.28 -> and then the role has the policy that allows access

2251.41 -> to the Secret.

2252.243 -> The only thing that's different in this scenario

2253.99 -> is there's an extra API call.

2256.75 -> So you have to do the AssumeRole call first,

2258.94 -> and then that will show up in both accounts.

2260.83 -> And the Secrets Manager API call itself

2263.47 -> will now only show up in Account B.

2266.23 -> So, again, for deprecation,

2268.06 -> there's that risk that if the client in Account A

2270.79 -> does not support 1.2, that this is going to fail,

2274.45 -> but there's also the second call,

2276.12 -> it has to make that IAM call.

2277.3 -> So both the call to IAM and the call to Secrets Manager

2281.35 -> have to support 1.2.

2283.51 -> If they don't, they'll both fail and just be aware

2287.62 -> that it's possible in some complicated environments

2289.45 -> it'll be two different clients.

2290.89 -> One client gets the STS token,

2293.11 -> and then another client actually interacts

2294.58 -> with a Secret Manager, maybe a totally different SDK

2296.92 -> or a different programming language.

2298 -> So there are two different calls you have to watch out for

2300.94 -> in that environment.

2303.46 -> All right.

2305.11 -> So, now I'm gonna do some demo

2307.57 -> and I'm going to show you how this works.

2312.19 -> When my laptop wakes up.

2322.93 -> Okay.

2326.29 -> Switch my input and hopefully everybody is seeing

2329.38 -> the browser screen.

2330.58 -> All right, so...

2331.99 -> Nope, go away.

2337.72 -> I just clicked off the console.

2338.68 -> Okay, so in this account I've got set up here,

2340.78 -> I'm starting off in my Health Dashboard.

2342.31 -> So this is the first step.

2344.95 -> I got one of our health notifications.

2347.41 -> Let me scroll down.

2348.76 -> So this is a notification that you get,

2350.95 -> it's marked "Security TLS deprecation notification."

2355.12 -> It has a bunch of information and a link to our blog there.

2359.08 -> And then, because this is a real call that I made.

2362.77 -> You can go to this affected resources section.

2364.84 -> Now in the affected resources section,

2366.97 -> right now, what we're able to do is send you

2368.843 -> a set of simplified data.

2371.68 -> This is basically the data that you'll see in CloudTrail,

2374.53 -> but it's simplified a bit

2375.43 -> because we have to do this at scale.

2377.05 -> So the most important information we're getting out of this

2379.9 -> is firstly, it's in the us-east-1 region.

2382.12 -> And I can see from the second data entry

2384.19 -> that this is the EC2 service.

2386.23 -> And then the third data entry is telling me

2387.97 -> the actual API call that I made.

2389.89 -> It was describe instances.

2392.2 -> And it's telling me it's TLS v1, which means TLS v1.0.

2396.31 -> It tells me I did it one time in the times detected.

2399.1 -> And then this last part, this is the user agent string.

2402.97 -> Now I mentioned the user agent strings are quite variable.

2405.4 -> This one is about short as you can get.

2407.47 -> If anybody here is a Python developer

2409.12 -> that may be looking at this and say,

2410.207 -> "Ah, Python-request, that's the request library,

2413.95 -> a common Python library that's used

2415.57 -> to make https connections.

2418.9 -> So, it says it's Python-requests/2.28.1.

2425.53 -> That does not mean it's Python version 2.28,

2428.47 -> which will be pretty, pretty old.

2430.03 -> It means it's actually the version of that specific library.

2432.61 -> So that's my starting point.

2433.81 -> I got one of these notifications.

2435.91 -> So I know that I'm looking for describe instance calls

2438.67 -> in EC2 in us-east-1.

2440.56 -> So now I'm gonna go into CloudTrail.

2442.63 -> I'm gonna filter this down.

2443.98 -> I'm gonna see if I can figure out maybe a bit more details

2447.04 -> about where this is coming from.

2451.57 -> Oh, and I must've type that wrong.

2459.37 -> All right, so I've got a bunch of describe instances calls.

2462.73 -> They are all coming from the username,

2464.65 -> that's an instance ID.

2465.64 -> So that tells me I've got an EC2 instance

2467.83 -> that's making these calls.

2469.69 -> It presumably has a role, a credential on it

2471.28 -> that allows it to make the calls.

2472.81 -> And one thing I want you to note,

2475.09 -> you're doing some analysis and you look at that,

2476.68 -> if you can see every one of those calls

2478.6 -> is exactly five minutes apart.

2480.4 -> So that's a little hint to me that this is probably

2482.41 -> some sort of automated process.

2485.62 -> So I can drill into one of these.

2487.21 -> I can pull this up.

2488.08 -> Obviously all the normal CloudTrail data is here.

2490.42 -> I can see the fact that this is coming from a role.

2493.75 -> It's assigned to an instance,

2494.8 -> so it's not an individual user most likely.

2497.59 -> And I can keep scrolling down.

2498.7 -> And down here in the TLS details section I can confirm,

2501.37 -> yes, this is TLS v1 that is being used to make these calls.

2507.25 -> So I went ahead and set up a CloudTrail Lake.

2511.21 -> One thing I wanna note about CloudTrail Lake,

2513.34 -> it's an awesome feature,

2514.69 -> but you have to turn it on before you need it.

2517.39 -> So when you turn it on, it will not go through and analyze

2520.18 -> your previous CloudTrail data.

2521.62 -> So don't wait until you get that PHD notice from us.

2524.56 -> If you want to use this feature, if you turn it on now,

2527.23 -> you'll be able to start searching data straight away.

2529.99 -> So I've got a query here that was set up.

2532.78 -> Hopefully that's legible.

2533.65 -> This is very similar to what you see in our blog.

2535.9 -> It has a date range in it.

2537.07 -> So because you might have a lot of data,

2538.78 -> you don't wanna necessarily see a ton of that.

2541.27 -> You can run that.

2542.103 -> It usually runs pretty quickly.

2544.3 -> And let me scroll down to the results.

2547.15 -> And here, so I can quickly do this.

2548.83 -> I could run this, there's an API for this.

2550.63 -> I could dump this out if necessary.

2553.09 -> I can see all of those calls.

2554.5 -> Obviously the columns here I've chosen in SQL.

2556.45 -> So you can put every single piece of data

2558.28 -> in there if you wanted or just keep it very simple

2560.38 -> to just show me all the TSL v1 calls

2562.75 -> and the name of the call.

2563.92 -> So, that's data lake.

2566.14 -> The other thing that I did is I configured

2568.42 -> the CloudWatch log integration for CloudTrail.

2572.89 -> So you can see here, the CloudTrail trail itself,

2577.75 -> that's my log integration.

2580.12 -> And if I go into that, I set up some metric filters.

2584.32 -> So this is a very useful way if you're searching

2586.51 -> for this at scale.

2587.65 -> So the metric filter there, I hope it's legible,

2590.2 -> is this filter pattern which is

2591.536 -> $.TLSdetails.TLSversion="TSLv1".

2596.92 -> So that will show me any v1.0 call.

2599.59 -> I'm not using a wildcard.

2600.82 -> I'm specifically using quotes there

2602.47 -> because I would need to match it exactly.

2605.47 -> By the way, we haven't mentioned this, I should mention it.

2607.87 -> TLS v1.0 is the most likely

2610 -> type of deprecated call you're gonna see.

2612.07 -> TLS v1.1 was never very popular because by the time

2616.09 -> it came out, a couple years later, v1.2 came out.

2619.15 -> Almost every client that can do 1.1, could do 1.2.

2622.24 -> So you'll very rarely see 1.1 traffic,

2624.82 -> except we learned last time we did this

2627.19 -> for monitoring agents.

2629.05 -> So we found last time,

2630.1 -> there was a class of like security agents

2632.65 -> that are scanning and just seeing what versions of TLS

2634.63 -> could I connect to in this environment

2636.79 -> and creating compliance reports.

2638.62 -> So every single actual 1.1 call we ever saw last time

2642.79 -> was just coming from a scanning agent.

2644.59 -> So, but for completeness, you do need to check that.

2648.52 -> So once you have this configured,

2650.74 -> you can create a metric and then, it's CloudWatch.

2653.595 -> So you can go in and look at the metric.

2655.48 -> You can look at the data over time.

2657.19 -> You can create alarms on this.

2660.397 -> Oh, look at that 12 hour period.

2661.81 -> Yep

2666.76 -> Turn that on.

2668.09 -> There we go.

2668.98 -> So these are all the TLS 1.0 calls

2671.95 -> that we've made in my environment

2673.12 -> in that last 12 hour period.

2674.8 -> And if you're familiar with CloudWatch,

2676.21 -> you know how to use this at scale; you can create an alarm.

2678.85 -> And then when it triggers above the threshold,

2680.41 -> you can have it send you a notification.

2682.27 -> You can have it send you a page, a Slack message.

2684.43 -> One piece of advice I would give everybody

2686.62 -> is don't do active notifications right off the bat.

2689.53 -> If you're an operations person, you're thinking, oh great,

2691.3 -> I'm gonna go home today and turn this on,

2692.32 -> and I'll start paging people.

2694.24 -> I wouldn't recommend it until you've at least

2696.85 -> looked at the data, because what you don't want to do,

2699.13 -> you could have thousands of TSL 1.0 calls

2701.32 -> happening right now,

2702.4 -> and you don't wanna start sending out thousands of pages

2704.53 -> on each and every one.

2705.91 -> That would generally be unpleasant.

2708.07 -> So, do a first pass, get an idea of the impacts

2712.39 -> you have right now.

2713.53 -> And the other great reason,

2715.27 -> once you've done that for setting up something like alarming

2717.34 -> is to watch out for regression.

2719.47 -> So I mentioned this earlier,

2720.88 -> you can go home now, you can check this.

2723.22 -> You can say, okay, we're 100% TLS 1.2 in our environment.

2726.13 -> Great.

2726.963 -> This project is running for the next 11 months.

2728.8 -> The cutoff date is June of 2023.

2731.05 -> So at any point in that 11 months,

2733.03 -> if somebody in your account or your AWS organization

2736.54 -> were to spin up some resource, some old resource,

2738.91 -> find some old image that for whatever reason

2740.65 -> they need to run that was using TLS 1.0.

2742.75 -> That could happen tomorrow, it could happen next month.

2744.37 -> It could happen six months time.

2745.57 -> So you do need to monitor for this over time.

2752.05 -> Another cool thing that you can do

2753.22 -> is the contributor insights.

2754.93 -> So you can actually set up a rule like this,

2757.21 -> where you actually, if you just wanna see

2758.56 -> all of the TLS calls and make sure that most of 'em are 1.2,

2761.5 -> I did this earlier.

2762.4 -> So you can see the vast majority are 1.2,

2764.02 -> but I do have some TLS 1.0 in there.

2766.39 -> That's under contributor insights here.

2768.25 -> And that's the same basic concept,

2770.71 -> I've just put in a filter for all versions of TLS.

2774.43 -> All right, so I know I've got a problem in my environment.

2776.683 -> I know I've got TLS 1.0 calls coming from this instance.

2779.44 -> So I'm gonna go ahead and remote into that instance

2782.17 -> and have a look and see what might be doing that.

2784.3 -> See if I can fix it.

2785.68 -> So I am going to use the Fleet Manager,

2788.59 -> a feature assistance manager to remote desktop in.

2791.14 -> This is nothing to do with TLS,

2792.4 -> but it's one of my favorite tools,

2795.19 -> because it means you don't have to stand up

2796.66 -> any kind of bastion host or remote desktop gateways.

2799.48 -> If you want get a nice green environment,

2801.91 -> just have to give it a key.

2803.17 -> Get this configured, same key you'd use

2805 -> for EC2 connections.

2807.73 -> And I can go right in here and get a nice desktop.

2814.99 -> Okay, so let's go ahead and make that full screen.

2817.39 -> All right, so this is a Windows desktop.

2819.1 -> This is I believe a 2019 build that's entirely up to date,

2822.22 -> but somehow I'm still making TLS 1.0 calls out of it.

2825.37 -> So I am gonna show you Wireshark here.

2828.76 -> If you have never seen Wireshark before,

2830.44 -> or if you've heard of things like network packet capture,

2832.36 -> and it terrifies you, it did me.

2834.49 -> I'll be honest when I first started

2836.32 -> doing this kind of engineering,

2837.25 -> but it is a very, very useful tool.

2839.29 -> if you ever have to troubleshoot TLS.

2840.88 -> Because it enables you to see what's really going on

2843.52 -> at the packet level.

2844.72 -> Now, when you first use a tool like this,

2846.94 -> and you just say, "Start packet capture.

2848.62 -> you get a bunch of data will start scrolling up the screen,

2850.78 -> and it looks very confusing.

2851.98 -> The good news is you don't have to look

2853.6 -> at all of that stuff.

2854.89 -> What you're gonna wanna do is filter it.

2856.66 -> And for this project,

2857.68 -> there's really only one thing you wanna do.

2860.24 -> I've got a filter set up here.

2861.7 -> Oh, come on.

2863.68 -> The mouse won't cooperate.

2868.03 -> Go apply this filter.

2869.35 -> So this filter here is using our property

2871.66 -> of the packet itself for our benefits.

2873.16 -> So when you make a connection to

2874.72 -> a modern TLS server these days,

2877.36 -> it passes through something called a server name,

2879.13 -> which enables you to filter down, excuse me,

2881.29 -> enables a server to know exactly which domain you are trying

2884.26 -> to connect you when you made that call.

2885.79 -> So I know that my calls are going to

2887.44 -> the EC2 service in us-east-1.

2889.51 -> So I've put in a filter that says,

2890.867 -> "tell me any packet where in the handshake

2894.04 -> it's making the server name contains EC2.us-east-1.

2897.79 -> Now, as soon as I turn that filter on,

2899.579 -> my traffic's disappeared.

2901.36 -> There's nothing going on right now.

2902.56 -> And I'm gonna sanity check that.

2904.48 -> I'm gonna bring up the command line here

2908.86 -> and I'm gonna run a CLI command.

2911.65 -> I'm gonna call, describe instances.

2912.94 -> And you'll see in just a second,

2914.44 -> there goes my outbound packet.

2915.91 -> So I've gotten rid of all the extraneous stuff.

2917.47 -> The only thing, once you start troubleshooting this,

2919.72 -> you generally care about is this first packet,

2921.49 -> the client hello.

2922.323 -> This is you making the outbound connection.

2924.22 -> If you made that outbound connection after deprecation

2927.13 -> with TLS 1.0, there will then be an error.

2929.38 -> There will be a connection reset that means you have that.

2931.24 -> But you would still see that outbound packet

2932.89 -> 'cause your computer would still be trying

2934.12 -> to make that connection.

2935.26 -> So here's the connection outbound.

2937.96 -> And Wireshark is one of my favorite tools for this

2939.82 -> because you can go down here.

2940.9 -> It's got a whole section here for transport layer security.

2943.78 -> You can drill into this and you can see every part

2945.88 -> of this that matters.

2948.07 -> You can see the protocol that's being used.

2950.86 -> You can see all the technical information.

2952.72 -> Very, very little of this is really gonna be necessary

2954.82 -> for this project.

2955.653 -> All you really care about is what version

2957.67 -> was used when I made that outbound call.

2959.92 -> So now what I have to do is track down what is doing that?

2962.32 -> 'Cause it's not my command line.

2964.96 -> So as we saw in our CloudTrail analysis,

2968.8 -> these calls will be made every five minutes.

2970.54 -> So it's a good bet that there's some process here

2973.12 -> that's running on a timer.

2974.29 -> So in Windows that will be Task Scheduler usually,

2976.21 -> on a Linux system it will often be Cron.

2978.16 -> So I'm gonna go in here and have a look and say,

2979.697 -> "Oh, what's this check EC2 instances?

2982 -> There's a program running here, demo.py on my desktop.

2985.42 -> I wonder if that might be the culprit.

2987.73 -> So let's have a look at demo.py.

2989.86 -> So this is a Python program.

2991.84 -> I'm not putting this up here to demonstrate

2993.55 -> my mad skills as a Python programmer

2995.29 -> 'cause they do not exist.

2996.79 -> Every time I get stuck, I have to ask my son,

2998.74 -> but this is a piece of sample code I stole from our website.

3001.56 -> This is how to write an API call to an AWS service

3004.53 -> in raw Python without using our Boto3 library,

3007.47 -> which is frankly much easier.

3008.7 -> The reason I'm using this is because I'm using

3011.52 -> just the basic Python,

3013.05 -> I added in this little piece of code here,

3016.41 -> this TLS adapter code.

3018.96 -> So this code is not standard.

3021.18 -> This code enabled me to manipulate the version of TLS

3023.853 -> that this Python client is using.

3025.83 -> Now, hopefully, if you use default configurations,

3029.4 -> you wouldn't usually to do this, but it does happen.

3031.74 -> People go in and hard code stuff.

3033.45 -> Maybe somebody had a compatibility issue once in the past,

3035.76 -> maybe somebody got told,

3036.99 -> oh you need to be using TLS 1.2 ,

3039.07 -> and they thought TLS 1.0 was any version of TLS.

3042 -> So this line here where it says SSLprotocol_TLSv1,

3048.15 -> that is the problem.

3049.23 -> That is telling this client to only use 1.0

3052.17 -> because it just says v1.

3054.06 -> So to sanity check that,

3055.68 -> I'm gonna go back to the Command Line.

3057.15 -> Let me get the Wireshark back up.

3060.78 -> Go back here to the Command line.

3062.25 -> So what I'm gonna do...

3064.56 -> No, now, hold on a second now.

3067.17 -> I'll send a control, get rid of the code.

3071.213 -> Okay, so I want Wireshark.

3074.67 -> Go.

3076.126 -> Go to the command prompt.

3079.17 -> Close out of that.

3081.3 -> Okay, so I wanna run that Python code directly here

3084.54 -> and it's doing the same thing,

3085.74 -> it's doing describe instances,

3086.88 -> but now you can see, as soon as I hit that, run that code,

3090.6 -> you can see another TLS 1.0 packet came up there.

3092.97 -> So we know this is the right culprit.

3094.8 -> So the good news is in this particular case,

3097.11 -> this is super easy to fix because the version of Python

3100.56 -> on my laptop is up to date.

3101.91 -> I know it supports 1.2.

3103.38 -> I know I can do it from the operating system level.

3105.39 -> The CLI is working.

3107.04 -> For this particular programming language,

3108.45 -> the syntax is 1_2, not 1.2.

3112.32 -> That's just the syntax it's using.

3113.7 -> Every programming language is gonna be different.

3116.4 -> So obviously you gotta find the right.

3118.02 -> This is, again, if you're a security engineer,

3120.36 -> you don't know Python, you gotta talk to your developers.

3122.19 -> You can't necessarily do this yourself.

3124.92 -> So I'm just gonna hit Save on that.

3129.69 -> Get the packet capture back up.

3131.82 -> Go back here, run that Python again.

3133.95 -> And now we should see it making that call.

3135.72 -> And now it's using 1.2.

3137.52 -> So that's a very simple example, but this can happen.

3141.24 -> Obviously I went and did this manually directly on the box.

3144.18 -> If you're doing this at scale,

3145.8 -> hopefully that Python code will be checked

3147.57 -> into a code repository and you can search that,

3149.79 -> find it, update it, test it and push that out to your fleet.

3155.07 -> All right.

3156.69 -> So, let us switch back.

3161.04 -> All right.

3161.88 -> So let's summarize everything we've seen.

3165.06 -> So, firstly, we've got a lot of resources available to you.

3167.46 -> Obviously you've come here to this presentation today.

3169.77 -> So if you want to read the blog

3171.75 -> about the TLS deprecation project itself,

3174.33 -> we've published this last month.

3175.38 -> If you've never read it,

3176.31 -> we've got a bunch of advice there.

3178.29 -> We've got a blog on using CloudTrail Lake,

3180.24 -> that search that showed you that a couple of my colleagues

3183.15 -> put together and we have a blog.

3184.62 -> If you've never had to do this, if you've never thought,

3186.57 -> how is SSL TLS configured in my operating system,

3190.38 -> if you're using an Amazon Linux build,

3191.97 -> we've got a blog about configuring that specifically there.

3196.08 -> All right, so what do you need to take away from this?

3198.54 -> So, the good news is everybody, every customer is gonna

3204.15 -> get this modern TLS configuration automatically.

3207.15 -> We're gonna push this change out on the backend of AWS.

3209.67 -> You're all gonna benefit from it.

3211.23 -> So that's the really good news here.

3214.5 -> You do have to watch out for these notifications.

3217.14 -> So we are gonna be sending these notifications.

3219.3 -> We're gonna be sending them out over this 11 month period.

3221.04 -> As I mentioned, regression can be an issue.

3223.23 -> So if you've got no notifications today, great,

3226.32 -> but you need to be monitoring them.

3227.55 -> You should be doing this anyway.

3229.17 -> Operational best practice,

3230.31 -> regardless of the TLS deprecation,

3231.96 -> you should be monitoring your Personal Health Dashboard

3234.03 -> notifications and taking action where necessary.

3238.56 -> And if you are impacted, you must take action.

3242.01 -> You must talk to the relevant stakeholders.

3244.02 -> This is not unfortunately often

3245.58 -> a one size fits all solution; there's gonna be code changes.

3249.06 -> There might be platform changes.

3250.35 -> There might be infrastructure changes.

3251.85 -> There might be product changes.

3253.11 -> There might even be hardware changes needed

3255.12 -> if you are impacted.

3256.56 -> So talk to the right people.

3258.39 -> Start that conversation and talk to us.

3261.03 -> If you are an AWS customer and you have a support contract,

3264.03 -> open a support case, if you need to,

3265.86 -> and we'll get the right technical people

3267.27 -> engaged on our side.

3269.813 -> If you even have a support contract,

3271.32 -> you can use our repost forum.

3273.39 -> If you have enterprise support,

3274.53 -> you can talk to your TAMs and your account teams.

3276.33 -> We love to talk about this and myself and Janelle,

3279.48 -> we'll be more than happy to continue this conversation

3281.7 -> afterwards, just come and see us over here

3283.89 -> on the side of the stage.

3286.8 -> And that's everything we have for you today.

3288.9 -> You can also contact us on LinkedIn and do please complete

3292.5 -> your surveys today.

3293.333 -> We'd love to get your feedback.

3295.56 -> All right.

3296.79 -> Thank you, everybody.

3297.623 -> Thank you, Janelle. (audience applauding)

Source: https://www.youtube.com/watch?v=8szxWSqYSaY