System Design for Beginners Course

Aug 15, 2023

System Design for Beginners Course

This course is a detailed introduction to system design for software developers and engineers.

Building large-scale distributed systems like Google, Facebook, Amazon, and Twitter requires an in-depth understanding of computer science principles. This allows systems to handle millions of users concurrently despite hardware failures.

We discuss the fundamental concepts of system design in this course like requirement selection, API design, Database Design, Network protocols, Fault-tolerance, design trade-offs, solution tradeoffs, and low-level design.

Gaurav Sen created this course. Check out his channel: @gkcs

You can learn about distributed systems and system design using the following resources:
Designing Data-Intensive Applications by Martin Kleppmann: https://amzn.to/3SyNAOy
System Design Simplified: https://interviewready.io

Let us know your thoughts and suggestions in the comments!

⭐️ Contents ⭐️
⌨️ (0:00:00) What is System Design
⌨️ (0:02:27) Design Patterns
⌨️ (0:04:07) Live Streaming System Design
⌨️ (0:07:42) Fault Tolerance
⌨️ (0:08:32) Extensibility
⌨️ (0:09:49) Testing
⌨️ (0:10:32) Summarizing the requirements
⌨️ (0:11:27) Core requirement - Streaming video
⌨️ (0:14:52) Diagramming the approaches
⌨️ (0:17:12) API Design
⌨️ (0:20:02) Database Design
⌨️ (0:22:32) Network Protocols
⌨️ (0:29:07) Choosing a Datastore
⌨️ (0:32:57) Uploading Raw Video Footage
⌨️ (0:34:37) Map Reduce for Video Transformation
⌨️ (0:39:24) WebRTC vs. MPEG DASH vs. HLS
⌨️ (0:41:55) Content Delivery Networks
⌨️ (0:42:27) High-Level Summary
⌨️ (0:44:48) Introduction to Low-Level Design
⌨️ (0:47:00) Video Player Design
⌨️ (0:49:17) Engineering requirements
⌨️ (0:50:32) Use case UML diagram
⌨️ (1:02:32) Class UML Diagram
⌨️ (1:06:27) Sequence UML Diagram
⌨️ (1:11:49) Coding the Server
⌨️ (1:24:23) Resources for System Design

🎉 Thanks to our Champion and Sponsor supporters:
👾 Nattira Maneerat
👾 Heather Wcislo
👾 Serhiy Kalinets
👾 Erdeniz Unvan
👾 Justin Hual
👾 Agustín Kussrow
👾 Otis Morgan

—

Learn to code for free and get a developer job: https://www.freecodecamp.org

Read hundreds of articles on programming: https://freecodecamp.org/news

Content

0.001 -> This course is a detailed introduction to system design for software developers and engineers. Gaurav Sen developed this course.

8.528 -> He is an experienced software engineer and he also has a popular YouTube channel.

13.225 -> You will learn about basic engineering design patterns that are used to build large-scale distributed systems.

19.544 -> In the second part of the course you will learn how to use the principles from the first part

23.976 -> to design and code a live streaming video app.

27.15 -> Hello everyone.

27.81 -> Welcome to the System Design course.

30.12 -> This course is for beginners in a sense that if you have never done

33.27 -> system design before or you have done very little system design or you have

37.11 -> just heard of it or read about it, but not got an opportunity to actually

40.62 -> do it, this course is right for you.

42.78 -> At the end of this course, you'll be able to identify some basic engineering

46.17 -> design patterns, which are used to design large scale distributed systems.

50.85 -> Let me define each of these terms in detail.

54.3 -> Large scale distributed systems, as the first part of the term, says

59.43 -> large scale, meaning that something which is being used a lot or is very

64.65 -> intensive in terms of compute or data or any, computer engineering principle.

70.68 -> For example Google, let's say Google Maps.

74.49 -> This is large scale because A: it has a lot of data.

78.51 -> The whole world's map has to be stored, incited.

81.22 -> B: it's being used by a lot of people.

83.83 -> C: it's being updated very frequently, and b, it has a lot of

88.84 -> performance expectations from it.

90.58 -> You don't expect Google Maps to go down.

92.47 -> You expect Google Maps to return results quickly.

95.26 -> You expect it to be accurate, so on and so forth.

98.02 -> So it's a very good product being used at a large scale.

101.42 -> Distributed systems means that the server or the code that is actually executing

107.685 -> this program is not in one place.

109.9 -> It's disputed all around the world.

112 -> So you might have one server in India, one server in the us, one in

115.63 -> Japan for tolerance, so that if one of these servers crashes, the rest

120.04 -> of the servers can take the load.

121.085 -> It's also for performance.

124.51 -> If the Indians want a Google Map result, they go and talk to the Indian server

128.205 -> because that gives a quick result back.

130.21 -> While if they go to the US server, they have to go across continents and then

134.35 -> get a response, which is a slow result.

137.62 -> Okay?

137.74 -> So that's the point of a large scale distributed system.

140.08 -> Many of these companies which provide these solutions, Expect

143.62 -> their engineers to know about system design, to know how to build these

146.8 -> large scale distributed systems.

148.539 -> So to do this, their engineering team depends on design patterns,

152.204 -> which we mentioned earlier.

153.669 -> Design patterns are particular practices, principles, or processes which are used

159.015 -> by engineers to build these systems.

161.71 -> For example, you have a common problem of a celebrity posting on

167.29 -> social media and that post being distributed to a lot of their followers.

172.03 -> For example, if Brad Pitt posts something on YouTube and then post on

175.84 -> LinkedIn, the problem is very similar.

178.42 -> It's one piece of content which has to be made into an event and notified

183.52 -> to millions of people potentially.

185.38 -> And what you want to do is you want to notify them quickly, but you also don't

189.34 -> want to put too much load on your service so that the rest of the requests, which

192.285 -> are coming into it continue being served.

194.59 -> This is a common problem, so you extract out a common problem and you

199.42 -> solve it using a common solution.

201.965 -> , this would be a design pattern.

203.24 -> Right now, you don't know what design patterns exist, but one popular design

207.37 -> pattern for this kind of a problem is a publisher subscriber model where Brad

212.05 -> Pit is a publisher and their event is subscribed to by millions of people.

217.36 -> And Intermediately decides the pace at which you send these notifications, which

222.25 -> keeps the server loaded low, and also make sure that all the notifications

226.56 -> actually reach all the subscribers.

227.92 -> Now, seeing this, you can tell that as an engineer you can take a lot of business

231.19 -> requirements and convert them into technical solutions, and engineers use

234.73 -> system design patterns to make reliable, scalable, and maintainable systems.

239.68 -> This helps them convert business requirements into technical solutions.

242.92 -> If this sounds exciting to you, watch's the rest of the video

245.535 -> and you won't be disappointed.

255.52 -> Okay, let's start with an example.

257.65 -> Let's say that you join a company which does broadcasting of

261.58 -> videos to millions of people.

263.74 -> This could be Hot Star, it could be YouTube, it could be Zoom also.

268.48 -> So some sort of events which are being broadcast to millions of people.

273.26 -> And you as an engineer have to devise the solution for this, come up with some

278.65 -> technology which can handle this problem.

280.96 -> So the first thing that we need to do as engineers is to define these

284.29 -> requirements from the user's perspective.

288.25 -> And often engineers don't do this.

291.49 -> They're product managers and a company which write a product requirement

294.75 -> document based on user feedback and data.

298.87 -> So that's good to have.

300.55 -> You have a well documented, thought business banked document, which the

307.845 -> engineer can then read and decide on how they can make it happen.

311.77 -> Amongst these, you pick up the most important features first.

315.34 -> Taking the example of a live streaming system, users who tune into your system

319.06 -> should be able to watch the stream live.

321.76 -> Whether you see it at HD quality or you see it at four 80 p, that's probably

325.78 -> a secondary issue, but being able to see the video is a primary issue.

330.49 -> So for that, you have to make sure that your server doesn't go down, and

333.37 -> you have to also make sure that your bandwidth requirements are sufficient.

336.7 -> Second, you reduce these features to data definitions.

341.02 -> So one of the features which might be defined is users should be able to like a

345.765 -> video or should be able to like a comment.

348.52 -> What you then do is you look at the concept of a, like the abstract

352.565 -> concept of somebody liking something.

354.61 -> What does that mean?

355.39 -> It means that a particular user likes a particular comment.

360.07 -> Okay, so it has a like id, it has a user ID who's done this.

363.19 -> It has a timestamp.

364.18 -> When was this video liked or when was this comment liked?

367.06 -> The comment itself is an abstract concept.

369.1 -> It's probably going to be mapped into an object.

371.2 -> It has an id, it has a user who has typed this comment.

373.86 -> It has a creation time and maybe it also has a thread in which it was posted.

380.89 -> So you're seeing that I'm taking features or I'm taking these abstract concepts

384.825 -> from the product requirement document and converting them into data definitions,

388.89 -> which are used, useful for an engineer.

391.27 -> These definitions can then be mapped into objects, which can

394 -> then be mapped into the database.

396.76 -> Once you've defined the data that you need to store, you need to

399.94 -> define endpoints through which this data can be manipulated or queried.

405.01 -> So I want to read comments, give me an API and sgp api ftp API for all WeCare.

412.6 -> The network protocol at this point is not important, but some method

415.63 -> by which I can send an electronic signal from one place to our server,

419.59 -> and our server can respond to that signal with the data that they need.

423.07 -> So our server is now encapsulating this data as per the user

427.15 -> requirement and defining some endpoints, which are called APIs.

431.785 -> So that external users can query and manipulate our data.

434.665 -> We do this for every feature that we have, core features and also

439.585 -> optional or good to have features.

441.475 -> Usually a product requirement document does not define optional

444.235 -> or good to have features.

445.495 -> These are all core features which are required in the document.

449.185 -> The good to have features will be probably picked up in the next

451.525 -> document, so you don't need to think of which features are optional,

456.055 -> but you do think of which features are most important as an engineer.

460.315 -> The second thing is you also have certain engineering requirements.

464.725 -> When you're creating these designs or coming up some code, you want to make

469.195 -> sure that none of your services fail If there is an outage, so for example,

474.865 -> if you have an outage in India and your common service is in India, then you

479.215 -> don't want the entire system to collapse.

481.165 -> We talked about this a little earlier, that you have multiple

484.165 -> server spread across the world to avoid a single point of failure.

488.965 -> . You may also have multiple servers in India itself.

492.195 -> So if one Indian server fails, another one picks up its responsibility.

497.905 -> So here you might have some sort of data duplication because of

500.785 -> which the other server is quickly able to pick up the responsibility.

503.935 -> Or you have some sort of partitioning that 50% of the users are going

506.965 -> to this server, 50% of the users are going to another server.

510.355 -> So just 50% of the users are affected under the common and often missed

513.835 -> out engineering requirement is extensibility, which means it's not

517.585 -> just about the technical solution that you come up with, but also how

522.535 -> easy it is to change that solution.

525.325 -> For example, if you write code to send a message to millions of users,

530.965 -> and you need to make a small tweak to that code because now you don't want

534.145 -> to just send it to 1 million users.

535.825 -> You also want to check whether they have read those messages.

539.365 -> So read

539.784 -> receipt.

540.678 -> The problem

542.515 -> with writing code, which is highly coupled with the feature.

545.875 -> Is that whenever there's a changing requirement, you have to put in a

549.025 -> lot of effort to redesign, test, and actually deploy that code again.

555.145 -> So you want your features, your engineering features.

557.785 -> To be extensible for this, you have to take out your engineering

560.755 -> pistol ball and gaze deeply

562.584 -> into it.

563.305 -> Actually what

564.025 -> you do is, as an experienced engineer, especially, you look back at

567.505 -> your past projects you look

570.415 -> back at things that you have done, you look back at things that people

573.474 -> have done.

574.17 -> You know what

575.214 -> the world is doing.

576.265 -> And based on that combined knowledge, you build a system so that you can

582.145 -> reasonably expect it to scale and extend as requirements change and as

586.464 -> more and more users join you, finally this design needs to be tested.

590.094 -> This is an important part of system design, which is

592.375 -> not really thought through

595.314 -> in interviews, but in the real world, when you have a system

599.035 -> which is designed by, let's say a

600.474 -> senior engineer that design

602.454 -> has to be tested.

603.875 -> You run through

605.74 -> a couple of requests with edge

608.17 -> cases, with common cases, and whether these

612.82 -> requests are having a sensible flow in the system at a high level.

617.17 -> The other thing that you can do is you can use sophisticated tools to load test

621.28 -> this design.

622.072 -> You can have

623.26 -> some sort of capacity estimation to actually see whether this design

626.74 -> is feasible, but importantly, you have to test this design before

630.52 -> you start getting into the code.

632.98 -> Okay. Let's recap with an example.

635.47 -> Let's say we have a live streaming system.

637.63 -> How would we go about designing this?

640.18 -> Requirements would be streaming video, processing video, sending

645.4 -> video to multiple customers.

647.52 -> So that is broadcasting not failing

650.62 -> is a requirement.

651.94 -> Showing advertisements, allowing reactions, showing disclaimers

656.02 -> or news flashes, having graceful degradation of video quality

660.13 -> in case you have low bandwidth, allowing multiple device support.

663.895 -> and so on.

664.675 -> Amongst these, you'll see that the product requirements are

667.194 -> mainly showing video to a lot

668.484 -> of users.

669.528 -> That's a major

671.094 -> requirement.

671.454 -> And also, of course, comments is a thing and being able to react as a thing.

674.785 -> You may also consider product requirements to be showing a banner in case there's a

677.935 -> problem.

678.564 -> It is, but the core requirement,

682.344 -> of course, is just showing a video, so I'll pick this feature first.

685.435 -> This means that I have to capture video from a source, let's say, which

690.354 -> is shooting at eight K, and I should be able to store it in someplace

695.574 -> in my server so that I can query it later in a live streaming system.

700.854 -> The later part, the query it later is probably milliseconds, so I may

706.675 -> not want to query that data at all.

708.714 -> I may want to directly stream it from the video camera onto millions

713.694 -> of people taking a step back.

715.584 -> It looks impractical because if I'm shooting at eight k at very high

720.144 -> quality, and this

722.319 -> is raw

722.86 -> footage.

723.58 -> Sending that

725.02 -> much data to people on their mobile phones is unreasonable.

729.43 -> So yes, I need to store it in some sort of a database or a file system,

733.99 -> and then I should be able to stream that or query that out so that I can

738.189 -> distribute it to all of my customers.

740.41 -> But I don't want my customers to know exactly how I'm

743.685 -> doing this or I don't

744.88 -> want them to know about, there's a change in implementation tomorrow.

747.79 -> I want this to be a black box so that they can pay me

750.8 -> and relax.

751.8 -> They just

752.8 -> hit an API and their problem is solved.

755.17 -> Of course, I might give them clients which are download on the

758.77 -> app store which take

760.24 -> care of all of this API and querying also so that as end users, they can

764.38 -> just enjoy watching an event instead of thinking about how the technical

768.28 -> part of this is actually happening.

770.079 -> These APIs have well defined signatures.

772.24 -> You can tell that if I want

773.74 -> this video Id in a particular

776.229 -> format, then I have to query a particular API called Get video.

780.31 -> , which is going to be returning me objects of

782.83 -> type frames.

783.715 -> And those

784.6 -> frames are also well defined.

786.46 -> So if you have ever written a program, you know that these API signatures

790.74 -> are very similar to method signatures.

793.15 -> The only difference being that these APIs might be queried not through a programming

796.935 -> language, but through a network protocol like G R P C, htp, ftp, any kind of

804.02 -> protocol which defines exactly how an electronic message is going to be taken

807.705 -> from one place and sent to another.

809.86 -> And also how the response is going to come back and how the

811.875 -> behavior of this interaction is

813.79 -> going to be that is defined

815.205 -> by the protocol.

816.49 -> So this

816.82 -> is great.

817.42 -> We have a

818.32 -> system which is storing some data, which is valuable to us

821.29 -> because we want to watch this

822.69 -> event live.

824.186 -> And this data

826.48 -> is going to be queried using APIs.

828.61 -> Also, these APIs are going to be tested beforehand

830.85 -> so that, the clients

832.63 -> who are using this work, paying for this service are not disappointed

835.48 -> when they actually query it.

836.8 -> As engineers, we have to think of various failures and our use.

840.22 -> . What if the database, which is stored in your videos crashes?

844.09 -> What if a particular firewall on the internet starts

847.905 -> blocking all of your requests?

849.37 -> What if one of the services that you've written in your entire

852.37 -> system, one program, one piece of code, starts misbehaving because

857.36 -> you have, introduced

858.73 -> a bug in it?

859.45 -> Or there is somebody who has maliciously entered the system and

863.05 -> changed the code in that system?

864.43 -> We have to use some design principles that we talked about earlier.

867.25 -> This challenge may also be a feature request, like if you want the musician

871.835 -> who's playing in this live event to be able to talk to the audience, do

875.74 -> a back and forth with some audience members who they select either

879.37 -> randomly or based on their activity.

882.22 -> So you have to display those users to this musician live and be able to broadcast two

889.39 -> parties to millions of people out there.

891.67 -> So taking these requirements, let's try to design a live streaming application.

895.45 -> So there's two ways to approach this.

896.68 -> One is, From customers or our clients to our server, which is

902.065 -> out there in different parts of

903.145 -> the world.

904.045 -> And then to

905.395 -> our database, which may be, again, in different parts of the world.

910.645 -> That's one way to think.

912.115 -> And the other way to think is from our database to our server.

916.465 -> So what kind of detail do I need to store to enable my server?

920.305 -> And what kind of APIs do I need to expose to enable my

924.515 -> customers to be able to use my

925.375 -> product?

925.999 -> Okay.

926.623 -> Both of these

928.495 -> approaches are fine.

929.845 -> They require different ways of thinking, though.

932.935 -> When it comes to a data based approach, you will need to consider what

937.855 -> kind of data do you need to store.

939.895 -> And often you'll be thinking of these

943.235 -> pieces of data as tables.

946.705 -> So you need to store a video

948.295 -> with an id.

949.522 -> The video

950.34 -> has a name, it has a size, and it has some data.

955.045 -> That's one way to think.

956.935 -> I prefer the other approach where customers define their problems, which are

962.214 -> then fulfilled using APIs on the server, which are then fulfilled by storing

969.175 -> some sort of data in the server, in the

972.115 -> database.

972.81 -> And that data

974.905 -> is then mapped onto

975.865 -> tables like and so for

977.814 -> this system, this is the product we'll be using.

981.865 -> Okay?

982.045 -> In our case, customers are live streaming customers, so they may be streaming from

987.655 -> their cell phones or laptops or the tv and we can't assume which device is being used

995.214 -> more often.

996.024 -> In certain

996.834 -> countries you'll have certain devices being

998.485 -> used more.

999.732 -> There, there

1000.979 -> could also be a tablet of course, from which they're streaming this

1003.439 -> video.

1003.798 -> But that there

1004.875 -> are multiple devices which need to be

1006.464 -> catered to.

1007.316 -> This is a

1008.594 -> front end UI design problem.

1011.775 -> System design is more to do with the distributed systems'

1015.285 -> backend part of things.

1017.714 -> There is some system design involved when it comes to API interactions and how to

1022.334 -> store or cash data on the front end, but that is not what we are focusing on here.

1027.194 -> We are focusing on the backend part of the system.

1029.925 -> Okay?

1030.645 -> So these clients need to be able to query our server in real time.

1035.474 -> So what does our server need to have?

1037.694 -> Its APIs are going to be something like get video and you pass in the video

1042.02 -> ID and you pass in your device type.

1045.885 -> So I send you the resolution based on your device type and you might

1049.395 -> have a particular offset that you know, I've already seen the

1052.215 -> first 10 minutes of the video.

1053.895 -> Show me everything after the 10th minute.

1056.474 -> If I have a video API like this, I also need to return something.

1061.275 -> So the return type could.

1063.869 -> Some frames, some

1065.04 -> video frames.

1066.004 -> Let's say

1067.02 -> each of these frames is 10 seconds long.

1069.305 -> So a single frame is what I sent back, such that it is off

1072.48 -> this video for this

1074.369 -> device.

1075.69 -> And after 10 minutes, so 10 minutes to 10 minutes and 10 seconds, that frame

1080.73 -> is picked up and sent to the client.

1082.59 -> If you know about

1083.58 -> API design, that this

1085.41 -> API is not

1086.82 -> well named.

1087.842 -> Get video

1088.865 -> means you get an entire video, but you're returning a frame.

1092.16 -> So maybe we can rename this to get video frame.

1095.94 -> Other thing is if you're using a

1097.085 -> REST protocol get is going

1100.62 -> to be defined in the request itself.

1102.36 -> So g e

1103.15 -> t or here is going

1105.78 -> to say that Get Me Video is enough to say that you want

1110.37 -> a video.

1111.078 -> In our case,

1112.139 -> like we said, it's a frame, so maybe get frame or get video

1115.565 -> frame is what we are looking at.

1117.75 -> You notice that Wimu also has a similar API when it comes to getting the next 10

1122.129 -> seconds.

1122.928 -> YouTube also has

1125.324 -> an api which tells you from which point to which point do you want to see the video?

1129.074 -> I'll give you all those things.

1130.635 -> Great. So we have this, what else?

1133.635 -> You should be able to comment on the video.

1135.614 -> Okay.

1136.364 -> So what you should be able to do is you should be able to post a comment.

1140.705 -> Again,

1141.324 -> I'm using well-known

1143.145 -> concept in rest post means you want to put something in the server, you

1148.695 -> want to manipulate some data, you want to add some data to the server.

1152.445 -> So if I say post a comment with an ID here, I don't care about which device

1157.459 -> it is coming from because the comment is just data, which has to be persisted in

1160.814 -> the server.

1161.475 -> Based on that,

1162.465 -> I'm not going to be, changing my

1164.175 -> response or changing the behavior in which I'm posting the comment.

1167.235 -> So I don't need that.

1168.284 -> I don't need an offset, but I do need the comment data.

1171.584 -> I need the author of the comment.

1173.715 -> I need the post on which this comment was made.

1175.995 -> So maybe the video on which this comment was made.

1178.304 -> So the video ID.

1180.764 -> And similarly for each requirement, we can expose APIs, which will allow us to

1186.794 -> query and manipulate the data as we want.

1189.254 -> So that roughly defines our server side capabilities.

1194.284 -> Okay, we haven't spoken

1195.494 -> about anything in detail.

1199.214 -> We haven't talked about the network protocol also that we use, but

1202.274 -> roughly this is what it's, okay.

1204.254 -> Now let's go for the next part, which is the database side of things.

1209.414 -> What kind of data we need to store to satisfy these APIs.

1214.004 -> Returning frames is something we want to do.

1216.194 -> Storing comments is something we want to do and also probably get those

1218.924 -> comments.

1219.584 -> And to get video

1222.224 -> frames, we also probably need to put video frames.

1225.554 -> What kind of database should we use?

1227.294 -> Comments are rather simple.

1228.374 -> You can store this in an SQL database such that an SQL table having an ID

1233.604 -> with you.

1234.572 -> The data of

1236.024 -> the comments are text.

1238.125 -> being stored as a

1238.784 -> problem.

1239.478 -> The author,

1240.915 -> which is a foreign key to a user table, right?

1243.975 -> So in a user table, you

1245.865 -> have an id and this

1247.155 -> author is going to be mapped to an ID over here.

1250.125 -> Along with that, we have a video id.

1251.774 -> So there's a video table also, which has an ID and some data.

1257.475 -> The video IDs are going to be mapped in this way, okay?

1261.645 -> An example of this would be video ID 10 is over here.

1266.445 -> This is a cricket match.

1268.725 -> Video ID 11 would be a musical

1270.524 -> event.

1271.082 -> Okay?

1271.64 -> And this is

1273.314 -> a comment table.

1274.635 -> So for this cricket match, a comment was made in which they said, Hey,

1281.175 -> and the author was author number one.

1283.935 -> Author number one happens to be called.

1286.905 -> That's their username.

1288.825 -> So if you have to display all comments, In the front end for a particular video.

1295.139 -> What you want to do is you want to get all of the comments in

1298.739 -> this table for that video 10.

1302.729 -> Then you want to get the data for that user because you want to display the

1305.494 -> username who has made that comment.

1308.009 -> And you also want to get some video data because you have to display that on top

1312.239 -> of the page where the comment was made.

1315.599 -> So in this way, we are able to satisfy our requirement of

1318.869 -> posting comments, posting video.

1320.549 -> Now, overall, this system is complete.

1323.099 -> We have a

1323.639 -> system which can answer

1326.789 -> queries.

1327.869 -> Okay? This is good enough to

1330.449 -> start with.

1331.304 -> This diagram

1332.159 -> is not

1332.709 -> incomplete.

1333.592 -> It's very rough

1336.239 -> or high level.

1337.427 -> There's nothing

1338.399 -> here which is concrete, nothing, which is, let's say, useful

1343.049 -> when it comes to implementation.

1344.909 -> A at this point in time, we have a rough idea of how we are going to be

1348.749 -> talking to each other, not what we are going to use to make this happen.

1352.589 -> So let's get into those implementation details.

1355.619 -> Firstly, on the client side, is there something we need to do?

1360.719 -> Yes, different APIs require different behaviors.

1365.999 -> Posting a comment means that I'm posting this once and I'll be querying that

1371.249 -> comment maybe soon, but it's not like I need something to keep happening.

1374.909 -> I don't need continuous updates on the comment.

1377.309 -> Those notifications can be given periodically to me, or maybe I

1381.689 -> don't need those notifications at all after a few months.

1384.959 -> I just don't need notifications on that comment.

1388.649 -> So you see over here, you have non real time behavior.

1393.749 -> What about a video frame?

1396.959 -> When I ask for a video frame, I usually need to ask for the next

1401.399 -> video frame immediately after that.

1403.919 -> So I'm watching a live video, I ask for a video frame.

1407.279 -> I am sorted for the next 10 seconds.

1409.769 -> In five seconds, I'll be asking the next video frame.

1413.7 -> . So that behavior is different.

1414.75 -> It's a more continuous behavior.

1417.09 -> Okay?

1417.48 -> So maybe we need to use different network protocols to make

1421.139 -> this happen.

1422.324 -> What would

1423.51 -> I use for a comment?

1424.59 -> I would use the most common network protocol when it

1430.565 -> comes to disability systems,

1431.615 -> which is hdp, HTP gives

1434.97 -> us the benefit that you have a stateless server.

1438.33 -> You don't need to store any information when you're handling a request.

1441.629 -> A stateless server is basically, I

1443.73 -> have no idea where you

1445.83 -> are from or what you want.

1448.11 -> Define everything in the request.

1450.12 -> Okay?

1451.86 -> Goon wants the next 10 seconds of video.

1454.44 -> What do you mean by the next 10 seconds?

1457.08 -> Who is God of San?

1458.85 -> Okay, so what should be there in the request is here's Gossen.

1463.95 -> This is his user id.

1464.94 -> You can actually go and look it up in your database.

1468.495 -> It's not the next 10

1469.395 -> seconds that he wants the

1470.955 -> video from minute number 10 to 10 minutes.

1473.264 -> 10 seconds.

1474.555 -> Okay.

1474.885 -> That video length is also well defined.

1477.075 -> There is no concept of the next point of view.

1479.325 -> And the video ID is mentioned.

1481.455 -> It's not like Gora wants the next 10 seconds of the video he was watching.

1485.205 -> No, I have no idea what we do.

1486.855 -> He was watching you define which video needs to be pulled up here.

1491.024 -> You might think that this is an obvious point, like what's so special about

1495.105 -> defining everything in the client

1496.365 -> itself.

1496.816 -> It looks a

1498.17 -> little tedious and stupid, but doing it

1500.955 -> is fine.

1501.75 -> Doesn't everyone

1502.545 -> do it?

1503.955 -> Not necessarily.

1504.682 -> When you ask

1506.325 -> for the next 10 seconds of video, for example, very often as a client, you don't

1512.445 -> know what the next 10 seconds should be.

1515.895 -> Take an example.

1518.625 -> Give me the video at 10 minutes, then 10.

1522.105 -> 10 seconds from 10 minutes.

1523.07 -> Okay?

1523.545 -> That's one chunk.

1525.375 -> Let's say you make that request.

1527.175 -> And the server is taking time to give a response.

1529.814 -> Now you come back and you say, give me the same video because

1532.814 -> you didn't receive it in time.

1535.185 -> The server is going to look at that request and serve

1538.875 -> you the same response again.

1540.645 -> The other approach to this would be the client making a request.

1544.514 -> Gimme the next 10 seconds, where next is known to the server.

1549.855 -> Okay, so here the definition is not 10 minutes to 10 minutes and 10 seconds.

1555.014 -> The definition here is the next 10 seconds.

1558.375 -> The server is now going to take this request, look at the user's

1562.28 -> current pointer, and then decide which 10 seconds to pull out.

1566.895 -> So in the first case, it'll take 10 minutes to 10 minutes, 10 seconds.

1570.435 -> In the second case, it's going to make a decision to send you

1574.665 -> the video from 10 minutes, 10 seconds to 10 minutes, 20 seconds.

1579.855 -> What's the benefit?

1580.814 -> The client didn't need to know which part of the video it wants to watch.

1586.139 -> , it lets the server handle it.

1588.36 -> It makes the overall network more efficient because the client is not

1592.98 -> making duplicate requests, and it makes the client code a little more simple.

1597.149 -> You don't need to define everything every single time you let the

1600.87 -> server figure it out with context.

1603.81 -> That's a difference between a stateful and a stateless protocol.

1607.139 -> HTP is stateless.

1609 -> Its benefit is that the server is kept simple.

1611.939 -> If the server crashes, there is no context on memory that is lost in

1615.389 -> the server.

1616.126 -> If you forget

1617.28 -> where the client currently was pointing to, then this is a serious

1620.495 -> problem, right?

1621.297 -> You can go

1622.5 -> and store it in your database instead and make a service stateless.

1626.31 -> Therefore, the protocol that you're going

1627.54 -> to use htp can be stateless.

1631.23 -> The best part about this is you can add new servers without

1633.54 -> any issue.

1634.322 -> You have no

1635.495 -> state being stored in server.

1636.87 -> So whenever a new

1637.62 -> server pops up a new request

1639.81 -> comes with total context in the request itself, which you use.

1644.159 -> To query your data, right?

1645.719 -> What about

1646.079 -> video frames?

1646.871 -> Do you want

1648.11 -> to query video frames?

1649.739 -> You want to get the next one?

1651.569 -> What kind of a protocol will be nice?

1654.509 -> You can use HTP here also, but a much better protocol would be

1660.09 -> something which is designed for video transmission, because in video you

1663.749 -> have to consider some other parts.

1666.689 -> What happens if I have a mobile device, which does not have much resolution?

1670.499 -> What happens if I have poor bandwidth?

1672.209 -> What kind of video am I transferring?

1673.62 -> If it's a live streaming thing, then the current video is the most important video.

1678.09 -> What happened previously does not matter anymore.

1679.769 -> If I missed that packet, if that packet was dropped in the

1682.289 -> network, let go.

1684.369 -> If it's a,

1686.489 -> if it's a live streaming lecture where you know, there needs to

1689.549 -> be full context to understand the next part of this lecture, then you

1693.239 -> probably want to send it properly.

1694.59 -> You want to send it over a

1695.554 -> reliable network and over

1697.264 -> a reliable protocol.

1699.314 -> Okay.

1699.554 -> So if you want a reliable protocol, a TCP back protocol is a good idea.

1704.414 -> If you want a realtime efficient protocol, then a UTB backed protocol is a good idea.

1711.104 -> So over here I'm going to use a protocol like Web RTC here.

1715.664 -> Web RTCs, a peer-to-peer protocol.

1717.344 -> So you actually are able to send video from the server to the client.

1721.094 -> Certain protocols have a client to server expectation.

1723.284 -> So the client is the only person who can

1725.894 -> send data who can make

1728.294 -> a request to the server.

1729.224 -> The server cannot send data by itself to the client.

1732.734 -> Okay?

1732.974 -> So what I'm doing again here is using a peer-to-peer protocol for

1736.964 -> the video, but for comments, it's always gonna be client to server.

1740.834 -> So you see that network protocols are also important when it

1745.334 -> comes to designing systems.

1747.314 -> Okay, finally, on the server side, what are some considerations we have to take?

1751.604 -> Similar to the

1753.114 -> server side we have to

1754.784 -> think about how are we going to talk to the server, but.

1758.294 -> Most database solutions, let's say MyQ or post case, define exactly how you're

1764.325 -> going to be talking to the databases.

1766.544 -> So this is a TCP backed protocol usually, but the protocol is defined

1770.75 -> well, so we don't need to think of that when it comes to talking to service.

1776.054 -> Elastic Search, for example, has HSTP

1778.365 -> based protocol Cassandra,

1781.575 -> Amazon db, Amazon

1783.315 -> Diameter db MySQL, post

1785.264 -> Grace, all

1785.774 -> of them, the protocols

1787.915 -> are identified.

1789.225 -> The problem then becomes which database solutions should we use?

1792.735 -> Because there's a ton of solutions out there, and they have tradeoffs.

1797.685 -> We could store data in

1799.185 -> the MySQL database

1800.774 -> also, but it's going to be expensive, and it could be potentially very slow also.

1807.405 -> So what is this

1808.605 -> video data?

1809.745 -> It's effectively

1810.885 -> a file.

1811.635 -> So storing it in a file system is not a bad idea.

1815.139 -> And you don't want to build a file system yourself.

1817.56 -> So you want to use a well-known file system

1820.27 -> solution for this maybe hdfs.

1823.439 -> That's one solution.

1824.639 -> You could also use a video hosting solution like wimu.

1829.439 -> Yeah, it's off the shelf.

1830.675 -> You can just use it.

1833.489 -> You can use wimu, by the way, to host events also.

1836.04 -> So it's best to mention this in the

1837.905 -> system design requirement,

1839.55 -> like maybe an enterprise solution from will take care of the entire thing.

1843.3 -> But if you can't do that, if you can't have a live

1844.979 -> streaming requirement

1846.719 -> and you don't wanna lose your job, maybe

1848.009 -> because then you can use a file system

1854.759 -> like sdfs or s3, Amazon s3.

1859.139 -> The benefit of S3 or sdfs is that it is cheap.

1862.62 -> It's

1863.56 -> easy to query and you can

1865.649 -> store very large files inside it, okay?

1868.504 -> In a database.

1869.644 -> Yes, it's

1871.454 -> also, it's

1872.324 -> not cheap but it is

1873.734 -> easy to query and you can store very large

1875.594 -> files inside.

1876.749 -> The capabilities

1877.904 -> that a database gives you in terms of updating data or querying data

1881.954 -> may or may not be very relevant to you when you have a static

1884.924 -> file, which is a video file, okay?

1887.689 -> You're primarily looking at low cost when it comes to storing video solutions.

1893.474 -> What about the user or the comment table?

1896.324 -> These two tables can have a skill solutions back in them,

1899.054 -> let's say MyQ or post quiz.

1901.874 -> You may say that a comment is a complex data structure.

1904.225 -> Every time, a requirement

1906.584 -> changes, the comment table also needs

1908.324 -> to change.

1909.062 -> Maybe you

1909.799 -> want to persist a lot of data in the comment table per entry.

1913.784 -> You want to persist all the replies of the comment also in the same entry.

1917.564 -> So on MySQL database or a SQL database is not what you're looking for.

1922.604 -> You're looking for a NoSQL database, which is not ideal when it comes

1927.134 -> to transactions or relational.

1929.79 -> Joins, but you don't have that requirement.

1933.09 -> You're looking for scale When it comes to comments, you just want to persist

1936.659 -> that data in a key value fashion.

1939.33 -> So that's no sequel.

1941.23 -> Great.

1941.826 -> This is very rough

1944.249 -> idea of how we are going to be designing, which is satisfying the requirements

1949.56 -> of the system, and also defining the protocols or the solutions that

1954.754 -> we'll be using to make it possible.

1957.239 -> For example, we talked about web RT Cstp when it comes to network

1960.239 -> protocols, and when it comes to database solutions, we talked about

1963.699 -> MySQL and a file system,

1966.3 -> a design can go in more and more depth based on how

1969.84 -> important that requirement is.

1971.58 -> So doing a recap, we see that we have a system, a very rough blueprint even

1977.1 -> now, which talks about how our customers are going to be accessing our APIs

1982.59 -> and how those APIs are going to be accessing the data in our database.

1987.359 -> The data in our database may be being filled by customers like in Facebook.

1990.839 -> Usually the customers are the people who are filling the data, or it may

1994.439 -> be filled by an external service.

1997.259 -> In a live streaming system, it's probably

1999.389 -> going to be a really

2001.614 -> high efficient camera, which is going to be recording the video

2003.769 -> live and persisting to our database.

2007.069 -> Okay.

2007.339 -> The network protocol that you probably want to look at is rtmp Realtime

2013.35 -> Media Protocol which is

2016.069 -> a guaranteed protocol.

2016.914 -> You don't lose any data when you are shooting video

2020.699 -> using this.

2021.475 -> And the idea

2022.639 -> is web rtc, you might lose some data.

2025.009 -> That's okay.

2025.669 -> It's the end user watching a

2027.049 -> live stream.

2027.864 -> They want

2028.729 -> data quick and real, but at the source of everything, you don't want to lose

2032.809 -> any data because everybody else will lose the data if you lose it at the source.

2035.899 -> So a highly reliable network can be set up over here in a high

2040.909 -> bandwidth, expensive network

2042.379 -> because, that video

2045.259 -> camera is really going to need a high bandwidth, expensive network to

2048.469 -> process that amount of data into your database and then this data can be sent.

2052.879 -> You're seeing that I've skipped a lot of

2054.019 -> requirements.

2054.469 -> That I can't

2055.819 -> take this high quality data and just send it, broadcast

2058.429 -> it to everybody on the planet.

2060.109 -> What I need to do is transform this data.

2062.929 -> Okay?

2063.469 -> So now what we are doing is we

2065.419 -> have looked at high level

2067.069 -> what the solutions are

2067.729 -> going to be and now really

2069.739 -> getting into the nitty gritties of designing the system.

2073.729 -> So the first part of this is how do we take this raw footage, this

2077.304 -> data over the higher level network, which comes to our database, our

2080.659 -> file system, and convert it into data that we have to serve our customers.

2084.8 -> Four adp, seven 20 P, so on and so forth.

2087.38 -> You can't serve eight K definition.

2089.179 -> So how do you do that?

2091.159 -> There needs to be some sort of a transformation service over here,

2095.3 -> which is going to be taking this live stream and converting it into different

2099.96 -> resolutions.

2100.654 -> 10 80 p, which

2103.429 -> is full hd.

2105.56 -> You have seven 20 P,

2106.71 -> which is hd four 80 P,

2109.37 -> which is decent for quite a few mobiles.

2112.459 -> And I don't know if this is something you want to do, but if the live

2116.929 -> part of your stream is really important, then 1 44 P is also okay.

2120.989 -> Because as long as people are getting information and they're able to see

2123.799 -> roughly what's going on, they're going to

2125.93 -> be happy.

2127.07 -> Sometimes

2127.64 -> these resolutions are defined well in the product requirement document itself,

2132.32 -> the p I d, because after speaking to customers and getting their real

2135.77 -> feedback, which resolutions

2137.93 -> are acceptable or tolerable for the customer, there might be some customers

2141.47 -> who ask for premium 4K transfer also.

2145.86 -> Okay, so that's fine.

2147.653 -> You're watching

2148.55 -> it on tv, then you probably wanna watch it at 4k.

2150.89 -> Okay.

2152.15 -> This raw video needs to be converted to these resolutions.

2156.65 -> How do you do that?

2157.64 -> So the first thing you'll do is you'll break this video to segments.

2160.535 -> This entire raw video is going to be broken into segments of 10 seconds.

2166.535 -> So you collect video for 10 seconds, chop it off, you break that into one segment,

2171.335 -> and then you give it for processing at point number 0, 1, 2, 3, and four.

2179.315 -> So let's say you have these zero to four, which is five programs,

2183.424 -> which take the raw video and convert it to different resolutions.

2187.625 -> So if you pass in 1 44 p, then a particular program

2191.825 -> picks it up.

2192.772 -> If you pass

2193.72 -> in four 80 p then another program

2195.67 -> picks it up.

2196.711 -> And maybe

2197.404 -> you have the same program, which is running in, in five concurrent pieces and

2201.215 -> five threads based on

2202.955 -> the resolution that you pass.

2204.935 -> It converts the video, the ten second video footage into that resolution.

2209.975 -> Similarly, you can think of different formats.

2214.535 -> So if you have a device, which is an Android device, , it probably

2219.22 -> has to see the video in a different

2220.865 -> format.

2221.255 -> Okay.

2221.645 -> It can't read

2222.815 -> all sorts of video.

2223.475 -> It has some formats which are well defined inside it in the device, so it can read

2228.095 -> that format.

2229.255 -> Similarly,

2229.835 -> apple devices might have a different format.

2232.085 -> So these formats define how you're going to read this piece of data.

2234.995 -> This video data, which is going to be sent over you through

2237.155 -> the network common video

2240.275 -> format is H 2 64.

2243.275 -> Apart from H 2 64, you might have your own proprietary

2247.035 -> formats.

2247.671 -> Let's say

2248.945 -> you have a format which is more efficient than H 2 64 or for your

2253.27 -> particular type of event, a music event.

2255.365 -> You research and you

2257.255 -> find that there's a

2258.73 -> way to store video more efficiently.

2261.125 -> So this is more of a very large scale problem, like Netflix has its own

2265.275 -> formats, but if you're

2267.185 -> a small company, edge 0.2 64 is very good.

2270.275 -> Okay?

2270.635 -> So different resolutions.

2272.81 -> Different formats.

2273.86 -> We need to take our raw video footage and convert this into a

2277.49 -> combination of a resolution and format.

2279.68 -> So over here we use a design pattern of map reduce.

2285.62 -> I won't go into too much detail of this design pattern, but the basic

2288.86 -> idea is you can take one video split into pieces, which is 10 seconds

2295.37 -> long, and send it to different servers to get different outputs.

2302.33 -> Okay?

2302.87 -> You might have something else also in the process, apart from just

2305.6 -> transforming the type of video.

2307.34 -> Maybe you want to compress it here in the next step.

2311.48 -> That can be in

2311.99 -> step two.

2312.8 -> The servers

2313.61 -> being used here might be s3, S two and S one.

2316.67 -> So you see S one is being used here and here.

2319.25 -> S3 is being used here and here.

2320.93 -> So

2321.26 -> there is no there is

2323.3 -> no guaranteed execution server that you have for one part of the process.

2328.82 -> Any server, which is free.

2330.47 -> We'll pick

2330.92 -> up a task and execute

2333.23 -> it, whether it's compression or transformation.

2336.86 -> Again, finally you get three different outputs, which you

2341.09 -> can store in your database.

2344.12 -> So this would be the map reduced pattern.

2346.46 -> It's a very high level overview of this.

2349.01 -> It's almost a ca of the map reduced pattern cause there's

2351.26 -> no reduce in this process over here.

2354.085 -> But have a look

2355.7 -> at this design pattern.

2356.48 -> It is useful when it comes to taking a single piece of data or

2360.74 -> a data lake and converting it into the data streams that you need.

2365.48 -> Next, how is this data actually going to go to the users

2368.42 -> who are looking to view it?

2370.61 -> So this data has to go over here to

2373.525 -> the server which is

2374.96 -> exposing these APIs of get put post.

2378.56 -> When you query this data using a protocol, you should be able to get it.

2382.58 -> We spoke about web RTC earlier, which is a peer-to-peer protocol and it's

2385.91 -> really good for video conferences.

2388.73 -> Because multiple people are streaming video together.

2392.33 -> However, in this case, it's a broadcast, not a conference.

2396.44 -> So we can take a step back and say, instead of web rtc, we are going to use

2400.37 -> a protocol which is more suitable for

2402.83 -> streaming.

2403.93 -> There's a

2406.13 -> couple of protocols here.

2406.91 -> Also.

2408.29 -> The most popular one would be EG dash, EG is a popular protocol.

2414.53 -> DASHER stands for Dynamic Adaptive Streaming or sstp, which means

2419.27 -> that depending on your bandwidth, depending on the network that

2422.93 -> you're on you are going

2424.195 -> to be able to see high quality video or low quality video.

2427.43 -> For example, you are going

2429.77 -> in a car, you're watching the video at 10 adp, you enter a tunnel, your

2433.85 -> network is really poor.

2435.32 -> So you start watching it at 1 44 p as a client, you don't want to handle it.

2439.67 -> You want the network protocol to handle it.

2442.13 -> It's defined in impact dash.

2445.22 -> Very similar protocol is hls.

2446.66 -> This is useful for iOS or MAC

2450.08 -> devices.

2451.1 -> Okay.

2451.31 -> And finally, what kind of data do you want to store on the server?

2454.55 -> Do you want to store any data at all?

2455.87 -> Do you wanna make it totally stateless?

2457.61 -> Yes, statelessness is useful when it comes to request

2460.31 -> serving and keeping context

2462.59 -> for every user.

2463.91 -> But for some things you can keep some

2466.97 -> state when it comes

2468.89 -> to video, you can cash the last 10 minutes of video on your server.

2474.14 -> So anybody asking for video in

2475.94 -> the last, that video,

2477.5 -> which is inside the last 10 minutes, is going to get it from the cash.

2480.59 -> Instead of you making a full network call to the database all the way over here,

2485.48 -> you instead server from server itself.

2487.16 -> So you avoid this network call saving time and bandwidth, both.

2490.97 -> So these are the high level considerations that you have

2492.95 -> when it comes to a system design.

2495.14 -> Like we said, this is a large scale distributed system.

2498.35 -> Our assumption is that you have a lot of users because of which.

2502.46 -> This much planning and this kind

2503.78 -> of a design makes sense

2505.46 -> cost-wise and engineering

2506.87 -> effort wise.

2507.845 -> We mentioned

2508.82 -> that there needs to be fall tolerance here.

2510.625 -> There needs to be performance here.

2512.09 -> So you can use things like CDN solutions.

2515.48 -> So you can use content delivery networks to persist some

2519.74 -> static data and have

2521.39 -> the clients actually pull the starting data from here.

2524.48 -> Webpages are a good idea.

2525.86 -> You can also have some

2526.67 -> video data posted on

2528.355 -> the CDNs, in which case authentication becomes a bit of a challenge because you

2532.13 -> don't want everyone to be able to see

2533.3 -> this data.

2534.238 -> And does the

2535.645 -> CDN do the authentication for you?

2537.08 -> Do you write some code and then host it on the cdn, which does the

2541.31 -> authentication?

2541.8 -> These are

2542.78 -> the challenges that you have, especially when you are designing

2545.21 -> a large scale distributed system where performance is key.

2548.51 -> Okay, so this is at a high

2550.585 -> level the things we need to consider when we are designing any sort of a system.

2555.305 -> Where the requirements are defined.

2556.715 -> Maybe in the product requirement document, maybe it's a

2558.935 -> startup.

2559.436 -> It's at a very, nascent stage.

2562.445 -> There is no requirement document.

2563.855 -> You're directly talking to the customers and coming up with a

2566.8 -> technological solution, which is going to satisfy the requirements.

2569.945 -> The important things to notice are that you define the requirements

2575.225 -> as abstract concepts like

2577.175 -> objects.

2577.825 -> These objects

2579.125 -> then need to be able to be manipulated and queried using APIs on your server.

2585.395 -> The data representations need to be stored in databases.

2589.685 -> Once this high level blueprint is done, we start thinking about what exactly do

2594.125 -> we need to make this system possible?

2597.125 -> So what protocols are we going to use?

2598.895 -> What kind of database solutions can we use?

2600.575 -> In some cases, you also think of what.

2603.725 -> Intermediate design pattern solutions you can use, like you have load

2606.845 -> balances, you have message queues.

2608.405 -> These are all design

2609.455 -> patterns.

2610.045 -> And these

2611.225 -> have been converted into tools by various companies.

2614.105 -> Redis, for example,

2614.92 -> provides cas load balances

2617.525 -> and many other things are also provided by cloud solution providers.

2620.335 -> Aws, you want to use these already well-built, well-tested solutions

2627.065 -> instead of rebuilding it yourself unless

2629.315 -> you know the trade

2630.815 -> off in terms of cost and performance is significant and it's worth it.

2634.235 -> Finally, once you have decided on the tools, you think about the interactions

2637.685 -> of these tools and the interactions of these services to meet your requirements.

2642.155 -> So we thought about how are we going to take a video and show it to the

2646.565 -> user?

2646.929 -> Oh, we have to

2648.435 -> transform that video.

2649.445 -> We have to convert it into different formats, otherwise it won't be visible

2653.48 -> in different operating systems.

2655.235 -> We then thought about how we are going to stream this video to the server.

2659.225 -> We then thought about how we are going to stream this video.

2661.85 -> To the client device.

2663.38 -> And finally, we are touched upon actually streaming the video through a CDN solution

2667.945 -> instead of from your server, because maybe your server is in the us and even

2671.78 -> if it is, close to the

2673.46 -> person who's streaming, you don't want your server to handle all that load, all

2677.875 -> of that load all for some static content.

2679.79 -> You want to be giving that load away to well-known solutions like

2684.5 -> content delivery networks, which can be tied up with their ISPs.

2687.65 -> You can have a look at how Netflix does it here.

2691.67 -> Another part of system design is low level design.

2694.61 -> This is in contrast with what we just saw, which is high

2697.855 -> level design where we

2699.59 -> took different components of the entire system and thought about how

2704.03 -> those components are going to interact with each other using network calls,

2707.64 -> using APIs.

2708.728 -> And then you're

2710.36 -> going to be sending data from one place to another.

2713.36 -> But largely if you look at these

2715.195 -> systems, they own the business

2719.09 -> data related to that service.

2721.205 -> Now what we are going to do is take certain functions of

2725.045 -> these services and try to

2726.965 -> code it out, which means we are going to go into much

2730.175 -> more depth.

2731.407 -> But because

2732.64 -> of that, we are probably going lose some breadth.

2735.005 -> Okay?

2735.215 -> When you, at the high design of a system, you are looking at

2738.065 -> everything from how the users will be

2739.565 -> interacting with your gateway.

2741.815 -> The gateway is going to be actually sending these requests

2743.89 -> to your internal services.

2745.475 -> The internal services are looking at databases.

2747.575 -> You can't code all of

2748.505 -> this out.

2749.449 -> Even in, at work.

2751.337 -> You

2751.81 -> can't just have this all in one document, one page.

2754.295 -> So you take small chunks of the system and you try to elaborate on each chunk.

2760.325 -> That would be the Louis design.

2762.065 -> The core functionality of live streaming is to be able to view a video as a

2768.545 -> customer, as a person who's paid.

2770.555 -> Maybe it's a subscription, maybe it's a one-time

2772.27 -> purchase.

2772.587 -> But what we are

2773.855 -> looking at here is how does a user fetch video?

2778.775 -> View video and fetch more of the video and continue doing so, video ends, right?

2784.415 -> We are not looking at onboarding the video, onboarding a

2787.715 -> movie or how the livestream

2790.39 -> is going to be moved from the source camera to our system.

2794.105 -> We are looking on the other side, the user side of how it's going

2797.225 -> to be consumed by the users.

2798.904 -> To do this, we again, have two approaches.

2801.515 -> One is to think of the code in the start.

2805.265 -> So if you are an object oriented programming language person,

2809.195 -> you might think of what kind of objects do I need to have in

2812.35 -> my system?

2813.058 -> How are these

2814.12 -> objects to interact with

2815.165 -> each other?

2815.717 -> Is there any

2816.545 -> kind of inheritance I have to take into consideration?

2819.395 -> But as earlier, this is a little difficult to think of unless the requirements

2823.81 -> are extremely well specified or they're so generic that you have to look at

2828.154 -> the data that you're storing for.

2829.355 -> Before you think of the function, I would suggest you think of the user.

2833.855 -> So even over here, we think about.

2836.735 -> , what are the actions that a user can perform?

2840.995 -> So what I'll be doing is on my phone or on my desktop, I'll be scrolling to

2845.885 -> a particular place, watch the video.

2849.365 -> Okay?

2849.55 -> So I'll be scrub the video up to a particular point that can happen.

2852.575 -> I can also click on play from the start.

2854.38 -> So start the video at the beginning at

2856.865 -> timestamp zero.

2857.945 -> Both of these

2859.565 -> are very close to each other.

2860.765 -> So

2861.585 -> effectively the action

2863.105 -> is play video at timestamp X under the functionality that

2867.665 -> I need is to pause the video.

2869.975 -> Okay.

2870.425 -> In which case, what do you do?

2872.615 -> Do you continue fetching more segments from the backend or

2876.305 -> do you stop touching segments?

2879.395 -> So the behavior is dependent on the user experience that you want to give.

2883.055 -> If you don't wanna clog up their mobile with the entire video, while they've just

2886.665 -> kept it, in the background then you

2888.985 -> have to be smart about it.

2889.805 -> Probably the next 20 or 30 seconds or one or two minutes can be buffered.

2895.37 -> into your device.

2896.75 -> So that behavior also has to be coded inside, despite pausing.

2900.47 -> We go for the next two minutes of video from the place that you have seated

2905.06 -> right now under the important requirement is that depending on your device, the

2910.91 -> video quality, which is going to be fetched, is going to be either hd.

2915.08 -> Let's say you are using another desktop or for atp,

2919.2 -> let's say for low resolution

2921.17 -> devices, and one final feature is up to what point have you played a video?

2926.36 -> Which means if you had a one hour long video, let's say there's a cricket match

2930.259 -> between India and Pakistan, and it's one hour long and you are seen up to

2934.34 -> the 20th minute, if you log out and come back and want to watch the same video,

2939.95 -> we should start at the 20th minute.

2942.14 -> We should store that somewhere so that you have a good user experience.

2945.74 -> You might want to cash the video like we were doing earlier, the next two minutes.

2949.43 -> You want to hit that

2950.06 -> buffer so that you start

2951.47 -> playing immediately when the user has come, or you may not want to

2954.265 -> do that.

2955.027 -> You might

2955.79 -> want to do that just for some videos, which have been recently watched.

2958.64 -> The older videos then can be kicked outta

2960.96 -> the cash.

2961.705 -> That What

2962.45 -> I'm considering here are memory optimizations, user

2967.04 -> behavior, and API calling.

2969.68 -> This is largely what lu design will mean when it comes to

2972.92 -> interacting with services.

2974.509 -> Depending on your level of seniority, and depending on the use case, you might

2978.02 -> have issues of concurrency, latency and throughput also come in here.

2984.14 -> An example use case will be the workflow that we had for

2987.86 -> chunking and transforming videos.

2990.89 -> I'll leave that as an

2991.64 -> exercise to you.

2992.684 -> But there

2993.38 -> are cases where you want to increase your throughput to the maximum, so

2997.91 -> you don't really care that much about

2999.29 -> latency.

2999.93 -> How quickly

3001.21 -> do you get those videos through the pipeline?

3002.68 -> That's not your concern.

3004.299 -> You want to make sure that your pipeline is continuously

3007.15 -> functioning right.

3008.575 -> Another cases

3010 -> where the moment a video comes in, the new video.

3013.45 -> You want to respond to it as quickly as possible with the

3016.18 -> assumption that video

3017.23 -> is really important.

3018.58 -> So there's going to be some sort of context switching over there.

3021.1 -> Okay. I'll leave that as

3021.85 -> an to you.

3022.85 -> I won't give

3023.35 -> you too many details, but it's interesting to think about how these

3027.25 -> different low level design requirements affect the code that we write.

3033.13 -> Similar to what we did earlier, we are going to take a structured

3035.535 -> approach to solving this problem.

3037.6 -> The first tool that we like to use is called a use case diagram.

3041.56 -> As a name suggests, we think about what are the use cases that we

3045.43 -> need to fulfill for every user.

3049.12 -> For example, you can see three actors in this system.

3052.81 -> An actor is a person who can do actions, so an admin can do actions in our system.

3058.39 -> They can add videos.

3059.89 -> Maybe a videographer is a person who can upload videos, so it may not be

3065.2 -> necessarily that every video which has been uploaded will be added to the system.

3069.025 -> Before it gets added, it has to have some metadata

3071.575 -> added to it.

3072.439 -> What's the

3073.015 -> description?

3073.645 -> What are the

3074.035 -> timestamps what kind

3075.775 -> of video is this?

3076.555 -> How do you

3077.104 -> tag it?

3078 -> That might

3078.895 -> be taken care of by an admin.

3080.545 -> The video rougher shoots the video,

3082.645 -> uploads it specifies

3084.72 -> the quality and everything else.

3086.815 -> So that's a separate actor and a customer.

3089.925 -> The person who consumes the video, we said that this is the most important

3093.835 -> person for us, so this is the only actor that we're going to be thinking about.

3098.305 -> Let's get rid of these two actors.

3100.105 -> We mentioned that the four things that we want the customer to be

3102.955 -> able to do, let's note them down.

3105.445 -> The first thing is to be able to play a video from her.

3111.815 -> Okay?

3113.425 -> Then comes another requirement, which is,

3121.48 -> This is them coming back to the old video that they watched

3124.42 -> partially and they

3125.74 -> want to start watching our game from the left of timestamp.

3128.29 -> We also want to, we are the

3130.056 -> maximum quality allowed by

3133.78 -> network and device.

3136.03 -> So if you're on a low quality network,

3137.63 -> that's okay.

3138.75 -> You low quality

3140.47 -> video is also fine.

3141.76 -> But if I'm on my

3142.94 -> home wifi I wanna see

3145.93 -> the best quality video that I possibly can have paid for the

3148.155 -> subscription.

3149.093 -> There's things

3150.97 -> which are also happening in the background.

3152.5 -> Of course, like we said, concurrency, fault tolerance, throughput.

3155.53 -> These are things which the end user does not need to

3157.75 -> think about.

3158.725 -> You might

3159.7 -> say latency is something that affects the

3161.53 -> user.

3162.225 -> It does.

3163.615 -> But we are

3165.7 -> not looking into it.

3166.42 -> We are assuming that all of our requests have to be answered within 10 seconds

3171.74 -> quickly enough.

3172.816 -> And this brings

3174.43 -> us to our next point.

3175.24 -> We need to

3175.97 -> continuously buffer.

3177.18 -> Our video,

3178.39 -> right? So have nonstop play when

3183.87 -> watching videos.

3184.97 -> This is assuming of course, that bandwidth

3188.83 -> is not messed up.

3189.64 -> If your bandwidth is messed up, you can't prefer, so that's not a problem.

3192.4 -> But if my bandwidth is fine, then for the next two minutes, I

3195.225 -> should be able to watch the video.

3197.31 -> Okay?

3197.8 -> This is

3198.256 -> a customer if we can

3199.72 -> fulfill these requirements, they're going to be a happy customer, which

3203.44 -> matters a lot as an engineer,

3206.77 -> okay?

3208.12 -> These would be called use cases, right?

3211.51 -> You can have many use cases.

3212.8 -> Some of them are core use cases, some of them are not.

3215.32 -> When it comes to system design, the expectation is that if there is a

3219.31 -> PRD that has been given to you, a product requirement document, usually

3222.555 -> it's just on one use case, right?

3224.595 -> You add a feature and each feature is important.

3228.88 -> So we'll take all of these and mark them as reasonably important.

3234.31 -> The next step is to convert these requirements.

3236.935 -> Into classes and objects.

3239.125 -> This is where things usually go wrong.

3241.225 -> When you look at the use case

3242.125 -> diagram it looks like

3244.975 -> everything is to do with the customer view at maximum quality

3249.955 -> allowed by networking device.

3251.455 -> But is the customer actually doing that?

3254.1 -> Is the customer saying that, please gimme the best quality video?

3258.555 -> No, that is obvious.

3260.725 -> That part is obvious.

3261.865 -> So who's going to handle that?

3264.505 -> If the customer's not going to handle that part of the system, then the

3267.625 -> system has to handle that part.

3270.445 -> Okay?

3270.595 -> So there needs to be another actor in this system, but it's not a, it's not somebody

3275.185 -> who does actions like physically, they're going to be interacting with your system.

3279.745 -> So we are maximum quality

3280.975 -> offered by the network.

3283.735 -> Who's going to handle that?

3285.085 -> There needs to be some sort of a

3286.105 -> controller or the brains

3288.175 -> behind.

3289.465 -> How much video should you send?

3291.415 -> So I'll just call this, okay.

3295.66 -> sometimes the entire service, the entire functionality can be

3300.76 -> handled just by using a tool.

3304.3 -> So we offload the problem to a tool, or in our case, what would

3308.71 -> be ideal is to use a network protocol, which takes care of this.

3314.53 -> Depending on my device and depending on my network requirements,

3318.01 -> you take care of the bandwidth.

3319.8 -> An adaptive protocol is going to handle this.

3323.71 -> Sstp dash will handle it.

3325.81 -> So we'll assume that this entire service, which was going to look at the user's

3330.345 -> requirements and then

3331.81 -> offer them video particular bandwidths, can be taken care of by simple protocol.

3337.3 -> So the speed limiter does not need

3338.59 -> to exist.

3339.655 -> Instead whenever

3340.72 -> you're connecting to our system, it's going to happen over

3343.9 -> TP dash and that is a pretty

3348.285 -> big deal.

3349.057 -> One entire

3349.87 -> use case taken care of just cause you know what tool to use now in an interview.

3355.165 -> Scenario you might be asked, how does S STP dash work exactly?

3359.515 -> How does it work internally For that, you can either read some

3363.055 -> papers which help you learn the

3365.595 -> protocol in terms database

3367.225 -> internal are also very similar.

3368.455 -> Operating system internals are also very similar.

3370.855 -> Or you can guess in this case, you should probably specify that,

3375.625 -> hey, I'm just guessing over here.

3377.065 -> But I think what's going to happen in this adaptive retreat resemble

3380.995 -> exponential back off and dcp.

3383.395 -> Now for the next bit clear video from a timestamp, this means that every video

3390.415 -> needs to actually store a corresponding timestamp for a particular user.

3396.805 -> Okay, that can

3397.285 -> be done.

3398.197 -> We need to

3399.565 -> have an object here.

3402.445 -> Our service, which is video server, a video consuming service

3407.935 -> is going to be used by end users.

3410.995 -> Play a video from a timestamp.

3413.235 -> Okay?

3414.4 -> So this has to be taken care of by the video zooming

3417.82 -> service.

3418.277 -> So that will

3419.65 -> be play video

3421.97 -> for a user.

3423.404 -> And this is

3424.84 -> the video I, okay, so I'll just remove the play for this.

3431.5 -> And I'll also mention the time stamp that is being busti.

3436.06 -> You see that?

3436.45 -> I'm still thinking in terms of APIs.

3438.67 -> The clearer your APIs are, the easier your lower design will be.

3442.99 -> The more you think about how the users or each feature is going to be

3446.805 -> implemented using the

3449.77 -> services that you have, the easier your design will

3452.62 -> be.

3453.382 -> Okay?

3454.144 -> Go back to

3456.43 -> video and watch from left off timestamp.

3458.11 -> This is quite

3459.04 -> straightforward.

3460.625 -> You have seek

3465.43 -> for this user and this video, what is a seek position?

3469.18 -> So let's say we call it get seek position, but

3471.61 -> like we say if you're

3472.69 -> using a.

3473.81 -> Rest api.

3474.73 -> Then you can just, you can mention in the method it's get, so that

3479.23 -> will send you back the seek

3480.62 -> position.

3481.066 -> It's best

3481.96 -> to mention the return type.

3483.1 -> Also,

3483.52 -> like I said the clearer

3487.6 -> you make your API, the better it usually is.

3490.12 -> So video frame will be sent back here.

3492.25 -> And similarly, what will be sent back here is the timestamp.

3499.66 -> So what's going to happen is when you come back to a video, which you have

3502.155 -> left off, you're going to first sync what is the position where you should go to.

3506.95 -> So that will give you a timestamp.

3508.03 -> And then you say, okay, play this video for this user from this timestamp.

3512.36 -> Alright?

3513.266 -> Have nonstop

3515.08 -> play when watching videos.

3516.64 -> This is interesting.

3518.23 -> We need to get frames in future.

3522.34 -> So what do we do?

3524.26 -> Do we save play here or do we save, get video frame for a

3531.07 -> particular user?

3533.752 -> And a video

3537.825 -> with a given timestamp.

3540.955 -> Do you see that these two are very similar since what this API is also

3546.565 -> going to return you is a video frame and you're gonna stack these video

3550.975 -> frames together and buffer them into the video, into the phone.

3555.305 -> Yes.

3556.431 -> I see a very close

3558.685 -> linking between this API and this api.

3562.565 -> At this point, you ask

3563.785 -> yourself, what does

3566.005 -> the product need?

3567.025 -> Is play a different action from fetching the future content.

3572.865 -> And here I'll make a decision of yes, it is different.

3578.275 -> It's strange to think of because they're doing the exact same thing.

3580.884 -> What they're doing is they're, the user is saying, claiming

3583.92 -> this video from this time stamp.

3586.165 -> That's okay.

3587.305 -> And the other one is that the player, the video player is saying, get me The

3592.045 -> video frame for this timestamp looks extremely similar, but the business use

3598.015 -> cases are very different.

3599.665 -> If the user says, get me a video from this timestamp, it means that they

3603.835 -> are probably seek to that position.

3605.995 -> They like what they saw their, or they found it exciting and they clicked on it.

3609.835 -> So it's very different behavior compared to, Hey, get me the next two

3613.765 -> minutes of video in the background.

3616.405 -> Okay.

3617.725 -> That's one very important thing.

3620.515 -> When you are

3621.055 -> designing any api think about

3624.535 -> who's using it, what is it being used for, right?

3627.895 -> And then think about the common functionalities.

3630.805 -> So these two APIs are very similar.

3633.085 -> And on the server side, what you would, I'd really like to do is just merge these

3639.175 -> two APIs.

3640.645 -> The client

3642.115 -> should figure out

3643.255 -> by itself that what

3645.055 -> video frame doesn't want to pick.

3647.065 -> , so maybe you cut down

3649.075 -> on the APIs that you

3650.735 -> have to maintain by replacing

3655.675 -> the current API of play with what we have here.

3661.405 -> So maybe every time you play from a particular point, we should sign an

3665.425 -> event to the backend service saying that this is an exciting part of the video.

3670.255 -> The user actually came to this timestamp, seek Q and clicked on the

3673.285 -> video.

3673.923 -> Okay.

3674.562 -> That there is a seek api,

3678.445 -> which is

3679.415 -> primarily for when you're

3681.055 -> left off a video.

3681.895 -> So if you left off, came back and started watching from there, maybe it's not super

3686.515 -> exciting, it's just the place that you're

3688.105 -> leading off from you, you wanna

3690.595 -> continue from.

3691.495 -> But during the video while watching it, if you came to a particular

3695.335 -> position and started viewing, that requires an event that

3699.115 -> shows that, this part

3700.375 -> of the video is sought after.

3703.195 -> So it feels like you're splitting hair here, but from the side of the

3707.395 -> business or from the side of analytics, they're very different use cases.

3711.595 -> From the side of engineering, it's the exact same

3713.215 -> behavior.

3713.785 -> Someone asks

3714.925 -> you for 10 seconds of video, you give them 10 seconds of video.

3717.325 -> But for analytics or for the business, this part should be

3721.95 -> converted into a three load.

3723.775 -> This part is just for user experience, so it depends on you As an engineer,

3730.375 -> I personally would take this API and I would see who's making this request.

3736.405 -> So a flag would probably say that the user made this request

3740.125 -> or the device, the mobile made the request all good buffering.

3745.975 -> And depending on that flag, I would then

3748.285 -> file an event for showing

3750.69 -> interest in this part of the video.

3753.475 -> Alright, so that is the use case

3754.765 -> diagram.

3755.132 -> We can now

3756.235 -> think about the class diagram.

3758.245 -> Let's draw the class

3759.565 -> diagram out.

3760.405 -> The first

3761.245 -> class that we need is a video.

3763.825 -> Two things that we need to store for every class are states and behaviors.

3769.345 -> States are data that an object needs to perform.

3774.685 -> Behaviors.

3776.075 -> For example I am speaking

3778.375 -> right now, so I need a throat.

3781.285 -> I need

3782.045 -> a tongue.

3782.845 -> I probably

3783.245 -> need a brain to speak and what I'm doing right now is teaching.

3788.665 -> So the behavior will be teaching while the data that

3792.025 -> I need is my body parts

3795.055 -> to actually work in tandem.

3796.975 -> Similarly, you might have a video which has certain data that it needs.

3801.775 -> So that would be the bites of the video.

3805.375 -> Let's say frames.

3806.395 -> Cause we have been using this term that's in

3808.195 -> every video.

3809.125 -> You might

3810.055 -> have some metadata who's the

3813.625 -> uploader of this video?

3815.065 -> How long is it?

3816.085 -> What kind of tag do you want to add?

3817.435 -> So on and so forth.

3819.64 -> , what operations can you perform on the

3821.11 -> video?

3822.07 -> You can get

3824.95 -> a frame, right?

3826.107 -> That's pretty

3827.235 -> much it.

3828.25 -> Okay.

3828.74 -> You can't add a frame after you've added

3830.34 -> a video.

3831.281 -> You can't do anything apart from

3834.105 -> just getting a frame from a particular point.

3836.32 -> So that's the simple class.

3838.87 -> You also have the class

3839.98 -> of user.

3841.362 -> That I'm not

3843.435 -> focusing on the class diagram.

3845.8 -> So user has a

3848.5 -> name, an email more metadata around them,

3854.53 -> but nothing else really.

3856.78 -> The most important thing, probably to be an ID in your case could be an email.

3860.8 -> A video also will have an id and for the user, you can probably get there.

3870.07 -> Id, okay.

3871.54 -> Very simple class again, which brings us to the most exciting

3874.15 -> class, which is watch video.

3879.279 -> Okay, so a watched video is going to be an action by a user

3883.87 -> who's basically watching a video.

3885.49 -> So this needs a video id.

3888.37 -> Which video have you watched?

3889.87 -> Who is watching it?

3890.745 -> That is the user id an ID of the action so that you can refer to

3895.509 -> it later.

3896.437 -> Up to what

3897.879 -> timestamp have they watched this video?

3899.379 -> So seek timestamp.

3902.259 -> And coming back to the use cases, we might want to buffer up to a

3907.18 -> particular point, but let's assume that the client handles that.

3909.75 -> The client knows how long, how much video you have buffered already in

3913.48 -> the device.

3915.105 -> Going back

3916.73 -> to a particular place is possible because of the timestamp.

3919.24 -> The final class is video consuming service.

3921.64 -> This also requires a class to be explicitly shown here.

3926.799 -> So that is video consuming service.

3933.73 -> Let's first define the behaviors that this class has.

3936.52 -> So that is API one.

3938.39 -> And if you two,

3945.7 -> and to do this, it just needs to set watch videos,

3955.28 -> which user, okay?

3958 -> That takes of our entire class.

3961.75 -> You've seen, this is a very simple class diagram.

3963.97 -> What is more challenging actually is the sequence diagram.

3969.04 -> Okay?

3969.61 -> How is a user going to watch a video?

3972.27 -> So this would be called a class diagram where we have defined what

3978.1 -> state and what behaviors are possible for each object in our system.

3984.82 -> Fine.

3986.29 -> These two diagrams of a class and use case are sufficient in most cases.

3992.38 -> However, in some places where the interaction is complex, like

3996.25 -> over here , you need

3999.115 -> another diagram which defines the sequence of actions.

4002.835 -> In this case, it's not very clear how the user is behaving.

4006.735 -> These are the actions that they can do.

4008.535 -> Okay?

4010.125 -> These are the things that you need to store and the behaviors you need

4014.115 -> to expose for the action to happen.

4017.265 -> That's also right.

4018.705 -> What is the sequence of actions?

4020.475 -> What happens first?

4021.285 -> What happens second?

4023.535 -> That's not clear.

4024.735 -> And for that you need a sequence icon.

4026.685 -> So let us try doing that.

4032.835 -> This is what you call a timeline.

4035.115 -> The y axis basically is time, but in

4038.115 -> descending order.

4038.895 -> So you have

4040.065 -> three timelines here.

4041.565 -> One is

4041.985 -> that if a user does certain actions things happen

4045.645 -> to a video, okay?

4047.445 -> And the video consuming service actually uses a video's current.

4051.92 -> Seek time to get the next frame.

4054.6 -> So this might not just be a video, it could be a video service, which is

4058.5 -> providing an api to consume

4060.72 -> the next frame of the video.

4063 -> We'll see how this

4063.725 -> happens.

4064.363 -> First the

4065.64 -> user sends a message.

4067.83 -> What message

4068.31 -> is this?

4069.03 -> Like we said earlier this is going

4071.55 -> to continue your information and the video information.

4085.2 -> This is responded to by the video consuming service immediately because

4089.009 -> it knows up to what point has a user,

4091.379 -> the video.

4092.475 -> So that is

4094.17 -> you return a timestamp.

4095.67 -> Now the user is going to make another interaction, which is

4099.99 -> play or get video frame, right?

4101.85 -> So the video frame as this user or

4106.535 -> this video at a timestamp,

4109.529 -> and the tool I'm using here

4110.49 -> is lucid.

4111.174 -> As you can

4112.2 -> see, if you use tools, it helps instead of reinventing the

4117.986 -> wheel or, doing it in

4119.07 -> a half way.

4119.874 -> This is much

4120.99 -> better.

4125.16 -> The video consuming service cannot get you the frame though.

4128.85 -> However, it's best that the user interacts with the video consuming service, so

4133.5 -> it could send a message to the video

4134.879 -> service and say get me the

4137.599 -> video frame

4146.109 -> Okay.

4146.599 -> The important thing to notice here is that the video service

4150.33 -> has no idea who the user

4153.929 -> is, whether they're authenticated or not, to watch

4156.33 -> this video.

4157.577 -> And it just

4159.45 -> gives you a timestamp.

4160.71 -> Okay.

4160.95 -> I could have taken this request and sent it directly to the video service.

4165.559 -> Okay.

4166.755 -> And the video service could

4167.505 -> say, sure, I don't care

4169.515 -> about what, who,

4170.285 -> which user is is trying

4172.274 -> to access this video.

4173.325 -> Just tell me the video and the time stamp and I'll give you the response.

4176.385 -> But authentication would be a bit of a problem.

4181.064 -> So that's the reason why I'm assuming authentication is going

4183.38 -> to happen here for every frame that

4185.684 -> you ask.

4186.465 -> And then the

4187.635 -> video timestamp, as we mentioned, I'm going to be sending back

4197.564 -> a response of a video frame.

4206.625 -> I could also send back multiple frames in the hope that you

4210.349 -> can use them later.

4212.457 -> But

4212.984 -> I'm assuming that the frame lasts for 10 seconds.

4214.934 -> And if you really feel like you can send a request in between and get another frame.

4219.194 -> So it's a single frame and this is then sent back.

4229.135 -> To the user.

4230.955 -> Okay, that's the interaction for a while.

4234.825 -> The moment you get a frame, you want to repeat this action.

4249.075 -> And so the data keeps

4250.155 -> flowing.

4251.125 -> Very similar

4253.065 -> to how a TCP connection works

4254.865 -> actually.

4255.54 -> Initially

4256.215 -> you have some sort of a handshake, right?

4259.1 -> You get some initial information, set up the connection, and then what's

4261.975 -> happening is you're constantly pinging.

4264.465 -> One of the

4264.795 -> drawbacks that I can

4266.115 -> see here in my diagram is that the video consuming services are intermediately.

4270.105 -> There's so much communication happening between the video service and the user.

4273.525 -> This is just wasting time.

4275.025 -> I don't know if that is worth it, if just authentication.

4280.03 -> is worth completing

4281.759 -> our flow

4282.66 -> this much.

4283.295 -> You can put

4284.249 -> some authentication here in the video service itself, so you'll

4288.599 -> save on two network calls, which

4292.259 -> is a lot.

4293.639 -> So yeah, maybe

4295.019 -> I should just take this, put

4297.03 -> this here and take

4298.83 -> this, put this

4300.209 -> over here.

4301.758 -> Don't go into

4304.08 -> the video consuming service.

4306.389 -> That's what we'll be doing in the code.

4308.82 -> So now we finally jump

4309.785 -> to coding.

4310.592 -> Remember to

4311.4 -> use the diagrams that we have made as a reference because most of our

4315.905 -> thoughts and most of the interactions are documented well over there,

4319.86 -> it would be a waste not to use it.

4321.929 -> The whole purpose of making these diagrams is to take away the

4325.745 -> thinking effort required by coding.

4327.57 -> Coding is basically us typing out or writing the things which

4331.83 -> have been mentioned or thought through in these diagrams.

4336.179 -> Okay, that

4336.509 -> will speed things and

4337.919 -> also help you avoid mistakes.

4340.74 -> So the

4341.099 -> first class that we talked

4342.48 -> about is a video class video.

4347.682 -> The state

4349.679 -> that we talked about is it has an id, it has frames, so maybe a set of frames here,

4359.129 -> but then the frames are also ordered.

4360.66 -> So an area of frames

4362.469 -> make sense.

4363.679 -> And then some

4365.495 -> metadata.

4366.629 -> So in our case, I'll just say meta data, json.

4372.33 -> Of course, in the real world, you would actually have the creator

4375.15 -> of the video, the uploader of the video, and many other parameters, the

4378.394 -> length of the video, everything else.

4380.429 -> But we don't need that

4381.259 -> right now.

4383.092 -> So I just

4385.889 -> create

4388.509 -> a class.

4390.606 -> That an ID

4393.75 -> helps you create classes

4394.95 -> much better.

4396.27 -> These classes

4397.59 -> that I'm creating are just plain Java objects.

4401.13 -> You can use the same thought

4403.47 -> process for c plus for C

4405.809 -> shop and many other languages.

4407.76 -> Any object or in programming language.

4409.8 -> If you're coding in

4410.46 -> Python or Scala it might

4413.55 -> be slightly different, right?

4415.68 -> But the logic is very similar.

4418.68 -> Come to the point of defining things as objects and the interactions between them.

4424.92 -> And finally, define the states and the behaviors of every

4428.644 -> object and then put them out.

4430.559 -> Okay? So this is largely language agnostic.

4432.87 -> This approach.

4435.18 -> You have a video which needs

4436.62 -> a thing.

4437.97 -> A frame is going to be a bunch of bites, so that's okay.

4448.44 -> And maybe it has a

4449.1 -> timestamp That is in

4453.855 -> thanks stamp.

4454.934 -> You also have a class of user from the diagram.

4457.634 -> A user has a string id, it has a string name string email.

4464.264 -> These are basically

4465.134 -> metadata.

4465.728 -> And we talked

4467.509 -> about some behaviors that these classes need.

4469.574 -> So written return of frame, when someone says get frame

4483.254 -> and which frame should we return?

4485.434 -> You should return

4486.214 -> the frame which is

4488.474 -> being called for a particular timestamp.

4490.364 -> So you see our class diagram missed that we can go back

4493.544 -> and fix it or we can

4496.034 -> fix it in code.

4496.804 -> I would suggest fixing it in both places because if the documentation

4501.194 -> is up to speed, it helps in timestamp.

4506.025 -> You can also have timestamp as an

4507.074 -> object.

4507.851 -> for, making sure

4510.18 -> that the inea is greater than zero and all that, all the validations that you want.

4514.23 -> But I'm just keeping

4515.19 -> things simple.

4515.91 -> Pass on a

4516.99 -> timestamp.

4517.41 -> Here we are going to be iterating over the frames.

4520.86 -> This can be improved of course, but what you want to do is you want to go

4526.34 -> over the frames and return the timestamp

4535.15 -> and return the frame.

4536.88 -> Only if the timestamp of the frame is less than the timestamp you

4542.73 -> have asked for less than equal to.

4546.84 -> And there also has to be some sort of an end times time.

4548.94 -> So we can assume that every frame is of

4550.95 -> 10 seconds in which

4552.305 -> case the start time is this.

4555.809 -> This has to be less than equal to, and the end time mentioned over

4561.13 -> here, plus 10, is greater than all

4565.47 -> equal to this time.

4567.054 -> Okay.

4567.45 -> Or rather greater than then this frame belongs to this.

4571.559 -> If you don't get the frame, then you just return none.

4576.75 -> You can also throw an

4577.38 -> exception here.

4579.305 -> Let's see.

4581.28 -> I would say throwing an exception makes

4582.54 -> sense because the meaning

4585.575 -> of an exception is that I didn't know how to react to this.

4590.88 -> None would mean that, oh, you get

4593.19 -> blank.

4593.705 -> So if a timestamp

4595.769 -> of 20 hours is sent for a video, having just one hour, a blank

4601.14 -> frame is not exactly what you want.

4602.519 -> You wanna say, oh, you're out of bounce.

4604.68 -> So

4609.15 -> index, under bounds

4609.93 -> exception being thrown

4612 -> here is going to be

4616.05 -> okay.

4617.43 -> Now there's another small problem I see here, which is

4620.519 -> the magic constant of plus 10.

4622.38 -> You don't really want to do this.

4623.4 -> It doesn't look good.

4624.929 -> . Because tomorrow

4626.159 -> if your frames get optimized and can store 30 seconds of video, then you

4630.48 -> have to go and change it in the code.

4632.159 -> So you might have this in a constant, in the video itself, let's say

4636.15 -> no, a class constant, so public static in frame time equal to 10.

4647.719 -> And then what you have is if you need to make any change, then you

4651.629 -> make a change in a single place.

4654.389 -> But

4655.269 -> even that would not

4657.419 -> be suggestible because this is something related to the frame.

4660.36 -> So let's take that, put it in the frame class, and then what

4665.369 -> you have is framed or framed.

4669.034 -> Time

4672.179 -> has to be this, but that is also,

4676.839 -> that's fine.

4677.624 -> It's good.

4678.45 -> There's just one.

4680.94 -> data point for the entire class.

4683.339 -> All of your frames

4683.999 -> have the same amount of

4685.44 -> frame time.

4686.849 -> It could be different, it could be that

4689.28 -> these frames some frames

4691.469 -> are really high quality.

4692.19 -> There's a lot of movement over there.

4693.509 -> So it's not even 10 seconds long.

4694.865 -> It's just two seconds long.

4696.089 -> And some

4696.509 -> frames are, like this

4698.639 -> coding frame

4699.209 -> over here.

4699.833 -> You can store

4700.769 -> 10, 15 seconds of video because there's not much happening over here.

4704.669 -> There's not any face moving or something.

4707.37 -> So I'll just take this and I'll leave it to the object,

4712.95 -> not the class, but the object.

4715.139 -> So it does not become a class property, which is going to be for every object.

4718.889 -> It is defined by each object.

4723.48 -> So any timestamp,

4730.139 -> and then this becomes start timestamp.

4735.309 -> Alright, so this is.

4737.58 -> This is better.

4738.408 -> I don't need

4738.84 -> to say, I don't need to mention start timestamp.

4741.42 -> I just said timestamp has to be greater.

4745.32 -> Okay.

4745.77 -> This is much more flexible code.

4748.26 -> This is what will set you apart from just

4751.38 -> the portal.

4752.012 -> You're going

4752.645 -> to be a engineer then.

4758.08 -> Okay.

4758.58 -> So now we

4759.63 -> have a user you can get

4762.99 -> their id.

4764.55 -> This is not really related to what we are doing, but that's

4770.43 -> okay in get ID and return the id.

4782.94 -> Okay. The next class is more interesting.

4786.275 -> There's watched video, which is a class here we have the ID.

4796.334 -> Of this action we have the video id.

4802.634 -> We also have the user who's actually watching this video, who's doing this

4806.954 -> action, and we have a seek bank stamp.

4809.985 -> So that is seek tank.

4814.754 -> Okay, we mentioned that there's going to be

4822.284 -> a method to get seat time.

4828.195 -> Important to notice that APIs having rest have the keyword, get inside them

4834.434 -> often, like you can't mention that you're posting something or getting

4837.589 -> something, but the objects here can have a get of their own because you don't

4842.624 -> exactly know what's happening, right?

4844.634 -> It's not a

4845.239 -> HTP API here it's a Java

4848.144 -> object, so it gets 10.

4849.855 -> Makes more sense here.

4851.534 -> So this defines

4852.344 -> everything.

4853.263 -> Except for

4855.15 -> the last couple of behaviors, which are actions by a user or actions by a system.

4861.87 -> So the video consuming service, show me the seek time.

4865.74 -> This is going to be a public ink, seek time, return for this watched video,

4878.16 -> seek time, had a user ID and a video id.

4889.06 -> Let's assume that watched video is a database call,

4907.89 -> and once you have this watch video, what you want to do is you want

4909.99 -> to get this done and return that.

4912.699 -> Okay.

4914.222 -> For this to

4918.839 -> work, we need to have a database,

4931.809 -> which we are going to create as a dummy object.

4935.19 -> Okay?

4936.9 -> Create that and turn on.

4942.449 -> Okay.

4942.779 -> This is great.

4943.799 -> It gives you the seek time.

4945.719 -> And then finally, the other API that we look for is class.

4952.019 -> We do service.

4955.379 -> So the final class is going to be using a file system, let's say, because videos

4960.929 -> are usually stored in the file system.

4964.919 -> And let's define that class, this class file

4968.4 -> system.

4969.442 -> Okay.

4970.485 -> Now there's

4972.57 -> a public method similar to the previous service, which is going to be returning

4978.18 -> a frame.

4979.236 -> And that is

4980.82 -> get frame for a particular video with a video ID and a timestamp.

4993.03 -> Okay. And how's the interaction going to happen?

4994.83 -> We first need the video, so let's say our file system gets

5001.34 -> us a video with this video id

5010.6 -> what?

5011.09 -> Once you have

5011.695 -> the video you need

5013.61 -> to return the frame corresponding to this tank stamp.

5022.76 -> And that's it.

5024.26 -> That's pretty much it.

5025.4 -> You have something backing your services.

5028.489 -> Some sort of database or

5029.839 -> file system.

5031.016 -> Once you have

5032.78 -> them persistent, you can always get to them and pull data from them.

5038.449 -> Once these systems are being backed by a database, then you

5042.565 -> can actually manipulate this data

5044.179 -> that you have by exposing

5046.129 -> APIs.

5047.269 -> And the best way to manipulate the data is to abstract them out into objects

5051.349 -> which have their own state and behavior.

5053.509 -> So the code is simplified and you can reuse a lot of the behaviors

5057.079 -> for different types of use cases.

5060.53 -> This would summarize a low level design process.

5063.469 -> Alright then that's a reasonably well detailed introduction to system design.

5068.809 -> If you want to look at more videos on system design, there's a free section

5073.19 -> that I have on Interview Ready, which is all about the design patterns and the

5077.78 -> basics of system design, load balancing,

5080.459 -> rate limiting charting and scaling.

5083.2 -> And if you

5084.844 -> want a more advanced version of system design, I have a paid

5088.415 -> section in that same course.

5090.394 -> I would suggest go ahead and check out the free resources first.

5094.205 -> If you really like it and you think that you know it's time to

5096.905 -> level up, go for the paid section.

5099.214 -> All the best.

Source: https://www.youtube.com/watch?v=m8Icp_Cid5o