[Backend #31] How to use kubectl & k9s to connect to a kubernetes cluster on AWS EKS

Aug 16, 2023

[Backend #31] How to use kubectl & k9s to connect to a kubernetes cluster on AWS EKS

In this lecture, we will learn how to use kubectl and k9s to connect to a kubernetes cluster on AWS.
- Join us on Discord: https://bit.ly/techschooldc
- Get the course on Udemy: https://bit.ly/backendudemy
- Buy us a coffee: https://www.buymeacoffee.com/techschool
- Full series playlist: https://bit.ly/backendmaster
- Github repository: https://github.com/techschool/simplebank
---

In this backend master class, we’re going to learn everything about how to design, develop, and deploy a complete backend system from scratch using PostgreSQL, Golang, and Docker.

TECH SCHOOL - From noob to pro
/ techschoolguru
At Tech School, we believe that everyone deserves a good and free education. We create high-quality courses and tutorials in Information Technology. If you like the videos, please feel free to share and subscribe to support the channel or buy us a coffee: https://www.buymeacoffee.com/techschool

Content

0.16 -> Hello everyone, and welcome back to the backend master class!

4 -> In the previous lecture, we’ve learned how to set up a kubernetes cluster using AWS EKS.

9.8 -> As you can see here, our simple bank cluster is already up and running.

14.03 -> So today I’m gonna show you how to connect to this kubernetes cluster

17.539 -> using 2 command line tools: kubectl and k9s

21.369 -> First, let’s search for kubectl in the browser.

24.84 -> Here it is, let’s open this page.

28.13 -> So kubectl is a Kubernetes command-line tool

31.009 -> which allows you to run commands against the Kubernetes clusters.

35.16 -> You can use it to deploy applications, inspect and manage cluster resources and view logs.

40.67 -> I’m on a Mac, so I will follow this link to know how to install the tool.

45.55 -> It’s pretty simple if you have Homebrew installed on your machine.

49.78 -> Just run: brew install kubectl

52.73 -> And that’s it!

54.199 -> We can run this kubectl version client command to see if it’s working well or not.

59.53 -> If it prints out the version of the client like this

62.92 -> Then it means the tool has been successfully installed.

66.2 -> Next step, we will need to verify kubectl configuration.

69.6 -> Basically, kubectl will need to read some information from a config file

74.78 -> In order to know how to access the Kubernetes cluster.

78.4 -> By default the file is located at this folder.

82.29 -> We can use this kubectl cluster-info command

85.26 -> To check if the config is properly set up or not.

89.12 -> Here we see an error because it’s trying to connect to a local cluster,

92.82 -> But we don’t have any cluster running in our local host.

96.11 -> We want it to connect to the AWS EKS cluster that we set up in the previous lecture.

101.47 -> So we have to tell kubectl how to look for that cluster.

104.61 -> In order to do that, here’s the command we can use

108 -> to fetch the AWS EKS cluster’s information and store it in the config file:

113.58 -> Aws eks update-kubeconfig --name

118.94 -> Then pass in the name of the cluster, which is simple-bank in our case.

124.62 -> Then the region where the cluster is located, which is eu-west-1.

129.45 -> And press enter.

130.88 -> Oops, an error occurred: Access denied exception.

135.95 -> And the reason is: user github-ci is not authorized

139.4 -> to perform the describe cluster operation on the simple-bank cluster.

143.72 -> So what we need to do now is to grant EKS permissions for this user.

148.07 -> Let’s go to AWS IAM service

152.54 -> Open the users page and select the github-ci user.

156.95 -> If you still remember, as we set up in previous lectures,

161.6 -> This user only has permissions to access ECR and secret manager services.

166.95 -> And these permissions are actually coming from the deployment user group

170.45 -> That the github-ci user is attached to.

173.73 -> So to add a new permission, let’s open this user group,

177.16 -> Change to the Permissions tab,

179.74 -> Click Add permissions, and select Create inline Policy.

183.39 -> In this form, we have to choose a service.

187.03 -> Let’s look for EKS.

189.819 -> Here it is.

191.18 -> There are many access levels,

193.03 -> And in the Read level, you can see the DescribeCluster permission.

197.069 -> You can select only this permission if you want,

199.25 -> But since I want the github-ci to be able to make changes to the cluster

203.56 -> and deploy applications to it later,

205.84 -> I will just allow it to perform all actions by checking this box.

209.61 -> Next, in the resources section,

212.62 -> You have the option to choose only some specific resources

215.62 -> you want this user group to manage.

217.97 -> Or you can simply allow it to manage all resources as I’m doing here.

223.44 -> OK now let’s click Review policy.

226.4 -> We must set a name for this policy.

228.39 -> I’m gonna call it EKS Full Access.

232.22 -> And click Create policy.

234.79 -> And voilà, now the deployment user group will have full access to EKS clusters.

240.68 -> If we go back to the Users page,

242.74 -> And check the github-ci user’s permissions,

246.05 -> We can see that now it has EKS Full Access permission,

249.64 -> That’s because this user belongs to the deployment group,

252.99 -> So it will be granted all permissions of the group.

256.5 -> OK cool!

257.56 -> Now let’s go back to the terminal and run the update-kubeconfig command again

262.4 -> This time it is successful.

264.57 -> A new context for the simple-bank cluster has been added to the .kube/config file.

268.71 -> Let’s check the content of the .kube folder

271.78 -> There’s a config file as expected.

273.98 -> Let’s print out the content of this file using the cat command.

278.29 -> At the top of the file it has some certificate authority data,

283.79 -> Then the URL of the server, where we can access the cluster.

287.13 -> Followed by the name of the cluster: simple-bank.

290.21 -> Next, we have the context to access this cluster.

293.98 -> It includes the ARN of the cluster and the user to access it.

298.33 -> Followed by the name of the context,

300.31 -> Which is also the current context, since we have only 1 context until now.

305.73 -> And at the bottom of the file,

307.02 -> There’s some more detail information of the user,

310.139 -> which kubectl will use to get the token to access the simple-bank cluster.

314.05 -> Now, in case you have multiple contexts for different clusters on your machine,

318.84 -> You can use this kubectl config use-context command

322.41 -> to select the context you want to connect to.

325.03 -> Alright, now let’s check if we can connect to our simple-bank cluster yet or not.

330.16 -> We can use the kubectl cluster-info for this purpose.

333.22 -> Oops, we’ve got an error: Unauthorized, you must be logged in to the server.

339.4 -> OK so if you face the same problem, then this might be the reason.

343.23 -> Your IAM entity isn’t authorized by the RBAC configuration of the EKS cluster.

349.24 -> This happens when the cluster is created by an IAM user or role

352.83 -> that’s different from the one you’re using to authenticate and connect to it.

357.45 -> Remember that the user I’m using on my local machine is github-ci.

361.08 -> But initially, only the creator of the Amazon EKS cluster

366.03 -> has the master permissions to configure the cluster.

369 -> If we want to extend this permission to other users,

373.07 -> we must add the aws-auth ConfigMap to the configuration of the EKS cluster.

379.19 -> OK so how can we fix this?

381.02 -> First, let’s run this command in the terminal to check the current user identity.

386.66 -> As you can see, it’s the github-ci user, not the root user we used to create the cluster.

391.75 -> That’s why it doesn’t have the permissions to access the cluster.

395.919 -> So if we run this kubectl get pods command,

399.12 -> It will throw the same error as when we run the cluster-info command before.

403.47 -> In order to fix this, we must use the access key ID and secret of the cluster creator account.

409.21 -> At the moment, we’re still using the access key of the github-ci user

413.98 -> as you can see in this aws credentials file.

417.28 -> Alright, now let’s create a new access credential for the root user: tech school.

422.36 -> I’m gonna open My Security Credentials page in a new tab.

426.27 -> Then select the Access keys section.

429.08 -> And click Create New Access Key.

431.43 -> The key has been successfully created.

434.1 -> We can click on this to show the access key.

436.889 -> I’m gonna copy the access key ID.

440.31 -> Then go back to the terminal,

441.9 -> And edit the aws credentials file using vim.

445.75 -> We can list multiple different access keys in this file.

449.05 -> So here I will put the old credentials under a profile named github.

452.83 -> And the new access keys will go into the default profile.

456.61 -> First, paste in the access key ID,

460.43 -> Followed by the secret access key,

462.29 -> I’m gonna copy its value from the aws console,

466.07 -> and paste it to the file.

467.22 -> Alright, let’s save this file,

470.25 -> Now I believe it will work.

471.93 -> Let’s try running the kubectl get pods command.

475.68 -> This time, there’s no authentication error as before.

478.81 -> It just say: no resources found in the default namespace.

482.24 -> That’s because we haven’t deployed anything to the cluster yet.

485.44 -> Let’s run the kubectl cluster-info command.

488.66 -> Here you go!

490.28 -> The info is successfully returned.

492.68 -> It contains the address of the control plane and core DNS of the cluster.

497.05 -> So it worked!

498.229 -> We can now access the cluster using the credentials of the root user.

501.31 -> Next, I’m gonna show you how to give github-ci user access to this cluster as well,

506.729 -> Because later we would want github to automatically deploy the app for us

510.449 -> Whenever we push new changes to the master branch.

513.009 -> But first, let’s learn how to tell aws cli to use the github credentials.

518.62 -> It’s pretty easy, we just need to export AWS_PROFILE=github

524.569 -> Note that the name of the profile must match the one we declare in the aws credentials

528.7 -> file.

529.91 -> Now if we run kubectl cluster-info again, it will fail because of Unauthorized access

535.509 -> as before.

537.279 -> If we want to get back to the default root user, just export AWS_PROFILE=default

542.459 -> Then this time, the cluster-info command will be successful again.

548.069 -> Alright, but how can we allow github user to access the cluster?

552.249 -> Well, to do that, we need to add this user to a special config map as shown in this example.

557.689 -> I’m gonna add this file to our simple-bank project.

561.839 -> So let’s open the project in visual studio code.

565.709 -> I will create a new folder eks to store all files related to kubernetes.

570.319 -> First, let’s add a new file called aws-auth.yaml

574.19 -> Next, we have to add the github-ci user to the map users section of this file.

579.949 -> So I’m gonna copy this example from step 7 of this tutorial.

584.07 -> And paste it to our aws-auth.yaml file.

587.509 -> We don’t need the map roles section, so let’s delete it.

590.779 -> Now, in the map users section,

592.92 -> We have to put the correct ARN of the github-ci user we want to give access to the cluster.

598.119 -> We can find its value in the IAM console,

600.619 -> Let’s open the Users page,

603.61 -> Select github-ci,

605.319 -> And click this button to copy its user ARN.

608.91 -> Then paste it here.

610.86 -> The username should be github-ci.

613.779 -> The groups should remain the same as system:masters.

616.869 -> Okay, now we need to apply this config map to the RBAC configuration of the cluster.

622.79 -> Let’s run this kubectl apply command in the terminal,

626.8 -> But we need to change the location of the file to eks/aws-auth.yaml

631.959 -> There’s a warning here, but it’s OK because this is the first time we apply new configuration.

637.889 -> As it clearly says here, the missing annotation will be patched automatically.

644.089 -> So let’s try switching to the github profile,

647.16 -> And run the kubectl cluster-info command.

649.649 -> Oh, we still got the same error: unauthorized.

653.35 -> So what happened?

654.35 -> Let’s go back to check the config map.

656.759 -> OK, I see.

658.459 -> Looks like we’ve got a typo here in the user ARN value.

661.95 -> The correct value should not have a duplicate ARN prefix.

665.749 -> So let’s fix it!

667.97 -> Then go back to the terminal

669.869 -> And apply the change again.

671.32 -> Oops, I forgot that we’re still using the github-ci profile.

674.999 -> We must change to the default root profile first.

678.899 -> Then apply the change again.

681.05 -> This time, it is successful.

683.069 -> Now let’s switch back to the github-ci profile.

685.98 -> And run kubectl cluster-info command one more time.

689.689 -> This time it worked!

691.249 -> Excellent!

692.36 -> So now you know how to grant access to the cluster

694.72 -> to a user who’s not the creator of the cluster.

697.899 -> This will be useful when we want to set up continuous deployment process later.

702.839 -> Now before we finish, I’m gonna show you 1 more tool,

705.26 -> Which, I think, is very cool and make it so much easier to interact with the kubenetes

709.88 -> cluster.

710.88 -> As you’ve already known, the normal way to interact with the cluster is using kubectl

715.119 -> tool,

716.119 -> For example, to get the services that are running on the cluster,

719.97 -> We run kubectl get service.

722.089 -> Or to get the list of running pods,

724.67 -> we use kubectl get pods command.

727.639 -> For short and simple commands like that, it’s ok to type,

731.44 -> But it would become very annoying when you have to run longer and more complex commands

736.429 -> That requires you to copy around the name or id of the services or pods.

741.22 -> To make it easier to interact with the cluster, we can use a tool called k9s.

745.779 -> It gives us a very nice user interface

748.679 -> and a set of convenient shortcuts to talk to the kubernetes cluster.

753.459 -> If you’re on a mac, we can install it with home brew.

756.98 -> Let’s run brew install k9s in the terminal.

759.769 -> After it’s successfully installed, we can run just k9s to access the cluster.

765.709 -> As you can see, a nice UI shows up,

768.129 -> and it’s currently listing all pods of the cluster.

771.11 -> They are 2 core-dns pods of the kube-system.

775.11 -> Because of some reasons, their status is still pending,

778.319 -> But we will deal with it later, in the next lecture.

781.47 -> For now, I’m gonna show you some useful shortcut commands

784.509 -> to navigate the cluster resources.

786.74 -> To switch the namespace, just type a colon, then ns, enter.

791.949 -> It will show you all available namespaces of the cluster.

796.329 -> You can use the arrows to select the namespace you want to access,

799.66 -> And simply press enter to go into that namespace.

804.16 -> And to get out to the previous screen, just press escape.

808.56 -> Similarly, to list all services, we type a colon, followed by service, then enter.

816.749 -> To list all pods, just type a colon, then pods, and enter.

822.029 -> If you have cronjobs running on the cluster,

825.459 -> You can list them out by typing a colon, followed by cj, enter.

831.579 -> Or to list all available nodes, just type a colon, then nodes, enter.

837.019 -> For now there’s no node in the cluster.

839.689 -> Perhaps that’s why the core-dns services cannot start, and stay in pending state.

844.55 -> Also, there are several more commands that are listed here on the UI,

849.92 -> For example, To delete a resource, press control d.

853.93 -> Or to describe a resource, just press d.

858.179 -> You will see all the information of your chosen resource.

862.179 -> And you can always use the escape key to go back to the previous screen.

867.35 -> Now let’s take a look at the config map that we’ve updated before.

870.8 -> It’s the aws-auth config map in the kube-system namespace.

875.889 -> If we press d to describe it,

878.54 -> We will see the github-ci user in the map users section.

882.22 -> And it belongs to the system:masters group, exactly as we wanted.

887.509 -> Now press escape to go back to the previous screen

891.439 -> And finally use the quit command to exit from k9s.

895.6 -> And that’s it for today’s lecture.

897.259 -> I hope you find it useful.

898.779 -> Thanks a lot for watching,

900.85 -> Happy learning, and see you in the next lecture!

Source: https://www.youtube.com/watch?v=hwMevai3_wQ