Control Plane Deployment Instructions

Network Checklist

There is a checklist to acknowledge first for the Nimbus version that we are going to deploy today -
EKS cluster will be deployed behind a publicly accessible load balancer (not protected by VPN)
Workspaces will be created under public subnets of chosen VPCs. It will have port 22 (ssh) open to the public (protected by ssh keys)
Additional ports opened in the workspace by engineers will also be public facing
For future deployment versions, Nimbus is fully extensible to adapt any VPN and VPC/subnets setup

Deployment Steps

1. CloudFormation (15~20 min)

Note: the following steps require the AWS policies listed here:
Applying the CloudFormation template is quite straight-forward, though AWS will take a quite long time to provide all the resources needed
  1. Download the CloudFormation template above. Log in to your AWS Account. Navigate to CloudFormation > Stacks > Create stack > With new resources
  1. The template is ready > Upload a template file
  1. Input stack name
  1. Input pClusterName , pDBPassword, and pDomainName
What Will Be Created?
  1. A new VPC with public and private subnets
  1. Route53 HostedZone, for example, nimbus.company.dev, the web app will be accessible via nimbus.company.dev, and the workspace will be accessible via abcde.company.dropbox.dev
  1. A new EKS cluster
  1. Two IAM Roles
    1. One for the EKS cluster, which contains quite a standard policy
    2. The other one is for the EKS node group, which contains all permissions that Nimbus needs to operate (policy nimbus), and permissions to create a load balancer via helm chart (policy eks, which we will cover later)
How do I know everything went well?
CloudFormation will run for about 15 minutes to finish. Once it completes, you will see a screen like this:
Image without caption

2. Domain Certificate

Eventually, we’d love engineers to access the Nimbus app via an HTTPS URL in the web, so in this step, we will create a certificate for HTTPS for our Webapp application
  1. In the AWS console, navigate to Certificate Manager
  1. Click on the Request button
  1. Select “Request a public certificate”
  1. “Fully qualified domain name” ⇒ pDomainName in the CloudFormation step ”Select validation method” ⇒ DNS ”Tags” ⇒ Input any tags you use to track
  1. Click on the Request button
  1. On the certificate details page, find the “Domains” section. Once the “CNAME name” and ”CNAME value” becomes non-empty. Click on “Create records in Route53” > Create records
  1. While the status is pending validation, you can move to the next step
    1. Image without caption
Why not include the certificate creation in CloudFormation? It’s definitely feasible. However, the domain validation was a bit error-prone when we did testing on our end. If the validation got stuck or failed, the whole CloudFormation stack will be rolled back. With some failures in between, it can lead to a super long time just waiting for the resources to be ready

3. EKS

Once the CloudFormation stack creation is complete, we can move to set up the cluster.
  1. In your terminal (with the correct AWS credentials), run aws eks update-kubeconfig --region {region} --name {cluster-name} The cluster-namecan be found in the AWS console, or it is {pClusterName}-eks, where pClusterName is the input you use in the last step. This will update the kubeconfig in your local so that you have access to the cluster (as the creator);
  1. Make sure you have helm installed, otherwise, install helm following https://helm.sh/. Once helm is installed, run helm repo add eks https://aws.github.io/eks-charts helm repo add nimbus http://helm.usenimbus.com/ helm upgrade --install nimbus nimbus/nimbus -n nimbus --create-namespace --set Host=<domain> --set aws-load-balancer-controller.clusterName=<cluster-name> --set ingress.aws.enabled=true The cluster-name is the same one as above and domainis the pDomainName that you put in as a parameter at the CloudFormation step. This will create an application load balancer for the EKS cluster, and install helm charts for Nimbus.
  1. Go to Route 53 > Hosted zones > {your domain} > Create record Record name ⇒ Leave empty Record type ⇒ “A” Alias ⇒ Toggle on Route Traffic to ⇒ “Alias to Application and Classic Load Balancer” Choose Region ⇒ Your region Choose load balancer ⇒ Find the load balancer create in the last step To find the load balancer: EC2 > Load Balancing > Load Balancers > Find the one created recently, and with the tag “elbv2.k8s.aws/cluster”: “{cluster_name}”
    1. Image without caption
Image without caption

4. Create Secrets and DB Schema Migration

We will walk through this part in the live deployment session
kubectl -n nimbus get pods
kubectl exec -it nimbus-alpine-56f5f9cd64-rvrmv -n nimbus -- ./db_migrate

5. User Authentication

As we mentioned in the last meeting, at the moment, we will use Auth0 to authenticate user logins. We have created an Auth0 tenant, and configured it in the last step. So no action needed for this part

6. Connect Hosted Zone Name Servers with Your Domain Provider

If your team is using a domain provider other than Route53 (like Cloudflare), you will need to create NS records in your domain provider to make Route53 accessible. After this is done, the deployment is finished! Congrats and welcome to the Self-Hosted Nimbus!
Image without caption

Powered by Notaku