ssh.md

VM SSH Setup

• Create an ssh key in your local system in the .ssh folder - Guide

• Add the public key (.pub) to your VM instance - Guide

• Create a config file in your .ssh folder

  touch ~/.ssh/config

• Copy the following snippet and replace with External IP of the Kafka, Spark (Master Node), Airflow VMs. Username and path to the ssh private key

  Host quantify-kafka
      HostName <External IP Address>
      User <username>
      IdentityFile <path/to/home/.ssh/keyfile>
  
  Host quantify-spark
      HostName <External IP Address Of Master Node>
      User <username>
      IdentityFile <path/to/home/.ssh/keyfile>
  
  Host quantify-airflow
      HostName <External IP Address>
      User <username>
      IdentityFile <path/to/home/.ssh/gcp>

• Once you are setup, you can simply SSH into the servers using the below commands in separate terminals. Do not forget to change the IP address of VM restarts.

  ssh quantify-kafka

  ssh quantify-spark

  ssh quantify-airflow

• You will have to forward ports from your VM to your local machine for you to be able to see Kafka, Airflow UI. Check how to do that here