The focus of this article is to explain how Azure Batch compute nodes existwith an Azure deployment, interactive SSH, and the concept of SSH tunnelingto a Docker Host on an Azure Batch compute node from your local machine.
Before adding a new SSH key to the ssh-agent to manage your keys, you should have checked for existing SSH keys and generated a new SSH key. When adding your SSH key to the agent, use the default macOS ssh-add command, and not an application installed by macports. I have connected to a server via SFTP using FileZilla and accepted adding the server's SSH key to the key cache in FileZilla. How can I extract this cached key to a keyfile so that may use it through other SFTP applications that require a keyfile be made available? Dec 02, 2019 SSH (Secure Shell) keys are an access credential that is used in the SSH protocol. Read the rest of this post to learn more about what are SSH keys or consider watching webinar below to find out more about the SSH protocol and the basics of SSH authentication.
Azure Batch compute nodes which comprise a pool are behind a NAT/load balancerwhich have certain endpoints exposed on the public IP of the deployment tospecific instances (i.e., compute nodes).
For instance, port 12345 may map to port 22 of the first instance of acompute node in the pool for the public IP address 1.2.3.4. The next computenode in the pool may have port 22 mapped to port 12346 on the load balancer.
This allows many compute nodes to sit behind one public IP address.
In order to use SSH, you will need to generate a public/private RSA keypairthat SSH requires for asymmetric key authentication. If you are runningBatch Shipyard on Linux/Mac (orWindows withssh-keygen
accessible in your %PATH%
or current working directory), youcan opt to leave ssh_public_key
and ssh_private_key
unspecified or emptyin ssh
configuration blocks and Batch Shipyard will automatically generatethe keypair for you. Alternatively, you can specify the location ofpre-generated keypairs that you may have on your system.
On Windows, if you don't have ssh-keygen
available as per above, you canuse PuTTYgento pre-generate public/private keys and then specify the file path inssh_public_key
and ssh_private_key
in ssh
configuration blocks. Tocreate compatible keys for use with Batch Shipyard, perform the followingactions:
Generate
button on the bottom rightConversions
file menu at the topExport OpenSSH key
Yes
.Public key for pasting into OpenSSH authorized_keys file:
and pressingCTRL+C or right-click and Copy.ssh
config property namedssh_public_key_data
with the key data from the boxssh
config property named ssh_public_key
and point it to the fileBy adding an SSH user to the pool (which can be automatically done for youvia the ssh
block in the pool config upon pool creation or through thepool user add
command), you can interactively log in to compute nodes in thepool and execute any command on the remote machine, including Dockercommands via sudo
.
You can utilize the pool ssh
command to automatically connect to anycompute node in the pool without having to manually resort to pool nodes grls
and issuing the ssh
command with the appropriate parameters. If you havethe SSH private key in the default location or as specified in thegenerated_file_export_path
, then an interactive SSH session will becreated to the compute node specified.
pool ssh
can accept either option --cardinal
or the option --nodeid
.If using --cardinal
it requires the natural counting number from zeroassociated with the list of nodes as enumerated by pool nodes grls
. If using--nodeid
, then the exact compute node id within the pool specified inthe pool config must be used. If neither option is specified, the defaultis --cardinal 0
. For example:
would create an interactive SSH session with the first compute node in thepool as listed by pool nodes grls
.
To take advantage of this feature, you must install Docker locally on yourmachine and have ssh
available. You can find guides to install Dockeron various operating systems here.
The typical recommendation is to secure the Docker daemon if beingaccessed remotely via certificates and TLS. Because SSH is already configuredon all of the nodes with authorized users to use the Docker daemon withBatch Shipyard, we can simply use SSH tunneling instead which simplifiesthe process and is less likely to be blocked in outbound firewall rules.This method is secure as the tunnel is opened and encrypted via ssh
witha public/private RSA key pair. Please note that the Docker daemon portis not mapped on the NAT/load balancer, so it is impossible to connect tothe port remotely without an SSH tunnel.
By specifying generate_docker_tunnel_script
as true
in the ssh
configuration block in the pool config, a file namedssh_docker_tunnel_shipyard.sh
will be generated on pool add
if anSSH user is specified, on pool user add
when a pool user is added, onpool resize
when a pool is resized, or on pool nodes grls
when a pool'sremote login settings are listed.
This script simplifies creating an SSH tunnel to the Docker socket fromyour local machine. It accepts a cardinal number of the node to connectto, similar to the --cardinal
option for pool ssh
. So if you wereconnecting to the first node in the pool, you would execute the dockertunnel script as:
This will background the SSH tunnel to the remote Docker daemon and outputsomething similar to the following:
Now you can run the docker
command locally but have these actionswork remotely through the tunnel on the compute node with the appropriate-H
option as noted above. For instance:
Microsoft word 2010 professional plus product key generator. would place the current shell context inside the busybox container runningremotely on the Batch compute node.
Alternatively you can export an environment variable named DOCKER_HOST
which will work for all docker
invocations until the environment variableis unset. For example:
would create a busybox container on the remote compute node similar tothe prior command.
To run a CUDA/GPU enabled docker image remotely with nvidia-docker, first youmust installnvidia-docker locallyin addition to docker as per the initial requirement. You can installnvidia-docker locally even without an Nvidia GPU or CUDA installed. It issimply required for the local command execution. If you do not have an NvidiaGPU available and install nvidia-docker
you will most likely encounter anerror with the nvidia docker service failing to start, but this is ok. Youcan then launch your CUDA-enabled Docker image on the remote compute nodeon Azure N-series VMs the same as any other Docker image except invokingwith the nvidia-docker
command instead:
Once you are finished with running your docker
and/or nvidia-docker
commands remotely, you can terminate the SSH tunnel by sending a SIGTERM tothe SSH tunnel process. In this example, the pid is 22204 as displayed bythe script, thus we would terminate the SSH tunnel with the following:
Finally, please remember that the ssh_docker_tunnel_shipyard.sh
scriptis generated and is specific for the pool as specified in the poolconfiguration file at the time of pool creation, resize, when an SSH useris added or when the remote login settings are listed.