THORChain Finance
Developers
Chain Clients
THORNodes
Get Involved
Setup - Hetzner Bare Metal
Setting up a Kubernetes Cluster with Hetzner Dedicated Servers
This guide for Hetzner Bare Metal is WIP and not currently recommended. Proceed with caution until an update is released and this warning removed.
The scripts in this repository will setup and maintain one or more kubernetes clusters consisting of dedicated Hetzner servers. Each cluster will also be provisioned to operate as a node in the THORCHain network.
Executing the scripts in combination with some manual procedures will get you highly available, secure clusters with the following features on bare metal.
Preparations
Servers
Acquire a couple of servers as the basis for a cluster (
AX41-NVME's are working well for instance). Visit the admin panel and name the servers appropriately.1
tc-k8s-node1
2
tc-k8s-node2
3
tc-k8s-node3
4
...
5
6
tc-k8s-master1
7
tc-k8s-master2
8
tc-k8s-worker1
9
tc-k8s-worker2
10
tc-k8s-worker3
11
...
Copied!
vSwitch
Create a vSwitch and order an appropriate subnet (it may take a while to show up after the order). Give the vSwitch a name (i.e.
tc-k8s-net) and assign this vSwitch to the servers.Usage
Clone this repository,
cd into it and download kubespray.1
git submodule init && git submodule update
Copied!
Create a Python virtual environment or similar.
1
# Optional
2
virtualenv -p python3 venv
Copied!
Install dependencies required by Python and Ansible Glaxy.
1
pip install -r requirements.python.txt
2
ansible-galaxy install -r requirements.ansible.yml
Copied!
Note: Mitogen does not work with ansible collections and the strategy must be changed (i.e.strategy: linear).
Provisioning
Create a deployment environment inventory file for each cluster you want to manage.
1
cp hosts.example inventory/production.yml
2
cp hosts.example inventory/test.yml
3
cp hosts.example inventory/environment.yml
4
...
5
6
cp hosts.example inventory/production-01.yml
7
cp hosts.example inventory/production-02.yml
8
...
9
10
cp hosts.example inventory/production-helsinki.yml
11
cp hosts.example inventory/whatever.yml
Copied!
Edit the inventory file with your server ip's and network information and customize everything to your needs.
1
# Manage a cluster
2
ansible-playbook cluster.init.yml -i inventory/environment.yml
3
ansible-playbook --become --become-user=root kubespray/cluster.yml -i inventory/environment.yml
4
ansible-playbook cluster.finish.yml -i inventory/environment.yml
5
6
# Run custom playbooks
7
ansible-playbook private-cluster.yml -i inventory/environment.yml
8
ansible-playbook private-test-cluster.yml -i inventory/environment.yml
9
ansible-playbook private-whatever-cluster.yml -i inventory/environment.yml
Copied!
THORChain
In order for the cluster to operate as a node in the THORCHain network deploy as instructed here. You can also refer to the node-launcher repository, if necessary, or the THORChain documentation as a whole.
Resetting the bare metal servers
This will install and use Ubuntu 20.04 on only one of the two internal NVMe drives. The unused ones will be used for persistent storage with ceph/rook. You can check the internal drive setup with
lsblk. Change it accordingly in the command shown above when necessary.Manually
Visit the console and put each server of the cluster into rescue mode. Then execute the following script.
1
installimage -a -r no -i images/Ubuntu-2004-focal-64-minimal.tar.gz -p /:ext4:all -d nvme0n1 -f yes -t yes -n hostname
Copied!
Automatically
Create a pristine state by running the playbooks in sequence.
1
ansible-playbook server.rescue.yml -i inventory/environment.yml
2
ansible-playbook server.bootstrap.yml -i inventory/environment.yml
Copied!
Instantiation
Instantiate the servers.
1
ansible-playbook server.instantiate.yml -i inventory/environment.yml
Copied!
Last modified 6mo ago
Export as PDF
Copy link
Edit on GitHub