Keep Network, local setup

This post will guide you how to setup Keep Network’s local environment. It will follow the original guidance but in more details. At the time of writing this guide I used CentOS 8, so it aims that OS, but I tried to cover the other ones too with links.

Install prerequisites

Go

wget https://golang.org/dl/go1.15.8.linux-amd64.tar.gz
tar -C /usr/local -xzf go1.15.8.linux-amd64.tar.gz
export PATH=$PATH:/usr/local/go/bin
go version

This will download the tarball and place it in the /usr/local, so we can add the binaries to the path. Keep in mind that you have to add the export to the .bashrc, so it will do the PATH update on each bash start.

Geth

wget https://github.com/ethereum/go-ethereum/archive/v1.9.25.zip
unzip v1.9.25.zip
cd go-ethereum-1.9.25/
go get # requires gcc
go mod vendor
go install github.com/ethereum/go-ethereum/cmd/geth

At this point you have a compiled Geth, but there is a missing GOPATH bin from the PATH. Go places the GOPATH in your home folder so you can set it up with the following command:

export PATH=$PATH:$HOME/go/bin
geth version

Docker

sudo yum install -y yum-utils
sudo yum-config-manager \
--add-repo \
https://download.docker.com/linux/centos/docker-ce.repo
sudo yum install docker-ce docker-ce-cli containerd.io
sudo systemctl enable --now docker
# Verify installation
docker version
docker ps # should get the empty list of containers

Solidity compiler

Keep in mind that the submodules require Solidity compiler 0.5.17, so you have to modify the Dockerfile. For this let’s open the Dockerfile and modify the FROM part to ethereum/solc:0.5.17-alpine.

wget https://github.com/PumpkinSeed/solc-rhel/archive/v0.1.0.zip
unzip v0.1.0.zip
cd solc-rhel-0.1.0
sh install.sh
# Verify installation
solc --version
# Output:
# solc, the solidity compiler commandline interface
# Version: 0.5.17+commit.d19bba13.Linux.g++

jq

sudo yum -y install epel-release
sudo yum -y install jq
# Verify installation
jq -Version

Protobuf compiler

sudo yum install -y autoconf automake libtool curl make gcc-c++
wget https://github.com/protocolbuffers/protobuf/releases/download/v3.14.0/protobuf-all-3.14.0.zip
unzip protobuf-all-3.14.0.zip
cd protobuf-3.14.0/
./autogen.sh
./configure
make
make check
sudo make install
sudo ldconfig
# Verify installation
protoc --version
# libprotoc 3.14.0

The hardest parts of the installations are already done.

protoc-gen-gogoslick

go get github.com/gogo/protobuf/protoc-gen-gogoslick

docker-compose

sudo curl -L "https://github.com/docker/compose/releases/download/1.28.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-composesudo chmod +x /usr/local/bin/docker-compose# Verify installation
docker-compose --version
# docker-compose version 1.28.2, build 67630359

pyenv

sudo yum groupinstall -y "Development Tools"sudo yum install -y zlib zlib-devel bzip2-devel openssl-devel sqlite-devel readline-develcurl -L https://github.com/pyenv/pyenv-installer/raw/master/bin/pyenv-installer | bash# Load pyenv automatically by adding
# the following to ~/.bashrc:
export PATH="/root/.pyenv/bin:$PATH"
eval "$(pyenv init -)"
eval "$(pyenv virtualenv-init -)"

pipenv

sudo yum install -y python3-pip
pip3 install pipenv

At this point we have all of the necessary tools so we can start to bootstrap and setup the Keep Network. Before we jump into this, let’s install one other tool which can help us a lot. Note that it’s only necessary if we want to have control over all parts of the system.

tmux

sudo yum install -y tmux

Setting up the Keep Network local environment

git clone https://github.com/keep-network/local-setup.git
cd local-setup/

Installing the system

So open 3 panes. That should be enough for this session and we will initialize and start a local Ethereum and Bitcoin chain.

./initialize-geth.sh

This script sets up the Ethereum node data location. It will be under $PWD/ethereum/data. By using this variable it will call geth init with the genesis files under $PWD/ethereum/genesis/genesis.json. So the geth data dir is initialized. We can safely start the geth instance. We can do it in the same tmux pane where we had initialized it.

./run-geth.sh

This sets up the same data directory like initialize does. Then it will start the geth long running process. This means that a single-node ethereum runs on our machine. The script starts Websocket and RPC endpoints as well. The mining also starts because this will execute the actual transactions. More details can be found in the shell script.

Since the ethereum is a long running process we move to the second tmux pane and start the bitcoin chain.

./run-bitcoin.sh

At this point I don’t want to go into details but this script starts the docker-compose file under the bitcoin directory. Both Bitcoin daemon and ElectrumX have their own Dockerfile.

So we also get a long running process with this script so we have to move to the third tmux pane.

3 pane of the system

We can run the install shell script like I have done in the picture above. There is a small chance that you haven’t install npm on your system, but it’s required for this script.

sudo yum install -y npm
./install.sh
# ....
# You should see this line at the and with the list of submodules:
# Installation script executed successfully...

This will take time… (probably hours).

We want to check out what happens inside the script. We will see that the first one is updating the corresponding submodules. You can find these submodules in the .gitmodules file. Then it calls the submodule installers. Install keep-core, keep-ecdsa, tbtc, tbtc-dapp and keep-dashboard. Since it compiles solidity and does npm install with a huge amount of C++ compilation, it takes time.

So at this point we have ethereum and bitcoin local nodes and the bootstrapped keep system. We have to start the keep related subsystems.

Setup Keep Network clients locally

# 1. pane:
./run-core-1.sh
# 2. pane:
./run-ecdsa-1.sh
# 3. pane:
./run-ecdsa-2.sh
# 4. pane:
./run-ecdsa-3.sh
Running clients

That was easy to set up. If they are running without any problem you should be good. Note that keep-ecdsa can burn the CPU. It uses all of my cores with ~90% load.

Keep-ecdsa CPU usage

We have to do a genesis block on Keep network and then we can start the dashboard.

cd keep-coreKEEP_ETHEREUM_PASSWORD="password" ./keep-core --config configs/config.local.1.toml relay genesis

Going back to the root of the repository and run the dashboard.

cd keep-core/solidity/dashboard
npx browserslist@latest --update-db
cd ../..
./run-keep-dashboard.sh

This will start a web application on port 3000. If you open it you should see something like the image below.

Keep dashboard

The last things are the end to end tests. First we have to install and switch to a newer version of node by using the node version manager.

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.35.3/install.sh | bash
source ~/.bashrc
nvm install 14.3.0cd e2e && npm install
cd ..
./run-e2e-test.sh

Yes, I know the shell scripts also contain the npm install, but it always runs into issues until I run it myself.

Ideally it should pass and basically we can start the development.

It is hard to write a guide like this since there lots of moving parts. If you run into an issue at any point which I haven’t covered in this article, please feel free to message me on twitter.

https://twitter.com/PumpkinSeed_dev

Gopher, Rustacean, Hobby Hacker