testnet
Hyperlane

Run Validators

Run Hyperlane validator and relayer agents for the BitSong crescendo-1 testnet to enable cross-chain message delivery.

This guide walks you through running Hyperlane validator and relayer agents that power cross-chain message delivery between BitSong and Base Sepolia. It follows the official Hyperlane validator documentation, adapted for the BitSong testnet.

Two deployment configurations are covered:

  • Local (Testnet) — Hex private keys with localStorage checkpoint storage. Simplest to set up, suitable for testing.
  • AWS (Production) — AWS KMS signing keys with S3 checkpoint storage. Hardware-backed security, recommended for production.

Prerequisites

Before running agents, ensure you have completed the previous steps:

  1. Agent keys generated and exported as environment variables
  2. Hyperlane bridge setup completed (all 7 on-chain transactions)
  3. Docker installed and running (see install Docker below)
  4. Foundry installed (provides cast)
  5. A funded Cosmos signer account on BitSong (see agent keys guide)

Install Docker

The Hyperlane agents run as Docker containers. Install Docker Engine on your server if you don't have it already:

Terminal
# Remove any old Docker packages
sudo apt-get remove -y docker docker-engine docker.io containerd runc 2>/dev/null

# Install prerequisites
sudo apt-get update
sudo apt-get install -y ca-certificates curl gnupg

# Add Docker's official GPG key
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg

# Add the Docker repository
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

# Install Docker Engine
sudo apt-get update
sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin

After installation, add your user to the docker group so you can run containers without sudo:

Terminal
sudo usermod -aG docker $USER
newgrp docker

Verify Docker is working:

Terminal
docker --version
# Docker version 28.x.x, build ...

docker run --rm hello-world
# Hello from Docker!
If you're on a different Linux distribution, follow the official Docker Engine installation guide for your OS.

System Requirements

ResourceMinimum
CPU2 cores
RAM2 GB
Storage4 GB
NetworkStable connection to both BitSong RPC and Base Sepolia RPC

Pre-Launch Checklist

Before starting the agents, verify every item:

  • :checked-box: Signing keys created — hex keys exported, or AWS KMS key created and verified (agent keys guide)
  • :checked-box: Checkpoint storage configured — local directories created, or S3 bucket set up with public read access (S3 setup)
  • :checked-box: Cosmos signer account funded with TBTSG — the validator cannot announce without gas tokens
  • :checked-box: Validator address funded with ETH on Base Sepolia — the Base Sepolia validator needs gas for on-chain announcements
  • :checked-box: Relayer key funded with ETH on Base Sepolia — the relayer pays gas on the destination chain
  • :checked-box: RPC endpoints reachable — use private RPC URLs, never public endpoints in production
  • :checked-box: Bridge setup completed — all 7 transactions from the setup guide succeeded
Need Base Sepolia ETH? Use the Coinbase Developer Platform faucet to fund both your validator and relayer addresses.

Required Environment Variables

# Agent keys (from the agent keys guide)
export VALIDATOR_KEY="0x<your-validator-private-key>"
export COSMOS_SIGNER_KEY="0x<your-cosmos-signer-key>"
export EVM_RELAYER_KEY="0x<your-relayer-key>"

# Derived addresses
export VALIDATOR_ADDR=$(cast wallet address --private-key $VALIDATOR_KEY)

# Chain configuration
export CHAIN_ID="crescendo-1"
export DOMAIN_ID="7171"
export REMOTE_DOMAIN="84532"
export NODE="tcp://localhost:26657"
export EVM_RPC="https://sepolia.base.org"
The AWS_REGION environment variable is required for all AWS operations. Without it, the agent fails with Invalid Configuration: Missing Region.

Pull the Agent Docker Image

Terminal
docker pull --platform linux/amd64 gcr.io/abacus-labs-dev/hyperlane-agent:agents-v2.0.0

Agent Configuration

The Hyperlane agents need a JSON configuration file that describes both chains in the bridge. Create this file using the IDs from your bridge setup.

Replace the placeholder values (<mailbox-id>, <merkle-hook-id>, <igp-id>) with the actual hex IDs returned during the bridge setup.

If you didn't save the IDs from the setup guide, you can query them from the chain:
Terminal
# Mailbox ID (also used for validatorAnnounce)
bitsongd query hyperlane mailboxes --output json --node $NODE | jq '.mailboxes'

# MerkleTreeHook ID
bitsongd query hyperlane hooks merkle-tree-hooks --output json --node $NODE | jq '.merkle_tree_hooks'

# IGP ID
bitsongd query hyperlane hooks igps --output json --node $NODE | jq '.igps'
The Base Sepolia values are official Hyperlane contracts from the Hyperlane registry — they are already filled in and do not need to be changed.

Create the directory and save the agent configuration file:

Terminal
mkdir -p $HOME/.bitsongd/hyperlane
$HOME/.bitsongd/hyperlane/agent-config.json
{
  "chains": {
    "bitsong": {
      "name": "bitsong",
      "chainId": "crescendo-1",
      "domainId": 7171,
      "protocol": "cosmosNative",
      "bech32Prefix": "bitsong",
      "slip44": 639,
      "contractAddressBytes": 32,
      "canonicalAsset": "utbtsg",
      "rpcUrls": [{ "http": "http://localhost:26657" }],
      "grpcUrls": [{ "http": "http://localhost:9090" }],
      "nativeToken": {
        "name": "BitSong",
        "symbol": "BTSG",
        "decimals": 6,
        "denom": "utbtsg"
      },
      "gasPrice": { "amount": "0.025", "denom": "utbtsg" },
      "gasMultiplier": "1.5",
      "blocks": {
        "confirmations": 1,
        "estimateBlockTime": 6,
        "reorgPeriod": 1
      },
      "index": { "from": 1, "chunk": 50 },
      "mailbox": "<mailbox-id>",
      "validatorAnnounce": "<mailbox-id>",
      "merkleTreeHook": "<merkle-hook-id>",
      "interchainGasPaymaster": "<igp-id>"
    },
    "basesepolia": {
      "name": "basesepolia",
      "chainId": 84532,
      "domainId": 84532,
      "protocol": "ethereum",
      "rpcUrls": [{ "http": "https://sepolia.base.org" }],
      "nativeToken": {
        "name": "Ether",
        "symbol": "ETH",
        "decimals": 18
      },
      "blocks": {
        "confirmations": 1,
        "estimateBlockTime": 2,
        "reorgPeriod": 1
      },
      "index": { "from": 13850000, "chunk": 9999 },
      "mailbox": "0x6966b0E55883d49BFB24539356a2f8A673E02039",
      "validatorAnnounce": "0x20c44b1E3BeaDA1e9826CFd48BeEDABeE9871cE9",
      "merkleTreeHook": "0x86fb9F1c124fB20ff130C41a79a432F770f67AFD",
      "interchainGasPaymaster": "0x28B02B97a850872C4D33C3E024fab6499ad96564"
    }
  }
}

Save this file to $HOME/.bitsongd/hyperlane/agent-config.json.

The validatorAnnounce field for BitSong uses the same ID as the mailbox. This is because on Cosmos-native Hyperlane, the mailbox module handles validator announcements internally.
Cosmos chains require bothrpcUrls and grpcUrls. EVM chains only need rpcUrls. If you omit grpcUrls for BitSong, the agent will fail to start.

Configuration Fields

bitsong.index.from
integer
Block height to start indexing from. Use 1 for a fresh chain.
bitsong.index.chunk
integer
Number of blocks to query per batch. 50 is recommended for Cosmos chains.
basesepolia.index.from
integer
Block height to start indexing Base Sepolia from. Should be near the Hyperlane mailbox deployment block (13850000). Setting this too high causes the relayer to miss earlier messages.
gasMultiplier
string
Gas multiplier for Cosmos transactions. 2.0 provides margin for gas estimation variance.
rpcUrls
array
List of RPC endpoints. For production, use private RPC URLs and configure multiple for redundancy. Never use public endpoints for production validators.

How Cross-Chain Validation Works

Before starting the agents, it helps to understand how messages are validated in each direction:

  • BitSong → Base Sepolia: The BitSong validator signs checkpoints over messages dispatched from BitSong. On the EVM side, a custom MultisigISM (deployed in the Warp Token guide) verifies these checkpoints before the relayer can deliver the message.
  • Base Sepolia → BitSong: The Base Sepolia validator signs checkpoints over messages dispatched from Base Sepolia. On the Cosmos side, the RoutingISM configured during bridge setup verifies these checkpoints.

This is why you must run two validator instances — one monitoring each origin chain. Each validator produces checkpoints that the destination chain's ISM uses for verification.

Start the Validator

The validator agent monitors the BitSong chain for dispatched messages, signs checkpoints, and stores them for relayers to read. You need one validator instance per origin chain.

BitSong Validator

mkdir -p $HOME/.bitsongd/hyperlane/validator-bitsong-db $HOME/.bitsongd/hyperlane/checkpoints-bitsong

docker run -d \
    --name hyperlane-validator-bitsong \
    --network host \
    --user $(id -u):$(id -g) \
    -e CONFIG_FILES=/config/agent-config.json \
    -v $HOME/.bitsongd/hyperlane/agent-config.json:/config/agent-config.json:ro \
    -v $HOME/.bitsongd/hyperlane/validator-bitsong-db:/hyperlane_db \
    -v $HOME/.bitsongd/hyperlane/checkpoints-bitsong:/checkpoints-bitsong \
    gcr.io/abacus-labs-dev/hyperlane-agent:agents-v2.0.0 \
    ./validator \
    --db /hyperlane_db \
    --originChainName bitsong \
    --reorgPeriod 1 \
    --interval 5 \
    --validator.type hexKey \
    --validator.key $VALIDATOR_KEY \
    --chains.bitsong.signer.type cosmosKey \
    --chains.bitsong.signer.key $COSMOS_SIGNER_KEY \
    --chains.bitsong.signer.prefix bitsong \
    --checkpointSyncer.type localStorage \
    --checkpointSyncer.path /checkpoints-bitsong
The BitSong validator always uses cosmosKey for the chain signer (to sign on-chain announcement transactions) regardless of whether the validator checkpoint signer uses hex or AWS KMS. The cosmosKey type uses the bitsong prefix to derive the correct Bech32 address.

Base Sepolia Validator

The Base Sepolia validator needs ETH on Base Sepolia to pay for the on-chain announcement transaction. Fund your validator address ($VALIDATOR_ADDR) using the Coinbase faucet before starting.
mkdir -p $HOME/.bitsongd/hyperlane/validator-basesepolia-db $HOME/.bitsongd/hyperlane/checkpoints-basesepolia

docker run -d \
    --name hyperlane-validator-basesepolia \
    --network host \
    --user $(id -u):$(id -g) \
    -e CONFIG_FILES=/config/agent-config.json \
    -v $HOME/.bitsongd/hyperlane/agent-config.json:/config/agent-config.json:ro \
    -v $HOME/.bitsongd/hyperlane/validator-basesepolia-db:/hyperlane_db \
    -v $HOME/.bitsongd/hyperlane/checkpoints-basesepolia:/checkpoints-basesepolia \
    gcr.io/abacus-labs-dev/hyperlane-agent:agents-v2.0.0 \
    ./validator \
    --db /hyperlane_db \
    --originChainName basesepolia \
    --reorgPeriod 1 \
    --interval 5 \
    --validator.type hexKey \
    --validator.key $VALIDATOR_KEY \
    --chains.basesepolia.signer.type hexKey \
    --chains.basesepolia.signer.key $EVM_RELAYER_KEY \
    --checkpointSyncer.type localStorage \
    --checkpointSyncer.path /checkpoints-basesepolia
The Base Sepolia validator uses hexKey (local) or aws (production) for both the checkpoint signer and the chain transaction signer. Unlike BitSong, EVM chains do not need a separate Cosmos signer.
When using AWS KMS with S3, you can use the same KMS key and the same S3 bucket for both validators. Differentiate them by using unique --checkpointSyncer.folder values (bitsong vs basesepolia).

Validator CLI Reference

--originChainName
string required
The chain this validator monitors. Run one validator instance per chain.
--db
string required
Path to the persistent database directory. Each validator instance must have its own database path.
--reorgPeriod
integer
Number of block confirmations before signing a checkpoint. 1 is appropriate for testnet. For production, match the chain's finality guarantees.
--interval
integer
Polling interval in seconds. 5 balances responsiveness with RPC load.
--validator.type
string required
Signing key type for checkpoints. hexKey for local hex private keys, aws for AWS KMS.
--validator.key
string
The hex private key for checkpoint signing. Required when --validator.type is hexKey.
--validator.region
string
AWS region of the KMS key. Required when --validator.type is aws.
--validator.id
string
AWS KMS key ID or alias (e.g. alias/hyperlane-validator-signer-bitsong). Required when --validator.type is aws.
--checkpointSyncer.type
string required
Storage backend for signed checkpoints. localStorage for local testing, s3 for production with AWS S3.
--checkpointSyncer.path
string
Local directory path for checkpoints. Required when --checkpointSyncer.type is localStorage.
--checkpointSyncer.bucket
string
S3 bucket name. Required when --checkpointSyncer.type is s3.
--checkpointSyncer.region
string
AWS region of the S3 bucket. Required when --checkpointSyncer.type is s3.
--checkpointSyncer.folder
string
Folder path inside the S3 bucket. Use different folders for each chain when sharing a bucket. Required when --checkpointSyncer.type is s3.
--chains.[chainName].signer.type
string
Transaction signer type. hexKey for EVM hex keys, cosmosKey for Cosmos chains, aws for AWS KMS.
--chains.[chainName].customRpcUrls
string
Override RPC URLs from the agent config. Comma-separated list of URLs. Useful for configuring private or redundant endpoints.

Wait for Validator Announcements

After starting, validators automatically announce their checkpoint storage locations on-chain. This is how relayers discover where to read signed checkpoints. The announcement requires the Cosmos signer account to have a TBTSG balance for gas fees.

Wait for announcements (this can take up to 2 minutes):

Terminal
# Check BitSong validator announcement
bitsongd query hyperlane ism announced-storage-locations \
    <mailbox-id> $(echo $VALIDATOR_ADDR | tr '[:upper:]' '[:lower:]') \
    --output json --node $NODE | jq '.storage_locations'

A non-empty storage_locations array confirms the validator has announced successfully. If the array is empty after 2 minutes, check the validator logs and ensure the Cosmos signer account is funded.

Start the Relayer

The relayer reads signed checkpoints from validators and delivers messages to their destination chains. It bridges messages in both directions between BitSong and Base Sepolia.

The relayer needs ETH on Base Sepolia to pay gas when delivering messages to the EVM chain. Fund your relayer address using the Coinbase faucet before starting.
mkdir -p $HOME/.bitsongd/hyperlane/relayer-db

docker run -d \
    --name hyperlane-relayer \
    --network host \
    --user $(id -u):$(id -g) \
    -e CONFIG_FILES=/config/agent-config.json \
    -v $HOME/.bitsongd/hyperlane/agent-config.json:/config/agent-config.json:ro \
    -v $HOME/.bitsongd/hyperlane/relayer-db:/hyperlane_db \
    -v $HOME/.bitsongd/hyperlane/checkpoints-bitsong:/checkpoints-bitsong:ro \
    -v $HOME/.bitsongd/hyperlane/checkpoints-basesepolia:/checkpoints-basesepolia:ro \
    gcr.io/abacus-labs-dev/hyperlane-agent:agents-v2.0.0 \
    ./relayer \
    --db /hyperlane_db \
    --relayChains bitsong,basesepolia \
    --allowLocalCheckpointSyncers true \
    --gaspaymentenforcement '[{"type": "none"}]' \
    --chains.bitsong.signer.type cosmosKey \
    --chains.bitsong.signer.key $COSMOS_SIGNER_KEY \
    --chains.bitsong.signer.prefix bitsong \
    --chains.basesepolia.signer.type hexKey \
    --chains.basesepolia.signer.key $EVM_RELAYER_KEY \
    --metricsPort 9091
The --allowLocalCheckpointSyncers true flag is required when validators use localStorage for checkpoint storage. It lets the relayer read checkpoints from the local filesystem. When using S3, this flag is not needed — the relayer reads directly from the S3 bucket.
The --gaspaymentenforcement '[{"type": "none"}]' flag disables gas payment enforcement for testnet. In production, configure this to enforce IGP payments so relayers are compensated.
The relayer always uses cosmosKey for the BitSong chain signer (same as the validator), since BitSong is a Cosmos-native chain. For Base Sepolia, use hexKey (local) or aws (production).

Relayer Whitelist

To limit the relayer to only relay your warp token messages (avoiding noise from other projects on shared testnets), add a whitelist:

Terminal
# Convert your EVM HypERC20 contract address to bytes32
EVM_HYP_ERC20="0x<your-evm-hyp-erc20-address>"
EVM_PADDED=$(printf "0x%064s" "${EVM_HYP_ERC20#0x}" | tr ' ' '0')

# Add --whitelist flag to the relayer docker run command:
--whitelist "[{\"senderAddress\":\"<token-id>\",\"destinationDomain\":\"$REMOTE_DOMAIN\"},{\"senderAddress\":\"$EVM_PADDED\",\"destinationDomain\":\"$DOMAIN_ID\"}]"

Replace <token-id> with the warp token ID from the bridge setup.

Running Multiple Validators

If you are running validators for multiple operators or coordinating a multi-validator setup:

  • A single signing key can serve multiple validator instances
  • A shared AWS account is permitted across validators
  • A shared S3 bucket is allowed — use unique --checkpointSyncer.folder values to separate checkpoint data (e.g. validator-1, validator-2)
  • Each validator instance must have its own --db path, unique metrics port, and separate log output

Manage Agents

Check agent status

Terminal
docker ps --filter "name=hyperlane" --format "table {{.Names}}\t{{.Status}}\t{{.Ports}}"

View logs

docker logs -f --tail 100 hyperlane-validator-bitsong

Verify checkpoints are being written

# BitSong checkpoints
ls -la $HOME/.bitsongd/hyperlane/checkpoints-bitsong/

# Base Sepolia checkpoints
ls -la $HOME/.bitsongd/hyperlane/checkpoints-basesepolia/

New checkpoint files are written every time a new outbound message is inserted into the mailbox.

Stop all agents

Terminal
docker stop hyperlane-validator-bitsong hyperlane-validator-basesepolia hyperlane-relayer

Restart agents

Terminal
docker start hyperlane-validator-bitsong hyperlane-validator-basesepolia hyperlane-relayer

Remove agents (preserves data)

Terminal
docker rm hyperlane-validator-bitsong hyperlane-validator-basesepolia hyperlane-relayer

Troubleshooting

Next Steps

Agent Keys Reference

Review key management options including AWS KMS and S3 checkpoint storage for production environments.

Bridge Setup

Review the on-chain bridge setup including ISM, Mailbox, and warp token configuration.

Hyperlane Setup

Configure the Hyperlane interoperability bridge on the BitSong crescendo-1 testnet to enable cross-chain messaging and token transfers.

Warp Token Deployment

Deploy a cross-chain warp route between BitSong and Base Sepolia using Hyperlane.

Copyright © 2026