Monitoring
Once your node is running, you should regularly verify it's healthy and syncing correctly. This guide covers the essential monitoring commands and common troubleshooting steps.
Check Sync Status
The most important check — verify your node is syncing and not falling behind:
curl -s http://localhost:26657/status | jq '.result.sync_info'
Key fields to monitor:
| Field | Healthy Value |
|---|---|
catching_up | false (fully synced) |
latest_block_height | Increasing over time |
latest_block_time | Recent (within last few seconds) |
catching_up is true, your node is still syncing. Wait for it to reach the latest block before creating a validator.Verify Network Connectivity
Check listening ports
Verify your node is listening on the expected ports:
# Check all bitsongd listening ports
ss -tlnp | grep bitsongd
Expected ports:
| Port | Service | Required |
|---|---|---|
26656 | P2P | Yes — node-to-node communication |
26657 | RPC | Yes — CometBFT RPC |
1317 | REST API | Optional — Cosmos SDK LCD |
9090 | gRPC | Optional — Cosmos SDK gRPC |
Check connected peers
curl -s http://localhost:26657/net_info | jq '.result.n_peers'
If the peer count is 0, your node cannot discover the network. Verify your persistent_peers configuration.
Check Validator Status
If you've created a validator, verify it's active and signing blocks:
# Check if your validator is in the active set
bitsongd query staking validators --output json | \
jq '.validators[] | select(.description.moniker == "<your-moniker>") | {moniker: .description.moniker, status: .status, jailed: .jailed, tokens: .tokens}'
Validator Status Values
| Status | Meaning |
|---|---|
BOND_STATUS_BONDED | Active — signing blocks |
BOND_STATUS_UNBONDING | Unbonding — leaving the active set |
BOND_STATUS_UNBONDED | Inactive — not signing blocks |
Check signing info
bitsongd query slashing signing-info $(bitsongd tendermint show-validator)
This shows your validator's missed block count and jailing status.
Node Health Checks
Check node version
bitsongd version --long
Check available disk space
df -h ~/.bitsongd
Check node logs
# If running in background with nohup
tail -100 bitsongd.log
# Follow logs in real time
tail -f bitsongd.log
# View recent logs
docker logs --tail 100 crescendo-1
# Follow logs in real time
docker logs -f crescendo-1
Troubleshooting
Check the logs for error messages. Common causes:
- Wrong genesis file — re-download from the networks repository
- Port already in use — another process is using the same port. Check with
ss -tlnp | grep <port> - Corrupt database — if the data directory is corrupted, you may need to reset:
bitsongd tendermint unsafe-reset-all
unsafe-reset-all deletes all chain data. You will need to sync from scratch.Your node may have lost its peers. Try:
- Check peer count:
curl -s http://localhost:26657/net_info | jq '.result.n_peers' - If peers is
0, updatepersistent_peersin~/.bitsongd/config/config.toml - Restart the node
This can happen if your machine doesn't have enough resources. Verify:
- CPU: At least 2 cores recommended
- RAM: At least 4 GB recommended
- Disk: SSD recommended for better I/O performance
- Network: Stable connection with low latency
Validators get jailed for missing too many blocks. To unjail:
- Make sure your node is fully synced (
catching_up: false) - Run the unjail transaction:
bitsongd tx slashing unjail \
--from <your-key-name> \
--keyring-backend test \
--chain-id crescendo-1 \
-y