Skip to content

chore: migrate motoko/basic_bitcoin to icp-cli#1372

Merged
marc0olo merged 36 commits into
masterfrom
chore/migrate-basic-bitcoin-to-icp-cli
Jun 15, 2026
Merged

chore: migrate motoko/basic_bitcoin to icp-cli#1372
marc0olo merged 36 commits into
masterfrom
chore/migrate-basic-bitcoin-to-icp-cli

Conversation

@marc0olo

@marc0olo marc0olo commented Jun 10, 2026

Copy link
Copy Markdown
Member

Summary

Migrates motoko/basic_bitcoin from dfx to icp-cli with a self-contained local Bitcoin testing environment.

Infrastructure:

  • Replace dfx.json with icp.yaml using @dfinity/motoko@v5.0.0; three environments: local (regtest), staging (testnet4), production (mainnet)
  • actor class BasicBitcoin(network : Types.Network) configured via init_args per environment
  • Custom Docker image (icp-cli-network-launcher-bitcoin) that extends the official network launcher with bitcoind in regtest mode — fully self-contained local testing, no external Bitcoin node required
  • bitcoind RPC bound to 127.0.0.1 (all callers run inside the container)
  • make build-image uses --pull to always fetch the latest base image, preventing stale local caches from causing issues
  • make test fails fast with a clear error if the network launcher container is not running
  • CI: runs on host runner (not inside a container) so icp-cli can bind-mount the Docker status dir; installs icp-cli + ic-wasm + ic-mops via npm; builds image with GHA layer cache
  • Delete dfx.json, BUILD.md, .devcontainer/

Canister code:

  • Move all source from src/basic_bitcoin/src/backend/; rename Main.moapp.mo
  • BitcoinApi.mo: calls the Bitcoin canister directly at its network-specific principal (not the management canister proxy). bitcoinCanister(network) selects ghsi2-... (mainnet) or g4xu7-... (testnet/regtest) at runtime. Actor type mirrors the official Bitcoin canister Candid. All Bitcoin API calls — including get_block_headers — go through BitcoinApi
  • EcdsaApi.mo / SchnorrApi.mo: use mo:ic directly (ic.ecdsa_public_key, ic.sign_with_ecdsa, ic.schnorr_public_key, ic.sign_with_schnorr) — threshold signing still goes through the management canister aaaaa-aa
  • KEY_NAME derived from network: key_1 (mainnet) or test_key_1 (testnet4 + local regtest)
  • Add get_block_headers and get_blockchain_info via BitcoinApi
  • Update mops.toml: moc 1.9.0, core 2.5.0, ic 4.0.0, --default-persistent-actors
  • Types.mo: reduced to types not available in mo:bitcoin or mo:ic; Utxo/OutPoint aliased from mo:bitcoin; use Blob for block hashes (not [Nat8])

Tests (Makefile):

  • 6 tests: address generation (P2PKH, P2TR key-only, P2TR full), fee percentiles, balance after mining, UTXOs after mining
  • Mining via docker exec into the running network launcher container (no port mapping needed)

Test plan

  • motoko-basic_bitcoin CI passes (ubuntu-24.04, amd64)

🤖 Generated with Claude Code

marc0olo and others added 29 commits June 10, 2026 17:29
Replace dfx.json with icp.yaml, restructure sources under backend/,
and migrate management canister calls to mo:ic@4.0.0. Actor class with
network parameter replaced by persistent actor with hardcoded regtest
network. ECDSA/Schnorr APIs updated to call ic.ecdsa_public_key,
ic.sign_with_ecdsa, ic.schnorr_public_key, and ic.sign_with_schnorr
directly instead of through typed actor handles.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…setup

- Convert actor to persistent actor class with network init arg
- Use test_key_1 for all networks (available in PocketIC)
- Add Dockerfile + docker/start.sh for self-contained bitcoind regtest
- Update icp.yaml with networks/environments and init_arg per env
- Update CI workflow with Docker build steps, version 0.3.2, --cycles 30t
- Expand Makefile with mining tests, topup target, BTC JSON-RPC tests
- Fix BitcoinApi.mo and Utils.mo for mops check compatibility
- Update README with Docker setup and multi-environment docs

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…keyword; drop type aliases

- Dockerfile: icp-cli-network-launcher:0.3.2 → v14.0.0-2026-06-04-04-52 (correct tag)
- app.mo: remove 'persistent' from actor class (redundant with --default-persistent-actors, M0217)
- app.mo: remove redundant type aliases inside actor body; use Types.X directly

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
bitcoind is not in Debian trixie's apt repos. Download the official
bitcoin-27.2 tarball from bitcoincore.org, verify the SHA256, extract
only the two needed binaries, then purge curl to keep the image clean.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When icp-cli runs inside a container (icp-dev-env-motoko), it creates a
temp status dir inside the container filesystem, then asks the host Docker
daemon to bind-mount that path — which the daemon can't see.

Run the job directly on ubuntu-24.04 (no container: wrapper) and install
icp-cli + ic-wasm via npm. Paths created on the host are visible to Docker.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…tings

icp-cli uses 'init_args' as a top-level environment field with canister
names as keys, not 'init_arg' nested inside 'settings'.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
mo:ic@4.0.0 defines BitcoinNetwork = { #mainnet; #testnet } — no #regtest.
The local PocketIC Bitcoin subnet runs in regtest mode and rejects calls
using #testnet. Define an inline management canister actor type that uses
our own Network type (which includes #regtest) for the Bitcoin API calls.
mo:ic is still used for ECDSA and Schnorr signing.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- GetUtxosResponse → re-export IC.BitcoinGetUtxosResult (Blob for hashes)
- BlockHash / Page type aliases removed (use Blob directly)
- MillisatoshiPerVByte → MillisatoshiPerByte (matches IC spec naming)
- BitcoinAddress → alias IC.BitcoinAddress
- UtxosFilter: lowercase variants (#min_confirmations/#page) + Blob, matching IC
- Cycles alias removed from all files (inlined as Nat)
- BitcoinApi.mo: no more [Nat8] conversion for tip_block_hash/next_page
- Types kept custom: Network (lowercase+regtest), NetworkCamelCase bridge,
  SendRequest, P2trDerivationPaths, EcdsaSignFunction, SchnorrSignFunction

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
-rpcbind=127.0.0.1 bound RPC only to localhost inside the container,
blocking connections from outside even with the port mapped in icp.yaml.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
icp-cli applies port mappings correctly, but relying on curl over a mapped
port introduces fragile network assumptions (port availability, NAT, IPv4/IPv6
resolution). docker exec runs bitcoin-cli directly inside the already-running
container — zero network configuration, always reliable.

- Makefile: mine blocks via docker exec (find container by ancestor image)
- icp.yaml: remove unused 18443:18443 port mapping

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Rename Docker image basic-bitcoin-launcher → icp-cli-network-launcher-bitcoin
  to make clear this is a network launcher variant
- Test 6: grep for 'tip_height = 101' instead of 'utxos' — the blob field
  in the Candid output contains binary data that interferes with grep;
  tip_height = 101 unambiguously confirms the Bitcoin blockchain synced

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- P2WPKH address generation (adapted from mo:bitcoin PR#9)
  Send function pending BIP143 sighash support in mo:bitcoin
- get_block_headers via IC management canister
- ecdsa_mock_signer → mock_sign_with_ecdsa (matches Rust naming)
- schnorr_mock_signer → mock_sign_with_schnorr
- Improve fee estimation comments to match Rust clarity

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ble)

- Remove get_p2wpkh_address and P2wpkh.mo — send_from_p2wpkh_address requires
  BIP143 sighash support not yet in mo:bitcoin; commented on PR#9 with details
- Add get_blockchain_info via inline management canister actor — returns chain
  height, block hash, timestamp, difficulty, and UTXO set size

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- BitcoinApi.mo: replace management canister proxy with direct Bitcoin canister
  calls. bitcoinCanister(network) selects the right principal at runtime:
    #mainnet  → ghsi2-tqaaa-aaaan-aaaca-cai
    #testnet/#regtest → g4xu7-jiaaa-aaaan-aaaaq-cai
  Actor type mirrors the official Bitcoin canister Candid:
  https://github.com/dfinity/bitcoin-canister/blob/master/canister/candid.did
- get_block_headers and get_blockchain_info now go through BitcoinApi
  (no inline actor workarounds needed)
- app.mo: derive KEY_NAME from NETWORK:
    #mainnet → 'key_1' (ICP mainnet Bitcoin production key)
    #testnet/#regtest → 'test_key_1' (ICP mainnet staging + local)
- Only ECDSA/Schnorr threshold signing still goes through aaaaa-aa
  (management canister, as intended)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Remove NetworkCamelCase and network_to_network_camel_case from Types.mo
  (implementation detail, not part of public API); inline the PascalCase
  conversion directly in P2pkh.mo where it is actually used
- Remove P2WpkhAddress type (P2WPKH was removed from the example)
- Drop PR#22 reference from Network type comment — that issue is closed;
  simplify to state the factual reason (mo:bitcoin uses PascalCase)
- Remove repeated 'See docs URL' doc comments from BitcoinApi.mo — one
  link in the README is sufficient; per-function links add noise

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
On Apple Silicon (arm64), Docker runs linux/arm64 containers natively.
Downloading the x86_64 binary silently fails and hangs the startup loop.
Use TARGETARCH to select the correct binary for amd64 or arm64.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The arm64 variant of icp-cli-network-launcher:latest does not support
--pocketic-config-bind (added in interface v1.1.0). Remove it — the
launcher binds the config port on all interfaces by default.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds a macos-latest (arm64) CI job with continue-on-error: true.
Currently expected to fail due to a canister_status Candid decode
error on the arm64 icp-cli-network-launcher image (log_memory_store_size
field not handled by icp-cli 0.3.2). Tracked in dfinity/icp-cli#601.

Uses colima to provide Docker on macOS runners. Remove continue-on-error
once the upstream icp-cli issue is resolved.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The default QEMU VM type fails on GitHub Actions arm64 runners.
Apple Virtualization Framework (vz) is required.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Standard GitHub-hosted macOS runners don't support VM creation (neither
QEMU nor Apple VZ), so Docker containers can't run. The arm64 issue is
tracked upstream in dfinity/icp-cli#601.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds an ubuntu-24.04-arm job that natively replicates the arm64
canister_status Candid decode failure (log_memory_store_size) seen on
Apple Silicon. Uses continue-on-error: true until dfinity/icp-cli#601
is resolved.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The arm64 Linux runner passes. The canister_status decode failure is
macOS arm64 icp-cli binary-specific, not a Linux arm64 issue.
Updated dfinity/icp-cli#601 with the confirmed root cause.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@raymondk raymondk marked this pull request as ready for review June 13, 2026 13:29
@raymondk raymondk requested review from a team as code owners June 13, 2026 13:29
marc0olo and others added 3 commits June 15, 2026 10:03
--pull ensures the icp-cli-network-launcher base image is always
refreshed from the registry before building, preventing stale local
caches from causing the log_memory_store_size decode error on macOS
arm64 (dfinity/icp-cli#601).

The arm64 CI job always passed because GHA runners have no prior
image cache — the issue is local-only. Remove it to reduce CI noise.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Restore educational content from the original README:
- Architecture section linking to ECDSA, Schnorr, and Bitcoin API specs
- Address types section explaining P2PKH vs P2TR key-only vs P2TR
  script-path and the BIP341 MAST pattern

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Comment thread motoko/basic_bitcoin/backend/app.mo
Comment thread motoko/basic_bitcoin/backend/app.mo Outdated
Comment thread motoko/basic_bitcoin/backend/app.mo Outdated
Comment thread motoko/basic_bitcoin/docker/start.sh Outdated

@gregorydemay gregorydemay left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor comments from Claude, but otherwise Claude is happy with it 😄

- get_block_headers: call BitcoinApi.get_block_headers instead of
  inline aaaaa-aa actor (eliminates dead code, adds cycles, stays
  consistent with the Bitcoin canister direct-call pattern)
- Remove stale "should be testnet" comment; network is set via init_args
- Fix doc comment: "millisatoshi/vbyte" → "millisatoshi per byte"
  to match the MillisatoshiPerByte type name
- Fix start.sh comment: port 18443 is used by bitcoin-cli, not curl

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Migrates the motoko/basic_bitcoin example from dfx to icp-cli, restructuring the Motoko canister code into backend/ and introducing a self-contained local Bitcoin regtest environment via a custom network-launcher Docker image.

Changes:

  • Replaced dfx.json-based configuration with icp.yaml environments and updated CI to build/run the custom network launcher image.
  • Moved/renamed Motoko sources into motoko/basic_bitcoin/backend/ and updated APIs to use mo:ic for threshold signing and to call the Bitcoin canister directly.
  • Added local test automation via a new Makefile and updated docs/toolchain dependencies (mops + lockfile).

Reviewed changes

Copilot reviewed 25 out of 26 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
motoko/basic_bitcoin/src/basic_bitcoin/src/Types.mo Removed old Types module as part of backend/ migration.
motoko/basic_bitcoin/src/basic_bitcoin/src/SchnorrApi.mo Removed old Schnorr wrapper (replaced by backend version using mo:ic).
motoko/basic_bitcoin/src/basic_bitcoin/src/Main.mo Removed old canister entrypoint (replaced by backend/app.mo).
motoko/basic_bitcoin/src/basic_bitcoin/src/EcdsaApi.mo Removed old ECDSA wrapper (replaced by backend version using mo:ic).
motoko/basic_bitcoin/src/basic_bitcoin/src/BitcoinApi.mo Removed management-canister Bitcoin API wrapper (replaced by direct Bitcoin canister actor).
motoko/basic_bitcoin/src/basic_bitcoin/basic_bitcoin.did Removed static candid (now configured via icp-cli/motoko recipe).
motoko/basic_bitcoin/README.md Updated documentation for icp-cli workflow, environments, and added usage examples.
motoko/basic_bitcoin/mops.toml Updated toolchain/deps and configured backend canister main.
motoko/basic_bitcoin/mops.lock Added lockfile for pinned Motoko dependencies.
motoko/basic_bitcoin/Makefile Added build-image/test/topup automation including regtest mining via Docker exec.
motoko/basic_bitcoin/icp.yaml Added icp-cli canister + network + environment configuration with init args.
motoko/basic_bitcoin/Dockerfile Added custom launcher image that installs bitcoind/bitcoin-cli.
motoko/basic_bitcoin/docker/start.sh Added entrypoint to start bitcoind regtest then exec the network launcher.
motoko/basic_bitcoin/dfx.json Removed dfx configuration.
motoko/basic_bitcoin/BUILD.md Removed dfx-based build instructions.
motoko/basic_bitcoin/backend/Utils.mo Updated utility helpers and renamed mock signers for fee estimation.
motoko/basic_bitcoin/backend/Types.mo Introduced new shared types based on mo:bitcoin + mo:ic.
motoko/basic_bitcoin/backend/SchnorrApi.mo Added mo:ic-based Schnorr public key + signing wrapper.
motoko/basic_bitcoin/backend/P2trKeyOnly.mo Updated P2TR key-only flow to use new signing API shapes.
motoko/basic_bitcoin/backend/P2tr.mo Updated P2TR send/sign logic and fee estimation plumbing for new signer function types.
motoko/basic_bitcoin/backend/P2pkh.mo Updated P2PKH send/sign logic and fee estimation plumbing for new signer function types.
motoko/basic_bitcoin/backend/EcdsaApi.mo Added mo:ic-based ECDSA public key + signing wrapper.
motoko/basic_bitcoin/backend/BitcoinApi.mo Added direct Bitcoin canister actor interface + helper functions.
motoko/basic_bitcoin/backend/app.mo Added new canister entrypoint with environment-driven network/key behavior + new query methods.
motoko/basic_bitcoin/.devcontainer/devcontainer.json Removed devcontainer setup.
.github/workflows/basic_bitcoin.yml Added CI workflow to build launcher image, deploy with icp-cli, and run tests.
Comments suppressed due to low confidence (3)

motoko/basic_bitcoin/backend/P2pkh.mo:68

  • Variable/comment still refer to “vbyte”, but the type is MillisatoshiPerByte. Renaming to fee_per_byte (and updating the comment) avoids unit confusion.
    motoko/basic_bitcoin/backend/P2tr.mo:268
  • fee_per_vbyte name and comment still say “vbyte”, but the type is MillisatoshiPerByte. Renaming to fee_per_byte (and updating the comment) keeps the unit consistent.
    motoko/basic_bitcoin/backend/P2tr.mo:318
  • Same unit mismatch here: fee_per_vbyte/comment refer to vbytes, but the type is MillisatoshiPerByte. Consider renaming to fee_per_byte and updating the comment to avoid confusion.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread motoko/basic_bitcoin/README.md Outdated
Comment thread motoko/basic_bitcoin/Makefile
Comment thread motoko/basic_bitcoin/backend/EcdsaApi.mo Outdated
Comment thread motoko/basic_bitcoin/backend/SchnorrApi.mo Outdated
Comment thread motoko/basic_bitcoin/docker/start.sh
- Fix typo: "Merkelized" → "Merkleized" in README
- Remove unused `import Types "Types"` from EcdsaApi.mo and SchnorrApi.mo
- Add BITCOIN_CONTAINER guard in Makefile to fail fast with a clear
  message when the network launcher container is not running
- Bind bitcoind RPC to 127.0.0.1 instead of 0.0.0.0 — all callers
  (bitcoin-cli in start.sh health check and docker exec from the host)
  connect from within the container, so localhost binding is sufficient
  and reduces attack surface

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 25 out of 26 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (3)

motoko/basic_bitcoin/backend/P2pkh.mo:68

  • The comment says the default fee is "1000 millisatoshis/vbyte" but the code actually uses 2000. This makes the documentation misleading when reasoning about fee selection.
    motoko/basic_bitcoin/backend/P2tr.mo:268
  • The comment says the default fee is "1000 millisatoshis/vbyte" but the code actually uses 2000, so the comment is inaccurate.
    motoko/basic_bitcoin/backend/P2tr.mo:318
  • The comment says the default fee is "1000 millisatoshis/vbyte" but the code actually uses 2000, so the comment is inaccurate.

Comment thread motoko/basic_bitcoin/backend/app.mo
…oshi/vbyte)

The code uses 2000 millisatoshi/vbyte; the comment said 1000.
Also fixes the unit in the parenthetical: "satoshi/byte" → "satoshi/vbyte".
Same wrong comment existed in both P2pkh.mo and P2tr.mo (twice).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@marc0olo marc0olo merged commit 936e794 into master Jun 15, 2026
6 checks passed
@marc0olo marc0olo deleted the chore/migrate-basic-bitcoin-to-icp-cli branch June 15, 2026 09:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants