Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 44 additions & 0 deletions .github/workflows/image-classification.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
name: image-classification

on:
push:
branches:
- master
pull_request:
paths:
- rust/image-classification/**
- .github/workflows/image-classification.yml

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
rust-image-classification:
runs-on: ubuntu-24.04
container: ghcr.io/dfinity/icp-dev-env-rust:1.0.0
env:
ICP_CLI_GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
- name: Install wasm32-wasi target
run: rustup target add wasm32-wasi
- name: Install wasi2ic
run: |
git clone https://github.com/wasm-forge/wasi2ic /tmp/wasi2ic
cargo install --path /tmp/wasi2ic --locked
- name: Install wasm-opt
run: |
version=117
curl -fsSLO "https://github.com/WebAssembly/binaryen/releases/download/version_${version}/binaryen-version_${version}-x86_64-linux.tar.gz"
tar -xzf "binaryen-version_${version}-x86_64-linux.tar.gz" --strip-components 1 -C /usr/local
rm "binaryen-version_${version}-x86_64-linux.tar.gz"
- name: Download model
working-directory: rust/image-classification
run: ./download_model.sh
- name: Deploy and test
working-directory: rust/image-classification
run: |
icp network start -d
icp deploy
make test
44 changes: 0 additions & 44 deletions .github/workflows/rust-image-classification-example.yaml

This file was deleted.

6 changes: 2 additions & 4 deletions rust/image-classification/.gitignore
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
.dfx/
build/
node_modules/
dist/
.DS_Store
Expand All @@ -8,5 +6,5 @@ _MACOSX
target/
*.old.did
.idea
src/backend/assets/mobilenetv2-7.onnx
.env
backend/assets/mobilenetv2-7.onnx
frontend/src/bindings/
2 changes: 1 addition & 1 deletion rust/image-classification/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
[workspace]
members = ["src/backend"]
members = ["backend"]
resolver = "2"
42 changes: 6 additions & 36 deletions rust/image-classification/Makefile
Original file line number Diff line number Diff line change
@@ -1,38 +1,8 @@
.PHONY: all
all: build

.PHONY: download_model
.SILENT: download_model
download_model:
bash ./download_model.sh

.PHONY: node_modules
.SILENT: node_modules
node_modules:
npm install

.PHONY: build
.SILENT: build
build: node_modules download_model
dfx canister create --all
dfx build

.PHONY: install
.SILENT: install
install: build
dfx deploy --yes

.PHONY: upgrade
.SILENT: upgrade
upgrade: build
dfx canister install --all --mode=upgrade

.PHONY: test
.SILENT: test
test: install
dfx canister call backend run | grep -w 'tractor' && echo 'PASS'

.PHONY: clean
.SILENT: clean
clean:
rm -fr .dfx
test:
@echo "=== Test 1: run() classifies the built-in test image ==="
@result=$$(icp canister call --query backend run '()') && \
echo "$$result" && \
echo "$$result" | grep -q 'tractor' && \
echo "PASS" || (echo "FAIL" && exit 1)
73 changes: 52 additions & 21 deletions rust/image-classification/README.md
Original file line number Diff line number Diff line change
@@ -1,33 +1,64 @@
# ICP image classification

This is an ICP smart contract that accepts an image from the user and runs image classification inference.
This example demonstrates running an ONNX machine-learning model inside an ICP canister.
The smart contract accepts an image from the user and runs image classification inference using the [Tract ONNX inference engine](https://github.com/sonos/tract) with the [MobileNet v2-7 model](https://github.com/onnx/models/tree/main/validated/vision/classification/mobilenet).

The example uses the WASI polyfill to run Tract (which relies on POSIX file I/O) inside the deterministic ICP runtime, and Wasm SIMD instructions for faster inference.

The smart contract consists of two canisters:

- the backend canister embeds the [the Tract ONNX inference engine](https://github.com/sonos/tract) with [the MobileNet v2-7 model](https://github.com/onnx/models/tree/main/validated/vision/classification/mobilenet).
It provides `classify()` and `classify_query()` endpoints for the frontend code to call.
The former endpoint is used for replicated execution (running on all nodes) whereas the latter runs only on a single node.
- the frontend canister contains the Web assets such as HTML, JS, CSS that are served to the browser.
- **backend** — embeds the Tract ONNX inference engine with the MobileNet v2-7 model.
It provides `classify()` and `classify_query()` endpoints:
the former runs under replicated execution (all nodes), the latter runs on a single node as a query call.
- **frontend** — serves the web UI (HTML/JS/CSS) from which users upload images and view results.

## Build and deploy from the command line

### Prerequisites

This example uses Wasm SIMD instructions that are available in `dfx` version `0.20.2-beta.0` or newer.
- Node.js
- icp-cli: `npm install -g @icp-sdk/icp-cli @icp-sdk/ic-wasm`
- wasi2ic: follow https://github.com/wasm-forge/wasi2ic and ensure `wasi2ic` is in your `$PATH`
- wasm-opt: `cargo install wasm-opt`
- Rust target `wasm32-wasi`: `rustup target add wasm32-wasi`

## Prerequisites
### Install

- [x] Install the [IC
SDK](https://internetcomputer.org/docs/current/developer-docs/getting-started/install). For local testing, `dfx >= 0.22.0` is required.
- [x] Clone the example dapp project: `git clone https://github.com/dfinity/examples`
- [x] Install WASI SDK 21:
- [x] Install `wasi-skd-21.0` from https://github.com/WebAssembly/wasi-sdk/releases/tag/wasi-sdk-21
- [x] Export `CC_wasm32_wasi` in your shell such that it points to WASI clang and sysroot. Example: `export CC_wasm32_wasi="/path/to/wasi-sdk-21.0/bin/clang --sysroot=/path/to/wasi-sdk-21.0/share/wasi-sysroot`
- [x] Install `wasi2ic`: Follow the steps in https://github.com/wasm-forge/wasi2ic and make sure that `wasi2ic` binary is in your `$PATH`.
- [x] Download MobileNet v2-7 to `src/backend/assets/mobilenetv2-7.onnx`: `./downdload_model.sh`
- [x] Install `wasm-opt`: `cargo install wasm-opt`
```bash
git clone https://github.com/dfinity/examples
cd examples/rust/image-classification
```

## Build the application
Download the MobileNet v2-7 model:

```bash
./download_model.sh
```
dfx start --background
dfx deploy

### Deploy and test

```bash
icp network start -d
icp deploy
make test
icp network stop
```

If the deployment is successful, the it will show the `frontend` URL.
Open that URL in browser to interact with the smart contract.
If the deployment is successful, the CLI will print the frontend URL.
Open that URL in a browser to interact with the smart contract.

For frontend development with hot reload:

```bash
npm run dev --prefix frontend
```

## Updating the Candid interface

```bash
icp build backend && candid-extractor target/wasm32-wasi/release/backend.wasm > backend/backend.did
```

## Security considerations and best practices

Refer to the [ICP security best practices](https://docs.internetcomputer.org/guides/security/overview) for guidance on securing your canister.
8 changes: 0 additions & 8 deletions rust/image-classification/build.sh

This file was deleted.

37 changes: 0 additions & 37 deletions rust/image-classification/dfx.json

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,17 @@
<meta name="viewport" content="width=device-width" />
<title>ICP image classification</title>
<base href="/" />
<link rel="icon" href="favicon.ico" />
<link type="text/css" rel="stylesheet" href="main.css" />
<link rel="icon" href="assets/favicon.ico" />
<link type="text/css" rel="stylesheet" href="assets/main.css" />
<script type="module" src="src/main.js"></script>
</head>

<body>
<form>
<div>
<h1>ICP image classification</h1>
<label id="filelabel" for="file" class="clickable">
<img id="image" class="image" src="logo_transparent.png"></img>
<img id="image" class="image" src="assets/logo_transparent.png"></img>
</label>
<input id="file" class="file" name="file" type="file" accept="image/png, image/jpeg" />
<div id="container">
Expand All @@ -29,11 +30,11 @@ <h1>ICP image classification</h1>
&nbsp; replicated execution
</label>
</div>
<img id="loader" src="loader.svg" class="loader invisible" />
<img id="loader" src="assets/loader.svg" class="loader invisible" />
<button id="classify" class="clean-button invisible" disabled>Go!</button>
</div>
</div>
</form>
</body>

</html>
</html>
17 changes: 17 additions & 0 deletions rust/image-classification/frontend/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{
"name": "frontend",
"private": true,
"type": "module",
"scripts": {
"prebuild": "npm i --include=dev",
"build": "vite build",
"dev": "vite"
},
"dependencies": {
"@icp-sdk/core": "~5.0.0"
},
"devDependencies": {
"@icp-sdk/bindgen": "~0.2.2",
"vite": "5.4.11"
}
}
26 changes: 26 additions & 0 deletions rust/image-classification/frontend/src/actor.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
import { safeGetCanisterEnv } from "@icp-sdk/core/agent/canister-env";
import { createActor } from "./bindings/backend";

// The ic_env cookie is set by the asset canister (SDK >=0.30.2) on all HTML
// responses. It contains the replica root key and any PUBLIC_* canister
// environment variables. In dev mode the vite dev server sets the same cookie
// via Set-Cookie header (see vite.config.js).
const canisterEnv = safeGetCanisterEnv();

// Resolve canister ID: cookie (icp-cli + dev server) → env var (dfx build-time)
const canisterId =
canisterEnv?.["PUBLIC_CANISTER_ID:backend"] ??
process.env.CANISTER_ID_BACKEND;

if (!canisterId) {
throw new Error(
"Canister ID for 'backend' not found. Run 'icp deploy' first."
);
}

export const backend = createActor(canisterId, {
agentOptions: {
host: window.location.origin,
rootKey: canisterEnv?.IC_ROOT_KEY,
},
});
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { backend } from "../../declarations/backend";
import { backend } from "./actor.js";

document.getElementById("classify").onclick = classify;
document.getElementById("file").onchange = onImageChange;
Expand Down
Loading
Loading