Skip to content

Commit dcd4687

Browse files
Merge branch 'staging' into waleedlatif1/email-verify-builtwith-enrichment
Resolve blocks/registry.ts conflict against staging's BlockMeta refactor: conform the new ZeroBounce/NeverBounce/MillionVerifier blocks to the new pattern — move `tags` off BlockConfig into `{Service}BlockMeta` exports and register them in BLOCK_META_REGISTRY. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2 parents f62d59a + 2c7b1ca commit dcd4687

1,629 files changed

Lines changed: 147327 additions & 50309 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.agents/skills/add-block/SKILL.md

Lines changed: 65 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -627,19 +627,78 @@ export const ServiceV2Block: BlockConfig = {
627627
}
628628
```
629629

630+
## Block Metadata (BlockMeta)
631+
632+
Every integration block **must** export a `{Service}BlockMeta` object at the bottom of the block file. This metadata drives the integration catalog, tag filters, and workflow template suggestions shown to users.
633+
634+
### Structure
635+
636+
```typescript
637+
import type { BlockConfig, BlockMeta } from '@/blocks/types'
638+
639+
// ... block definition above ...
640+
641+
export const {Service}BlockMeta = {
642+
tags: ['messaging', 'automation'], // Same tags as the block's tags field
643+
templates: [ // Optional but strongly encouraged
644+
{
645+
icon: {Service}Icon,
646+
title: '{Service} use-case title',
647+
prompt: 'Build a workflow that ...',
648+
modules: ['agent', 'workflows'], // Modules the template uses
649+
category: 'productivity', // Template category
650+
tags: ['automation'], // Template-level tags
651+
alsoIntegrations: ['slack'], // Other blocks referenced in the prompt (optional)
652+
},
653+
],
654+
} as const satisfies BlockMeta
655+
```
656+
657+
### Rules
658+
659+
- **Import `BlockMeta`** from `@/blocks/types` alongside `BlockConfig`
660+
- **`tags`** must match the `tags` array on the block config exactly
661+
- **Templates are optional** but should be added for any integration that has a recognizable use case — aim for 2–4 templates per block
662+
- **Template `prompt`** should start with "Build a workflow that..." or "Create a workflow that..." and be concrete enough to generate a real workflow in Mothership
663+
- **Template `modules`** lists the Sim modules the template relies on: `'knowledge-base' | 'tables' | 'files' | 'workflows' | 'scheduled' | 'agent'`
664+
- **Template `category`** is one of: `'popular' | 'sales' | 'support' | 'engineering' | 'marketing' | 'productivity' | 'operations'`
665+
- **`alsoIntegrations`** names other block types (e.g. `'slack'`, `'linear'`) referenced in the template prompt — helps the catalog surface this template when those blocks are selected
666+
- Place the export **after** the main `{Service}Block` export, at the very bottom of the file
667+
668+
### Register in the blocksMeta object
669+
670+
After adding `{Service}BlockMeta` to the block file, register it in `apps/sim/blocks/registry.ts`:
671+
672+
```typescript
673+
// Add import (alongside the block import, alphabetically)
674+
import { ServiceBlock, ServiceBlockMeta } from '@/blocks/blocks/service'
675+
676+
// Add to blocksMeta object (alphabetically)
677+
export const blocksMeta = {
678+
// ... existing entries ...
679+
service: ServiceBlockMeta,
680+
}
681+
```
682+
630683
## Registering Blocks
631684

632685
After creating the block, remind the user to:
633-
1. Import in `apps/sim/blocks/registry.ts`
686+
1. Import `{Service}Block` and `{Service}BlockMeta` in `apps/sim/blocks/registry.ts`
634687
2. Add to the `registry` object (alphabetically):
688+
3. Add to the `blocksMeta` object (alphabetically):
635689

636690
```typescript
637-
import { ServiceBlock } from '@/blocks/blocks/service'
691+
import { ServiceBlock, ServiceBlockMeta } from '@/blocks/blocks/service'
638692

639693
export const registry: Record<string, BlockConfig> = {
640694
// ... existing blocks ...
641695
service: ServiceBlock,
642696
}
697+
698+
export const blocksMeta = {
699+
// ... existing entries ...
700+
service: ServiceBlockMeta,
701+
}
643702
```
644703

645704
## Complete Example
@@ -827,7 +886,10 @@ All tool IDs referenced in `tools.access` and returned by `tools.config.tool` MU
827886
- [ ] Tools.access lists all tool IDs (snake_case)
828887
- [ ] Tools.config.tool returns correct tool ID (snake_case)
829888
- [ ] Outputs match tool outputs
830-
- [ ] Block registered in registry.ts
889+
- [ ] Block registered in `registry.ts` blocks object (alphabetically)
890+
- [ ] `{Service}BlockMeta` exported at bottom of block file with `tags` and `templates`
891+
- [ ] `BlockMeta` imported from `@/blocks/types` alongside `BlockConfig`
892+
- [ ] Block meta registered in `registry.ts` blocksMeta object (alphabetically)
831893
- [ ] If icon missing: asked user to provide SVG
832894
- [ ] If triggers exist: `triggers` config set, trigger subBlocks spread
833895
- [ ] Optional/rarely-used fields set to `mode: 'advanced'`
Lines changed: 142 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,142 @@
1+
---
2+
name: add-enrichment
3+
description: Add a code-defined table enrichment (registry entry) under `apps/sim/enrichments/` backed by an ordered provider cascade, ensuring every provider tool it calls has hosted-key support. Use when adding a per-row table enrichment that fills cells via existing Sim tools.
4+
---
5+
6+
# Adding a Table Enrichment
7+
8+
Enrichments are code-defined entries in `apps/sim/enrichments/` that run **directly per table row** (no workflow). Each enrichment declares inputs, outputs, and an ordered list of **providers**; the cascade runner tries providers in order and the first non-empty result fills the cell. Each provider calls one existing Sim tool via `executeTool`, which injects the workspace's BYOK key or a **hosted key** and bills usage automatically.
9+
10+
Because enrichments run on Sim's hosted keys by default, **every provider tool you reference must have hosted-key support** — otherwise it can only run when the workspace brings its own key. This command makes that check a required step.
11+
12+
## Overview
13+
14+
| Step | What | Where |
15+
|------|------|-------|
16+
| 1 | Pick the data-source tool(s) for each output | `tools/{service}/` + `tools/registry.ts` |
17+
| 2 | **Verify each tool has `hosting`; if not, run `/add-hosted-key`** | `tools/{service}/{action}.ts` |
18+
| 3 | Write the enrichment definition | `enrichments/{name}/{name}.ts` + `index.ts` |
19+
| 4 | Register it | `enrichments/registry.ts` |
20+
| 5 | Verify | tsc / biome / manual run |
21+
22+
## Architecture (what you're plugging into)
23+
24+
- **`enrichments/types.ts`**`EnrichmentConfig { id, name, description, icon, inputs, outputs, providers }` and `EnrichmentProvider { id, label, toolId, buildParams, mapOutput }`. Providers are **plain data** (no `@/tools` import) so the catalog stays client-safe.
25+
- **`enrichments/providers.ts`**`toolProvider(...)` (typed passthrough) plus shared input helpers: `str(v)`, `normalizeDomain(v)`, `firstNonEmpty(arr)`, `splitName(fullName)`.
26+
- **`enrichments/run.ts`** — the server-only cascade runner. Calls `executeTool(provider.toolId, { ...params, _context: { workspaceId } })`, accumulates hosted-key cost, returns the first non-empty mapped result. **You do not edit this** — it works for any registry entry.
27+
- **`enrichments/registry.ts`**`ENRICHMENT_REGISTRY` / `ALL_ENRICHMENTS` / `getEnrichment`. Register new entries here.
28+
29+
Outputs automatically become table columns; billing, the catalog/sidebar UI, the column meta-header icon, and per-row execution all work with no extra wiring.
30+
31+
## Step 1: Pick the data-source tool(s)
32+
33+
For each output the enrichment produces, decide which existing tool provides it. Look up the service's API and the tool in `apps/sim/tools/{service}/` (e.g. `hunter_email_finder`, `pdl_person_enrich`, `pdl_company_enrich`). Confirm:
34+
35+
- The tool id is registered in `apps/sim/tools/registry.ts`.
36+
- Its `params` accept what you can derive from table columns (read the tool's `params`).
37+
- Its `outputs` / `transformResponse` actually expose the field you need (read the real output shape — don't assume).
38+
39+
Order providers **cheapest / most-likely-to-hit first**; the cascade stops at the first non-empty result. Apollo / LinkedIn are not hosted-safe (ToS) — don't use them.
40+
41+
## Step 2: Verify hosted-key support — chain to `/add-hosted-key` if missing
42+
43+
**This is the required gate.** For every tool a provider calls, open `apps/sim/tools/{service}/{action}.ts` and check for a `hosting` block:
44+
45+
```typescript
46+
hosting: {
47+
envKeyPrefix: 'SERVICE_API_KEY',
48+
apiKeyParam: 'apiKey',
49+
byokProviderId: 'service',
50+
pricing: { /* ... */ },
51+
rateLimit: { /* ... */ },
52+
}
53+
```
54+
55+
- **If `hosting` is present** — good. Note the `envKeyPrefix`; the deployment needs `{PREFIX}_COUNT` + `{PREFIX}_1..N` env vars set for the hosted key to actually resolve at runtime (ops concern, not code). If those env vars aren't set in the target environment, the provider will only run with a workspace BYOK key.
56+
- **If `hosting` is absent** — the tool can't use a Sim-provided key, so the enrichment would silently produce blank cells on hosted Sim. **Stop and run `/add-hosted-key <service>`** to add hosted-key support to that tool first, then come back. Do this for every provider tool that lacks it.
57+
58+
Why it matters: the cascade runner only bills (and only reads `output.cost.total`) when `executeTool` injected a hosted key, which requires the tool's `hosting` config. No `hosting` → no hosted key → the enrichment depends entirely on per-workspace BYOK.
59+
60+
## Step 3: Write the enrichment definition
61+
62+
Create `apps/sim/enrichments/{name}/{name}.ts` and a barrel `index.ts`. Mirror the existing entries (`work-email`, `phone-number`, `company-domain`, `company-info`).
63+
64+
```typescript
65+
import { SomeIcon } from 'lucide-react'
66+
import { filterUndefined } from '@sim/utils/object'
67+
import { normalizeDomain, splitName, str, toolProvider } from '@/enrichments/providers'
68+
import type { EnrichmentConfig } from '@/enrichments/types'
69+
70+
export const myEnrichment: EnrichmentConfig = {
71+
id: 'my-enrichment',
72+
name: 'My Enrichment',
73+
description: 'One concise sentence describing what it finds.',
74+
icon: SomeIcon,
75+
inputs: [
76+
// Person enrichments take a single canonical `fullName` (Clay-style);
77+
// split it with splitName() for tools that need first/last.
78+
{ id: 'fullName', name: 'Full name', type: 'string', required: true },
79+
{ id: 'companyDomain', name: 'Company domain', type: 'string' },
80+
],
81+
outputs: [{ id: 'value', name: 'value', type: 'string' }],
82+
providers: [
83+
toolProvider({
84+
id: 'provider-a',
85+
label: 'Provider A',
86+
toolId: 'service_action', // must have `hosting` (Step 2)
87+
buildParams: (inputs) => {
88+
// Return null when there aren't enough inputs → cascade skips this provider.
89+
const name = splitName(inputs.fullName)
90+
const domain = normalizeDomain(inputs.companyDomain)
91+
if (!name || !domain) return null
92+
return { domain, first_name: name.firstName, last_name: name.lastName }
93+
},
94+
mapOutput: (output) => {
95+
// Return { [outputId]: value } on a hit, or null to fall through.
96+
const value = str(output.value)
97+
return value ? { value } : null
98+
},
99+
}),
100+
// ...additional fallback providers, in priority order.
101+
],
102+
}
103+
```
104+
105+
```typescript
106+
// apps/sim/enrichments/{name}/index.ts
107+
export { myEnrichment } from './my-enrichment'
108+
```
109+
110+
Rules:
111+
- Keep the file **client-safe**: import only `lucide-react`, `@sim/utils/*`, `@/enrichments/providers`, and the types. **Never import `@/tools`** here — the runner does the tool call.
112+
- `buildParams` returns `null` when inputs are insufficient (provider skipped). `mapOutput` returns `null`/empty for a miss (falls through). Use `filterUndefined` when assembling optional tool params; coerce numbers explicitly (don't pass `''` to number outputs).
113+
- Output `id`s are the keys `mapOutput` returns; output `name`s are the default column names (the user can rename them in the config).
114+
115+
## Step 4: Register it
116+
117+
In `apps/sim/enrichments/registry.ts`, import and add the entry (catalog order is registration order):
118+
119+
```typescript
120+
import { myEnrichment } from '@/enrichments/my-enrichment'
121+
122+
export const ENRICHMENT_REGISTRY: EnrichmentRegistry = {
123+
// ...existing
124+
[myEnrichment.id]: myEnrichment,
125+
}
126+
```
127+
128+
## Step 5: Verify
129+
130+
1. `bunx tsc --noEmit` (from `apps/sim`, `NODE_OPTIONS=--max-old-space-size=8192`) and `bunx biome check` on the changed files.
131+
2. In a table → **+ New column → Enrichments** → pick the new enrichment, map its inputs to columns, name the output column(s), Save. Confirm it appears in the catalog with its icon/description.
132+
3. With hosted keys (or a workspace BYOK key) configured for each provider's service, run a row and confirm the cell fills; the dev-server log shows `Enrichment hit { provider }`. A row whose providers all miss completes blank; a row where every provider errored shows an error cell.
133+
134+
## Checklist
135+
136+
- [ ] Each output mapped to a real tool field (verified against the tool's `params`/`outputs`)
137+
- [ ] **Every provider tool has a `hosting` block — ran `/add-hosted-key` for any that didn't**
138+
- [ ] Providers ordered cheapest / most-likely-first; Apollo/LinkedIn not used
139+
- [ ] Enrichment file is client-safe (no `@/tools` import); uses `toolProvider` + shared helpers
140+
- [ ] `buildParams` returns `null` on insufficient inputs; `mapOutput` returns `null` on a miss
141+
- [ ] Registered in `enrichments/registry.ts`
142+
- [ ] tsc + biome clean; created and ran the column end-to-end
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
interface:
2+
display_name: "Add Enrichment"
3+
short_description: "Build a table enrichment cascade"
4+
brand_color: "#16A34A"
5+
default_prompt: "Use $add-enrichment to add a code-defined Sim table enrichment backed by a provider cascade."

0 commit comments

Comments
 (0)