Skip to content

Implement remaining OpenAI-compatible AI stubs #478

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Jorel97 wants to merge 13 commits into
base: master
Choose a base branch
from
Open

Implement remaining OpenAI-compatible AI stubs #478

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
6e20562
fix(ai): implement kimi chat completions
Jorel97 May 29, 2026
bffd78c
fix(ai): implement cohere chat completions
Jorel97 May 29, 2026
d9147e5
fix(ai): implement novita chat completions
Jorel97 May 29, 2026
d63b418
fix(ai): implement venice chat completions
Jorel97 May 29, 2026
58a1afe
fix(ai): implement parasail chat completions
Jorel97 May 29, 2026
63ed1d8
fix(ai): implement kimi chat completions
Jorel97 May 29, 2026
28b18be
fix(ai): implement cohere chat completions
Jorel97 May 29, 2026
c5ddf0f
fix(ai): implement novita chat completions
Jorel97 May 29, 2026
8e1a852
fix(ai): implement venice chat completions
Jorel97 May 29, 2026
b3dd548
fix(ai): implement parasail chat completions
Jorel97 May 29, 2026
4185190
fix(setup): preserve adapter config on token reuse
Jorel97 May 29, 2026
1e3b3d6
fix(setup): allow clearing optional setup config
Jorel97 May 29, 2026
6a8c517
fix(setup): allow clearing optional setup config
Jorel97 May 29, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 55 additions & 17 deletions packages/ai/cohere/src/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,27 +4,65 @@ interface Config {
baseUrl?: string;
}

const DEFAULT_BASE = "https://api.cohere.ai/compatibility";

function chatCompletionsUrl(baseUrl: string): string {
const base = baseUrl.replace(/\/+$/, '');
return base.endsWith('/v1') ? `${base}/chat/completions` : `${base}/v1/chat/completions`;
}
Comment on lines +9 to +12
Copy link
Copy Markdown
Contributor

@greptile-apps greptile-apps Bot May 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 chatCompletionsUrl duplicated verbatim in all five adapters

The exact same function is copy-pasted into cohere, kimi, novita, parasail, and venice. Any future fix must be applied in five places. Extracting it once into @profullstack/sh1pt-core alongside the existing tokenSetup / defineAi exports would make it reusable and keep these thin adapters truly thin.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!


export default defineAi<Config>({
id: 'ai-cohere',
label: 'Cohere',
defaultModel: 'command-r-plus',
models: ['command-r-plus'],

async generate(ctx, prompt, _opts, _config) {
const apiKey = ctx.secret('COHERE_API_KEY');
if (!apiKey) throw new Error('COHERE_API_KEY not in vault — run `sh1pt promote ai setup`');
ctx.log(`[stub] ai-cohere · ${prompt.length} chars in — integration pending`);
return { text: '[stub — ai-cohere integration not yet implemented]', model: 'command-r-plus' };
id: "ai-cohere",
label: "Cohere",
defaultModel: "command-a-plus-05-2026",
models: ["command-a-plus-05-2026", "command-a-03-2025", "command-r-plus", "command-r"],

async generate(ctx, prompt, opts, config) {
const apiKey = ctx.secret("COHERE_API_KEY");
if (!apiKey) throw new Error('COHERE_API_KEY not in vault - run `sh1pt promote ai setup`');
const model = opts.model ?? "command-a-plus-05-2026";
ctx.log(`ai-cohere · model=${model} · ${prompt.length} chars in`);
if (ctx.dryRun) return { text: '[dry-run]', model };

const messages: Array<{ role: string; content: string }> = [];
if (opts.system) messages.push({ role: 'system', content: opts.system });
messages.push({ role: 'user', content: prompt });

const res = await fetch(chatCompletionsUrl(config.baseUrl ?? DEFAULT_BASE), {
method: 'POST',
headers: {
authorization: `Bearer ${apiKey}`,
'content-type': 'application/json',
},
body: JSON.stringify({
model,
messages,
...(opts.maxTokens !== undefined ? { max_tokens: opts.maxTokens } : {}),
...(opts.temperature !== undefined ? { temperature: opts.temperature } : {}),
...opts.extra,
}),
});
if (!res.ok) throw new Error(`Cohere ${res.status}: ${(await res.text()).slice(0, 200)}`);
const data = (await res.json()) as {
choices: Array<{ message?: { content?: string } }>;
model?: string;
usage?: { prompt_tokens?: number; completion_tokens?: number };
};
return {
text: data.choices[0]?.message?.content ?? '',
model: data.model ?? model,
inputTokens: data.usage?.prompt_tokens,
outputTokens: data.usage?.completion_tokens,
};
},

setup: tokenSetup<Config>({
secretKey: 'COHERE_API_KEY',
label: 'Cohere',
vendorDocUrl: 'https://dashboard.cohere.com',
steps: [
'Sign in at https://dashboard.cohere.com and create an API key',
'Copy the key — usually shown once',
'Paste below; sh1pt encrypts it in the vault',
secretKey: "COHERE_API_KEY",
label: "Cohere",
vendorDocUrl: "https://docs.cohere.com/docs/compatibility-api",
steps: ["Sign in at https://dashboard.cohere.com and create an API key", "Copy the key - usually shown once", "Paste below; sh1pt encrypts it in the vault"],
fields: [
{ key: 'baseUrl', message: 'OpenAI-compatible base URL (optional; leave blank for the default):' },
],
}),
Comment thread
greptile-apps[bot] marked this conversation as resolved.
});
72 changes: 55 additions & 17 deletions packages/ai/kimi/src/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,27 +4,65 @@ interface Config {
baseUrl?: string;
}

const DEFAULT_BASE = "https://api.moonshot.ai";

function chatCompletionsUrl(baseUrl: string): string {
const base = baseUrl.replace(/\/+$/, '');
return base.endsWith('/v1') ? `${base}/chat/completions` : `${base}/v1/chat/completions`;
}

export default defineAi<Config>({
id: 'ai-kimi',
label: 'Kimi (Moonshot)',
defaultModel: 'kimi-k2-0905-preview',
models: ['kimi-k2-0905-preview'],

async generate(ctx, prompt, _opts, _config) {
const apiKey = ctx.secret('MOONSHOT_API_KEY');
if (!apiKey) throw new Error('MOONSHOT_API_KEY not in vault — run `sh1pt promote ai setup`');
ctx.log(`[stub] ai-kimi · ${prompt.length} chars in — integration pending`);
return { text: '[stub — ai-kimi integration not yet implemented]', model: 'kimi-k2-0905-preview' };
id: "ai-kimi",
label: "Kimi (Moonshot)",
defaultModel: "kimi-k2.6",
models: ["kimi-k2.6", "kimi-k2-0905-preview"],

async generate(ctx, prompt, opts, config) {
const apiKey = ctx.secret("MOONSHOT_API_KEY");
if (!apiKey) throw new Error('MOONSHOT_API_KEY not in vault - run `sh1pt promote ai setup`');
const model = opts.model ?? "kimi-k2.6";
ctx.log(`ai-kimi · model=${model} · ${prompt.length} chars in`);
if (ctx.dryRun) return { text: '[dry-run]', model };

const messages: Array<{ role: string; content: string }> = [];
if (opts.system) messages.push({ role: 'system', content: opts.system });
messages.push({ role: 'user', content: prompt });

const res = await fetch(chatCompletionsUrl(config.baseUrl ?? DEFAULT_BASE), {
method: 'POST',
headers: {
authorization: `Bearer ${apiKey}`,
'content-type': 'application/json',
},
body: JSON.stringify({
model,
messages,
...(opts.maxTokens !== undefined ? { max_tokens: opts.maxTokens } : {}),
...(opts.temperature !== undefined ? { temperature: opts.temperature } : {}),
...opts.extra,
}),
});
if (!res.ok) throw new Error(`Kimi (Moonshot) ${res.status}: ${(await res.text()).slice(0, 200)}`);
const data = (await res.json()) as {
choices: Array<{ message?: { content?: string } }>;
model?: string;
usage?: { prompt_tokens?: number; completion_tokens?: number };
};
return {
text: data.choices[0]?.message?.content ?? '',
model: data.model ?? model,
inputTokens: data.usage?.prompt_tokens,
outputTokens: data.usage?.completion_tokens,
};
},

setup: tokenSetup<Config>({
secretKey: 'MOONSHOT_API_KEY',
label: 'Kimi (Moonshot)',
vendorDocUrl: 'https://platform.moonshot.ai',
steps: [
'Sign in at https://platform.moonshot.ai and create an API key',
'Copy the key — usually shown once',
'Paste below; sh1pt encrypts it in the vault',
secretKey: "MOONSHOT_API_KEY",
label: "Kimi (Moonshot)",
vendorDocUrl: "https://platform.kimi.ai/docs/api/overview",
steps: ["Sign in at https://platform.kimi.ai and create an API key", "Copy the key - usually shown once", "Paste below; sh1pt encrypts it in the vault"],
fields: [
{ key: 'baseUrl', message: 'OpenAI-compatible base URL (optional; leave blank for the default):' },
],
}),
});
72 changes: 55 additions & 17 deletions packages/ai/novita/src/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,27 +4,65 @@ interface Config {
baseUrl?: string;
}

const DEFAULT_BASE = "https://api.novita.ai/openai";

function chatCompletionsUrl(baseUrl: string): string {
const base = baseUrl.replace(/\/+$/, '');
return base.endsWith('/v1') ? `${base}/chat/completions` : `${base}/v1/chat/completions`;
}

export default defineAi<Config>({
id: 'ai-novita',
label: 'NovitaAI',
defaultModel: 'meta-llama/llama-3.3-70b-instruct',
models: ['meta-llama/llama-3.3-70b-instruct'],

async generate(ctx, prompt, _opts, _config) {
const apiKey = ctx.secret('NOVITA_API_KEY');
if (!apiKey) throw new Error('NOVITA_API_KEY not in vault — run `sh1pt promote ai setup`');
ctx.log(`[stub] ai-novita · ${prompt.length} chars in — integration pending`);
return { text: '[stub — ai-novita integration not yet implemented]', model: 'meta-llama/llama-3.3-70b-instruct' };
id: "ai-novita",
label: "NovitaAI",
defaultModel: "meta-llama/llama-3.3-70b-instruct",
models: ["meta-llama/llama-3.3-70b-instruct", "deepseek/deepseek-v3-0324", "qwen/qwen3-235b-a22b-fp8"],

async generate(ctx, prompt, opts, config) {
const apiKey = ctx.secret("NOVITA_API_KEY");
if (!apiKey) throw new Error('NOVITA_API_KEY not in vault - run `sh1pt promote ai setup`');
const model = opts.model ?? "meta-llama/llama-3.3-70b-instruct";
ctx.log(`ai-novita · model=${model} · ${prompt.length} chars in`);
if (ctx.dryRun) return { text: '[dry-run]', model };

const messages: Array<{ role: string; content: string }> = [];
if (opts.system) messages.push({ role: 'system', content: opts.system });
messages.push({ role: 'user', content: prompt });

const res = await fetch(chatCompletionsUrl(config.baseUrl ?? DEFAULT_BASE), {
method: 'POST',
headers: {
authorization: `Bearer ${apiKey}`,
'content-type': 'application/json',
},
body: JSON.stringify({
model,
messages,
...(opts.maxTokens !== undefined ? { max_tokens: opts.maxTokens } : {}),
...(opts.temperature !== undefined ? { temperature: opts.temperature } : {}),
...opts.extra,
}),
});
if (!res.ok) throw new Error(`NovitaAI ${res.status}: ${(await res.text()).slice(0, 200)}`);
const data = (await res.json()) as {
choices: Array<{ message?: { content?: string } }>;
model?: string;
usage?: { prompt_tokens?: number; completion_tokens?: number };
};
return {
text: data.choices[0]?.message?.content ?? '',
model: data.model ?? model,
inputTokens: data.usage?.prompt_tokens,
outputTokens: data.usage?.completion_tokens,
};
},

setup: tokenSetup<Config>({
secretKey: 'NOVITA_API_KEY',
label: 'NovitaAI',
vendorDocUrl: 'https://novita.ai',
steps: [
'Sign in at https://novita.ai and create an API key',
'Copy the key — usually shown once',
'Paste below; sh1pt encrypts it in the vault',
secretKey: "NOVITA_API_KEY",
label: "NovitaAI",
vendorDocUrl: "https://novita.ai/docs/api-reference/model-apis-llm-create-chat-completion",
steps: ["Sign in at https://novita.ai and create an API key", "Copy the key - usually shown once", "Paste below; sh1pt encrypts it in the vault"],
fields: [
{ key: 'baseUrl', message: 'OpenAI-compatible base URL (optional; leave blank for the default):' },
],
}),
});
72 changes: 55 additions & 17 deletions packages/ai/parasail/src/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,27 +4,65 @@ interface Config {
baseUrl?: string;
}

const DEFAULT_BASE = "https://api.parasail.io";

function chatCompletionsUrl(baseUrl: string): string {
const base = baseUrl.replace(/\/+$/, '');
return base.endsWith('/v1') ? `${base}/chat/completions` : `${base}/v1/chat/completions`;
}

export default defineAi<Config>({
id: 'ai-parasail',
label: 'Parasail',
defaultModel: 'PARASAIL_API_KEY',
models: ['PARASAIL_API_KEY'],

async generate(ctx, prompt, _opts, _config) {
const apiKey = ctx.secret('https://parasail.io');
if (!apiKey) throw new Error('https://parasail.io not in vault — run `sh1pt promote ai setup`');
ctx.log(`[stub] ai-parasail · ${prompt.length} chars in — integration pending`);
return { text: '[stub — ai-parasail integration not yet implemented]', model: 'PARASAIL_API_KEY' };
id: "ai-parasail",
label: "Parasail",
defaultModel: "parasail-llama-33-70b-fp8",
models: ["parasail-llama-33-70b-fp8", "parasail-deepseek-r1", "parasail-qwen3-32b"],

async generate(ctx, prompt, opts, config) {
const apiKey = ctx.secret("PARASAIL_API_KEY");
if (!apiKey) throw new Error('PARASAIL_API_KEY not in vault - run `sh1pt promote ai setup`');
const model = opts.model ?? "parasail-llama-33-70b-fp8";
ctx.log(`ai-parasail · model=${model} · ${prompt.length} chars in`);
if (ctx.dryRun) return { text: '[dry-run]', model };

const messages: Array<{ role: string; content: string }> = [];
if (opts.system) messages.push({ role: 'system', content: opts.system });
messages.push({ role: 'user', content: prompt });

const res = await fetch(chatCompletionsUrl(config.baseUrl ?? DEFAULT_BASE), {
method: 'POST',
headers: {
authorization: `Bearer ${apiKey}`,
'content-type': 'application/json',
},
body: JSON.stringify({
model,
messages,
...(opts.maxTokens !== undefined ? { max_completion_tokens: opts.maxTokens } : {}),
...(opts.temperature !== undefined ? { temperature: opts.temperature } : {}),
...opts.extra,
}),
});
if (!res.ok) throw new Error(`Parasail ${res.status}: ${(await res.text()).slice(0, 200)}`);
const data = (await res.json()) as {
choices: Array<{ message?: { content?: string } }>;
model?: string;
usage?: { prompt_tokens?: number; completion_tokens?: number };
};
return {
text: data.choices[0]?.message?.content ?? '',
model: data.model ?? model,
inputTokens: data.usage?.prompt_tokens,
outputTokens: data.usage?.completion_tokens,
};
},

setup: tokenSetup<Config>({
secretKey: 'https://parasail.io',
label: 'Parasail',
vendorDocUrl: '',
steps: [
'Sign in at and create an API key',
'Copy the key — usually shown once',
'Paste below; sh1pt encrypts it in the vault',
secretKey: "PARASAIL_API_KEY",
label: "Parasail",
vendorDocUrl: "https://docs.parasail.io/parasail-docs/cookbooks/chat-completions",
steps: ["Sign in at https://parasail.io and create an API key", "Copy the key - usually shown once", "Paste below; sh1pt encrypts it in the vault"],
fields: [
{ key: 'baseUrl', message: 'OpenAI-compatible base URL (optional; leave blank for the default):' },
],
}),
});
Loading