Example

const model = new Portkey({
mode: "single",
llms: [
{
provider: "openai",
virtual_key: "open-ai-key-1234",
model: "gpt-3.5-turbo-instruct",
max_tokens: 2000,
},
],
});

// Stream the output of the model and process it
const res = await model.stream(
"Question: Write a story about a king\nAnswer:"
);
for await (const i of res) {
process.stdout.write(i);
}

Hierarchy

  • BaseLLM
    • Portkey

Constructors

Properties

apiKey?: string = undefined
baseURL?: string = undefined
llms?: null | [LLMOptions] = undefined
mode?: string = undefined

Generated using TypeDoc