CloudNature | Amazon Bedrock For JavaScript and TypeScript Developers

Introduction

Hey! If you are here, it means you are planning to use Amazon Bedrock, and every example you see uses Python. So, you might have typed on Google something like '"Amazon" "Bedrock" "Typescript"' and somehow you ended up here (not complaining though 😜).

This blog post covers examples in Node.js, both JavaScript and TypeScript, for using Amazon Bedrock SDK V3. You can expect two things from this blog post: examples and my thoughts.

Did I mention there is one example for each model available in Amazon Bedrock? 🚀

Let's jump right through it.

What? Wait a sec! If you have never used Amazon Bedrock, I have an article waiting for you after this one: The Complete Guide to Amazon Bedrock for Generative AI.

Initializing Clients

"Clients"! Yes, you read it right, this time it's not my typo. We need to install two clients:

npm install @aws-sdk/client-bedrock
npm install @aws-sdk/client-bedrock-runtime

client-bedrock: SDK for creating and managing Amazon Bedrock models;
client-bedrock-runtime: SDK for invoking Amazon Bedrock models and running inference on them.

Now, let's import the libraries. Amazon Bedrock is available in different regions, so we need to select one, or it will use the default one.

import { BedrockRuntimeClient, InvokeModelCommand, InvokeModelCommandInput, InvokeModelCommandOutput, InvokeModelWithResponseStreamCommand, InvokeModelWithResponseStreamCommandInput, InvokeModelWithResponseStreamCommandOutput } from "@aws-sdk/client-bedrock-runtime";

import { BedrockClient, CreateModelCustomizationJobCommand, GetModelCustomizationJobCommand, ListFoundationModelsCommand, CreateModelCustomizationJobCommandInput, CreateModelCustomizationJobCommandOutput, GetModelCustomizationJobCommandInput, GetModelCustomizationJobCommandOutput, ListFoundationModelsCommandInput, ListFoundationModelsCommandOutput } from '@aws-sdk/client-bedrock';

const client = new BedrockRuntimeClient({ region: process.env.REGION || 'us-east-1' });
const client = new BedrockClient({ region: process.env.REGION || 'us-east-1' });

With that done, we're ready to start with the client-bedrock-runtime SDK.

Invoke Model With Response Stream

This API streams the Generative AI model response to us. For this example, I used the model Anthropic Claude Instant V1.

const MODEL_ID = process.env.MODEL_ID || 'anthropic.claude-instant-v1';
const PROMPT = process.env.PROMPT || 'Hi, who are you?';

const params: InvokeModelWithResponseStreamCommandInput = {
    modelId: MODEL_ID,
    contentType: "application/json",
    accept: "application/json",
    body: JSON.stringify({
      prompt: `

Human:${PROMPT}

Assistant:`,
      max_tokens_to_sample: 300,
      temperature: 0.5,
      top_k: 250,
      top_p: 1,
    }),
};
const command = new InvokeModelWithResponseStreamCommand(param);
const res = await client.send(command);

const chunks = [];

for await (const event of res.body) {
    if (event.chunk && event.chunk.bytes) {
        const chunk = JSON.parse(Buffer.from(event.chunk.bytes).toString("utf-8"));
        chunks.push(chunk.completion); // change this line
    } else if (
        event.internalServerException ||
        event.modelStreamErrorException ||
        event.throttlingException ||
        event.validationException
    ) {
        console.error(event);
        break;
    }
};
console.log({
    prompt: PROMPT,
    completion: chunks.join(''),
})

Above, we can see how to integrate the 'InvokeModelWithResponseStreamCommand' with TypeScript. One thing worth mentioning is that if we are using a Lambda Function, this integration may not be as useful if we simply take it from this example. It becomes more useful when integrated with Lambda response streaming so we can effectively stream the response from the Generative AI model back to our users. You can find an example here: Lambda Streaming First Byte (TTFB) Pipeline.

Invoke Model

Starting from the basics, we will go through every single model provider available in Amazon Bedrock as of the time I'm writing this blog post (I'll probably maintain this blog post updated if I see you like it 🙂).

Here is the complete structure to invoke the model:

const MODEL_ID = process.env.MODEL_ID || '';
const PROMPT = process.env.PROMPT || '';

const params: InvokeModelCommandInput = {
    modelId: MODEL_ID,
    contentType: "application/json",
    accept: "application/json",
    body: JSON.stringify(/*Here we place prompt and inference parameters, every model has its own structure 😩*/),
};
const command = new InvokeModelCommand(params);
const res = await client.send(command);
const jsonString = new TextDecoder().decode(res.body);
const modelRes = JSON.parse(jsonString);

const bodyRes = {
    prompt: PROMPT,
    completion: /* Here we place the response from "modelRes", every model has its own response 😩*/,
};
console.debug(bodyRes);

Each model has its own body and response parameters. Starting from this code I'll show the 'params' and how to retrieve the 'response' for each model provider.

I understand that these are all different providers, and generative AI is still too young to have a strong standard, but having to map different parameters and different response objects can be quite confusing. Please, model providers, talk to each other and provide a single, user-friendly way for parameters and model responses 🙏.

⚠️Depending on the region you are in, you may not see every model available.

Invoke Model: Anthropic

⚠️ This content has been updated with the latest Anthropic models and SDK.

Here are the models available for Anthropic:

```
"anthropic.claude-instant-v1"
```
```
"anthropic.claude-v2"
```
```
"anthropic.claude-v2:1"
```

"anthropic.claude-3-sonnet-20240229-v1:0"

"anthropic.claude-3-haiku-20240307-v1:0"

And here is the code:

const MODEL_ID = process.env.MODEL_ID || 'anthropic.claude-instant-v1';
const PROMPT = process.env.PROMPT || 'Hi, who are you?';
const params: InvokeModelCommandInput = {
    modelId: MODEL_ID,
    contentType: "application/json",
    accept: "application/json",
    body: JSON.stringify({
    body: JSON.stringify({
      anthropic_version: "bedrock-2023-05-31",
      max_tokens: 4096,
      temperature: 0.5,
      top_k: 250,
      top_p: 1,
      messages: [
        {
          role: "user",
          content: [
            // {
            //   type: "image",
            //   source: {
            //     type: "base64",
            //     media_type: "image/png",
            //     data: 'YOUR_BASE_64_IMAGE',
            //   },
            // },
            {
              type: "text",
              text: PROMPT
            },
          ],
        }
      ],
    }),
};
const res = await invokeModel(params);
const jsonString = new TextDecoder().decode(res.body);
const modelRes = JSON.parse(jsonString);
    
const bodyRes = {
        prompt: PROMPT,
        completion: modelRes.content[0].text,
};
console.debug(bodyRes);

Invoke Model: AI21 Labs

Here are the models available for AI21 Labs:

```
"ai21.j2-mid-v1"
```
```
"ai21.j2-ultra-v1"
```

And here is the code:

const MODEL_ID = process.env.MODEL_ID || 'ai21.j2-mid';
const PROMPT = process.env.PROMPT || 'Hi, who are you?';
const params: InvokeModelCommandInput = {
    modelId: MODEL_ID,
    contentType: "application/json",
    accept: "application/json",
    body: JSON.stringify({
      prompt: PROMPT,
      maxTokens: 200,
      temperature: 0.7,
      topP: 1,
      stopSequences: [],
      countPenalty: { scale: 0 },
      presencePenalty: { scale: 0 },
      frequencyPenalty: { scale: 0 },
    }),
};
const res = await invokeModel(params);
const jsonString = new TextDecoder().decode(res.body);
const modelRes = JSON.parse(jsonString);
    
const bodyRes = {
        prompt: PROMPT,
        completion: modelRes.completions[0].data.text,
};
console.debug(bodyRes);

Invoke Model: Cohere

Here are the models available for Cohere:

```
"cohere.command-text-v14"
```

And here is the code:

const MODEL_ID = process.env.MODEL_ID || 'cohere.command-text-v14';
const PROMPT = process.env.PROMPT || 'Hi, who are you?';
const params: InvokeModelCommandInput = {
    modelId: MODEL_ID,
    contentType: "application/json",
    accept: "application/json",
    body: JSON.stringify({
      prompt: PROMPT,
      max_tokens: 400,
      temperature: 0.75,
      p: 0.01,
      k: 0,
      stop_sequences: [],
      return_likelihoods: "NONE",
    }),
};
const res = await invokeModel(params);
const jsonString = new TextDecoder().decode(res.body);
const modelRes = JSON.parse(jsonString);
    
const bodyRes = {
        prompt: PROMPT,
        completion: modelRes.generations[0].text,
};
console.debug(bodyRes);

Invoke Model: Amazon Text

Here are the models available for Amazon Text:

```
"amazon.titan-text-lite-v1"
```
```
"amazon.titan-text-express-v1"
```
```
"amazon.titan-text-agile-v1"
```

And here is the code:

const MODEL_ID = process.env.MODEL_ID || 'amazon.titan-text-lite-v1';
const PROMPT = process.env.PROMPT || 'Hi, who are you?';
const params: InvokeModelCommandInput = {
    modelId: MODEL_ID,
    contentType: "application/json",
    accept: "application/json",
    body: JSON.stringify({
      inputText: PROMPT,
      textGenerationConfig: {
        maxTokenCount: 300,
        stopSequences: [],
        temperature: 0,
        topP: 0.9,
      }
    }),
};
const res = await invokeModel(params);
const jsonString = new TextDecoder().decode(res.body);
const modelRes = JSON.parse(jsonString);
    
const bodyRes = {
        prompt: PROMPT,
        completion: modelRes.results[0].outputText,
};
console.debug(bodyRes);

⚠️Those models are still in preview, but the documentation show a detailed overview on what Amazon' models need and give.

Invoke Model: Meta

Here are the models available for Meta:

```
"meta.llama2-70b-chat-v1"
```
```
"meta.llama2-13b-chat-v1"
```

And here is the code:

const MODEL_ID = process.env.MODEL_ID || 'meta.llama2-70b-chat-v1';
const PROMPT = process.env.PROMPT || 'Hi, who are you?';
const params: InvokeModelCommandInput = {
    modelId: MODEL_ID,
    contentType: "application/json",
    accept: "application/json",
    body: JSON.stringify({
      prompt: PROMPT,
      temperature: 0.5,
      top_p: 0.9,
      max_gen_len: 512
    }),
};
const res = await invokeModel(params);
const jsonString = new TextDecoder().decode(res.body);
const modelRes = JSON.parse(jsonString);
    
const bodyRes = {
        prompt: PROMPT,
        completion: modelRes.generation,
};
console.debug(bodyRes);

Invoke Model: Amazon Embedding

Here are the models available for Amazon Embedding:

```
"amazon.titan-embed-text-v1"
```

And here is the code:

const MODEL_ID = process.env.MODEL_ID || 'amazon.titan-embed-text-v1';
const PROMPT = process.env.PROMPT || 'Hi, who are you?';
const params: InvokeModelCommandInput = {
    modelId: MODEL_ID,
    contentType: "application/json",
    accept: "application/json",
    body: JSON.stringify({
      inputText: PROMPT,
    }),
};
const res = await invokeModel(params);
const jsonString = new TextDecoder().decode(res.body);
const modelRes = JSON.parse(jsonString);
    
const bodyRes = {
        prompt: PROMPT,
        embedding: modelRes.embedding,
};
console.debug(bodyRes);

Invoke Model: Stability AI

Here are the models available for Stability AI:

```
"stability.stable-diffusion-xl-v0"
```
```
"stability.stable-diffusion-xl-v1"
```

And here is the code:

const MODEL_ID = process.env.MODEL_ID || 'stability.stable-diffusion-xl-v0';
const PROMPT = process.env.PROMPT || 'Hi, who are you?';
const params: InvokeModelCommandInput = {
    modelId: MODEL_ID,
    contentType: "application/json",
    accept: "application/json",
    body: JSON.stringify({
      text_prompts: [{ text: PROMPT }],
      cfg_scale: 10,
      seed: 0,
      steps: 50,
    }),
};
const res = await invokeModel(params);
const jsonString = new TextDecoder().decode(res.body);
const modelRes = JSON.parse(jsonString);
    
const bodyRes = {
        prompt: PROMPT,
        image: modelRes.artifacts[0].base64,
};
console.debug(bodyRes);

Additionally, we can save the image in a file. I'll leave this code to the AI 🤖... I'm joking, it's in my repository 😜.

On a side note, don't you want to know what the Stable Diffusion XL V0 response is to the question: "Hi, who are you?"? Here's the result 👇

Personification of Stability Diffusion XL V0

Invoke Model: Amazon Image

Here are the models available for Amazon Image:

```
"amazon.titan-image-generator-v1"
```

And here is the code:

const MODEL_ID = process.env.MODEL_ID || 'amazon.titan-image-generator-v1';
const PROMPT = process.env.PROMPT || 'Hi, who are you?';
const params: InvokeModelCommandInput = {
    modelId: MODEL_ID,
    contentType: "application/json",
    accept: "application/json",
    body: JSON.stringify({
      // TEXT_IMAGE
      taskType: "TEXT_IMAGE",
      textToImageParams: {
        text: PROMPT,
      },
      imageGenerationConfig: {
        cfgScale: 8,
        seed: 0,
        quality: "standard", //"premium"
        width: 1024,
        height: 1024,
        numberOfImages: 1
      }
    }),
};
const res = await invokeModel(params);
const jsonString = new TextDecoder().decode(res.body);
const modelRes = JSON.parse(jsonString);
    
const bodyRes = {
        prompt: PROMPT,
        images: base64Images,
};
console.debug(bodyRes);

Additionally with Amazon Titan Image Generator we can edit images starting from another images, here's just the body for doing just that:

⚠️ Remember to remove previous textToImageParams and taskType parameters.

Image variation: to generate new image varying style and background

        // IMAGE_VARIATION
        // taskType: "IMAGE_VARIATION",
        // imageVariationParams: {
        //   text: PROMPT,
        //   negativeText: "bad quality, low resolution, cartoon",
        //   images: [
        //     IMAGE_BASE_64
        //   ]
        // },

Inpainting: to reconstructs a specified region within the mask given as input with the starting image.

        // INPAINTING
        // taskType: "INPAINTING",
        // inPaintingParams: {
        //   text: PROMPT, // Required
        //   negativeText: "bad quality, low res", // Optional
        //   image: IMAGE_BASE_64, // Required
        //   maskPrompt: "windows", // One of "maskImage" or "maskPrompt" is required
        //   // maskImage: IMAGE_BASE_64, // Input maskImage based on the values 0 or 1 only 
        // },

Outpainting: to create new pixels into a specific region within the mask

        // OUTPAINTING
        // taskType: "OUTPAINTING",
        // outPaintingParams: {
        //   text: PROMPT, // Required
        //   negativeText: "bad quality, low res", // Optional
        //   image: IMAGE_BASE_64, // Required
        //   maskPrompt: "windows", // One of "maskImage" or "maskPrompt" is required
        //   // maskImage: IMAGE_BASE_64, // One of "maskImage" or "maskPrompt" is required 
        //   outPaintingMode: "DEFAULT"
        // },

Out of curiosity we saw Stable Diffusion response to the question: "Hi, who are you?". What about Amazon Titan Image? 👇 Personification of Amazon Titan Image

Perfect, now that we have explored everything there is in the client-bedrock-runtime SDK, it's time to learn how we can use the client-bedrock SDK 🚅.

Invoke Model: Mistral AI

Here are the models available for Mistral AI:

```
"mistral.mistral-7b-instruct-v0:2"
```
```
"mistral.mixtral-8x7b-instruct-v0:1"
```

And here is the code:

const MODEL_ID = process.env.MODEL_ID || 'mistral.mixtral-8x7b-instruct-v0:1';
const PROMPT = process.env.PROMPT || 'Hi, who are you?';
const params: InvokeModelCommandInput = {
    modelId: MODEL_ID,
    contentType: "application/json",
    accept: "application/json",
    body: JSON.stringify({
      prompt: PROMPT,
      max_tokens: 400,
      temperature: 0.75,
      top_p: 0.01,
      // top_k: 0,
      // stop: [],
    }),
};
const res = await invokeModel(params);
const jsonString = new TextDecoder().decode(res.body);
const modelRes = JSON.parse(jsonString);
    
const bodyRes = {
        prompt: PROMPT,
        completion: modelRes.outputs[0].text,
};
console.debug(bodyRes);

List Foundation Models

The easiest of them all, it will list foundation models available on Amazon Bedrock. Here's the code:

const params: ListFoundationModelsCommandInput = {}
const command = new ListFoundationModelsCommand(param);
const res = await client.send(command);

⚠️Update: After an initial bug was corrected, it's now possible to list foundation models without workarounds

Unfortunately, this API has a bug, and without parameters, it throws errors in the 'byProvider' parameter. Also, using the 'byProvider' parameter will throw error status 400 because the regex for the model provider name is not correct. Using 'byInferenceType' is the least impacting parameter if you are starting with Amazon Bedrock 😊.

Create Model Customization Job

This command was taken right from the V3 documentation, which is really well done.

Here's the code:

const BUCKET_URI = process.env.BUCKET_URI || 's3://S3_BUCKET_NAME';
const ROLE_ARN = process.env.ROLE_ARN || 'arn:aws:iam::ACCOUNT_ID:role/ROLE_NAME';
const BASE_MODEL_IDENTIFIER = process.env.BASE_MODEL_IDENTIFIER || 'arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-text-express-v1';

const now = new Date();
const params: CreateModelCustomizationJobCommandInput = {
    jobName: `job-${now.getTime()}`, // required
    customModelName: `titan-text-express-v1-${now.getTime()}`, // required
    roleArn: ROLE_ARN, // required
    baseModelIdentifier: BASE_MODEL_IDENTIFIER, // required
    jobTags: [ // TagList
      { // Tag
        key: 'bedrock', // required
        value: 'true', // required
      },
    ],
    customModelTags: [
      {
        key: 'custom-bedrock', // required
        value: 'true', // required
      },
    ],
    trainingDataConfig: {
      s3Uri: `${BUCKET_URI}/training/dataset.jsonl`, // required
    },
    outputDataConfig: {
      s3Uri: `${BUCKET_URI}/output`, // required
    },
    hyperParameters: { // required
      'epochCount': '1',
      'batchSize': '4',
      'learningRate': '0.02',
      'learningRateWarmupSteps': '0',
    },
    // customModelKmsKeyId: 'STRING_VALUE',
    // clientRequestToken: 'STRING_VALUE',
    // validationDataConfig: { // ValidationDataConfig
    //   validators: [ // Validators // required
    //     { // Validator
    //       s3Uri: 'STRING_VALUE', // required
    //     },
    //   ],
    // },
    // vpcConfig: { // VpcConfig
    //   subnetIds: [ // SubnetIds // required
    //     'STRING_VALUE',
    //   ],
    //   securityGroupIds: [ // SecurityGroupIds // required
    //     'STRING_VALUE',
    //   ],
    // },
  };
const command = new CreateModelCustomizationJobCommand(param);
const res = await client.send(command);

console.log(res.jobArn)

Fine tuning is still in preview, but from this documentation https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/client/bedrock/command/CreateModelCustomizationJobCommand/, we can have a detailed sneak peek 👀 of what's coming.

Get Model Customization Job

Nothing too special about this one, it gets the job information using its identifier.

const JOB_ARN = process.env.JOB_NAME || 'JOB_NAME';
const params: GetModelCustomizationJobCommandInput = {
    jobIdentifier: JOB_ARN,
};
const command = new GetModelCustomizationJobCommand(params);
const res = await client.send(command);

Here we can check the 'status' and also the 'failureMessage', which is really handy to receive by email on Friday at 17.55 😈.

⚠️There are other APIs from 'client-bedrock' that I won't cover because they are really simple or not as useful as these 3.

Conclusion

There you have it, folks! With these code snippets in mind, you can use Amazon Bedrock like a pro 💻. We went through both SDKs and found workarounds for bugs, I think this was a nice ride, and hopefully, you too will be able to enjoy your ride better after this article.

Here you can find the GitHub repository: https://github.com/Depaa/amazon-bedrock-nodejs 😉

If you enjoyed this article, please let me know in the comment section or send me a DM. I'm always happy to chat! ✌️

Thank you so much for reading! 🙏 Keep an eye out for more AWS-related posts, and feel free to connect with me on LinkedIn 👉 https://www.linkedin.com/in/matteo-depascale/.

Reference

Disclaimer: opinions expressed are solely my own and do not represent the views or opinions of my employer.

Amazon Bedrock For JavaScript and TypeScript Developers

Introduction

Initializing Clients

Invoke Model With Response Stream

Invoke Model

Invoke Model: Anthropic

Invoke Model: AI21 Labs

Invoke Model: Cohere

Invoke Model: Amazon Text

Invoke Model: Meta

Invoke Model: Amazon Embedding

Invoke Model: Stability AI

Invoke Model: Amazon Image

Invoke Model: Mistral AI

List Foundation Models

Create Model Customization Job

Get Model Customization Job

Conclusion

Reference

Related posts