[ad_1]
|
Replace October 10, 2023 — Amazon Bedrock is now accessible in 3 areas globally: US East (N. Virginia), US West (Oregon), and Asia Pacific (Tokyo).
This April, we introduced Amazon Bedrock as a part of a set of latest instruments for constructing with generative AI on AWS. Amazon Bedrock is a totally managed service that gives a alternative of high-performing basis fashions (FMs) from main AI corporations, together with AI21 Labs, Anthropic, Cohere, Stability AI, and Amazon, together with a broad set of capabilities to construct generative AI functions, simplifying the event whereas sustaining privateness and safety.
At this time, I’m completely happy to announce that Amazon Bedrock is now usually accessible! I’m additionally excited to share that Meta’s Llama 2 13B and 70B parameter fashions will quickly be accessible on Amazon Bedrock.
Amazon Bedrock’s complete capabilities aid you experiment with a wide range of high FMs, customise them privately together with your knowledge utilizing strategies comparable to fine-tuning and retrieval-augmented era (RAG), and create managed brokers that carry out complicated enterprise duties—all with out writing any code. Take a look at my earlier posts to study extra about brokers for Amazon Bedrock and join FMs to your organization’s knowledge sources.
Be aware that some capabilities, comparable to brokers for Amazon Bedrock, together with information bases, proceed to be accessible in preview. I’ll share extra particulars on what capabilities proceed to be accessible in preview in the direction of the top of this weblog publish.
Since Amazon Bedrock is serverless, you don’t need to handle any infrastructure, and you’ll securely combine and deploy generative AI capabilities into your functions utilizing the AWS companies you’re already accustomed to.
Amazon Bedrock is built-in with Amazon CloudWatch and AWS CloudTrail to help your monitoring and governance wants. You should use CloudWatch to trace utilization metrics and construct personalized dashboards for audit functions. With CloudTrail, you’ll be able to monitor API exercise and troubleshoot points as you combine different techniques into your generative AI functions. Amazon Bedrock additionally lets you construct functions which might be in compliance with the GDPR and you need to use Amazon Bedrock to run delicate workloads regulated below the U.S. Well being Insurance coverage Portability and Accountability Act (HIPAA).
Get Began with Amazon Bedrock
You possibly can entry accessible FMs in Amazon Bedrock by the AWS Administration Console, AWS SDKs, and open-source frameworks comparable to LangChain.
Within the Amazon Bedrock console, you’ll be able to browse FMs and discover and cargo instance use circumstances and prompts for every mannequin. First, you must allow entry to the fashions. Within the console, choose Mannequin entry within the left navigation pane and allow the fashions you wish to entry. As soon as mannequin entry is enabled, you’ll be able to check out totally different fashions and inference configuration settings to discover a mannequin that matches your use case.
For instance, right here’s a contract entity extraction use case instance utilizing Cohere’s Command mannequin:
The instance exhibits a immediate with a pattern response, the inference configuration parameter settings for the instance, and the API request that runs the instance. If you choose Open in Playground, you’ll be able to discover the mannequin and use case additional in an interactive console expertise.
Amazon Bedrock presents chat, textual content, and picture mannequin playgrounds. Within the chat playground, you’ll be able to experiment with numerous FMs utilizing a conversational chat interface. The next instance makes use of Anthropic’s Claude mannequin:
As you consider totally different fashions, you must attempt numerous immediate engineering strategies and inference configuration parameters. Immediate engineering is a brand new and thrilling talent centered on higher perceive and apply FMs to your duties and use circumstances. Efficient immediate engineering is about crafting the proper question to get essentially the most out of FMs and acquire correct and exact responses. Usually, prompts needs to be easy, simple, and keep away from ambiguity. It’s also possible to present examples within the immediate or encourage the mannequin to purpose by extra complicated duties.
Inference configuration parameters affect the response generated by the mannequin. Parameters comparable to Temperature, Prime P, and Prime Ok provide you with management over the randomness and variety, and Most Size or Max Tokens management the size of mannequin responses. Be aware that every mannequin exposes a distinct however typically overlapping set of inference parameters. These parameters are both named the identical between fashions or comparable sufficient to purpose by while you check out totally different fashions.
We focus on efficient immediate engineering strategies and inference configuration parameters in additional element in week 1 of the Generative AI with Giant Language Fashions on-demand course, developed by AWS in collaboration with DeepLearning.AI. It’s also possible to verify the Amazon Bedrock documentation and the mannequin supplier’s respective documentation for extra ideas.
Subsequent, let’s see how one can work together with Amazon Bedrock by way of APIs.
Utilizing the Amazon Bedrock API
Working with Amazon Bedrock is so simple as choosing an FM on your use case after which making just a few API calls. Within the following code examples, I’ll use the AWS SDK for Python (Boto3) to work together with Amazon Bedrock.
Record Accessible Basis Fashions
First, let’s arrange the boto3 consumer after which use list_foundation_models() to see essentially the most up-to-date checklist of obtainable FMs:
import boto3
import json
bedrock = boto3.consumer(
service_name="bedrock",
region_name="us-east-1"
)
bedrock.list_foundation_models()Run Inference Utilizing Amazon Bedrock’s InvokeModel API
Subsequent, let’s carry out an inference request utilizing Amazon Bedrock’s InvokeModel API and boto3 runtime consumer. The runtime consumer manages the information aircraft APIs, together with the InvokeModel API.
The InvokeModel API expects the next parameters:
The modelId parameter identifies the FM you wish to use. The request physique is a JSON string containing the immediate on your activity, along with any inference configuration parameters. Be aware that the immediate format will differ primarily based on the chosen mannequin supplier and FM. The contentType and settle for parameters outline the MIME kind of the information within the request physique and response and default to utility/json. For extra data on the newest fashions, InvokeModel API parameters, and immediate codecs, see the Amazon Bedrock documentation.
Instance: Textual content Era Utilizing AI21 Lab’s Jurassic-2 Mannequin
Here’s a textual content era instance utilizing AI21 Lab’s Jurassic-2 Extremely mannequin. I’ll ask the mannequin to inform me a knock-knock joke—my model of a Hi there World.
bedrock_runtime = boto3.consumer(
service_name="bedrock-runtime",
region_name="us-east-1"
)
modelId = 'ai21.j2-ultra-v1'
settle for="utility/json"
contentType="utility/json"
physique = json.dumps(
{"immediate": "Knock, knock!",
"maxTokens": 200,
"temperature": 0.7,
"topP": 1,
}
)
response = bedrock_runtime.invoke_model(
physique=physique,
modelId=modelId,
settle for=settle for,
contentType=contentType
)
response_body = json.hundreds(response.get('physique').learn())Right here’s the response:
It’s also possible to use the InvokeModel API to work together with embedding fashions.
Instance: Create Textual content Embeddings Utilizing Amazon’s Titan Embeddings Mannequin
Textual content embedding fashions translate textual content inputs, comparable to phrases, phrases, or presumably giant models of textual content, into numerical representations, often called embedding vectors. Embedding vectors seize the semantic that means of the textual content in a high-dimension vector house and are helpful for functions comparable to personalization or search. Within the following instance, I’m utilizing the Amazon Titan Embeddings mannequin to create an embedding vector.
immediate = "Knock-knock jokes are hilarious."
physique = json.dumps({
"inputText": immediate,
})
model_id = 'amazon.titan-embed-text-v1'
settle for="utility/json"
content_type="utility/json"
response = bedrock_runtime.invoke_model(
physique=physique,
modelId=model_id,
settle for=settle for,
contentType=content_type
)
response_body = json.hundreds(response['body'].learn())
embedding = response_body.get('embedding')The embedding vector (shortened) will look just like this:
[0.82421875, -0.6953125, -0.115722656, 0.87890625, 0.05883789, -0.020385742, 0.32421875, -0.00078201294, -0.40234375, 0.44140625, ...]
Be aware that Amazon Titan Embeddings is obtainable at this time. The Amazon Titan Textual content household of fashions for textual content era continues to be accessible in restricted preview.
Run Inference Utilizing Amazon Bedrock’s InvokeModelWithResponseStream API
The InvokeModel API request is synchronous and waits for all the output to be generated by the mannequin. For fashions that help streaming responses, Bedrock additionally presents an InvokeModelWithResponseStream API that permits you to invoke the desired mannequin to run inference utilizing the supplied enter however streams the response because the mannequin generates the output.
Streaming responses are notably helpful for responsive chat interfaces to maintain the person engaged in an interactive utility. Here’s a Python code instance utilizing Amazon Bedrock’s InvokeModelWithResponseStream API:
response = bedrock_runtime.invoke_model_with_response_stream(
modelId=modelId,
physique=physique)
stream = response.get('physique')
if stream:
for occasion in stream:
chunk=occasion.get('chunk')
if chunk:
print(json.hundreds(chunk.get('bytes').decode))Knowledge Privateness and Community Safety
With Amazon Bedrock, you’re accountable for your knowledge, and all of your inputs and customizations stay non-public to your AWS account. Your knowledge, comparable to prompts, completions, and fine-tuned fashions, shouldn’t be used for service enchancment. Additionally, the information is rarely shared with third-party mannequin suppliers.
Your knowledge stays within the Area the place the API name is processed. All knowledge is encrypted in transit with a minimal of TLS 1.2 encryption. Knowledge at relaxation is encrypted with AES-256 utilizing AWS KMS managed knowledge encryption keys. It’s also possible to use your personal keys (buyer managed keys) to encrypt the information.
You possibly can configure your AWS account and digital non-public cloud (VPC) to make use of Amazon VPC endpoints (constructed on AWS PrivateLink) to securely connect with Amazon Bedrock over the AWS community. This permits for safe and personal connectivity between your functions working in a VPC and Amazon Bedrock.
Governance and Monitoring
Amazon Bedrock integrates with IAM that will help you handle permissions for Amazon Bedrock. Such permissions embody entry to particular fashions, playground, or options inside Amazon Bedrock. All AWS-managed service API exercise, together with Amazon Bedrock exercise, is logged to CloudTrail inside your account.
Amazon Bedrock emits knowledge factors to CloudWatch utilizing the AWS/Bedrock namespace to trace widespread metrics comparable to InputTokenCount, OutputTokenCount, InvocationLatency, and (variety of) Invocations. You possibly can filter outcomes and get statistics for a selected mannequin by specifying the mannequin ID dimension while you seek for metrics. This close to real-time perception helps you observe utilization and value (enter and output token rely) and troubleshoot efficiency points (invocation latency and variety of invocations) as you begin constructing generative AI functions with Amazon Bedrock.
Billing and Pricing Fashions
Listed here are a few issues round billing and pricing fashions to remember when utilizing Amazon Bedrock:
Billing – Textual content era fashions are billed per processed enter tokens and per generated output tokens. Textual content embedding fashions are billed per processed enter tokens. Picture era fashions are billed per generated picture.
Pricing Fashions – Amazon Bedrock offers two pricing fashions, on-demand and provisioned throughput. On-demand pricing lets you use FMs on a pay-as-you-go foundation with out having to make any time-based time period commitments. Provisioned throughput is primarily designed for big, constant inference workloads that want assured throughput in trade for a time period dedication. Right here, you specify the variety of mannequin models of a selected FM to fulfill your utility’s efficiency necessities as defined by the utmost variety of enter and output tokens processed per minute. For detailed pricing data, see Amazon Bedrock Pricing.
Now Accessible
Amazon Bedrock is obtainable at this time in AWS Areas US East (N. Virginia) and US West (Oregon). To study extra, go to Amazon Bedrock, verify the Amazon Bedrock documentation, discover the generative AI house at group.aws, and get hands-on with the Amazon Bedrock workshop. You possibly can ship suggestions to AWS re:Publish for Amazon Bedrock or by your normal AWS contacts.
(Accessible in Preview) The Amazon Titan Textual content household of textual content era fashions, Stability AI’s Secure Diffusion XL picture era mannequin, and brokers for Amazon Bedrock, together with information bases, proceed to be accessible in preview. Attain out by your normal AWS contacts when you’d like entry.
(Coming Quickly) The Llama 2 13B and 70B parameter fashions by Meta will quickly be accessible by way of Amazon Bedrock’s absolutely managed API for inference and fine-tuning.
Begin constructing generative AI functions with Amazon Bedrock, at this time!
— Antje
[ad_2]