AWS Bedrock LLM
Use full-parameter foundational and custom models hosted on AWS via Bedrock for RAG + Agents.
AWS Bedrock LLM
AWS Bedrock offers a very simple deployment service of full state of the art un-quantized foundational LLMs for you to run on your AWS account. While these models and their data should be private to your account each model does have its own EULA and can vary from model to model.
Connecting to AWS Bedrock
Valid account setup required!
You must have a valid AWS Account to use AWS Bedrock - this service is not free.
You must be an administrator or root user of the account to follow the below steps. Otherwise, consult with your account admin.
Obtain IAM credentials to use for Bedrock
Heads up! You can optionally use session tokens for your AWS account if
you do not want to create a new IAM user. Both forms of authentication are
supported for SyphaChat's AWS Bedrock integration. Learn more about
temporary session tokens on AWS
ā
the following below is for using IAM user authentication. If you are using session tokens this section is not relevant.
Do not use your root AWS access ID and Key for Bedrock. You should always use the least privileged policy possible when using AWS keys in any third-party applications, not just SyphaChat.
Ideally, you should create a new user with at least the following permission and policy below. The below policy does not limit access to any particular model, but is good for starting. It is up to you to modify the access level of whatever credentials will be used with AWS Bedrock depending on your use-case.
The below credentials allow a user to see models and stream responses as well as default synchronous response - which is used for @agents.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"bedrock:InvokeModel",
"bedrock:ListCustomModels",
"bedrock:GetPrompt",
"bedrock:GetFoundationModel",
"bedrock:InvokeModelWithResponseStream",
"bedrock:GetCustomModel",
"bedrock:ListFoundationModels"
],
"Resource": "*"
}
]
}Once a user with the correct permissions is defined or found you will need their AWS ACCESS ID and AWS ACCESS SECRET for connecting.
Copy the desired model you wish to use.
Please only use a chat capable model for LLM selection. Incompatible model types will not produce chats in SyphaChat.
All models on Bedrock require permission from the model owner for use on your AWS account. Access is usually granted very quickly.
You can request model permission from a link like this (replace us-west-2 with you region)
https://us-west-2.console.aws.amazon.com/bedrock/home?region=us-west-2#/modelaccess
Below are where you can find model ids that you can use in SyphaChat.
Now that you have set up and have AWS Bedrock ready for LLM chatting, you can input the credentials to SyphaChat.
AWS Bedrock IAM Access ID
This is the ACCESS ID of the user who can invoke models on Bedrock
AWS Bedrock IAM Access Key
This is the ACCESS SECRET of the user who can invoke models on Bedrock
AWS Bedrock Session Token
This is the session token of the user who can invoke models on Bedrock. You must be using temporary session tokens to use this and have selected the Session Token form of authentication in the UI.
The default form of authentication is using IAM user credentials.
AWS region
This is the region that your account is enabled to use with AWS Bedrock - this depends on your region preference and account settings.
An example is us-west-2 Oregon. Yours may be elsewhere.
Model ID
This is the model id copied from the screenshot above. If you are using a custom model, your key will be available elsewhere in the Bedrock UI.
Model context window
This is the maximum amount of tokens that can exist in a single query for a model. This is a model specific parameter and will vary from model to model. Consult with AWS Bedrock's UI or the model provider documentation for what this limit may be.