Ends in
00
days
00
hrs
00
mins
00
secs
ENROLL NOW

📚 $2.99 eBooks and Get 30% OFF AWS Pro, Specialty, & Claude Practice Exams & Courses

Search Your Notes by Meaning with Amazon Bedrock Embeddings

Home » AWS » Search Your Notes by Meaning with Amazon Bedrock Embeddings

Search Your Notes by Meaning with Amazon Bedrock Embeddings

Search your notes for “cloud bill” and keyword search comes back empty, even if you wrote a note about exactly that. Your note says “AWS Budgets” and “spending”, but keyword search only matches words that appear in the text. Semantic search matches meaning instead, and Amazon Bedrock embeddings let you build it with a few short Python scripts. In this tutorial, you index ten study notes with Amazon Titan Text Embeddings V2, then run semantic search and BM25 keyword search against the same eleven questions. Semantic search put the right passage in its top 3 for all ten answerable questions; keyword search missed one, and it was the cloud bill question.

  • What you build: a command-line search over a folder of Markdown notes, in six short Python scripts.
  • What you compare: semantic search with Titan Text Embeddings V2 against BM25 keyword search, scored against answers written down before the test.
  • What you need: an AWS account you can sign in to through the AWS Management Console, and Python 3.12 or later. You don’t need the AWS CLI.
  • What it costs: 659 input tokens for the whole tutorial, about $0.000013, with no AWS resources to clean up.

What Are Amazon Bedrock Embeddings?

An embedding is a list of numbers that represents what a piece of text means. Amazon Bedrock serves embedding models through the same runtime API as its text models: you send text and get a vector back. Texts with similar meaning produce vectors that point in similar directions, even when they share no words. Semantic search uses that property to find a note that answers a question written in different words.

This tutorial uses Amazon Titan Text Embeddings V2, model ID amazon.titan-embed-text-v2:0. It accepts up to 8,192 tokens or 50,000 characters per request and returns a vector of 1,024, 512, or 256 numbers. The model returns the vector and nothing else. Storing and comparing vectors is therefore up to you, in your own code or in a vector database.

Search returns passages from your notes, but it does not write answers. That retrieval step is the first half of retrieval-augmented generation (RAG), and this tutorial stops before the generation half. Because nothing is generated, a bad result shows up as the wrong passage instead of a plausible-sounding answer. You also see which passage matched each question and its score.

Amazon Bedrock embeddings semantic search flow: notes are embedded once into index.json, and each question is embedded and compared with cosine similarity

What You Will Build with Amazon Bedrock Embeddings

You will build a command-line tool that searches a folder of Markdown notes. It splits each note into passages, one per paragraph, embeds every passage once with Amazon Bedrock, and saves the vectors in a local index.json file. A second script ranks the same passages with BM25, a long-established keyword-ranking formula, so you can compare both methods on the same questions. You use the Amazon Bedrock console to look up the model and generate a short-term API key, and the scripts send that key with each request. Everything runs on your computer except the embedding calls, and no AWS resources are created.

File What it does
passages.py Splits each note into passages, one per paragraph
embedding.py Calls Titan Text Embeddings V2 through Amazon Bedrock
index_notes.py Embeds every passage once and saves index.json
semantic_search.py Ranks passages by meaning
keyword_search.py Ranks passages by shared words (BM25)
queries.json Eleven test questions, ten of them with the passage that should answer them
compare.py Runs the test questions through both searches and prints a table

Prerequisites

  • An AWS account and an IAM user or role that can sign in to the AWS Management Console. Avoid the root user for everyday work.
  • Permission to use Amazon Bedrock in us-east-1. The AWS managed policy AmazonBedrockLimitedAccess, which the console’s API keys page recommends, covers the console pages and the API calls in this tutorial.
  • Python 3.12 or later. This tutorial was tested with Python 3.14.6 and 3.12.12.
  • A code editor and a terminal. The screenshots use Visual Studio Code on macOS.

If you prefer a narrower policy, the scripts need two permissions. bedrock:InvokeModel lets the identity send text to the model, and bedrock:CallWithBearerToken lets it authenticate that call with an Amazon Bedrock API key:

JSON
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "bedrock:InvokeModel",
      "Resource": "arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v2:0"
    },
    {
      "Effect": "Allow",
      "Action": "bedrock:CallWithBearerToken",
      "Resource": "*"
    }
  ]
}

However, this policy covers the scripts only. Browsing the console in Steps 1 and 2 needs more, such as the bedrock:Get* and bedrock:List* read permissions in AmazonBedrockLimitedAccess. You can test the policy in the IAM policy simulator before you attach it. In the test run for this tutorial, the simulator allowed bedrock:InvokeModel on Titan Text Embeddings V2 in us-east-1 and denied it for the older Titan Embeddings G1 model and for us-west-2. If you change the model or the Region, change the ARN to match, or the first call fails with AccessDeniedException.

Titan Text Embeddings V2 costs $0.02 per 1 million input tokens on demand in us-east-1 (Amazon Bedrock pricing, checked November 16, 2026). The complete run in this tutorial used 659 input tokens, which comes to about $0.000013. Your notes are sent to Amazon Bedrock as plain text. For this reason, use the sample notes, or notes that contain nothing sensitive.

Implementation Steps

Step 1: Find Titan Text Embeddings V2 in the Amazon Bedrock Console

Sign in to the AWS Management Console and set the Region selector at the top of the page to US East (N. Virginia), which is us-east-1. Open the Amazon Bedrock console, choose Model catalog in the left navigation pane, and search for Titan Text Embeddings V2. Then choose the model to open its details page. The Details section lists what the scripts depend on: the model ID amazon.titan-embed-text-v2:0, text input of up to 8k tokens, and an embedding output of 1,024, 512, or 256 numbers.

Amazon Bedrock embeddings model Titan Text Embeddings V2 in the Amazon Bedrock console model catalog, with the model ID highlighted

You don’t need to request access to this model first, because Amazon Bedrock enables access to Amazon’s serverless models by default. The model ID on this page is the MODEL_ID value that embedding.py uses later, in Step 5. The Open in playground button is unavailable for this model, so you call it from Python instead. The model returns a vector of numbers rather than text, and the later steps compare those vectors.

Step 2: Generate a Short-Term Amazon Bedrock API Key

Before the scripts can call Amazon Bedrock, they need a way to prove who is calling. Instead of installing the AWS CLI and signing in from the terminal, you can generate an Amazon Bedrock API key in the console. A short-term key inherits the permissions of your console session and works only in the Region where you generate it. It expires when your console session does, after 12 hours at most, and it creates no IAM user or other AWS resource.

Tutorials dojo strip

In the left navigation pane, choose API keys. The page offers two key types: a long-term key creates an IAM user, and AWS recommends long-term keys only for exploration. This tutorial uses a short-term key instead. Then, on the Short-term API keys tab, choose Generate short-term API keys.

API keys page in the Amazon Bedrock console with the Generate short-term API keys button highlighted

The dialog that opens shows the key only once. Under Use your API key, keep the MacOS/Linux shell tab selected and choose Copy API Key next to the export AWS_BEARER_TOKEN_BEDROCK= command. You paste that command into your terminal in Step 3. The dialog warns that you can’t retrieve the key after you leave it, so if you lose it, generate a new one.

Short-term API keys dialog in the Amazon Bedrock console with a sample key and the macOS and Linux export command highlighted

Treat the key like a password. Don’t paste it into your code, commit it to Git, or show it in a screenshot; the key in the screenshot above was replaced with a sample value. The dialog shows an expiry of 12 hours, but the key stops working sooner if your console session ends first. A copy left in your shell history stops working at the same time.

Step 3: Set Up Python for Amazon Bedrock Embeddings

Create a project folder and a Python virtual environment, then install boto3, the AWS SDK for Python. Boto3 1.39.0 and later read an Amazon Bedrock API key from the AWS_BEARER_TOKEN_BEDROCK environment variable. The scripts therefore need no AWS credentials file, CLI profile, or access keys. Run these commands in a terminal:

Terminal
mkdir bedrock-notes-search
cd bedrock-notes-search
python3 -m venv .venv
source .venv/bin/activate
python -m pip install boto3

Next, paste the export command you copied in Step 2 and press Enter. It has this form, with your key in place of the placeholder:

Terminal
export AWS_BEARER_TOKEN_BEDROCK=<your-api-key>

Then check that the terminal has the key, without printing the key itself:

Terminal
python -c "import os; print('API key set:', 'AWS_BEARER_TOKEN_BEDROCK' in os.environ)"
Output
API key set: True

VS Code terminal after creating the virtual environment, installing boto3, and confirming that the API key is set

Open the bedrock-notes-search folder in your editor and run every later command from it. The variable lasts only as long as this terminal session. When you open a new terminal, run source .venv/bin/activate and paste the export command again. If the key has expired by then, generate a new one in Step 2.

Step 4: Create the Sample Notes

The sample set has ten short study notes on AWS topics, each with one or two paragraphs. Using the same notes also lets you compare your Amazon Bedrock embeddings results with the ones in this tutorial. Run this script from the project folder. It creates the notes folder and all ten files:

Terminal
mkdir -p notes

cat > notes/aws-budgets.md <<'EOF'
AWS Budgets sends an alert when actual or forecasted spending crosses a
threshold you set. A monthly cost budget is one of the first things to
create in a new account.

Budget alerts can go to email addresses or to an Amazon SNS topic.
EOF

cat > notes/bedrock-access-denied.md <<'EOF'
AccessDeniedException from InvokeModel means the caller is not allowed
to use that model. Check that the IAM policy allows bedrock:InvokeModel
on the model's ARN and that no service control policy blocks it.

Also confirm that the code uses the exact model ID and a Region where
the model is available.
EOF

cat > notes/cloudwatch-alarms.md <<'EOF'
A CloudWatch alarm watches one metric and changes state when the value
crosses a threshold for a number of evaluation periods. Point the alarm
at an SNS topic to get notified.
EOF

cat > notes/dynamodb-partition-keys.md <<'EOF'
Choose a DynamoDB partition key with many distinct values so requests
spread across partitions. A key with few values, such as a status field,
sends most traffic to the same partition and can cause throttling.
EOF

cat > notes/few-shot-prompting.md <<'EOF'
Few-shot prompting puts a handful of input and output examples in the
prompt so the model copies the pattern. It helps when you need a fixed
format, such as one label per line.
EOF

cat > notes/iam-least-privilege.md <<'EOF'
Least privilege means each user or role gets only the permissions its
job needs. Start from a narrow policy and add actions when a task fails,
instead of starting from administrator access and removing things later.

IAM Access Analyzer can generate a policy from the actions a role
actually used, which helps trim permissions that were never needed.
EOF

cat > notes/inference-parameters.md <<'EOF'
Temperature controls how random a model's output is. Lower values give
more predictable answers; higher values give more varied ones.

Maximum tokens caps the length of the response, which also caps the
output cost of each request.
EOF

cat > notes/lambda-timeouts.md <<'EOF'
A Lambda function stops when it reaches its configured timeout, even if
the work is not finished. The timeout has an upper limit, so raising it
does not help every long-running job.

Work that runs longer can be split into smaller steps coordinated by AWS
Step Functions, or moved to a container on Amazon ECS.
EOF

cat > notes/rag.md <<'EOF'
Retrieval-augmented generation (RAG) first retrieves passages related to
the question, then adds them to the prompt. The model answers from those
passages instead of relying only on what it learned during training.

Retrieval quality limits answer quality: if the right passage is not
retrieved, the model cannot use it.
EOF

cat > notes/s3-versioning.md <<'EOF'
With S3 Versioning turned on, overwriting or deleting an object keeps
the earlier version. You can restore a file you replaced by mistake by
copying an older version back.

Old versions still cost storage, so pair versioning with a lifecycle
rule that expires noncurrent versions after a set number of days.
EOF

Next, create passages.py. It splits each note on blank lines and turns every paragraph into a passage with an ID made from the file name and the paragraph number, such as s3-versioning.md#1. A paragraph is a useful unit for notes because it usually covers one idea. Much smaller pieces lose context, while much larger ones mix several ideas into one vector.

Python
"""Split Markdown notes into passages: one passage per paragraph."""

from pathlib import Path

NOTES_DIR = Path("notes")


def load_passages(notes_dir: Path = NOTES_DIR) -> list[dict]:
    passages = []
    for path in sorted(notes_dir.glob("*.md")):
        text = path.read_text(encoding="utf-8")
        paragraphs = [p for p in text.split("\n\n") if p.strip()]
        for number, paragraph in enumerate(paragraphs, start=1):
            passages.append({
                # file name + paragraph number, such as s3-versioning.md#1
                "id": f"{path.name}#{number}",
                # collapse line breaks and extra spaces
                "text": " ".join(paragraph.split()),
            })
    return passages

Check the split:

Terminal
python -c "from passages import load_passages; print(len(load_passages()), 'passages')"
Output
17 passages

Step 5: Generate Your First Amazon Bedrock Embedding

Create embedding.py. The embed() function sends one InvokeModel request and returns the vector with the number of input tokens you are billed for. The code contains no key, because boto3 reads it from the AWS_BEARER_TOKEN_BEDROCK variable you set in Step 3. Setting normalize to True scales every vector to a length of 1, which makes the comparison in Step 7 a plain dot product. DIMENSIONS = 1024 is the model’s default size; 512 and 256 produce smaller vectors.

Python
"""Turn text into an embedding with Amazon Titan Text Embeddings V2."""

import json

import boto3

REGION = "us-east-1"
MODEL_ID = "amazon.titan-embed-text-v2:0"
DIMENSIONS = 1024  # Titan V2 also accepts 512 or 256

bedrock = boto3.client("bedrock-runtime", region_name=REGION)


def embed(text: str) -> tuple[list[float], int]:
    """Return the embedding vector and the number of input tokens it used."""
    body = json.dumps({
        "inputText": text,
        "dimensions": DIMENSIONS,
        "normalize": True,
    })
    response = bedrock.invoke_model(
        modelId=MODEL_ID,
        contentType="application/json",
        accept="application/json",
        body=body,
    )
    payload = json.loads(response["body"].read())
    return payload["embedding"], payload["inputTextTokenCount"]


if __name__ == "__main__":
    vector, tokens = embed("Grant each role only the permissions it needs.")
    print(f"Dimensions:   {len(vector)}")
    print(f"Input tokens: {tokens}")
    print(f"First values: {[round(v, 4) for v in vector[:5]]}")

Run it:

Terminal
python embedding.py
Output
Dimensions:   1024
Input tokens: 10
First values: [-0.0246, 0.0205, 0.0126, 0.0229, -0.014]

Amazon Bedrock embeddings: embedding.py in VS Code and the first embedding printed in the terminal

The vector has 1,024 numbers, and no single number means anything on its own; the meaning is in the comparison with another vector. Step 7 then makes that comparison. The sentence passed to embed() in embedding.py used 10 input tokens. At $0.02 per million input tokens, that call therefore cost $0.0000002.

Step 6: Build an Index of Amazon Bedrock Embeddings

Embedding every passage again on every search would be slow, and it would also cost tokens each time. Instead, create index_notes.py, which embeds every passage once and saves the vectors in index.json together with the model ID and vector size. Later, the search script checks those two values and refuses to compare vectors made with different settings. Run this script again whenever you add or change a note.

Python
"""Embed every passage once and save the vectors to index.json."""

import json
from pathlib import Path

from embedding import DIMENSIONS, MODEL_ID, embed
from passages import NOTES_DIR, load_passages

INDEX_FILE = Path("index.json")

passages = load_passages()
if not passages:
    raise SystemExit(f"No .md notes found in {NOTES_DIR}/")

total_tokens = 0
for passage in passages:
    passage["embedding"], tokens = embed(passage["text"])
    total_tokens += tokens

index = {"model_id": MODEL_ID, "dimensions": DIMENSIONS, "passages": passages}
INDEX_FILE.write_text(json.dumps(index), encoding="utf-8")
print(f"Indexed {len(passages)} passages from {NOTES_DIR}/ "
      f"({total_tokens} input tokens)")

Build the index:

Terminal
python index_notes.py
Output
Indexed 17 passages from notes/ (536 input tokens)

index_notes.py open in VS Code after indexing 17 note passages into index.json

Step 7: Search Your Notes with Amazon Bedrock Embeddings

Create semantic_search.py. It embeds the question with the same model and then compares that vector with every saved vector. Because all vectors have a length of 1, their dot product equals their cosine similarity: values closer to 1 mean closer meaning. The score ranks passages against each other, but it is not a probability that a passage is correct.

Python
"""Rank passages by meaning: embed the question, then compare vectors."""

import json
import sys
import textwrap
from pathlib import Path

from embedding import DIMENSIONS, MODEL_ID, embed

INDEX_FILE = Path("index.json")


def load_index() -> list[dict]:
    if not INDEX_FILE.exists():
        raise SystemExit("index.json not found. Run: python index_notes.py")
    index = json.loads(INDEX_FILE.read_text(encoding="utf-8"))
    if index["model_id"] != MODEL_ID or index["dimensions"] != DIMENSIONS:
        raise SystemExit("index.json was built with different settings. "
                         "Run: python index_notes.py")
    return index["passages"]


def semantic_search(
    question: str, passages: list[dict], top_k: int = 3
) -> list[tuple[float, dict]]:
    query_vector, _ = embed(question)
    scored = []
    for passage in passages:
        vector = passage["embedding"]
        # Both vectors are normalized, so the dot product is cosine similarity.
        score = sum(q * p for q, p in zip(query_vector, vector, strict=True))
        scored.append((score, passage))
    scored.sort(key=lambda item: item[0], reverse=True)
    return scored[:top_k]


if __name__ == "__main__":
    question = " ".join(sys.argv[1:]).strip()
    if not question:
        raise SystemExit('Usage: python semantic_search.py "your question"')
    for score, passage in semantic_search(question, load_index()):
        print(f"{score:.3f}  {passage['id']}")
        wrapped = textwrap.fill(passage["text"], 72)
        print(textwrap.indent(wrapped, "       "), end="\n\n")

Ask a Paraphrased Question

Try a question that describes least privilege without naming it:

Terminal
python semantic_search.py "How can I avoid giving someone more access than needed?"
Output
0.287  iam-least-privilege.md#2
       IAM Access Analyzer can generate a policy from the actions a role
       actually used, which helps trim permissions that were never needed.

0.233  iam-least-privilege.md#1
       Least privilege means each user or role gets only the permissions its
       job needs. Start from a narrow policy and add actions when a task fails,
       instead of starting from administrator access and removing things later.

0.162  rag.md#2
       Retrieval quality limits answer quality: if the right passage is not
       retrieved, the model cannot use it.

Amazon Bedrock embeddings semantic search results for a paraphrased question in the VS Code terminal

Both top results come from the least-privilege note, although the question never mentions IAM or least privilege. The paragraph about IAM Access Analyzer ranks first at 0.287, just ahead of the paragraph that defines least privilege at 0.233. The third result, from rag.md, scores 0.162 but has nothing to do with permissions. So read the scores as an order and a gap between results, not as percentages.

Step 8: Add a BM25 Keyword Search Baseline

Create keyword_search.py, the baseline that the Amazon Bedrock embeddings search is compared against. It runs locally without calling Amazon Bedrock and implements BM25, which scores a passage by the question words it contains. Rare words count more than common ones, repeated words add less each time, and long passages are scored down slightly. K1 sets how fast repeated words stop adding to the score, and B sets how much passage length matters. The script has no stop-word list, so common words such as “how” count too.

Write the BM25 Script

Apart from passages.py, the script uses only the Python standard library:

Python
"""Rank passages by shared words with BM25, a classic ranking formula."""

import math
import re
import sys
import textwrap
from collections import Counter

from passages import load_passages

K1 = 1.5   # how fast repeats of the same word stop adding to the score
B = 0.75   # how much longer passages are penalized


def tokenize(text: str) -> list[str]:
    return re.findall(r"[a-z0-9]+", text.lower())


def keyword_search(
    question: str, passages: list[dict], top_k: int = 3
) -> list[tuple[float, dict]]:
    docs = [tokenize(p["text"]) for p in passages]
    avg_len = sum(len(words) for words in docs) / len(docs)
    doc_freq = Counter(term for words in docs for term in set(words))
    n = len(docs)

    scored = []
    for passage, words in zip(passages, docs):
        counts = Counter(words)
        length_norm = 1 - B + B * len(words) / avg_len
        score = 0.0
        for term in set(tokenize(question)):
            if term not in counts:
                continue
            df = doc_freq[term]
            idf = math.log((n - df + 0.5) / (df + 0.5) + 1)
            tf = counts[term]
            score += idf * tf * (K1 + 1) / (tf + K1 * length_norm)
        # passages that share no word with the question are not results
        if score > 0:
            scored.append((score, passage))
    scored.sort(key=lambda item: item[0], reverse=True)
    return scored[:top_k]


if __name__ == "__main__":
    question = " ".join(sys.argv[1:]).strip()
    if not question:
        raise SystemExit('Usage: python keyword_search.py "your question"')
    results = keyword_search(question, load_passages())
    if not results:
        print("No passage shares a word with the question.")
    for score, passage in results:
        print(f"{score:5.2f}  {passage['id']}")
        wrapped = textwrap.fill(passage["text"], 72)
        print(textwrap.indent(wrapped, "       "), end="\n\n")

Run the Keyword Search

Run the same question:

Terminal
python keyword_search.py "How can I avoid giving someone more access than needed?"
Output
 6.54  inference-parameters.md#1
       Temperature controls how random a model's output is. Lower values give
       more predictable answers; higher values give more varied ones.

 6.10  iam-least-privilege.md#2
       IAM Access Analyzer can generate a policy from the actions a role
       actually used, which helps trim permissions that were never needed.

 1.70  iam-least-privilege.md#1
       Least privilege means each user or role gets only the permissions its
       job needs. Start from a narrow policy and add actions when a task fails,
       instead of starting from administrator access and removing things later.

BM25 keyword search results in the VS Code terminal for the least-privilege question

However, keyword search ranks the temperature passage from inference-parameters.md first. That passage shares only “how” and “more” with the question, and BM25 cannot tell that those words carry little meaning. The two least-privilege passages follow because they contain “access”, and one of them also contains “needed”. Still, keeping every word is deliberate: the point is to test plain BM25, not a tuned version.

Step 9: Compare Keyword Search with Semantic Search

One question proves little, so create queries.json with eleven questions. Seven are paraphrases; one is an exact term, one is an error code, one is an acronym, and one has no answer in the notes. Every question except the unrelated one names the passage that should answer it, and those expected passages were chosen before running either search. Write them down first in your own experiments too; otherwise, the test will drift toward whatever the search returns.

JSON
[
  {"type": "paraphrase", "expected": "iam-least-privilege.md#1",
   "question": "How can I avoid giving someone more access than needed?"},
  {"type": "paraphrase", "expected": "s3-versioning.md#1",
   "question": "How do I get back a file I overwrote by accident?"},
  {"type": "paraphrase", "expected": "aws-budgets.md#1",
   "question": "Warn me before my cloud bill gets too high"},
  {"type": "paraphrase", "expected": "lambda-timeouts.md#1",
   "question": "My serverless function gets cut off on long jobs"},
  {"type": "paraphrase", "expected": "dynamodb-partition-keys.md#1",
   "question": "Why is one database partition getting most of the traffic?"},
  {"type": "paraphrase", "expected": "few-shot-prompting.md#1",
   "question": "Show the model a few examples of the format I want"},
  {"type": "paraphrase", "expected": "inference-parameters.md#1",
   "question": "How do I make the answers less random?"},
  {"type": "exact term", "expected": "iam-least-privilege.md#2",
   "question": "IAM Access Analyzer"},
  {"type": "error code", "expected": "bedrock-access-denied.md#1",
   "question": "AccessDeniedException"},
  {"type": "acronym", "expected": "rag.md#1",
   "question": "RAG"},
  {"type": "unrelated", "expected": null,
   "question": "How do I bake banana bread?"}
]

Create the Comparison Script

Create compare.py. It runs every question through both searches and prints the rank of the expected passage in each search’s top 3, or miss when the passage is not there. For the unrelated question, it prints the top score instead, because there is no right answer to find. Long questions are shortened with “…” so the table fits a terminal. Two summary lines follow: how many answerable questions found their passage in the top 3, and the fastest and slowest semantic search times, including the Amazon Bedrock call.

Python
"""Run every test question through both searches and compare the ranks."""

import json
import time
from pathlib import Path

from keyword_search import keyword_search
from semantic_search import load_index, semantic_search

passages = load_index()
queries = json.loads(Path("queries.json").read_text(encoding="utf-8"))


def rank(results: list[tuple[float, dict]], expected: str | None) -> str:
    if expected is None:  # unrelated question: show the top score instead
        return f"top {results[0][0]:.2f}" if results else "no result"
    ids = [passage["id"] for _, passage in results]
    return f"#{ids.index(expected) + 1}" if expected in ids else "miss"


def short(text: str, width: int = 44) -> str:
    return text if len(text) <= width else text[:width - 1] + "…"


print(f"{'TYPE':<11} {'QUESTION':<44} {'KEYWORD':>9} {'SEMANTIC':>9}")
hits = {"keyword": 0, "semantic": 0}
semantic_ms = []
for q in queries:
    keyword = keyword_search(q["question"], passages)
    start = time.perf_counter()
    semantic = semantic_search(q["question"], passages)
    semantic_ms.append((time.perf_counter() - start) * 1000)

    k, s = rank(keyword, q["expected"]), rank(semantic, q["expected"])
    hits["keyword"] += k.startswith("#")
    hits["semantic"] += s.startswith("#")
    print(f"{q['type']:<11} {short(q['question']):<44} {k:>9} {s:>9}")

answerable = sum(q["expected"] is not None for q in queries)
print(f"\nExpected passage in top 3: keyword {hits['keyword']}/{answerable}, "
      f"semantic {hits['semantic']}/{answerable}")
print(f"Semantic search time per question: {min(semantic_ms):.0f}-"
      f"{max(semantic_ms):.0f} ms (includes the Bedrock call)")

Amazon Bedrock embeddings compared with BM25 keyword search: compare.py checks whether the expected passage ranks in the top 3

Run the Comparison

Then start the comparison from the project folder:

Terminal
python compare.py
Output
TYPE        QUESTION                                       KEYWORD  SEMANTIC
paraphrase  How can I avoid giving someone more access …        #3        #2
paraphrase  How do I get back a file I overwrote by acc…        #1        #1
paraphrase  Warn me before my cloud bill gets too high        miss        #1
paraphrase  My serverless function gets cut off on long…        #1        #2
paraphrase  Why is one database partition getting most …        #1        #1
paraphrase  Show the model a few examples of the format…        #1        #1
paraphrase  How do I make the answers less random?              #1        #1
exact term  IAM Access Analyzer                                 #1        #1
error code  AccessDeniedException                               #1        #1
acronym     RAG                                                 #1        #1
unrelated   How do I bake banana bread?                   top 2.74  top 0.07

Expected passage in top 3: keyword 9/10, semantic 10/10
Semantic search time per question: 351-1387 ms (includes the Bedrock call)

compare.py results table in the VS Code terminal: keyword search vs semantic search on eleven questions

Results: Keyword Search vs. Semantic Search

Question type Questions Keyword search (top 3) Semantic search (top 3)
Paraphrase 7 6 of 7 7 of 7
Exact term, error code, acronym 3 3 of 3 3 of 3
Unrelated (no right answer) 1 Returned a passage, score 2.74 Best score only 0.07
TD for Business

Across the ten questions with an expected passage, semantic search put it in the top 3 every time and ranked it first eight times. Keyword search found it in the top 3 nine times and also ranked it first eight times. When semantic search ranked the expected passage second, the first result was the other paragraph of the same note. Step 7 shows this for the least-privilege question, and running the Lambda question through semantic_search.py also shows it. Keyword search missed only “Warn me before my cloud bill gets too high”, which shares no word with the AWS Budgets note.

The unrelated question shows a different weakness in each method. Keyword search returned the temperature passage for “How do I bake banana bread?” because both contain “how”. Semantic search always returns its top 3, but the best score for that question was 0.07, far below the 0.287 that the least-privilege question scored in Step 7. That gap comes from a small test, so it is not a tested cutoff.

Most paraphrases still share a word or two with their notes, such as “file”, “partition”, and “examples”, because the same person wrote the notes and the questions. As a result, keyword search ranked five of the seven paraphrased passages first. Semantic searches took 351 to 1,387 ms per question, including the Amazon Bedrock call. In contrast, keyword search runs locally and makes no network call.

What Worked and What Didn’t

Semantic search with Amazon Bedrock embeddings handled the case that keyword search cannot: it matched “Warn me before my cloud bill gets too high” to the AWS Budgets note even though the two share no words. The unrelated question scored 0.07, against 0.287 for a real match in Step 7, and because semantic search always returns its top 3, that low score is the only sign that nothing matched. Exact terms worked with both methods, since “IAM Access Analyzer”, “AccessDeniedException”, and “RAG” ranked first in both searches. In other words, semantic search matched the paraphrases without giving up exact-term matches.

The gap was one question: keyword search found 9 of 10 expected passages, and semantic search found 10 of 10. The test itself explains why the gap is small, because the same person wrote the notes and the questions, so most paraphrases still share a word with their note. Twice, semantic search ranked the other paragraph of the right note first, which is fine for finding a note but matters if a RAG pipeline passes only the top passage to a model. Eleven questions, ten notes, and one model are enough to see the pattern, not to settle which method is better; a fairer test would use questions written by someone who has not read the notes.

Troubleshooting Amazon Bedrock Embeddings

Each item starts with the error or symptom you see:

  • NoCredentialsError: Unable to locate credentials. The terminal has no API key, which happens in a new terminal or when the export command was skipped. Paste the export command from Step 2 again, or generate a new key if you closed the dialog.
  • AccessDeniedException with Invalid API Key format. The variable holds only part of the key, or extra characters came along with it. Generate a new key and copy the whole command with Copy API Key instead of selecting the text by hand.
  • AccessDeniedException with Authentication failed: Please make sure your API Key is valid. The key can’t be used for this request. In the test run, this happened when a key generated in us-east-1 was used for us-west-2, because a short-term key works only in its own Region. An expired key also stops working, so generate a new key in us-east-1 and run the new export command.
  • AccessDeniedException: Your account is currently being verified. The AWS account is new, and AWS has not finished verifying it. Wait and retry; AWS says verification normally takes less than 2 hours.
  • ThrottlingException: Too many requests on the very first call. The account’s on-demand quota for the model is 0, which can happen on new accounts. In the Service Quotas console, open Amazon Bedrock and check On-demand model inference requests per minute for Amazon Titan Text Embeddings V2. If it shows 0, ask AWS Support for an increase or use an account with a non-zero quota.
  • AccessDeniedException about authorization. The identity that generated the key is not allowed bedrock:InvokeModel on this model and Region, or not allowed bedrock:CallWithBearerToken. Compare the ARN in your policy with MODEL_ID and REGION in embedding.py.
  • index.json not found. The index was never built. Run python index_notes.py.
  • index.json was built with different settings. MODEL_ID or DIMENSIONS changed after indexing. Run python index_notes.py again.

Clean Up and Cost of Amazon Bedrock Embeddings

This tutorial creates no AWS resources, so there is nothing to delete in your AWS account. The short-term API key creates no IAM user either, and it expires on its own when your console session ends, after 12 hours at most. If a key leaks before then, you can block it by attaching a policy to the identity that generated it that denies bedrock:CallWithBearerToken and bedrock-mantle:CallWithBearerToken. Your costs come only from the embedding calls, which used 659 input tokens in the test run.

When you are done, remove the key from the terminal and delete the project folder if you no longer need it. The unset command clears the variable from the current terminal only. The index.json file contains your note text, so delete it if you indexed anything private. Run these commands from the project folder:

Terminal
unset AWS_BEARER_TOKEN_BEDROCK
cd ..
rm -rf bedrock-notes-search

Conclusion

Amazon Bedrock embeddings turn a folder of notes into something you can search by meaning, with one Bedrock call per question. The whole tool fits in six small Python files and a local index file. Semantic search put every expected passage in its top 3, including one that shared no words with its question, while keyword search missed that question and matched “how” for a question that had no answer at all. Keyword search still matched every exact-term question, needs no network call, and costs nothing per request.

One next step is a hybrid search that combines both scores, so a passage can rank well through shared words, shared meaning, or both. Another is to pass the top passages to a text model on Amazon Bedrock and build the generation half of RAG on top of this index, as in Building a Cost-Aware RAG Application with Amazon Bedrock. Either way, keep the habit from Step 9 and write down the expected answers before you run the search. Otherwise, it is easy to accept whatever the search returns as correct.

References

📚 $2.99 eBooks and Get 30% OFF AWS Pro, Specialty, & Claude Practice Exams & Courses

Tutorials Dojo portal

Turn Your Team Into Cloud-Ready Professionals Today

Tutorials Dojo for Business

Learn AWS with our PlayCloud Hands-On Labs

$2.99 AWS and Azure Exam Study Guide eBooks

tutorials dojo study guide eBook

Learn GCP By Doing! Try Our GCP PlayCloud

Learn Azure with our Azure PlayCloud

FREE AI and AWS Digital Courses

FREE AWS, Azure, GCP Practice Test Samplers

SAA-C03 Exam Guide SAA-C03 examtopics AWS Certified Solutions Architect Associate

Subscribe to our YouTube Channel

Tutorials Dojo YouTube Channel

Follow Us On Linkedin

Written by: Danilo Casim Jr.

Dan is a fourth-year BS Computer Science student at Pamantasan ng Lungsod ng Muntinlupa and a Software Engineer at Tutorials Dojo. As Lead of the AWS Student Builder Group - Tempest, he builds meaningful digital solutions while giving back to the community.

AWS, Azure, and GCP Certifications are consistently among the top-paying IT certifications in the world, considering that most companies have now shifted to the cloud. Earn over $150,000 per year with an AWS, Azure, or GCP certification!

Follow us on LinkedIn, YouTube, Facebook, or join our Slack study group. More importantly, answer as many practice exams as you can to help increase your chances of passing your certification exams on your first try!

View Our AWS, Azure, and GCP Exam Reviewers Check out our FREE courses

Our Community

~98%
passing rate
Around 95-98% of our students pass the AWS Certification exams after training with our courses.
200k+
students
Over 200k enrollees choose Tutorials Dojo in preparing for their AWS Certification exams.
~4.8
ratings
Our courses are highly rated by our enrollees from all over the world.

What our students say about us?