How to Reduce Embedding Costs Without Hurting Retrieval

Reducing embedding cost is not simply a matter of selecting the cheapest model. A retrieval system pays in several places: tokens sent to the embedding API, vectors stored in the database, index memory, write operations, and repeated processing when documents change. An inexpensive embedding call can still feed an oversized and frequently rebuilt index.

The safe objective is therefore lower cost per useful retrieval, not lower cost per vector. Start with a cost ledger, remove work that carries no retrieval value, and test every model or dimension change against representative queries before it reaches production.

Price and specification statements in this article were checked against official OpenAI sources on 2026-07-23. The current OpenAI model pages list text-embedding-3-small at $0.02 per million input tokens and text-embedding-3-large at $0.13 per million input tokens (small model page, large model page). Prices and model behavior can change, so recheck the linked pages before making a purchasing or migration decision.

Cost levers at a glance

Lever	Direct cost effect	Retrieval risk	What to verify
Remove duplicate or low-value text	Fewer input tokens, vectors, and writes	Low to medium	Excluded samples still contain no unique evidence
Reduce chunk overlap	Fewer repeated tokens and vectors	Medium	Questions that cross chunk boundaries still retrieve sufficient evidence
Shorten vector dimensions	Lower raw vector storage and often lower index memory	Medium to high	Recall, ranking, latency, and database support at each tested dimension
Batch offline embedding work	Lower request overhead; provider discounts may apply	Low if inputs are unchanged	Current pricing, completion window, limits, and failed-item handling
Re-embed only changed chunks	Fewer repeat API calls and writes	Low	Fingerprints include every vector-producing setting
Switch embedding models	Different API price and potentially different storage needs	High	Quality by query segment, migration storage, and rollback path

Start with deduplication and incremental indexing because they remove avoidable work without intentionally changing the semantic representation. Treat dimension and model changes as migrations that require retrieval evaluation.

Measure the current embedding bill

Create a monthly ledger before optimizing anything. Record at least:

• source tokens processed for new documents;
• tokens processed again because of updates or rebuilds;
• accepted, rejected, and retried API requests;
• number of stored vectors and their dimensions;
• metadata, index, replica, and backup storage;
• write or ingestion charges;
• retrieval traffic and any read-related charges;
• engineering time spent on failed or unnecessary rebuilds.

Count tokens after cleaning and chunking rather than estimating from file sizes. Headers, overlap, boilerplate, and markup can change the input sent to the model. Keep original, overlap, and re-embedded tokens separate.

For OpenAI workloads, the organization usage API can report aggregated embedding input_tokens and num_model_requests, with filters or grouping for fields such as model and project (official usage API reference, checked 2026-07-24). Use provider-side usage as one ledger input, then reconcile it with your ingestion logs and invoice rather than relying on a single counter.

A useful top-line metric is:

monthly embedding API cost = embedded input tokens / 1,000,000 × provider price per million tokens

For an illustrative workload, 500 million input tokens at $0.02 per million would cost $10 for embedding calls. Vector storage, database operations, and replicas remain separate. The guide to vector database pricing explains how those meters combine.

Add quality beside cost. Track recall at a fixed result count, success on answerable questions, irrelevant-result rate, and the fraction of queries that return no usable evidence. Without that baseline, a lower invoice can conceal a retrieval regression.

Remove duplicate and low-value text

The cheapest embedding is the one you do not need to create. Deduplicate before chunking so that repeated documents do not produce slightly different chunk boundaries and escape detection.

Use several layers:

1. Canonical identity: map mirrors, exports, and alternate URLs to one document ID.
2. Exact content hash: hash normalized text after removing volatile timestamps or transport markup.
3. Section-level hash: detect repeated legal notices, navigation, footers, and copied introductions.
4. Near-duplicate review: use a conservative similarity method for documents that are almost, but not exactly, the same.

Near-duplicate removal needs safeguards: two policy pages may differ by one sentence that matters. Keep a reason for every exclusion and sample the removed set.

Low-value text is not the same as short text. A three-line error definition may be useful while a long navigation block adds nothing. Exclude content by function, but preserve titles, headings, and table labels that make passages understandable.

Chunking can create another form of duplication. A 20% overlap means part of the corpus is deliberately embedded more than once. Reduce overlap only after testing boundary questions. The RAG chunk-size guide shows how to estimate overlap overhead without treating one chunk size as universal.

Choose dimensions deliberately

Vector dimensions influence raw storage and often affect index memory and query work. For float32 vectors, the unindexed vector payload is approximately:

vector count × dimensions × 4 bytes

Ten million 3,072-dimensional float32 vectors are about 122.88 GB before metadata, index overhead, replicas, and backups. At 1,024 dimensions, the raw vectors are about 40.96 GB. These are arithmetic estimates, not a database bill; compression, quantization, index type, and provider metering can change actual consumption.

OpenAI states that text-embedding-3-large can produce up to 3,072 dimensions and that its v3 embedding models support shortening through the dimensions parameter. Its launch article gives a provider-run example in which a 256-dimensional shortened large-model embedding outperformed an unshortened 1,536-dimensional older model on MTEB (official announcement). That benchmark is evidence about those models and that test suite, not proof that 256 dimensions will preserve quality for your corpus.

Treat dimension as an evaluation variable. Test a small grid such as the current dimension, one moderate reduction, and one aggressive reduction. Keep the model, chunks, index settings, filters, and test queries fixed. Compare retrieval quality and latency, then calculate the storage reduction. Do not change model and dimension in the same experiment unless you also run combinations that isolate each effect.

OpenAI also states that its embeddings remain L2-normalized after shortening, so cosine similarity and Euclidean distance produce the same ranking for those outputs (official embeddings FAQ). Do not assume the same normalization behavior for another provider without checking its documentation.

Batch ingestion efficiently

Batching reduces request overhead and can improve ingestion throughput, but ordinary synchronous batching does not automatically reduce token-based charges. A provider may price batched inputs identically or offer a separate asynchronous batch product.

OpenAI’s Batch API currently supports /v1/embeddings, uses a 24h completion window, advertises a 50% discount, and limits embedding batches to 50,000 inputs across the requests in a batch (official Batch API reference, checked 2026-07-24). Those terms are specific to that product and may change. Confirm the current documentation and account limits before calculating savings. The OpenAI Batch API comparison covers the operational trade-offs of asynchronous processing.

Build batches by measured tokens, not document count. Set a token ceiling below the provider limit, cap input count, and retain a stable ID for every chunk.

A robust ingestion worker should:

• assemble batches from already cleaned and deduplicated chunks;
• record the model, dimensions, normalization version, and content hash;
• use idempotent upserts so a retry does not create duplicate vectors;
• retry only failed items when the API identifies them;
• apply exponential backoff with jitter to transient failures;
• checkpoint completed batches;
• separate validation failures from rate-limit and service failures.

Throughput optimization must not alter chunk boundaries or silently truncate inputs. Report accepted and rejected tokens, retry rate, and rate-limit wait time.

Re-embed only changed content

Full-corpus rebuilds are simple, but they make every edit as expensive and risky as an initial import. Use content-addressed ingestion instead.

For each chunk, store a deterministic fingerprint derived from the cleaned text plus every setting that affects the vector: embedding provider, model, dimensions, preprocessing version, and chunking version. On the next run:

• skip chunks whose fingerprint already has a valid vector;
• embed new or changed chunks;
• delete vectors whose source chunks no longer exist;
• update metadata without re-embedding when vector-relevant content is unchanged;
• queue a full migration only when the model or vector-producing configuration changes.

Keep document identity separate from the chunk fingerprint. Otherwise, inserting one paragraph can renumber later chunks and trigger a rebuild. Section- and content-based IDs are more durable than array positions.

For a model migration, write to a versioned index while the current index serves traffic. Temporary duplicate storage preserves rollback. Switch reads only after evaluation; do not overwrite the only known-good vectors.

Test retrieval before switching models

Provider benchmarks are useful screening evidence, but your release decision should use your documents and queries. Build a test set that represents the workload: common questions, rare terminology, acronyms, multilingual queries if applicable, tables, code, near-duplicate pages, filters, and questions with no relevant document.

For each query, label the passages that would provide sufficient evidence. Then run the old and proposed configurations against the same frozen corpus. At a minimum, compare:

• recall at the result count passed to the next stage;
• mean reciprocal rank when the first useful result matters;
• irrelevant results in the final candidate set;
• zero-result and no-answer behavior;
• query latency at median and tail percentiles;
• vector storage and estimated monthly cost.

Set release gates before viewing results: no loss on critical queries, no material decline in aggregate recall, and a limit on new failures. Thresholds must reflect the application.

Inspect disagreements, not only averages. A cheaper configuration may improve common queries while losing product codes or another language. Preserve the current configuration for a regressing segment if necessary.

For a more detailed test-set and ranking-metric workflow, see how to evaluate RAG retrieval. If the current model remains the main cost or quality constraint after removing waste, compare the documented options in OpenAI vs Cohere vs Voyage embeddings before running a corpus-specific benchmark.

Cost-control checklist

Use this sequence for a controlled reduction:

1. Export a baseline of tokens, vectors, writes, storage, latency, and retrieval quality.
2. Deduplicate documents and repeated sections before changing the model.
3. Measure overlap overhead and remove only overlap that evaluation shows is unnecessary.
4. Test lower dimensions with all other variables fixed.
5. Batch by token count and make retries idempotent.
6. Fingerprint chunks so unchanged content bypasses embedding.
7. Build model migrations beside the current index, with a rollback path.
8. Run a frozen retrieval test set and review failures by query segment.
9. Recalculate total cost, including the vector database and temporary migration storage.
10. Roll out gradually and monitor both spend and retrieval failures.

The order matters. Deduplication and incremental indexing remove waste without changing the semantic model, so they usually carry less retrieval risk. Dimension and model changes can produce larger savings, but they belong behind an evaluation gate. If database architecture is the dominant expense after these steps, compare the operational trade-offs described in Pinecone versus turbopuffer for RAG rather than expecting the embedding API alone to solve the bill.

Frequently asked questions

Does batching embeddings reduce the number of billed tokens?

Not by itself. Combining inputs can reduce request overhead, but token-based billing still depends on the provider’s pricing rules. A separate asynchronous batch product may offer a discount; verify its current price, time window, and limits.

How much storage do lower-dimensional embeddings save?

For uncompressed float32 vectors, raw payload scales linearly with dimensions. Reducing from 3,072 to 1,024 dimensions cuts that payload by about two-thirds. Actual database savings differ because indexes, metadata, replicas, backups, compression, and minimum billing units add overhead.

Can existing embeddings be shortened without calling the model again?

Do not assume so. A model’s dimensions parameter affects the produced embedding, and arbitrary truncation of stored vectors may not match the provider-supported method. Follow the model documentation and treat a dimension change as a versioned re-embedding migration unless the provider explicitly documents another path.

What should be optimized first: chunk size, dimensions, or the model?

First remove duplicate text and avoid re-embedding unchanged chunks. Next test overlap and dimensions while holding other variables fixed. Change models only behind a corpus-specific evaluation and rollback plan, because model changes can alter retrieval behavior across every query segment.

Primary sources

• OpenAI, text-embedding-3-small model page, checked 2026-07-23.
• OpenAI, text-embedding-3-large model page, checked 2026-07-23.
• OpenAI, New embedding models and API updates, checked 2026-07-23.
• OpenAI Help Center, Embeddings FAQ, checked 2026-07-23.
• OpenAI, Organization usage API reference for embeddings, checked 2026-07-24.
• OpenAI, Batch API reference, checked 2026-07-24.