Optimizing for Cost

ContextGem offers several strategies to optimize for cost efficiency while maintaining extraction quality:

• 💸 Select Cost-Efficient Models: Use smaller/distilled LLMs for extracting aspects and basic concepts (e.g. titles, payment amounts, dates).

• ⚖️ Consider a Decision Model for Closed Questions: Boolean, label and rating concepts, as well as aspects, can be extracted with a decision model, which generates no text. (See Decision Models for when a decision model is a good fit, and for what its requests cost.)

• ⚙️ Use Default Parameters: All the extractions will be processed in as few model calls as possible.

• 📉 Enable Justifications Only When Necessary: Do not use justifications for simple aspects or concepts. This will reduce the number of tokens generated.

• 📊 Monitor Usage and Cost: Track model calls, token consumption, and cost to identify optimization opportunities.

Example of optimizing extraction for cost

# Example of optimizing extraction for cost

import os

from contextgem import DocumentLLM


llm = DocumentLLM(
    model="openai/gpt-6-luna",
    api_key=os.environ.get("CONTEXTGEM_OPENAI_API_KEY"),
    auto_pricing=True,  # calculate the costs with `genai-prices`, to track them
    # or set `pricing_details=ModelPricing(...)` with the prices of your provider
)

# ... use the LLM for extraction ...

# ... monitor usage and cost ...
usage = llm.get_usage()  # get the usage details, including tokens and calls' details.
cost = llm.get_cost()  # get the cost details, including input, output, and total costs.
print(usage)
print(cost)