# LLMs

Module for handling processing logic using LLMs.

This module provides classes and utilities for interacting with LLMs in document processing
workflows. It includes functionality for managing LLM configurations, handling API calls,
processing text and image inputs, tracking token usage and costs, and managing rate limits
for LLM requests.

The module supports various LLM providers through the litellm library, enabling both
text-only and multimodal (vision) capabilities. It implements efficient asynchronous
processing patterns and provides detailed usage statistics for monitoring and cost management.

### *class* contextgem.public.llms.DocumentLLM(\*\*data)

Bases: `_DocumentLLM`

Handles processing documents with a specific LLM.

This class serves as an abstraction for interacting with an LLM. It provides functionality
for querying the LLM with text or image inputs, and manages prompt preparation and token
usage tracking. An LLM can be used on its own or, with a role assigned, as a part of
a model group.

* **Variables:**
  * **model** ([*str*](https://docs.python.org/3/builtins/stdtypes.html#str)) – Model identifier in format {model_provider}/{model_name}.
    See [https://docs.litellm.ai/docs/providers](https://docs.litellm.ai/docs/providers) for supported providers.
  * **deployment_id** ([*str*](https://docs.python.org/3/builtins/stdtypes.html#str) *|* *None*) – Deployment ID for the LLM. Primarily used with Azure OpenAI.
  * **api_key** ([*str*](https://docs.python.org/3/builtins/stdtypes.html#str) *|* *None*) – API key for LLM authentication. Not required for local models (e.g., Ollama).
    Not included in the serialized LLM data, i.e. to be set again after deserialization.
    Not shown when the LLM is printed, nor in the text of a validation error of the LLM,
    which names the invalid parameter and the reason without showing the values passed.
    The values are still returned by the `errors()` and `json()` methods of the error,
    unless `include_input=False` is passed, with `'**********'` in place of a key
    that is passed as a value, e.g. a rejected key or the key among the parameters that
    an LLM is created with, and in place of the value of any parameter that an LLM does
    not have, as it may be the key passed under a misspelled name. An LLM that is among
    the values is returned by `errors()` as it is, i.e. with the key as its attribute,
    and is written out by `json()` without its key. A key that is passed as the value
    of another parameter, e.g. of `model`, is returned as any other value.
  * **api_base** ([*str*](https://docs.python.org/3/builtins/stdtypes.html#str) *|* *None*) – Base URL of the API endpoint. Not included in the serialized LLM data,
    i.e. to be set again after deserialization.
  * **api_version** ([*str*](https://docs.python.org/3/builtins/stdtypes.html#str) *|* *None*) – API version. Primarily used with Azure OpenAI.
  * **role** ([*str*](https://docs.python.org/3/builtins/stdtypes.html#str) *|* *None*) – Free-form role label of the LLM (up to 64 printable characters), used for routing in a `DocumentModelGroup`:
    the LLM extracts the aspects/concepts with the same `model_role`. Defaults to None,
    in which case the LLM extracts the aspects/concepts without `model_role`. Ignored
    when the LLM is used on its own, as it then extracts all aspects/concepts.
  * **system_message** ([*str*](https://docs.python.org/3/builtins/stdtypes.html#str) *|* *None*) – Preparatory system-level message to set context for LLM responses.
    Defaults to None, in which case the default system message for extraction is set,
    rendered for the output language, also when the output language is changed later.
    Assigning None restores the default system message, while an empty string disables
    the system message, i.e. none is sent.
  * **temperature** ([*float*](https://docs.python.org/3/builtins/functions.html#float) *|* *None*) – Sampling temperature (0.0 or higher; the upper limit depends on
    the model) controlling response creativity.
    Lower values produce more predictable outputs, higher values generate more varied responses.
    Defaults to None, in which case the parameter is not sent, and the default of the model
    applies. A value is sent to any model; if `litellm` reports it as not supported
    for the model, it is dropped from the request, with a warning logged.
  * **max_tokens** ([*int*](https://docs.python.org/3/builtins/functions.html#int) *|* *None*) – Maximum output tokens allowed per request. For reasoning (CoT-capable)
    models, the reasoning tokens count towards this limit. Defaults to None, in which case
    no limit is sent with the requests, and the default of the model applies (for most
    providers, the maximum output of the model). A response cut off at the limit
    fails the extraction, and is returned with a warning logged in a chat.
  * **reasoning_effort** (*ReasoningEffort* *|* *None*) – The effort level for the LLM to reason about the input. Can be set to
    `"none"`, `"minimal"`, `"low"`, `"medium"`, `"high"`, `"xhigh"`, or
    `"max"`. The parameter is sent only to the models detected as supporting reasoning.
    Support for specific values varies by model and provider: if `litellm` reports
    a value as not supported for the model, it is dropped from the request, with a warning
    logged, while a value that is rejected otherwise, e.g. by the provider API, fails
    the request. Note that
    `"none"` explicitly disables reasoning on models that support this value, which
    differs from the default `None` (parameter not sent, provider default applies).
    Relevant for reasoning (CoT-capable) models. Defaults to None.
  * **top_p** ([*float*](https://docs.python.org/3/builtins/functions.html#float) *|* *None*) – Nucleus sampling value (0.0 to 1.0) controlling output focus/randomness.
    Lower values make output more deterministic, higher values produce more diverse outputs.
    Defaults to `None` (the parameter is not sent). Some providers (e.g. Claude 4.x)
    reject requests that set both `temperature` and `top_p`.
  * **num_retries_failed_request** ([*int*](https://docs.python.org/3/builtins/functions.html#int)) – Number of retries when LLM request fails with a transient
    error (request timeout, rate limit, server or connection error). Retries are performed
    with exponential backoff and jitter. Non-transient errors (e.g. invalid request,
    failed authentication, exceeded context window) are not retried. Defaults to 3.
  * **max_retries_failed_request** ([*int*](https://docs.python.org/3/builtins/functions.html#int)) – LLM provider-specific retry count for failed requests,
    i.e. retries performed by the provider’s SDK, where supported. Applies to the first
    attempt of a request only. Defaults to 0.
  * **max_retries_invalid_data** ([*int*](https://docs.python.org/3/builtins/functions.html#int)) – Number of retries when LLM request succeeds but LLM returns
    invalid data. Each retry is a new request, i.e. the LLM is asked for the same
    data up to `1 + max_retries_invalid_data` times. A fallback LLM applies its own
    setting in the same way: it is asked once, and then again up to its
    `max_retries_invalid_data` times. If the content of a call is sent
    in parts, a retry is a request for the part that the invalid data was returned
    for, i.e. the parts answered with valid data are not sent again. Defaults to 3.
  * **timeout** ([*int*](https://docs.python.org/3/builtins/functions.html#int)) – Timeout in seconds for LLM API calls. As responses are not streamed,
    the whole response has to be received within the timeout. Defaults to 600 seconds.
  * **pricing_details** ([*ModelPricing*](https://contextgem.shcherbak.ai/api/data_models/index.html.md#contextgem.public.data_models.ModelPricing) *|* *None*) – ModelPricing object with pricing details for cost calculation.
    Defaults to None.
  * **auto_pricing** ([*bool*](https://docs.python.org/3/builtins/functions.html#bool)) – Enable automatic LLM cost calculation using genai-prices.
    Cannot be set together with `pricing_details`, and is not supported for
    local models (e.g. `ollama/`, `ollama_chat/`, `lm_studio/`): both
    raise `ValueError`. Defaults to `False`.
  * **auto_pricing_refresh** ([*bool*](https://docs.python.org/3/builtins/functions.html#bool)) – Whether genai-prices should auto-refresh its cached
    pricing data, e.g. to price a recently released model that the bundled
    pricing data does not include yet. Defaults to `False`.
  * **is_fallback** ([*bool*](https://docs.python.org/3/builtins/functions.html#bool)) – Indicates whether the LLM is a fallback model. Defaults to False.
  * **fallback_models** ([*list*](https://docs.python.org/3/builtins/stdtypes.html#list) *[*[*DocumentLLM*](#contextgem.public.llms.DocumentLLM) *|* [*DocumentDecisionModel*](https://contextgem.shcherbak.ai/api/decision_models/index.html.md#contextgem.public.decision_models.DocumentDecisionModel) *]*) – DocumentLLMs to use as fallbacks if the current one fails, tried
    in the given order until one of them succeeds. Each must have `is_fallback=True`
    and the same role, output language, system message, tools, `tool_choice` and
    `parallel_tool_calls` as the current LLM, must differ from the current LLM and
    from each other in at least one config param, and must support vision if
    the current LLM is used to extract concepts from images. A decision model
    (`DocumentDecisionModel`) with `is_fallback=True` and the same role can
    also be a fallback: it replaces the LLM only in the extraction, provided that
    a decision model can extract everything the LLM is to extract, and is skipped
    in a chat. Each fallback applies its own retry settings: a fallback LLM that
    returns invalid data during extraction is asked again up to its
    `max_retries_invalid_data` times, as the current LLM is, and a fallback
    decision model is asked once. Defaults to an empty list.
  * **output_language** (*LanguageRequirement*) – Language for produced output text (justifications, explanations).
    Can be “en” (English) or “adapt” (adapts to document/image language). Defaults to “en”.
    Applies to the extraction prompts, and to the default system message: with a custom
    system message, it applies to the extraction prompts only.
  * **async_limiter** (*AsyncLimiter*) – Controls frequency of async LLM API requests for concurrent tasks.
    Defaults to allowing 10 acquisitions per 10-second period to prevent rate limit issues.
    The rate applies to all the requests to the LLM that are sent concurrently, i.e.
    the extraction calls that run at the same time share it.
    Requests to a fallback model are controlled by the limiter of that fallback model.
    See [https://github.com/mjpieters/aiolimiter](https://github.com/mjpieters/aiolimiter) for configuration details.
  * **seed** ([*int*](https://docs.python.org/3/builtins/functions.html#int) *|* *None*) – Seed for random number generation to help produce more consistent outputs
    across multiple runs. When set to a specific integer value, the LLM will attempt
    to use this seed for sampling operations. However, deterministic output is still
    not guaranteed even with the same seed, as other factors may influence the model’s
    response. Support depends on the model provider: a seed is mostly of use with local
    and open-weight models, while some providers (e.g. Anthropic) do not support it.
    If `litellm` reports the seed as not supported for the model, it is dropped from
    the request, with a warning logged. Defaults to None (the parameter is not sent).
* **Parameters:**
  * **model** ([*Annotated*](https://docs.python.org/3/library/typing.html#typing.Annotated) *[*[*str*](https://docs.python.org/3/builtins/stdtypes.html#str) *,* [*Strict*](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.Strict) *(**strict=True* *)* *,* [*StringConstraints*](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.StringConstraints) *(**strip_whitespace=True* *,* *to_upper=None* *,* *to_lower=None* *,* *strict=None* *,* *min_length=1* *,* *max_length=None* *,* *pattern=None* *,* *ascii_only=None* *)* *]*)
  * **api_key** ([*Annotated*](https://docs.python.org/3/library/typing.html#typing.Annotated) *[*[*str*](https://docs.python.org/3/builtins/stdtypes.html#str) *,* [*Strict*](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.Strict) *(**strict=True* *)* *,* [*StringConstraints*](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.StringConstraints) *(**strip_whitespace=True* *,* *to_upper=None* *,* *to_lower=None* *,* *strict=None* *,* *min_length=1* *,* *max_length=None* *,* *pattern=None* *,* *ascii_only=None* *)* *]*  *|* *None*)
  * **api_base** ([*Annotated*](https://docs.python.org/3/library/typing.html#typing.Annotated) *[*[*str*](https://docs.python.org/3/builtins/stdtypes.html#str) *,* [*Strict*](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.Strict) *(**strict=True* *)* *,* [*StringConstraints*](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.StringConstraints) *(**strip_whitespace=True* *,* *to_upper=None* *,* *to_lower=None* *,* *strict=None* *,* *min_length=1* *,* *max_length=None* *,* *pattern=None* *,* *ascii_only=None* *)* *]*  *|* *None*)
  * **role** ([*Annotated*](https://docs.python.org/3/library/typing.html#typing.Annotated) *[*[*str*](https://docs.python.org/3/builtins/stdtypes.html#str) *,* [*Strict*](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.Strict) *(**strict=True* *)* *,* [*AfterValidator*](https://pydantic.dev/docs/validation/latest/api/pydantic/functional_validators/#pydantic.functional_validators.AfterValidator) *(**func=~contextgem.internal.typings.validators._validate_model_role* *)* *]*  *|* *None*)
  * **num_retries_failed_request** ([*Annotated*](https://docs.python.org/3/library/typing.html#typing.Annotated) *[*[*int*](https://docs.python.org/3/builtins/functions.html#int) *,* [*Strict*](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.Strict) *(**strict=True* *)* *]*)
  * **timeout** ([*Annotated*](https://docs.python.org/3/library/typing.html#typing.Annotated) *[*[*int*](https://docs.python.org/3/builtins/functions.html#int) *,* [*Strict*](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.Strict) *(**strict=True* *)* *]*)
  * **pricing_details** ( *\_ModelPricing* *|* *None*)
  * **is_fallback** ([*Annotated*](https://docs.python.org/3/library/typing.html#typing.Annotated) *[*[*bool*](https://docs.python.org/3/builtins/functions.html#bool) *,* [*Strict*](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.Strict) *(**strict=True* *)* *]*)
  * **fallback_models** ([*list*](https://docs.python.org/3/builtins/stdtypes.html#list) *[* *\_DocumentModel* *]*)
  * **auto_pricing** ([*Annotated*](https://docs.python.org/3/library/typing.html#typing.Annotated) *[*[*bool*](https://docs.python.org/3/builtins/functions.html#bool) *,* [*Strict*](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.Strict) *(**strict=True* *)* *]*)
  * **auto_pricing_refresh** ([*Annotated*](https://docs.python.org/3/library/typing.html#typing.Annotated) *[*[*bool*](https://docs.python.org/3/builtins/functions.html#bool) *,* [*Strict*](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.Strict) *(**strict=True* *)* *]*)
  * **deployment_id** ([*Annotated*](https://docs.python.org/3/library/typing.html#typing.Annotated) *[*[*str*](https://docs.python.org/3/builtins/stdtypes.html#str) *,* [*Strict*](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.Strict) *(**strict=True* *)* *,* [*StringConstraints*](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.StringConstraints) *(**strip_whitespace=True* *,* *to_upper=None* *,* *to_lower=None* *,* *strict=None* *,* *min_length=1* *,* *max_length=None* *,* *pattern=None* *,* *ascii_only=None* *)* *]*  *|* *None*)
  * **api_version** ([*Annotated*](https://docs.python.org/3/library/typing.html#typing.Annotated) *[*[*str*](https://docs.python.org/3/builtins/stdtypes.html#str) *,* [*Strict*](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.Strict) *(**strict=True* *)* *,* [*StringConstraints*](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.StringConstraints) *(**strip_whitespace=True* *,* *to_upper=None* *,* *to_lower=None* *,* *strict=None* *,* *min_length=1* *,* *max_length=None* *,* *pattern=None* *,* *ascii_only=None* *)* *]*  *|* *None*)
  * **system_message** ([*str*](https://docs.python.org/3/builtins/stdtypes.html#str) *|* *None*)
  * **max_tokens** ([*Annotated*](https://docs.python.org/3/library/typing.html#typing.Annotated) *[*[*int*](https://docs.python.org/3/builtins/functions.html#int) *,* [*Strict*](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.Strict) *(**strict=True* *)* *]*  *|* *None*)
  * **reasoning_effort** ([*Literal*](https://docs.python.org/3/library/typing.html#typing.Literal) *[* *'none'* *,*  *'minimal'* *,*  *'low'* *,*  *'medium'* *,*  *'high'* *,*  *'xhigh'* *,*  *'max'* *]*  *|* *None*)
  * **max_retries_failed_request** ([*Annotated*](https://docs.python.org/3/library/typing.html#typing.Annotated) *[*[*int*](https://docs.python.org/3/builtins/functions.html#int) *,* [*Strict*](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.Strict) *(**strict=True* *)* *]*)
  * **max_retries_invalid_data** ([*Annotated*](https://docs.python.org/3/library/typing.html#typing.Annotated) *[*[*int*](https://docs.python.org/3/builtins/functions.html#int) *,* [*Strict*](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.Strict) *(**strict=True* *)* *]*)
  * **output_language** ([*Literal*](https://docs.python.org/3/library/typing.html#typing.Literal) *[* *'en'* *,*  *'adapt'* *]*)
  * **temperature** ([*Annotated*](https://docs.python.org/3/library/typing.html#typing.Annotated) *[*[*float*](https://docs.python.org/3/builtins/functions.html#float) *,* [*Strict*](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.Strict) *(**strict=True* *)* *]*  *|* *None*)
  * **top_p** ([*Annotated*](https://docs.python.org/3/library/typing.html#typing.Annotated) *[*[*float*](https://docs.python.org/3/builtins/functions.html#float) *,* [*Strict*](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.Strict) *(**strict=True* *)* *]*  *|* *None*)
  * **seed** ([*Annotated*](https://docs.python.org/3/library/typing.html#typing.Annotated) *[*[*int*](https://docs.python.org/3/builtins/functions.html#int) *,* [*Strict*](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.Strict) *(**strict=True* *)* *]*  *|* *None*)
  * **tools** ([*Sequence*](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence) *[*[*Callable*](https://docs.python.org/3/library/collections.abc.html#collections.abc.Callable) *[* *[* *...* *]* *,* [*str*](https://docs.python.org/3/builtins/stdtypes.html#str) *]*  *|* [*Callable*](https://docs.python.org/3/library/collections.abc.html#collections.abc.Callable) *[* *[* *...* *]* *,* [*Coroutine*](https://docs.python.org/3/library/collections.abc.html#collections.abc.Coroutine) *[*[*Any*](https://docs.python.org/3/library/typing.html#typing.Any) *,* [*Any*](https://docs.python.org/3/library/typing.html#typing.Any) *,* [*str*](https://docs.python.org/3/builtins/stdtypes.html#str) *]* *]* *]*  *|* *None*)
  * **tool_choice** ([*str*](https://docs.python.org/3/builtins/stdtypes.html#str) *|* [*Annotated*](https://docs.python.org/3/library/typing.html#typing.Annotated) *[*[*dict*](https://docs.python.org/3/builtins/stdtypes.html#dict) *[*[*str*](https://docs.python.org/3/builtins/stdtypes.html#str) *,* [*Any*](https://docs.python.org/3/library/typing.html#typing.Any) *]* *,* [*BeforeValidator*](https://pydantic.dev/docs/validation/latest/api/pydantic/functional_validators/#pydantic.functional_validators.BeforeValidator) *(**func=~contextgem.internal.typings.validators._validate_is_json_dict* *,* *json_schema_input_type=PydanticUndefined* *)* *]*  *|* *None*)
  * **parallel_tool_calls** ([*bool*](https://docs.python.org/3/builtins/functions.html#bool) *|* *None*)
  * **tool_max_rounds** ([*Annotated*](https://docs.python.org/3/library/typing.html#typing.Annotated) *[*[*int*](https://docs.python.org/3/builtins/functions.html#int) *,* [*Strict*](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.Strict) *(**strict=True* *)* *]*)

Note:

> - Model groups
>   : Refer to the [`DocumentModelGroup`](https://contextgem.shcherbak.ai/api/models/index.html.md#contextgem.public.models.DocumentModelGroup) class for more information on constructing model groups,
>     which are a collection of models with unique roles, used for complex document processing tasks.
> - Model role
>   : The `role` of an LLM is a free-form label used to route aspects/concepts to the models
>     of a [`DocumentModelGroup`](https://contextgem.shcherbak.ai/api/models/index.html.md#contextgem.public.models.DocumentModelGroup). For example, if an aspect/concept is assigned
>     `model_role="complex"`, it means that the aspect/concept is extracted from the document
>     using the model in the group with the `role` set to “complex”, while the aspects/concepts
>     without `model_role` are extracted using the model without `role`. This helps to channel
>     different tasks to different models, ensuring that the task is handled by the most
>     appropriate model. Usually, domain expertise is required to determine the most appropriate
>     model for a specific aspect/concept. A role is just a label and implies no capabilities
>     of the model. When an LLM is used on its own, i.e. not in a group, you can skip the role
>     assignment completely: the LLM extracts all aspects/concepts, regardless of their `model_role`.
> - Explicit capability declaration
>   : Model vision capabilities are automatically detected using
>     `litellm.supports_vision()`. If this function does not correctly identify your model’s capabilities,
>     an error will be raised when the model is to be sent images, and you can explicitly declare
>     the capability by setting `_supports_vision=True` on the LLM instance.

Example:
: ```python
  from contextgem import DocumentLLM
  <br/>
  <br/>
  # Create a single LLM, which extracts all aspects and concepts it is given
  # (a role is needed only for routing in a model group)
  llm = DocumentLLM(
      model="openai/gpt-6-luna",
      api_key="your-api-key",  # Replace with your actual API key
      auto_pricing=True,  # optional, to calculate the costs with `genai-prices`
  )
  <br/>
  # Create a fallback LLM in case the primary model fails
  fallback_llm = DocumentLLM(
      model="anthropic/claude-sonnet-5-5",
      api_key="your-anthropic-api-key",  # Replace with your actual API key
      is_fallback=True,  # must have the same role as the primary LLM (here: no role)
      auto_pricing=True,  # optional, to calculate the costs with `genai-prices`
  )
  # Assign the fallback LLM to the primary LLM
  # (several fallback models can be assigned, to be tried in the given order)
  llm.fallback_models = [fallback_llm]
  ```

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be
validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

#### \_eq_deserialized_llm_config(other)

Custom config equality method to compare this \_DocumentLLM with a deserialized instance.

Compares the \_\_dict_\_ of both instances and performs specific checks for
certain attributes that require special handling.

Note that, by default, the reconstructed deserialized \_DocumentLLM will be only partially
equal (==) to the original one, as the api credentials are redacted, and the async limiter
and the async lock are not serialized and point to different objects in memory
post-initialization. The attached prompt templates are not serialized either:
the deserialized \_DocumentLLM has the built-in ones, which the LLMs share, i.e. a custom
prompt has to be set again. Also, usage and cost are reset by default pre-serialization.

* **Parameters:**
  **other** (`contextgem.internal.base.llms._DocumentLLM`) – Another \_DocumentLLM instance to compare with
* **Returns:**
  True if the instances are equal, False otherwise
* **Return type:**
  [`bool`](https://docs.python.org/3/builtins/functions.html#bool)

#### \_update_default_prompt(prompt_path, prompt_type)

For advanced users only!

Update the default Jinja2 prompt template for the LLM.

This method allows you to replace the built-in prompt templates with custom ones
for specific extraction types. The framework uses these templates to guide the LLM
in extracting structured information from documents.

The custom prompt must be a valid Jinja2 template and include all the necessary
variables that are present in the default prompt. Otherwise, the extraction may fail.
Default prompts are located under `contextgem/internal/prompts/`

IMPORTANT NOTES:

The default prompts are complex and specifically designed for
various steps of LLM extraction with the framework. Such prompts include the
necessary instructions, template variables, nested structures and loops, etc.

Only use custom prompts if you MUST have a deeper customization and adaptation of the
default prompts to your specific use case. Otherwise, the default prompts should be
sufficient for most use cases.

Use at your own risk!

* **Parameters:**
  * **prompt_path** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str) | [`pathlib.Path`](https://docs.python.org/3/library/pathlib.html#pathlib.Path)) – Path to the Jinja2 template file (.j2 extension required)
  * **prompt_type** ([`typing.Literal`](https://docs.python.org/3/library/typing.html#typing.Literal)[`'aspect'`, `'concept'`]) – Type of prompt to update (“aspect” or “concept”)
* **Returns:**
  None
* **Return type:**
  [`None`](https://docs.python.org/3/builtins/constants.html#None)

#### *property* async_limiter *: AsyncLimiter*

Gets the async rate limiter for this model.

* **Returns:**
  The AsyncLimiter instance controlling request rate limits.
* **Return type:**
  AsyncLimiter

#### chat(prompt, \*, images=None, chat_session=None)

Synchronously sends a prompt to the LLM and gets a response.
For models supporting vision, attach images to the prompt if needed.

This method allows direct interaction with the LLM by submitting your own prompt.

* **Parameters:**
  * **prompt** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str)) – The input prompt to send to the LLM
  * **images** ([`list`](https://docs.python.org/3/builtins/stdtypes.html#list)[`contextgem.internal.base.images._Image`] | [`None`](https://docs.python.org/3/builtins/constants.html#None)) – Optional list of Image instances for vision queries
  * **chat_session** (`contextgem.internal.base.llms._ChatSession` | [`None`](https://docs.python.org/3/builtins/constants.html#None)) – Optional stateful chat session to preserve and use history.
* **Returns:**
  The LLM’s response. A response cut off at the max output tokens
  is returned incomplete, with a warning logged.
* **Return type:**
  [`str`](https://docs.python.org/3/builtins/stdtypes.html#str)
* **Raises:**
  * [**ValueError**](https://docs.python.org/3/builtins/exceptions.html#ValueError) – If the prompt is empty or not a string
  * [**ValueError**](https://docs.python.org/3/builtins/exceptions.html#ValueError) – If images parameter is not a list of Image instances
  * [**ValueError**](https://docs.python.org/3/builtins/exceptions.html#ValueError) – If images are provided but the model doesn’t support vision
  * [**ValueError**](https://docs.python.org/3/builtins/exceptions.html#ValueError) – If the request params of a fallback LLM are invalid, e.g.
    parallel tool calls are set for a fallback LLM that does not support them
  * [**ValueError**](https://docs.python.org/3/builtins/exceptions.html#ValueError) – If the prompt exceeds the context window of the LLM, and
    no fallback LLM is left to try
  * [**ModelAPIError**](https://contextgem.shcherbak.ai/api/exceptions/index.html.md#contextgem.public.exceptions.ModelAPIError) – If the request to the LLM fails, or the LLM returns
    an empty response, and no fallback LLM is left to try
  * [**LLMToolLoopLimitError**](https://contextgem.shcherbak.ai/api/exceptions/index.html.md#contextgem.public.exceptions.LLMToolLoopLimitError) – If the LLM keeps calling tools after
    `tool_max_rounds` rounds of tool calls

#### *async* chat_async(prompt, \*, images=None, chat_session=None)

Asynchronously sends a prompt to the LLM and gets a response.
For models supporting vision, attach images to the prompt if needed.

This method allows direct interaction with the LLM by submitting your own prompt.

* **Parameters:**
  * **prompt** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str)) – The input prompt to send to the LLM
  * **images** ([`list`](https://docs.python.org/3/builtins/stdtypes.html#list)[`contextgem.internal.base.images._Image`] | [`None`](https://docs.python.org/3/builtins/constants.html#None)) – Optional list of Image instances for vision queries
  * **chat_session** (`contextgem.internal.base.llms._ChatSession` | [`None`](https://docs.python.org/3/builtins/constants.html#None)) – Optional stateful chat session to preserve and use history.
* **Returns:**
  The LLM’s response. A response cut off at the max output tokens
  is returned incomplete, with a warning logged.
* **Return type:**
  [`str`](https://docs.python.org/3/builtins/stdtypes.html#str)
* **Raises:**
  * [**ValueError**](https://docs.python.org/3/builtins/exceptions.html#ValueError) – If the prompt is empty or not a string
  * [**ValueError**](https://docs.python.org/3/builtins/exceptions.html#ValueError) – If images parameter is not a list of Image instances
  * [**ValueError**](https://docs.python.org/3/builtins/exceptions.html#ValueError) – If images are provided but the model doesn’t support vision
  * [**ValueError**](https://docs.python.org/3/builtins/exceptions.html#ValueError) – If the request params of a fallback LLM are invalid, e.g.
    parallel tool calls are set for a fallback LLM that does not support them
  * [**ValueError**](https://docs.python.org/3/builtins/exceptions.html#ValueError) – If the prompt exceeds the context window of the LLM, and
    no fallback LLM is left to try
  * [**ModelAPIError**](https://contextgem.shcherbak.ai/api/exceptions/index.html.md#contextgem.public.exceptions.ModelAPIError) – If the request to the LLM fails, or the LLM returns
    an empty response, and no fallback LLM is left to try
  * [**LLMToolLoopLimitError**](https://contextgem.shcherbak.ai/api/exceptions/index.html.md#contextgem.public.exceptions.LLMToolLoopLimitError) – If the LLM keeps calling tools after
    `tool_max_rounds` rounds of tool calls

#### extract_all(document, \*, overwrite_existing=False, max_items_per_call=0, use_concurrency=False, max_paragraphs_to_analyze_per_call=0, max_images_to_analyze_per_call=0, raise_exception_on_extraction_error=True)

Extracts all aspects and concepts from a document and its aspects.

This method performs comprehensive extraction by processing the document for aspects
and concepts, then extracting concepts from each aspect. The operation can be
configured for concurrent processing and customized extraction parameters.

This is the synchronous version of extract_all_async().

* **Parameters:**
  * **document** (`contextgem.internal.base.documents._Document`) – The document to analyze.
  * **overwrite_existing** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool)) – Whether to overwrite already processed aspects and concepts with
    newly extracted information. Defaults to False.
  * **max_items_per_call** ([`int`](https://docs.python.org/3/builtins/functions.html#int)) – Maximum number of items with the same extraction params to process
    in each call. Defaults to 0 (all items in one call). If concurrency is enabled, defaults to 1.
    For complex tasks, you should not set a high value, in order to avoid prompt overloading.
  * **use_concurrency** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool)) – If True, enables concurrent processing of multiple items.
    Concurrency can considerably reduce processing time, but may cause rate limit errors
    with model providers. Use this option when API rate limits allow for multiple concurrent
    requests. Defaults to False.
  * **max_paragraphs_to_analyze_per_call** ([`int`](https://docs.python.org/3/builtins/functions.html#int)) – Maximum paragraphs to include in a
    single LLM prompt. A decision model is sent each paragraph in a request of
    its own for aspects, whatever the value. For concepts, it is sent a text
    with more paragraphs in parts of at most that many paragraphs, each in
    a request of its own: a concept then gets an extracted item for each part,
    and a concept with `singular_occurrence=True` is rejected with
    `ValueError`. Defaults to 0 (all paragraphs). Cannot be negative.
  * **max_images_to_analyze_per_call** ([`int`](https://docs.python.org/3/builtins/functions.html#int)) – Maximum images to include in a single LLM prompt.
    Defaults to 0 (all images). Cannot be negative. Relevant only for document-level
    concepts.
  * **raise_exception_on_extraction_error** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool)) – Whether to raise an exception if the extraction fails
    due to invalid data returned by a model or an error in the model API. If False, a warning will
    be issued instead, and no extracted items will be returned. Defaults to True.
* **Returns:**
  The document with extracted aspects and concepts.
* **Return type:**
  `contextgem.internal.base.documents._Document`

#### *async* extract_all_async(document, \*, overwrite_existing=False, max_items_per_call=0, use_concurrency=False, max_paragraphs_to_analyze_per_call=0, max_images_to_analyze_per_call=0, raise_exception_on_extraction_error=True)

Asynchronously extracts all aspects and concepts from a document and its aspects.

This method performs comprehensive extraction by processing the document for aspects
and concepts, then extracting concepts from each aspect. The operation can be
configured for concurrent processing and customized extraction parameters.

* **Parameters:**
  * **document** (`contextgem.internal.base.documents._Document`) – The document to analyze.
  * **overwrite_existing** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool)) – Whether to overwrite already processed aspects and concepts with
    newly extracted information. Defaults to False.
  * **max_items_per_call** ([`int`](https://docs.python.org/3/builtins/functions.html#int)) – Maximum number of items with the same extraction params to process
    in each call. Defaults to 0 (all items in one call). If concurrency is enabled,
    defaults to 1. For complex tasks, you should not set a high value, in order to avoid
    prompt overloading.
  * **use_concurrency** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool)) – If True, enables concurrent processing of multiple items.
    Concurrency can considerably reduce processing time, but may cause rate limit errors
    with model providers. Use this option when API rate limits allow for multiple concurrent
    requests. Defaults to False.
  * **max_paragraphs_to_analyze_per_call** ([`int`](https://docs.python.org/3/builtins/functions.html#int)) – Maximum paragraphs to include in a
    single LLM prompt. A decision model is sent each paragraph in a request of
    its own for aspects, whatever the value. For concepts, it is sent a text
    with more paragraphs in parts of at most that many paragraphs, each in
    a request of its own: a concept then gets an extracted item for each part,
    and a concept with `singular_occurrence=True` is rejected with
    `ValueError`. Defaults to 0 (all paragraphs). Cannot be negative.
  * **max_images_to_analyze_per_call** ([`int`](https://docs.python.org/3/builtins/functions.html#int)) – Maximum images to include in a single LLM prompt.
    Defaults to 0 (all images). Cannot be negative. Relevant only for document-level
    concepts.
  * **raise_exception_on_extraction_error** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool)) – Whether to raise an exception if the extraction fails
    due to invalid data returned by a model or an error in the model API. If False, a warning will
    be issued instead, and no extracted items will be returned. Defaults to True.
* **Returns:**
  The document with extracted aspects and concepts.
* **Return type:**
  `contextgem.internal.base.documents._Document`

#### extract_aspects_from_document(document, \*, from_aspects=None, overwrite_existing=False, max_items_per_call=0, use_concurrency=False, max_paragraphs_to_analyze_per_call=0, raise_exception_on_extraction_error=True)

Extracts aspects from the provided document using predefined models.

If an aspect instance has `extracted_items` populated, the `reference_paragraphs` field will be
automatically populated from these items. For an aspect with the “sentences” reference depth,
the `reference_sentences` field is populated from them as well: with the extracted sentences,
and not with the other sentences of the reference paragraphs.

This is the synchronous version of extract_aspects_from_document_async().

* **Parameters:**
  * **document** (`contextgem.internal.base.documents._Document`) – The document from which aspects are to be extracted.
  * **from_aspects** ([`collections.abc.Sequence`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence)[`contextgem.internal.base.aspects._Aspect`] | [`None`](https://docs.python.org/3/builtins/constants.html#None)) – Existing aspects to use as a base for extraction. If None, uses all
    document’s aspects. An aspect that is given more than once is extracted once.
  * **overwrite_existing** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool)) – Whether to overwrite already processed aspects with newly extracted information.
    Defaults to False.
  * **max_items_per_call** ([`int`](https://docs.python.org/3/builtins/functions.html#int)) – Maximum items with the same extraction params to process per call.
    Defaults to 0 (all items in single call). For complex tasks, you should not set a value, to avoid
    prompt overloading. If concurrency is enabled, defaults to 1 (each item processed separately).
  * **use_concurrency** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool)) – If True, enables concurrent processing of multiple items.
    Concurrency can considerably reduce processing time, but may cause rate limit errors
    with model providers. Use this option when API rate limits allow for multiple concurrent
    requests. Defaults to False.
  * **max_paragraphs_to_analyze_per_call** ([`int`](https://docs.python.org/3/builtins/functions.html#int)) – Maximum paragraphs to analyze in a single LLM
    prompt. Changes nothing for a decision model, which is sent each paragraph in
    a request of its own. Defaults to 0 (all paragraphs). Cannot be negative.
  * **raise_exception_on_extraction_error** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool)) – Whether to raise an exception if the extraction fails
    due to invalid data returned by a model or an error in the model API. If False, a warning will
    be issued instead, and no extracted items will be returned. Defaults to True.
* **Returns:**
  List of processed \_Aspect objects with extracted items.
* **Return type:**
  [`list`](https://docs.python.org/3/builtins/stdtypes.html#list)[`contextgem.internal.base.aspects._Aspect`]

#### *async* extract_aspects_from_document_async(document, \*, from_aspects=None, overwrite_existing=False, max_items_per_call=0, use_concurrency=False, max_paragraphs_to_analyze_per_call=0, raise_exception_on_extraction_error=True)

Extracts aspects from the provided document using predefined models asynchronously.

If an aspect instance has `extracted_items` populated, the `reference_paragraphs` field will be
automatically populated from these items. For an aspect with the “sentences” reference depth,
the `reference_sentences` field is populated from them as well: with the extracted sentences,
and not with the other sentences of the reference paragraphs.

* **Parameters:**
  * **document** (`contextgem.internal.base.documents._Document`) – The document from which aspects are to be extracted.
  * **from_aspects** ([`collections.abc.Sequence`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence)[`contextgem.internal.base.aspects._Aspect`] | [`None`](https://docs.python.org/3/builtins/constants.html#None)) – Existing aspects to use as a base for extraction. If None, uses all
    document’s aspects. An aspect that is given more than once is extracted once.
  * **overwrite_existing** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool)) – Whether to overwrite already processed aspects with newly extracted information.
    Defaults to False.
  * **max_items_per_call** ([`int`](https://docs.python.org/3/builtins/functions.html#int)) – Maximum number of items with the same extraction params to process
    per call. Defaults to 0 (all items in one call). If concurrency is enabled,
    defaults to 1. For complex tasks, you should not set a high value, in order to avoid
    prompt overloading.
  * **use_concurrency** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool)) – If True, enables concurrent processing of multiple items.
    Concurrency can considerably reduce processing time, but may cause rate limit errors
    with model providers. Use this option when API rate limits allow for multiple concurrent
    requests. Defaults to False.
  * **max_paragraphs_to_analyze_per_call** ([`int`](https://docs.python.org/3/builtins/functions.html#int)) – Maximum paragraphs to analyze in a single LLM
    prompt. Changes nothing for a decision model, which is sent each paragraph in
    a request of its own. Defaults to 0 (all paragraphs). Cannot be negative.
  * **raise_exception_on_extraction_error** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool)) – Whether to raise an exception if the extraction fails
    due to invalid data returned by a model or an error in the model API. If False, a warning will
    be issued instead, and no extracted items will be returned. Defaults to True.
* **Returns:**
  List of processed \_Aspect objects with extracted items.
* **Return type:**
  [`list`](https://docs.python.org/3/builtins/stdtypes.html#list)[`contextgem.internal.base.aspects._Aspect`]

#### extract_concepts_from_aspect(aspect, document, \*, from_concepts=None, overwrite_existing=False, max_items_per_call=0, use_concurrency=False, max_paragraphs_to_analyze_per_call=0, raise_exception_on_extraction_error=True)

Extracts concepts associated with a given aspect in a document.

This method processes an aspect to extract related concepts using models.
If the aspect has not been previously processed, a ValueError is raised.

This is the synchronous version of extract_concepts_from_aspect_async().

* **Parameters:**
  * **aspect** (`contextgem.internal.base.aspects._Aspect`) – The aspect from which to extract concepts.
  * **document** (`contextgem.internal.base.documents._Document`) – The document that contains the aspect.
  * **from_concepts** ([`collections.abc.Sequence`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence)[`contextgem.internal.base.concepts._Concept`] | [`None`](https://docs.python.org/3/builtins/constants.html#None)) – List of existing concepts to process. Defaults to None.
    A concept that is given more than once is extracted once.
  * **overwrite_existing** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool)) – Whether to overwrite already processed concepts with newly
    extracted information. Defaults to False.
  * **max_items_per_call** ([`int`](https://docs.python.org/3/builtins/functions.html#int)) – Maximum number of items with the same extraction params to process
    in each call. Defaults to 0 (all items in one call). If concurrency is enabled,
    defaults to 1. For complex tasks, you should not set a high value, in order to avoid
    prompt overloading.
  * **use_concurrency** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool)) – If True, enables concurrent processing of multiple items.
    Concurrency can considerably reduce processing time, but may cause rate limit errors
    with model providers. Use this option when API rate limits allow for multiple concurrent
    requests. Defaults to False.
  * **max_paragraphs_to_analyze_per_call** ([`int`](https://docs.python.org/3/builtins/functions.html#int)) – Maximum paragraphs to include in a
    single LLM prompt. A decision model is sent a text with more
    paragraphs in parts of at most that many paragraphs, each in a request of
    its own: a concept then gets an extracted item for each part, and
    a concept with `singular_occurrence=True` is rejected with `ValueError`.
    Defaults to 0 (all paragraphs). Cannot be negative.
  * **raise_exception_on_extraction_error** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool)) – Whether to raise an exception if the extraction fails
    due to invalid data returned by a model or an error in the model API. If False, a warning will
    be issued instead, and no extracted items will be returned. Defaults to True.
* **Returns:**
  List of processed concept objects.
* **Return type:**
  [`list`](https://docs.python.org/3/builtins/stdtypes.html#list)[`contextgem.internal.base.concepts._Concept`]

#### *async* extract_concepts_from_aspect_async(aspect, document, \*, from_concepts=None, overwrite_existing=False, max_items_per_call=0, use_concurrency=False, max_paragraphs_to_analyze_per_call=0, raise_exception_on_extraction_error=True)

Asynchronously extracts concepts from a specified aspect using models.

This method processes an aspect to extract related concepts using models.
If the aspect has not been previously processed, a ValueError is raised.

* **Parameters:**
  * **aspect** (`contextgem.internal.base.aspects._Aspect`) – The aspect from which to extract concepts.
  * **document** (`contextgem.internal.base.documents._Document`) – The document that contains the aspect.
  * **from_concepts** ([`collections.abc.Sequence`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence)[`contextgem.internal.base.concepts._Concept`] | [`None`](https://docs.python.org/3/builtins/constants.html#None)) – List of existing concepts to process. Defaults to None.
    A concept that is given more than once is extracted once.
  * **overwrite_existing** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool)) – Whether to overwrite already processed concepts with newly
    extracted information. Defaults to False.
  * **max_items_per_call** ([`int`](https://docs.python.org/3/builtins/functions.html#int)) – Maximum number of items with the same extraction params to process
    in each call. Defaults to 0 (all items in one call). If concurrency is enabled,
    defaults to 1. For complex tasks, you should not set a high value, in order to avoid
    prompt overloading.
  * **use_concurrency** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool)) – If True, enables concurrent processing of multiple items.
    Concurrency can considerably reduce processing time, but may cause rate limit errors
    with model providers. Use this option when API rate limits allow for multiple concurrent
    requests. Defaults to False.
  * **max_paragraphs_to_analyze_per_call** ([`int`](https://docs.python.org/3/builtins/functions.html#int)) – Maximum paragraphs to include in a
    single LLM prompt. A decision model is sent a text with more
    paragraphs in parts of at most that many paragraphs, each in a request of
    its own: a concept then gets an extracted item for each part, and
    a concept with `singular_occurrence=True` is rejected with `ValueError`.
    Defaults to 0 (all paragraphs). Cannot be negative.
  * **raise_exception_on_extraction_error** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool)) – Whether to raise an exception if the extraction fails
    due to invalid data returned by a model or an error in the model API. If False, a warning will
    be issued instead, and no extracted items will be returned. Defaults to True.
* **Returns:**
  List of processed concept objects.
* **Return type:**
  [`list`](https://docs.python.org/3/builtins/stdtypes.html#list)[`contextgem.internal.base.concepts._Concept`]

#### extract_concepts_from_document(document, \*, from_concepts=None, overwrite_existing=False, max_items_per_call=0, use_concurrency=False, max_paragraphs_to_analyze_per_call=0, max_images_to_analyze_per_call=0, raise_exception_on_extraction_error=True)

Extracts concepts from the provided document using predefined models.

This is the synchronous version of extract_concepts_from_document_async().

* **Parameters:**
  * **document** (`contextgem.internal.base.documents._Document`) – The document from which concepts are to be extracted.
  * **from_concepts** ([`collections.abc.Sequence`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence)[`contextgem.internal.base.concepts._Concept`] | [`None`](https://docs.python.org/3/builtins/constants.html#None)) – Existing concepts to use as a base for extraction. If None, uses all
    document’s concepts. A concept that is given more than once is extracted once.
  * **overwrite_existing** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool)) – Whether to overwrite already processed concepts with
    newly extracted information. Defaults to False.
  * **max_items_per_call** ([`int`](https://docs.python.org/3/builtins/functions.html#int)) – Maximum items with the same extraction params to process per call.
    Defaults to 0 (all items in single call). For complex tasks, you should not set a value, to avoid
    prompt overloading. If concurrency is enabled, defaults to 1 (each item processed separately).
  * **use_concurrency** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool)) – If True, enables concurrent processing of multiple items.
    Concurrency can considerably reduce processing time, but may cause rate limit errors
    with model providers. Use this option when API rate limits allow for multiple concurrent
    requests. Defaults to False.
  * **max_paragraphs_to_analyze_per_call** ([`int`](https://docs.python.org/3/builtins/functions.html#int)) – Maximum paragraphs to analyze in a single LLM
    prompt. A decision model is sent a text with more
    paragraphs in parts of at most that many paragraphs, each in a request of
    its own: a concept then gets an extracted item for each part, and
    a concept with `singular_occurrence=True` is rejected with `ValueError`.
    Defaults to 0 (all paragraphs). Cannot be negative.
  * **max_images_to_analyze_per_call** ([`int`](https://docs.python.org/3/builtins/functions.html#int)) – Maximum images to include in a single LLM prompt.
    Defaults to 0 (all images). Cannot be negative.
  * **raise_exception_on_extraction_error** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool)) – Whether to raise an exception if the extraction fails
    due to invalid data returned by a model or an error in the model API. If False, a warning will
    be issued instead, and no extracted items will be returned. Defaults to True.
* **Returns:**
  List of processed Concept objects with extracted items.
* **Return type:**
  [`list`](https://docs.python.org/3/builtins/stdtypes.html#list)[`contextgem.internal.base.concepts._Concept`]

#### *async* extract_concepts_from_document_async(document, \*, from_concepts=None, overwrite_existing=False, max_items_per_call=0, use_concurrency=False, max_paragraphs_to_analyze_per_call=0, max_images_to_analyze_per_call=0, raise_exception_on_extraction_error=True)

Extracts concepts from the provided document using predefined models asynchronously.

This method processes a document to extract concepts using configured models.

* **Parameters:**
  * **document** (`contextgem.internal.base.documents._Document`) – The document from which concepts are to be extracted.
  * **from_concepts** ([`collections.abc.Sequence`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence)[`contextgem.internal.base.concepts._Concept`] | [`None`](https://docs.python.org/3/builtins/constants.html#None)) – Existing concepts to use as a base for extraction. If None, uses all
    document’s concepts. A concept that is given more than once is extracted once.
  * **overwrite_existing** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool)) – Whether to overwrite already processed concepts with
    newly extracted information. Defaults to False.
  * **max_items_per_call** ([`int`](https://docs.python.org/3/builtins/functions.html#int)) – Maximum number of items with the same extraction params to process
    per call. Defaults to 0 (all items in one call). If concurrency is enabled,
    defaults to 1. For complex tasks, you should not set a high value, in order to avoid
    prompt overloading.
  * **use_concurrency** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool)) – If True, enables concurrent processing of multiple items.
    Concurrency can considerably reduce processing time, but may cause rate limit errors
    with model providers. Use this option when API rate limits allow for multiple concurrent
    requests. Defaults to False.
  * **max_paragraphs_to_analyze_per_call** ([`int`](https://docs.python.org/3/builtins/functions.html#int)) – Maximum paragraphs to analyze in a single LLM
    prompt. A decision model is sent a text with more
    paragraphs in parts of at most that many paragraphs, each in a request of
    its own: a concept then gets an extracted item for each part, and
    a concept with `singular_occurrence=True` is rejected with `ValueError`.
    Defaults to 0 (all paragraphs). Cannot be negative.
  * **max_images_to_analyze_per_call** ([`int`](https://docs.python.org/3/builtins/functions.html#int)) – Maximum images to include in a single LLM prompt.
    Defaults to 0 (all images). Cannot be negative.
  * **raise_exception_on_extraction_error** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool)) – Whether to raise an exception if the extraction fails
    due to invalid data returned by a model or an error in the model API. If False, a warning will
    be issued instead, and no extracted items will be returned. Defaults to True.
* **Returns:**
  List of processed Concept objects with extracted items.
* **Return type:**
  [`list`](https://docs.python.org/3/builtins/stdtypes.html#list)[`contextgem.internal.base.concepts._Concept`]

#### *classmethod* from_dict(obj_dict)

Reconstructs an instance of the class from a dictionary representation.

This method deserializes a dictionary containing the object’s attributes and values
into a new instance of the class. It handles complex nested structures like aspects,
concepts, and extracted items, properly reconstructing each component.

A deserialized model or model group has no API credentials, as they are not
included in the serialized data, i.e. they are to be set again.

* **Parameters:**
  **obj_dict** ([`dict`](https://docs.python.org/3/builtins/stdtypes.html#dict)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str), [`typing.Any`](https://docs.python.org/3/library/typing.html#typing.Any)]) – Dictionary containing the serialized object data.
* **Returns:**
  A new instance of the class with restored attributes.
* **Return type:**
  [`typing.Self`](https://docs.python.org/3/library/typing.html#typing.Self)

#### *classmethod* from_disk(file_path)

Loads an instance of the class from a JSON file stored on disk.

This method reads the JSON content from the specified file path and
deserializes it into an instance of the class using the from_json
method.

* **Parameters:**
  **file_path** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str) | [`pathlib.Path`](https://docs.python.org/3/library/pathlib.html#pathlib.Path)) – Path to the JSON file to load (must end with ‘.json’).
  Can be a string or a Path object.
* **Returns:**
  An instance of the class populated with the data from the file.
* **Return type:**
  [`typing.Self`](https://docs.python.org/3/library/typing.html#typing.Self)
* **Raises:**
  * [**ValueError**](https://docs.python.org/3/builtins/exceptions.html#ValueError) – If the file path doesn’t end with ‘.json’, or if the file
    does not hold valid serialized data, e.g. it is not a valid UTF-8 text
    (`UnicodeDecodeError`) or JSON (`json.JSONDecodeError`), or the data
    fails validation (pydantic’s `ValidationError`). These errors are
    subclasses of `ValueError`.
  * [**TypeError**](https://docs.python.org/3/builtins/exceptions.html#TypeError) – If the serialized data is of another class.
  * [**OSError**](https://docs.python.org/3/builtins/exceptions.html#OSError) – If the file cannot be read, e.g. it does not exist.
    The error is the one raised by the file operation, e.g.
    `FileNotFoundError` or `PermissionError`.

#### *classmethod* from_json(json_string)

Creates an instance of the class from a JSON string representation.

This method deserializes the provided JSON string into a dictionary and uses
the from_dict method to construct the class instance. It validates that the
class name in the serialized data matches the current class.

* **Parameters:**
  **json_string** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str)) – JSON string containing the serialized object data.
* **Returns:**
  A new instance of the class with restored state.
* **Return type:**
  [`typing.Self`](https://docs.python.org/3/library/typing.html#typing.Self)
* **Raises:**
  * [**json.JSONDecodeError**](https://docs.python.org/3/library/json.html#json.JSONDecodeError) – If the string is not a valid JSON. The error
    gives the position of what is invalid, and does not hold the string.
  * [**ValueError**](https://docs.python.org/3/builtins/exceptions.html#ValueError) – If the JSON value is not an object with a class name,
    i.e. not serialized data.
  * [**TypeError**](https://docs.python.org/3/builtins/exceptions.html#TypeError) – If the class name in the serialized data doesn’t match.

#### get_cost()

Retrieves the accumulated cost information of the model and of its fallback
models, if configured.

This method collects cost statistics for the current model instance and for its
fallback models (if configured), providing insights into API usage expenses.

* **Returns:**
  A list of cost statistics containers for the model and its
  fallback models.
* **Return type:**
  [`list`](https://docs.python.org/3/builtins/stdtypes.html#list)[`contextgem.internal.data_models._ModelCostOutputContainer`]

#### get_usage()

Retrieves the usage information of the model and of its fallback models,
if configured.

This method collects token usage statistics for the current model instance and
for its fallback models (if configured), providing insights into API consumption.

* **Returns:**
  A list of usage statistics containers for the model and its
  fallback models.
* **Return type:**
  [`list`](https://docs.python.org/3/builtins/stdtypes.html#list)[`contextgem.internal.data_models._ModelUsageOutputContainer`]

#### *property* is_group *: [bool](https://docs.python.org/3/builtins/functions.html#bool)*

Returns False indicating this is a single model, not a group.

* **Returns:**
  Always False for single models.
* **Return type:**
  [bool](https://docs.python.org/3/builtins/functions.html#bool)

#### *property* list_roles *: [list](https://docs.python.org/3/builtins/stdtypes.html#list)[[str](https://docs.python.org/3/builtins/stdtypes.html#str) | [None](https://docs.python.org/3/builtins/constants.html#None)]*

Returns a list containing the role of this model.

(For a single model, this returns a list with just one element - the model’s role,
or None if the model has no role. For model groups, the method implementation returns
roles of all models in the group.)

* **Returns:**
  A list containing the role of this model.
* **Return type:**
  [list](https://docs.python.org/3/builtins/stdtypes.html#list)[[str](https://docs.python.org/3/builtins/stdtypes.html#str) | None]

#### reset_usage_and_cost()

Resets the usage and cost statistics for the model and its fallback models
(if configured).

This method clears accumulated usage and cost data, which is useful when processing
multiple documents sequentially and tracking metrics for each document separately.

* **Returns:**
  None
* **Return type:**
  [`None`](https://docs.python.org/3/builtins/constants.html#None)

#### to_dict()

Transforms the current object into a dictionary representation.

Converts the object to a dictionary that includes:
- All public attributes
- Special handling for specific public and private attributes

When a model or model group is serialized, its API credentials and usage/cost stats are removed.

* **Returns:**
  A dictionary representation of the current object with all necessary data for serialization
* **Return type:**
  [`dict`](https://docs.python.org/3/builtins/stdtypes.html#dict)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str), [`typing.Any`](https://docs.python.org/3/library/typing.html#typing.Any)]

#### to_disk(file_path)

Saves the serialized instance to a JSON file at the specified path.

This method converts the instance to a dictionary representation using to_dict(),
then writes it to disk as a formatted JSON file with UTF-8 encoding.

* **Parameters:**
  **file_path** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str) | [`pathlib.Path`](https://docs.python.org/3/library/pathlib.html#pathlib.Path)) – Path where the JSON file should be saved (must end with ‘.json’).
  Can be a string or a Path object.
* **Return type:**
  [`None`](https://docs.python.org/3/builtins/constants.html#None)
* **Returns:**
  None
* **Raises:**
  * [**ValueError**](https://docs.python.org/3/builtins/exceptions.html#ValueError) – If the file path doesn’t end with ‘.json’.
  * [**OSError**](https://docs.python.org/3/builtins/exceptions.html#OSError) – If the file cannot be written, e.g. its directory does not
    exist. The error is the one raised by the file operation, e.g.
    `FileNotFoundError` or `PermissionError`.

#### to_json()

Converts the object to its JSON string representation.

Serializes the object into a JSON-formatted string using the dictionary
representation provided by the to_dict() method.

* **Returns:**
  A JSON string representation of the object.
* **Return type:**
  [`str`](https://docs.python.org/3/builtins/stdtypes.html#str)

#### deployment_id *: [Annotated](https://docs.python.org/3/library/typing.html#typing.Annotated)[[str](https://docs.python.org/3/builtins/stdtypes.html#str), [Strict](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.Strict)(strict=True), [StringConstraints](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.StringConstraints)(strip_whitespace=True, to_upper=[None](https://docs.python.org/3/builtins/constants.html#None), to_lower=[None](https://docs.python.org/3/builtins/constants.html#None), strict=[None](https://docs.python.org/3/builtins/constants.html#None), min_length=1, max_length=[None](https://docs.python.org/3/builtins/constants.html#None), pattern=[None](https://docs.python.org/3/builtins/constants.html#None), ascii_only=[None](https://docs.python.org/3/builtins/constants.html#None))] | [None](https://docs.python.org/3/builtins/constants.html#None)*

#### api_version *: [Annotated](https://docs.python.org/3/library/typing.html#typing.Annotated)[[str](https://docs.python.org/3/builtins/stdtypes.html#str), [Strict](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.Strict)(strict=True), [StringConstraints](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.StringConstraints)(strip_whitespace=True, to_upper=[None](https://docs.python.org/3/builtins/constants.html#None), to_lower=[None](https://docs.python.org/3/builtins/constants.html#None), strict=[None](https://docs.python.org/3/builtins/constants.html#None), min_length=1, max_length=[None](https://docs.python.org/3/builtins/constants.html#None), pattern=[None](https://docs.python.org/3/builtins/constants.html#None), ascii_only=[None](https://docs.python.org/3/builtins/constants.html#None))] | [None](https://docs.python.org/3/builtins/constants.html#None)*

#### system_message *: [str](https://docs.python.org/3/builtins/stdtypes.html#str) | [None](https://docs.python.org/3/builtins/constants.html#None)*

#### max_tokens *: [Annotated](https://docs.python.org/3/library/typing.html#typing.Annotated)[[int](https://docs.python.org/3/builtins/functions.html#int), [Strict](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.Strict)(strict=True)] | [None](https://docs.python.org/3/builtins/constants.html#None)*

#### reasoning_effort *: [Literal](https://docs.python.org/3/library/typing.html#typing.Literal)['none', 'minimal', 'low', 'medium', 'high', 'xhigh', 'max'] | [None](https://docs.python.org/3/builtins/constants.html#None)*

#### max_retries_failed_request *: [Annotated](https://docs.python.org/3/library/typing.html#typing.Annotated)[[int](https://docs.python.org/3/builtins/functions.html#int), [Strict](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.Strict)(strict=True)]*

#### max_retries_invalid_data *: [Annotated](https://docs.python.org/3/library/typing.html#typing.Annotated)[[int](https://docs.python.org/3/builtins/functions.html#int), [Strict](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.Strict)(strict=True)]*

#### output_language *: [Literal](https://docs.python.org/3/library/typing.html#typing.Literal)['en', 'adapt']*

#### temperature *: [Annotated](https://docs.python.org/3/library/typing.html#typing.Annotated)[[float](https://docs.python.org/3/builtins/functions.html#float), [Strict](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.Strict)(strict=True)] | [None](https://docs.python.org/3/builtins/constants.html#None)*

#### top_p *: [Annotated](https://docs.python.org/3/library/typing.html#typing.Annotated)[[float](https://docs.python.org/3/builtins/functions.html#float), [Strict](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.Strict)(strict=True)] | [None](https://docs.python.org/3/builtins/constants.html#None)*

#### seed *: [Annotated](https://docs.python.org/3/library/typing.html#typing.Annotated)[[int](https://docs.python.org/3/builtins/functions.html#int), [Strict](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.Strict)(strict=True)] | [None](https://docs.python.org/3/builtins/constants.html#None)*

#### tools *: [Sequence](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence)[[Callable](https://docs.python.org/3/library/collections.abc.html#collections.abc.Callable)[[...], [str](https://docs.python.org/3/builtins/stdtypes.html#str)] | [Callable](https://docs.python.org/3/library/collections.abc.html#collections.abc.Callable)[[...], [Coroutine](https://docs.python.org/3/library/collections.abc.html#collections.abc.Coroutine)[[Any](https://docs.python.org/3/library/typing.html#typing.Any), [Any](https://docs.python.org/3/library/typing.html#typing.Any), [str](https://docs.python.org/3/builtins/stdtypes.html#str)]]] | [None](https://docs.python.org/3/builtins/constants.html#None)*

#### tool_choice *: [str](https://docs.python.org/3/builtins/stdtypes.html#str) | [Annotated](https://docs.python.org/3/library/typing.html#typing.Annotated)[[dict](https://docs.python.org/3/builtins/stdtypes.html#dict)[[str](https://docs.python.org/3/builtins/stdtypes.html#str), [Any](https://docs.python.org/3/library/typing.html#typing.Any)], [BeforeValidator](https://pydantic.dev/docs/validation/latest/api/pydantic/functional_validators/#pydantic.functional_validators.BeforeValidator)(func=\_validate_is_json_dict, json_schema_input_type=PydanticUndefined)] | [None](https://docs.python.org/3/builtins/constants.html#None)*

#### parallel_tool_calls *: [bool](https://docs.python.org/3/builtins/functions.html#bool) | [None](https://docs.python.org/3/builtins/constants.html#None)*

#### tool_max_rounds *: [Annotated](https://docs.python.org/3/library/typing.html#typing.Annotated)[[int](https://docs.python.org/3/builtins/functions.html#int), [Strict](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.Strict)(strict=True)]*

#### model *: [Annotated](https://docs.python.org/3/library/typing.html#typing.Annotated)[[str](https://docs.python.org/3/builtins/stdtypes.html#str), [Strict](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.Strict)(strict=True), [StringConstraints](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.StringConstraints)(strip_whitespace=True, to_upper=[None](https://docs.python.org/3/builtins/constants.html#None), to_lower=[None](https://docs.python.org/3/builtins/constants.html#None), strict=[None](https://docs.python.org/3/builtins/constants.html#None), min_length=1, max_length=[None](https://docs.python.org/3/builtins/constants.html#None), pattern=[None](https://docs.python.org/3/builtins/constants.html#None), ascii_only=[None](https://docs.python.org/3/builtins/constants.html#None))]*

#### api_key *: [Annotated](https://docs.python.org/3/library/typing.html#typing.Annotated)[[str](https://docs.python.org/3/builtins/stdtypes.html#str), [Strict](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.Strict)(strict=True), [StringConstraints](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.StringConstraints)(strip_whitespace=True, to_upper=[None](https://docs.python.org/3/builtins/constants.html#None), to_lower=[None](https://docs.python.org/3/builtins/constants.html#None), strict=[None](https://docs.python.org/3/builtins/constants.html#None), min_length=1, max_length=[None](https://docs.python.org/3/builtins/constants.html#None), pattern=[None](https://docs.python.org/3/builtins/constants.html#None), ascii_only=[None](https://docs.python.org/3/builtins/constants.html#None))] | [None](https://docs.python.org/3/builtins/constants.html#None)*

#### api_base *: [Annotated](https://docs.python.org/3/library/typing.html#typing.Annotated)[[str](https://docs.python.org/3/builtins/stdtypes.html#str), [Strict](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.Strict)(strict=True), [StringConstraints](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.StringConstraints)(strip_whitespace=True, to_upper=[None](https://docs.python.org/3/builtins/constants.html#None), to_lower=[None](https://docs.python.org/3/builtins/constants.html#None), strict=[None](https://docs.python.org/3/builtins/constants.html#None), min_length=1, max_length=[None](https://docs.python.org/3/builtins/constants.html#None), pattern=[None](https://docs.python.org/3/builtins/constants.html#None), ascii_only=[None](https://docs.python.org/3/builtins/constants.html#None))] | [None](https://docs.python.org/3/builtins/constants.html#None)*

#### role *: [Annotated](https://docs.python.org/3/library/typing.html#typing.Annotated)[[str](https://docs.python.org/3/builtins/stdtypes.html#str), [Strict](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.Strict)(strict=True), [AfterValidator](https://pydantic.dev/docs/validation/latest/api/pydantic/functional_validators/#pydantic.functional_validators.AfterValidator)(func=\_validate_model_role)] | [None](https://docs.python.org/3/builtins/constants.html#None)*

#### num_retries_failed_request *: [Annotated](https://docs.python.org/3/library/typing.html#typing.Annotated)[[int](https://docs.python.org/3/builtins/functions.html#int), [Strict](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.Strict)(strict=True)]*

#### timeout *: [Annotated](https://docs.python.org/3/library/typing.html#typing.Annotated)[[int](https://docs.python.org/3/builtins/functions.html#int), [Strict](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.Strict)(strict=True)]*

#### pricing_details *: \_ModelPricing | [None](https://docs.python.org/3/builtins/constants.html#None)*

#### is_fallback *: [Annotated](https://docs.python.org/3/library/typing.html#typing.Annotated)[[bool](https://docs.python.org/3/builtins/functions.html#bool), [Strict](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.Strict)(strict=True)]*

#### fallback_models *: [list](https://docs.python.org/3/builtins/stdtypes.html#list)[\_DocumentModel]*

#### auto_pricing *: [Annotated](https://docs.python.org/3/library/typing.html#typing.Annotated)[[bool](https://docs.python.org/3/builtins/functions.html#bool), [Strict](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.Strict)(strict=True)]*

#### auto_pricing_refresh *: [Annotated](https://docs.python.org/3/library/typing.html#typing.Annotated)[[bool](https://docs.python.org/3/builtins/functions.html#bool), [Strict](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.Strict)(strict=True)]*

### *class* contextgem.public.llms.ChatSession(\*\*data)

Bases: `_ChatSession`

Stateful chat session that preserves message history across turns.

To be used as `chat_session=...` parameter for `DocumentLLM.chat(...)`
or `DocumentLLM.chat_async(...)`.

Note:
: A chat session keeps the first system message it is given: the system message
  of an LLM is added to the session only if the session has none yet. The system
  message of the session is sent with each request of the session, also by any
  other LLM that the session is used with, instead of the system message
  of that LLM.
  <br/>
  In a chat with tools, the history of the session keeps the tool calls
  of each answered turn and their results, which are sent to the LLM with
  the next turns. The results include the errors of the tools, which the LLM
  receives as it calls them. A turn that ends without an answer, e.g. on
  an error of the LLM API, is started over when it is sent again, i.e. its
  tool calls are not kept. `messages` lists the messages of the user and
  the answers of the LLM, without the tool calls.
  <br/>
  The history also keeps the thinking blocks that an LLM returns with its tool
  calls and answers, e.g. those of Claude when `reasoning_effort` is set.
  They are sent back unchanged, so that the LLM keeps its reasoning across
  tool rounds and turns, but only to the same model with the same tools and
  system message: a provider may reject them otherwise.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be
validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

* **Parameters:**
  **custom_data** ([*Annotated*](https://docs.python.org/3/library/typing.html#typing.Annotated) *[*[*dict*](https://docs.python.org/3/builtins/stdtypes.html#dict) *[*[*str*](https://docs.python.org/3/builtins/stdtypes.html#str) *,* [*Any*](https://docs.python.org/3/library/typing.html#typing.Any) *]* *,* [*BeforeValidator*](https://pydantic.dev/docs/validation/latest/api/pydantic/functional_validators/#pydantic.functional_validators.BeforeValidator) *(**func=~contextgem.internal.typings.validators._validate_is_json_dict* *,* *json_schema_input_type=PydanticUndefined* *)* *]*)

#### clone()

Creates and returns a deep copy of the current instance.

* **Return type:**
  [`typing.Self`](https://docs.python.org/3/library/typing.html#typing.Self)
* **Returns:**
  A deep copy of the current instance.

#### *classmethod* from_dict(obj_dict)

Reconstructs an instance of the class from a dictionary representation.

This method deserializes a dictionary containing the object’s attributes and values
into a new instance of the class. It handles complex nested structures like aspects,
concepts, and extracted items, properly reconstructing each component.

A deserialized model or model group has no API credentials, as they are not
included in the serialized data, i.e. they are to be set again.

* **Parameters:**
  **obj_dict** ([`dict`](https://docs.python.org/3/builtins/stdtypes.html#dict)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str), [`typing.Any`](https://docs.python.org/3/library/typing.html#typing.Any)]) – Dictionary containing the serialized object data.
* **Returns:**
  A new instance of the class with restored attributes.
* **Return type:**
  [`typing.Self`](https://docs.python.org/3/library/typing.html#typing.Self)

#### *classmethod* from_disk(file_path)

Loads an instance of the class from a JSON file stored on disk.

This method reads the JSON content from the specified file path and
deserializes it into an instance of the class using the from_json
method.

* **Parameters:**
  **file_path** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str) | [`pathlib.Path`](https://docs.python.org/3/library/pathlib.html#pathlib.Path)) – Path to the JSON file to load (must end with ‘.json’).
  Can be a string or a Path object.
* **Returns:**
  An instance of the class populated with the data from the file.
* **Return type:**
  [`typing.Self`](https://docs.python.org/3/library/typing.html#typing.Self)
* **Raises:**
  * [**ValueError**](https://docs.python.org/3/builtins/exceptions.html#ValueError) – If the file path doesn’t end with ‘.json’, or if the file
    does not hold valid serialized data, e.g. it is not a valid UTF-8 text
    (`UnicodeDecodeError`) or JSON (`json.JSONDecodeError`), or the data
    fails validation (pydantic’s `ValidationError`). These errors are
    subclasses of `ValueError`.
  * [**TypeError**](https://docs.python.org/3/builtins/exceptions.html#TypeError) – If the serialized data is of another class.
  * [**OSError**](https://docs.python.org/3/builtins/exceptions.html#OSError) – If the file cannot be read, e.g. it does not exist.
    The error is the one raised by the file operation, e.g.
    `FileNotFoundError` or `PermissionError`.

#### *classmethod* from_json(json_string)

Creates an instance of the class from a JSON string representation.

This method deserializes the provided JSON string into a dictionary and uses
the from_dict method to construct the class instance. It validates that the
class name in the serialized data matches the current class.

* **Parameters:**
  **json_string** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str)) – JSON string containing the serialized object data.
* **Returns:**
  A new instance of the class with restored state.
* **Return type:**
  [`typing.Self`](https://docs.python.org/3/library/typing.html#typing.Self)
* **Raises:**
  * [**json.JSONDecodeError**](https://docs.python.org/3/library/json.html#json.JSONDecodeError) – If the string is not a valid JSON. The error
    gives the position of what is invalid, and does not hold the string.
  * [**ValueError**](https://docs.python.org/3/builtins/exceptions.html#ValueError) – If the JSON value is not an object with a class name,
    i.e. not serialized data.
  * [**TypeError**](https://docs.python.org/3/builtins/exceptions.html#TypeError) – If the class name in the serialized data doesn’t match.

#### *property* messages *: [list](https://docs.python.org/3/builtins/stdtypes.html#list)[\_Message]*

Returns the list of messages in the session: the system message, if any,
and the messages of the user and the answers of the LLM. The tool calls
of the LLM and their results are not listed, although they are kept in
the history of the chat that is sent to the LLM.

* **Returns:**
  The list of messages in the session.
* **Return type:**
  [list](https://docs.python.org/3/builtins/stdtypes.html#list)[\_Message]

#### reset()

Clears conversation history by removing all messages.

* **Returns:**
  None
* **Return type:**
  [`None`](https://docs.python.org/3/builtins/constants.html#None)

#### to_dict()

Transforms the current object into a dictionary representation.

Converts the object to a dictionary that includes:
- All public attributes
- Special handling for specific public and private attributes

When a model or model group is serialized, its API credentials and usage/cost stats are removed.

* **Returns:**
  A dictionary representation of the current object with all necessary data for serialization
* **Return type:**
  [`dict`](https://docs.python.org/3/builtins/stdtypes.html#dict)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str), [`typing.Any`](https://docs.python.org/3/library/typing.html#typing.Any)]

#### to_disk(file_path)

Saves the serialized instance to a JSON file at the specified path.

This method converts the instance to a dictionary representation using to_dict(),
then writes it to disk as a formatted JSON file with UTF-8 encoding.

* **Parameters:**
  **file_path** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str) | [`pathlib.Path`](https://docs.python.org/3/library/pathlib.html#pathlib.Path)) – Path where the JSON file should be saved (must end with ‘.json’).
  Can be a string or a Path object.
* **Return type:**
  [`None`](https://docs.python.org/3/builtins/constants.html#None)
* **Returns:**
  None
* **Raises:**
  * [**ValueError**](https://docs.python.org/3/builtins/exceptions.html#ValueError) – If the file path doesn’t end with ‘.json’.
  * [**OSError**](https://docs.python.org/3/builtins/exceptions.html#OSError) – If the file cannot be written, e.g. its directory does not
    exist. The error is the one raised by the file operation, e.g.
    `FileNotFoundError` or `PermissionError`.

#### to_json()

Converts the object to its JSON string representation.

Serializes the object into a JSON-formatted string using the dictionary
representation provided by the to_dict() method.

* **Returns:**
  A JSON string representation of the object.
* **Return type:**
  [`str`](https://docs.python.org/3/builtins/stdtypes.html#str)

#### *property* unique_id *: [str](https://docs.python.org/3/builtins/stdtypes.html#str)*

Returns the ULID of the instance.

#### custom_data *: [Annotated](https://docs.python.org/3/library/typing.html#typing.Annotated)[[dict](https://docs.python.org/3/builtins/stdtypes.html#dict)[[str](https://docs.python.org/3/builtins/stdtypes.html#str), [Any](https://docs.python.org/3/library/typing.html#typing.Any)], [BeforeValidator](https://pydantic.dev/docs/validation/latest/api/pydantic/functional_validators/#pydantic.functional_validators.BeforeValidator)(func=\_validate_is_json_dict, json_schema_input_type=PydanticUndefined)]*