# Serializing objects and results
ContextGem provides multiple serialization methods to preserve your document processing pipeline components and results. These methods enable you to save your work, transfer data between systems, or integrate with other applications.
## πΎ Serialization Methods
The following ContextGem objects support serialization:
* [`Document`](https://contextgem.shcherbak.ai/api/documents/index.html.md#contextgem.public.documents.Document) - Contains document content and extracted information
* [`ExtractionPipeline`](https://contextgem.shcherbak.ai/api/pipelines/index.html.md#contextgem.public.pipelines.ExtractionPipeline) - Defines extraction structure and logic
* [`DocumentLLM`](https://contextgem.shcherbak.ai/api/llms/index.html.md#contextgem.public.llms.DocumentLLM) - Stores LLM configuration for document processing
* [`DocumentDecisionModel`](https://contextgem.shcherbak.ai/api/decision_models/index.html.md#contextgem.public.decision_models.DocumentDecisionModel) - Stores decision model configuration for document processing
* [`DocumentModelGroup`](https://contextgem.shcherbak.ai/api/models/index.html.md#contextgem.public.models.DocumentModelGroup) - Stores the configuration of the models in the group
Each object supports three serialization methods:
* `to_json()` - Converts the object to a JSON string for cross-platform compatibility
* `to_dict()` - Converts the object to a Python dictionary for in-memory operations
* `to_disk(file_path)` - Saves the object to a JSON file at the specified path, which must end with `.json`
## π Deserialization Methods
To reconstruct objects from their serialized forms, use the corresponding class methods:
* `from_json(json_string)` - Creates an object from a JSON string
* `from_dict(dict_object)` - Creates an object from a Python dictionary
* `from_disk(file_path)` - Loads an object from a file on disk
`to_disk()` and `from_disk()` raise the error of a file that cannot be written or read as it is, e.g. `FileNotFoundError` or `PermissionError` (subclasses of `OSError`). `from_disk()` raises the error of what the file holds as `from_json()` raises it, with a note that names the file: `ValueError` for data that is not valid JSON or fails validation (`json.JSONDecodeError` and pydanticβs `ValidationError` are subclasses of `ValueError`), and `TypeError` for the data of another class, e.g. an aspect loaded with `Document.from_disk()`.
#### NOTE
Deserialization reconstructs the full state of the original document or pipeline: all fields, extracted data, and unique IDs are preserved, so a round-tripped object compares equal to the original (e.g. `Document.from_dict(document.to_dict()) == document`). The serialized data stores each reference paragraph and reference sentence of the aspects and of the extracted items on its own, while in the restored document they are the paragraphs and sentences of the document itself, i.e. the same objects as in `document.paragraphs`, as after an extraction (see [π Locating Paragraphs and Sentences](https://contextgem.shcherbak.ai/documents/document_config/index.html.md#locating-paras-and-sents)). For models, see [π API Credentials of Models](#serialization-llm-credentials-label).
## π API Credentials of Models
The API credentials of a model, i.e. its `api_key` and `api_base`, are never included in the serialized data, so that they do not end up in the files, logs or other systems that the serialized data is passed to. The usage and cost statistics of a model are not included either. The other configuration parameters of the model are preserved, together with its fallback models.
A deserialized model therefore has no API credentials. Set them again before using the model, both for the model and for each of its fallback models:
```python
from contextgem import DocumentLLM
llm = DocumentLLM.from_json(llm_config_json) # or from_dict() / from_disk()
llm.api_key = ""
llm.api_base = "" # if applicable
for fallback_llm in llm.fallback_models:
fallback_llm.api_key = ""
fallback_llm.api_base = "" # if applicable
```
For a deserialized [`DocumentModelGroup`](https://contextgem.shcherbak.ai/api/models/index.html.md#contextgem.public.models.DocumentModelGroup), do the same for each model in its `models`.
## π Example Usage
```python
# Example of serializing and deserializing ContextGem document,
# extraction pipeline, and LLM config.
import os
from pathlib import Path
from contextgem import (
Aspect,
BooleanConcept,
Document,
DocumentLLM,
ExtractionPipeline,
StringConcept,
)
# Create a document object
text_path = (
Path(__file__).resolve().parents[4] / "tests" / "ndas" / "en_nda_with_anomalies.txt"
) # your file path here (Path adapted for testing)
doc = Document(raw_text=text_path.read_text(encoding="utf-8"))
# Create an extraction pipeline
extraction_pipeline = ExtractionPipeline(
aspects=[
Aspect(
name="Categories of confidential information",
description="Clauses describing confidential information covered by the NDA",
concepts=[
StringConcept(
name="Types of disclosure",
description="Types of disclosure of confidential information",
),
# ...
],
),
# ...
],
concepts=[
BooleanConcept(
name="Is mutual",
description="Whether the NDA is mutual (both parties act as discloser/recipient)",
add_justifications=True,
),
# ...
],
)
# Attach the pipeline to the document
doc.assign_pipeline(extraction_pipeline)
# Configure a document LLM with your API parameters
llm = DocumentLLM(
model="openai/gpt-6-luna",
api_key=os.getenv("CONTEXTGEM_OPENAI_API_KEY"),
)
# Extract data from the document
doc = llm.extract_all(doc)
# Serialize the LLM config, pipeline and document
llm_config_json = llm.to_json() # or to_dict() / to_disk()
extraction_pipeline_json = extraction_pipeline.to_json() # or to_dict() / to_disk()
processed_doc_json = doc.to_json() # or to_dict() / to_disk()
# Deserialize the LLM config, pipeline and document
llm_deserialized = DocumentLLM.from_json(
llm_config_json
) # or from_dict() / from_disk()
extraction_pipeline_deserialized = ExtractionPipeline.from_json(
extraction_pipeline_json
) # or from_dict() / from_disk()
processed_doc_deserialized = Document.from_json(
processed_doc_json
) # or from_dict() / from_disk()
# All extracted data is preserved!
assert processed_doc_deserialized.aspects[0].concepts[0].extracted_items
# API credentials (API key and API base) are not included in the serialized
# LLM config, so set them again before using the deserialized LLM
llm_deserialized.api_key = os.getenv("CONTEXTGEM_OPENAI_API_KEY")
```
## π Use Cases
* **Caching Results**: Save processed documents to avoid repeating expensive model calls
* **Transfer Between Systems**: Export results from one environment and import in another
* **API Integration**: Convert objects to JSON for API responses
* **Workflow Persistence**: Save pipeline configurations for later reuse