# RatingConcept [`RatingConcept`](https://contextgem.shcherbak.ai/api/concepts/index.html.md#contextgem.public.concepts.RatingConcept) is a specialized concept type that calculates, infers, and derives rating values from documents within a clearly defined numerical scale. ## 📝 Overview [`RatingConcept`](https://contextgem.shcherbak.ai/api/concepts/index.html.md#contextgem.public.concepts.RatingConcept) enables sophisticated rating analysis from documents, allowing you to: * **Derive implicit ratings**: Calculate ratings based on sentiment analysis, key criteria, or contextual evaluation * **Generate evaluative scores**: Produce numerical assessments that quantify quality, relevance, or performance * **Normalize diverse signals**: Convert qualitative assessments into consistent numerical ratings within your defined scale * **Synthesize overall scores**: Combine multiple factors or opinions into comprehensive rating assessments This concept type is particularly valuable for generating evaluative information from documents such as: * Product and service reviews where sentiment must be quantified on a standardized scale * Performance assessments requiring numerical quality or satisfaction scoring * Risk evaluations needing severity or probability measurements * Content analyses where subjective characteristics must be rated objectively ## 💻 Usage Example Here’s a simple example of how to use [`RatingConcept`](https://contextgem.shcherbak.ai/api/concepts/index.html.md#contextgem.public.concepts.RatingConcept) to extract a product rating: ```python # ContextGem: RatingConcept Extraction import os from contextgem import Document, DocumentLLM, RatingConcept # Create a Document object from text describing a product without an explicit rating smartphone_description = ( "This smartphone features a 5000mAh battery that lasts all day with heavy use. " "The display is 6.7 inch AMOLED with 120Hz refresh rate. " "Camera system includes a 50MP main sensor, 12MP ultrawide, and 8MP telephoto lens. " "The phone runs on the latest processor with 8GB RAM and 256GB storage. " "It has IP68 water resistance and Gorilla Glass Victus protection." ) doc = Document(raw_text=smartphone_description) # Define a RatingConcept that requires analysis to determine a rating product_quality = RatingConcept( name="Product Quality Rating", description=( "Evaluate the overall quality of the smartphone based on its specifications, " "features, and adherence to industry best practices" ), rating_scale=(1, 10), add_justifications=True, # include justification for the rating justification_depth="balanced", justification_max_sents=5, ) # Attach the concept to the document doc.add_concepts([product_quality]) # Configure DocumentLLM with your API parameters llm = DocumentLLM( model="openai/gpt-6-luna", api_key=os.getenv("CONTEXTGEM_OPENAI_API_KEY"), ) # Extract the concept from the document - the LLM will analyze and assign a rating product_quality = llm.extract_concepts_from_document(doc)[0] # Print the calculated rating print(f"Quality Rating: {product_quality.extracted_items[0].value}") # Print the justification print(f"Justification: {product_quality.extracted_items[0].justification}") ``` ## ⚙️ Parameters When creating a [`RatingConcept`](https://contextgem.shcherbak.ai/api/concepts/index.html.md#contextgem.public.concepts.RatingConcept), you can specify the following parameters: | Parameter | Type | Default Value | Description | |---------------------------|-------------------|-----------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `name` | `str` | (Required) | A unique name identifier for the concept | | `description` | `str` | (Required) | A clear description of what should be evaluated and rated, including the criteria for assigning different values within the rating scale (e.g., “Evaluate product quality based on features, durability, and performance where 1 represents poor quality and 10 represents exceptional quality”). The more specific the description, the more consistent and accurate the ratings will be. | | `rating_scale` | `tuple[int, int]` | (Required) | Defines the boundaries for valid ratings as a tuple of (start, end) values (e.g., `(1, 5)` for a 1-5 star rating, or `(0, 100)` for a percentage-based evaluation with an LLM). This parameter establishes the numerical range within which all ratings must fall, ensuring consistency across evaluations. The start must be 0 or more, and the end greater than the start. A decision model accepts only as many levels as `litellm` allows in a question, and rejects a larger scale with `ValueError` before any request is sent (see [📏 Limits](https://contextgem.shcherbak.ai/decision_models/decision_models/index.html.md#decision-models-limits-label)). | | `model_role` | `str | None` | `None` | The role label of the model that extracts the concept when a model group is used: the concept is routed to the model in the group with the same `role`. With the default `None`, the concept is routed to the model without a role. Ignored when a single model is used, as it extracts every concept it is given. For more details, see [🏷️ Model Roles](https://contextgem.shcherbak.ai/optimizations/optimization_choosing_llm/index.html.md#model-roles-label). | | `modality` | `str` | `"text"` | The document content the concept is extracted from. Available values: `"text"`, `"image"`, `"multimodal"` (text and/or images, whichever the document has). `"image"` and `"multimodal"` are supported for document-level concepts only, and require an LLM that supports vision when the document has images. A concept with `"image"` cannot have `add_references=True`. A decision model extracts from text only, and rejects `"image"` and `"multimodal"` with `ValueError` before any request is sent (see [✅ Supported Aspects and Concepts](https://contextgem.shcherbak.ai/decision_models/decision_models/index.html.md#decision-models-supported-label)). For more details, see [🖼️ Concept Modality](https://contextgem.shcherbak.ai/optimizations/optimization_choosing_llm/index.html.md#concept-modality-label). | | `add_justifications` | `bool` | `False` | Whether to include justifications for extracted items. Justifications provide explanations of why the LLM assigned specific rating values and the reasoning behind the evaluation, which is especially useful for understanding the factors that influenced the rating. For example, a justification might explain that a smartphone received an 8/10 quality rating based on its premium build materials, advanced camera system, and long battery life, despite lacking expandable storage. A decision model provides no justifications, and rejects `True` with `ValueError` before any request is sent (see [✅ Supported Aspects and Concepts](https://contextgem.shcherbak.ai/decision_models/decision_models/index.html.md#decision-models-supported-label)). | | `justification_depth` | `str` | `"brief"` | Justification detail level. Available values: `"brief"`, `"balanced"`, `"comprehensive"`. | | `justification_max_sents` | `int` | `2` | Maximum sentences in a justification. | | `add_references` | `bool` | `False` | Whether to include source references for extracted items. References indicate the specific locations in the document that provided information or evidence used to determine the rating. This is particularly useful for understanding which parts of the document influenced the rating assessment, allowing to trace back evaluations to relevant content that supports the numerical value assigned. A decision model provides no references for concepts, and rejects `True` with `ValueError` before any request is sent (see [✅ Supported Aspects and Concepts](https://contextgem.shcherbak.ai/decision_models/decision_models/index.html.md#decision-models-supported-label)). | | `reference_depth` | `str` | `"paragraphs"` | Source reference granularity. Available values: `"paragraphs"`, `"sentences"`. | | `singular_occurrence` | `bool` | `False` | Whether this concept has at most one extracted item. If `True`, at most one rating is extracted, and none if there is nothing to extract: an LLM that returns more than one is asked again, as for any other invalid data. For rating concepts, this parameter is particularly useful when you want to extract a single overall score (e.g., *“overall product quality”*) rather than identifying multiple ratings throughout the document for different features. This helps distinguish between a global evaluation versus component-specific ratings. Note that with advanced LLMs, this constraint may not be required as they can often infer the appropriate number of ratings to extract from the concept’s name, description, and rating context. With a decision model ([`DocumentDecisionModel`](https://contextgem.shcherbak.ai/api/decision_models/index.html.md#contextgem.public.decision_models.DocumentDecisionModel)), a concept gets at most one item with or without this parameter, unless the text is sent in parts, i.e. `max_paragraphs_to_analyze_per_call` is passed to the extraction method: a decision model then returns an item for each part, and rejects a concept with `singular_occurrence=True` with `ValueError` before any request is sent. See [sending a long text in parts](https://contextgem.shcherbak.ai/decision_models/decision_models/index.html.md#decision-models-parts-label). | | `custom_data` | `dict` | `{}` | Optional. Dictionary for storing any additional data that you want to associate with the concept. This data must be JSON-serializable. This data is not used for extraction but can be useful for custom processing or downstream tasks. | ## 🚀 Advanced Usage ### 🔍 References and Justifications for Extraction When extracting a [`RatingConcept`](https://contextgem.shcherbak.ai/api/concepts/index.html.md#contextgem.public.concepts.RatingConcept), it’s often useful to include justifications, to understand the reasoning behind the score, and references to the text that informed it: ```python # ContextGem: RatingConcept Extraction with References and Justifications import os from contextgem import Document, DocumentLLM, RatingConcept # Sample document text about a software product with various aspects software_review = """ Software Review: ProjectManager Pro 5.0 User Interface: The interface is clean and modern, with intuitive navigation. New users can quickly find what they need without extensive training. The dashboard provides a comprehensive overview of project status. Performance: The application loads quickly even with large projects. Resource-intensive operations like generating reports occasionally cause minor lag on older systems. The mobile app performs exceptionally well, even on limited bandwidth. Features: Project templates are well-designed and cover most common project types. Task dependencies are easily managed, and the Gantt chart visualization is excellent. However, the software lacks advanced risk management tools that competitors offer. Support: The documentation is comprehensive and well-organized. Customer service response time averages 4 hours, which is acceptable but not industry-leading. The knowledge base needs more video tutorials. """ # Create a Document from the text doc = Document(raw_text=software_review) # Create a RatingConcept with justifications and references enabled usability_rating_concept = RatingConcept( name="Software usability rating", description="Evaluate the overall usability of the software on a scale of 1-10 based on UI design, intuitiveness, and learning curve", rating_scale=(1, 10), add_justifications=True, # enable justifications to explain the rating justification_depth="comprehensive", # provide detailed reasoning justification_max_sents=5, # allow up to 5 sentences for justification add_references=True, # include references to source text reference_depth="sentences", # reference specific sentences rather than paragraphs ) # Attach the concept to the document doc.add_concepts([usability_rating_concept]) # Configure DocumentLLM with your API parameters llm = DocumentLLM( model="openai/gpt-6-luna", api_key=os.getenv("CONTEXTGEM_OPENAI_API_KEY"), ) # Extract the concept usability_rating_concept = llm.extract_concepts_from_document(doc)[0] # Print the extracted rating item with justification and references extracted_item = usability_rating_concept.extracted_items[0] print(f"Software Usability Rating: {extracted_item.value}/10") print(f"\nJustification: {extracted_item.justification}") print("\nSource references:") for sent in extracted_item.reference_sentences: print(f"- {sent.raw_text}") ``` ### ⭐⭐ Multiple Rating Categories You can extract multiple rating categories from a document by creating separate rating concepts: ```python # ContextGem: Multiple RatingConcept Extraction import os from contextgem import Document, DocumentLLM, RatingConcept # Sample document text about a restaurant review with multiple quality aspects to rate restaurant_review = """ Restaurant Review: Bella Cucina Atmosphere: The restaurant has a warm, inviting ambiance with soft lighting and comfortable seating. The décor is elegant without being pretentious, and the noise level allows for easy conversation. Food Quality: The ingredients were fresh and high-quality. The pasta was perfectly cooked al dente, and the sauces were flavorful and well-balanced. The seafood dish had slightly overcooked shrimp, but the fish was excellent. Service: Our server was knowledgeable about the menu and wine list. Water glasses were kept filled, and plates were cleared promptly. However, there was a noticeable delay between appetizers and main courses. Value: Portion sizes were generous for the price point. The wine list offers selections at various price points, though markup is slightly higher than average for comparable restaurants in the area. """ # Create a Document from the text doc = Document(raw_text=restaurant_review) # Define a consistent rating scale to be used across all rating categories restaurant_rating_scale = (1, 5) # Define multiple rating concepts for different quality aspects of the restaurant atmosphere_rating = RatingConcept( name="Atmosphere Rating", description="Rate the restaurant's atmosphere and ambiance", rating_scale=restaurant_rating_scale, ) food_rating = RatingConcept( name="Food Quality Rating", description="Rate the quality, preparation, and taste of the food", rating_scale=restaurant_rating_scale, ) service_rating = RatingConcept( name="Service Rating", description="Rate the efficiency, knowledge, and attentiveness of the service", rating_scale=restaurant_rating_scale, ) value_rating = RatingConcept( name="Value Rating", description="Rate the value for money considering portion sizes and pricing", rating_scale=restaurant_rating_scale, ) # Attach all concepts to the document doc.add_concepts([atmosphere_rating, food_rating, service_rating, value_rating]) # Configure DocumentLLM with your API parameters llm = DocumentLLM( model="openai/gpt-6-luna", api_key=os.getenv("CONTEXTGEM_OPENAI_API_KEY"), ) # Extract all concepts from the document extracted_concepts = llm.extract_concepts_from_document(doc) # Print all ratings print("Restaurant Ratings (1-5 scale):") for concept in extracted_concepts: if concept.extracted_items: print(f"{concept.name}: {concept.extracted_items[0].value}/5") # Calculate and print overall average rating of the extracted ratings ratings = [ concept.extracted_items[0].value for concept in extracted_concepts if concept.extracted_items ] if ratings: average_rating = sum(ratings) / len(ratings) print(f"\nOverall Rating: {average_rating:.1f}/5") ``` ## 📊 Extracted Items When a [`RatingConcept`](https://contextgem.shcherbak.ai/api/concepts/index.html.md#contextgem.public.concepts.RatingConcept) is extracted, it is populated with **a list of extracted items** accessible through the `.extracted_items` property. Each item is an instance of the `_IntegerItem` class with the following attributes: | Attribute | Type | Description | |------------------------|---------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `value` | int | The extracted rating value as an integer within the defined rating scale | | `justification` | str | Explanation of why this rating was extracted (only if `add_justifications=True`) | | `probabilities` | dict[int, float] | Probability of each integer of the rating scale, e.g. `{1: 0.05, 2: 0.25, 3: 0.7}` (only if the concept is extracted by a decision model, `None` otherwise; see [🎲 Probabilities of Extracted Items](https://contextgem.shcherbak.ai/decision_models/decision_models/index.html.md#decision-models-probabilities-label)) | | `reference_paragraphs` | list[[`Paragraph`](https://contextgem.shcherbak.ai/api/paragraphs/index.html.md#contextgem.public.paragraphs.Paragraph)] | List of paragraph objects that influenced the rating determination (only if `add_references=True`) | | `reference_sentences` | list[[`Sentence`](https://contextgem.shcherbak.ai/api/sentences/index.html.md#contextgem.public.sentences.Sentence)] | List of sentence objects that influenced the rating determination (only if `add_references=True` and `reference_depth="sentences"`) | #### NOTE To find the position of a reference paragraph or sentence in the document, e.g. to cite it or to retrieve its surrounding context, see [📍 Locating Paragraphs and Sentences](https://contextgem.shcherbak.ai/documents/document_config/index.html.md#locating-paras-and-sents). ## 💡 Best Practices - Create descriptive names for your rating concepts that clearly indicate what is being evaluated (e.g., *“Product Usability Rating”* rather than just *“Rating”*). - Enhance extraction quality by including clear definitions of what each point on the scale represents in your concept description (e.g., *“1 = poor, 3 = average, 5 = excellent”*). - Provide specific evaluation criteria in your concept description to guide the LLM’s assessment process. For example, when rating software usability, specify that factors like interface intuitiveness, learning curve, and navigation efficiency should be considered. - Enable justifications (using `add_justifications=True`) when you need to understand the reasoning behind a rating, which is particularly valuable for evaluations that involve complex criteria where the rationale may not be immediately obvious from the score alone. - Enable references (using `add_references=True`) to trace ratings back to specific evidence in the document that informed the evaluation. - Apply `singular_occurrence=True` for concepts that should yield a single comprehensive rating (like an overall product score) rather than multiple ratings throughout the document.