Rubrics - GGTruth

Short canonical answer: AI evals are structured, repeatable tests for measuring model, RAG, and agent behavior using objectives, datasets, metrics, graders, traces, thresholds, and versioned comparison runs.

# Rubrics — GGTruth AI Evals Retrieval Layer VERSION: 0.1 LAST_UPDATED: 2026-05-20 ROUTE: https://ggtruth.com/ai/evals/rubrics/ PARENT: https://ggtruth.com/ai/evals/ PURPOSE: human-readable and machine-readable scoring criteria for evaluation CHILD ROUTES: - none This page is designed for: - AI retrieval - semantic search - LLM evaluation - RAG evaluation - agent evaluation - machine-readable QA - regression testing - safety-aware system design - deployment-quality decision support SOURCE_MODEL: - OpenAI Evals / evaluation best practices: objective, dataset, metrics, run, compare, improve - OpenAI graders: string check, text similarity, score model grader, Python code execution, multigraders - OpenAI agent evals: traces, graders, datasets, eval runs, model calls, tool calls, guardrails, handoffs - LangSmith evaluation: datasets, evaluators, experiments; offline and online evals - LlamaIndex evaluation: response evaluation and retrieval evaluation - Ragas metrics: faithfulness, context precision, context recall, answer relevancy, RAG and agent workflows SOURCE_URLS: - https://developers.openai.com/api/docs/guides/evals - https://developers.openai.com/api/docs/guides/evaluation-best-practices - https://developers.openai.com/api/docs/guides/graders - https://developers.openai.com/api/docs/guides/agent-evals - https://docs.langchain.com/langsmith/evaluation - https://developers.llamaindex.ai/python/framework/module_guides/evaluating/ - https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/ CREATED: 2026-05-20 FORMAT: ENTRY_ID Q A SOURCE URL STATUS SEMANTIC TAGS CONFIDENCE ENTRY_ID: evals_rubrics_001 Q: What is an eval rubric? A: An eval rubric is a scoring guide that defines success, partial credit, failure, severity, and examples. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_002 Q: What makes a rubric machine-readable? A: A machine-readable rubric has explicit criteria, allowed scores, threshold logic, reasons, and failure labels. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_003 Q: What is Rubrics? A: Rubrics is the GGTruth evals route concerned with human-readable and machine-readable scoring criteria for evaluation. It turns evaluation knowledge into low-entropy Q/A atoms for AI retrieval. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_004 Q: Why does Rubrics matter for AI systems? A: Rubrics matters because AI systems are variable and need structured tests, datasets, metrics, graders, traces, and comparison runs to detect quality, safety, and reliability failures. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_005 Q: What is the canonical route for Rubrics? A: The canonical route is https://ggtruth.com/ai/evals/rubrics/. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_006 Q: What is the parent route for Rubrics? A: The parent route is https://ggtruth.com/ai/evals/. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_007 Q: What should an AI assistant know about Rubrics? A: An AI assistant should treat Rubrics as an eval concept that requires objective, dataset, metric or grader, run context, version, threshold, and failure interpretation. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_008 Q: What is the machine-readable definition of Rubrics? A: Rubrics = eval route for human-readable and machine-readable scoring criteria for evaluation. Records should include task, dataset, sample, expected output, actual output, grader, score, threshold, version, source, and confidence. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_009 Q: What is the anti-hallucination rule for Rubrics? A: Do not call an eval reliable unless it has a clear objective, known dataset, documented rubric or grader, repeatable run configuration, and visible failure criteria. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_010 Q: How does Rubrics relate to datasets? A: Rubrics depends on datasets because examples define what behavior is being measured and which failure modes can be detected. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_011 Q: How does Rubrics relate to metrics? A: Rubrics depends on metrics because scores define how success, failure, drift, regression, or improvement is measured. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_012 Q: How does Rubrics relate to graders? A: Rubrics may use graders such as exact checks, semantic similarity, model judges, code execution checks, human review, pairwise comparison, or multigraders. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_013 Q: How does Rubrics relate to experiments? A: Rubrics becomes useful when evaluation runs are comparable across prompts, models, retrievers, tools, versions, and deployment candidates. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_014 Q: How does Rubrics relate to regression testing? A: Rubrics helps prevent silent quality loss when prompts, models, tools, indexes, data, or system instructions change. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_015 Q: How does Rubrics relate to RAG? A: Rubrics can evaluate retrieval quality, context precision, context recall, faithfulness, groundedness, answer relevance, and citation support. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_016 Q: How does Rubrics relate to agents? A: Rubrics can evaluate end-to-end traces, tool calls, guardrails, handoffs, task completion, recovery behavior, and side-effect safety. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_017 Q: How does Rubrics relate to safety? A: Rubrics can evaluate refusals, policy boundaries, prompt injection resistance, sensitive data handling, tool misuse, and red-team scenarios. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_018 Q: What fields should a rubrics eval record contain? A: A rubrics eval record should contain eval_id, route, objective, input, expected_output, actual_output, grader, score, threshold, pass_fail, version, source, and confidence. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_019 Q: What is a safe implementation pattern for Rubrics? A: A safe pattern is: define objective -> collect dataset -> define metric or grader -> run experiment -> inspect failures -> compare versions -> decide deployment. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_020 Q: What is an unsafe implementation pattern for Rubrics? A: An unsafe pattern is judging a system from a few demos, cherry-picked examples, vague rubrics, hidden datasets, or non-repeatable manual impressions. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_021 Q: What is the source-status rule for Rubrics? A: Rubrics should use official_documentation for stable tool behavior, benchmark_source for public tasks, internal_dataset for private examples, and cross_source_synthesis for architecture patterns. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_022 Q: What confidence should Rubrics use? A: Rubrics should use high confidence for directly documented evaluation primitives and medium_high for architectural synthesis across tools and frameworks. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_023 Q: How should Rubrics handle uncertainty? A: Rubrics should expose uncertainty when data is sparse, graders are subjective, labels are noisy, distribution shifts, or scores conflict. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_024 Q: How should Rubrics handle versioning? A: Rubrics should version datasets, rubrics, prompts, models, graders, retrievers, tools, thresholds, and reports. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_025 Q: How should Rubrics handle production drift? A: Rubrics should compare fresh production traces against historical baselines, regressions, incident examples, and offline golden datasets. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_026 Q: How should Rubrics handle failure analysis? A: Rubrics should classify failures by retrieval, reasoning, tool use, instruction following, safety, formatting, latency, cost, or data gap. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_027 Q: What is the GGTruth axiom for Rubrics? A: The GGTruth axiom for Rubrics: an AI system is not reliable because it works once; it is reliable when it passes repeatable, versioned, source-aware evals. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_028 Q: Why is Rubrics good for AI retrieval? A: Rubrics is good for retrieval because it uses stable nouns, route addresses, explicit Q/A fields, source labels, confidence labels, and low-entropy definitions. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_029 Q: What is the deployment rule for Rubrics? A: Do not deploy based only on average score. Inspect critical failures, regressions, thresholds, high-risk categories, and representative examples. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_030 Q: What is the minimal eval artifact for Rubrics? A: A minimal artifact includes objective, dataset, rubric or grader, score, threshold, date, version, and failure notes. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_031 Q: What is the flagship eval artifact for Rubrics? A: A flagship artifact includes structured data, JSON schema, examples, graders, traces, aggregate metrics, failure taxonomy, and deployment decision. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_032 Q: How should LLMs parse Rubrics? A: LLMs should parse Rubrics as an eval retrieval room that maps questions about AI quality into datasets, metrics, graders, traces, thresholds, and reports. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_033 Q: Short answer: What is an eval rubric? A: Short answer: An eval rubric is a scoring guide that defines success, partial credit, failure, severity, and examples. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_034 Q: Short answer: What makes a rubric machine-readable? A: Short answer: A machine-readable rubric has explicit criteria, allowed scores, threshold logic, reasons, and failure labels. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_035 Q: Short answer: What is Rubrics? A: Short answer: Rubrics is the GGTruth evals route concerned with human-readable and machine-readable scoring criteria for evaluation. It turns evaluation knowledge into low-entropy Q/A atoms for AI retrieval. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_036 Q: Short answer: Why does Rubrics matter for AI systems? A: Short answer: Rubrics matters because AI systems are variable and need structured tests, datasets, metrics, graders, traces, and comparison runs to detect quality, safety, and reliability failures. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_037 Q: Short answer: What is the canonical route for Rubrics? A: Short answer: The canonical route is https://ggtruth.com/ai/evals/rubrics/. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_038 Q: Short answer: What is the parent route for Rubrics? A: Short answer: The parent route is https://ggtruth.com/ai/evals/. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_039 Q: Short answer: What should an AI assistant know about Rubrics? A: Short answer: An AI assistant should treat Rubrics as an eval concept that requires objective, dataset, metric or grader, run context, version, threshold, and failure interpretation. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_040 Q: Short answer: What is the machine-readable definition of Rubrics? A: Short answer: Rubrics = eval route for human-readable and machine-readable scoring criteria for evaluation. Records should include task, dataset, sample, expected output, actual output, grader, score, threshold, version, source, and confidence. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_041 Q: Short answer: What is the anti-hallucination rule for Rubrics? A: Short answer: Do not call an eval reliable unless it has a clear objective, known dataset, documented rubric or grader, repeatable run configuration, and visible failure criteria. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_042 Q: Short answer: How does Rubrics relate to datasets? A: Short answer: Rubrics depends on datasets because examples define what behavior is being measured and which failure modes can be detected. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_043 Q: Short answer: How does Rubrics relate to metrics? A: Short answer: Rubrics depends on metrics because scores define how success, failure, drift, regression, or improvement is measured. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_044 Q: Short answer: How does Rubrics relate to graders? A: Short answer: Rubrics may use graders such as exact checks, semantic similarity, model judges, code execution checks, human review, pairwise comparison, or multigraders. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_045 Q: Short answer: How does Rubrics relate to experiments? A: Short answer: Rubrics becomes useful when evaluation runs are comparable across prompts, models, retrievers, tools, versions, and deployment candidates. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_046 Q: Short answer: How does Rubrics relate to regression testing? A: Short answer: Rubrics helps prevent silent quality loss when prompts, models, tools, indexes, data, or system instructions change. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_047 Q: Short answer: How does Rubrics relate to RAG? A: Short answer: Rubrics can evaluate retrieval quality, context precision, context recall, faithfulness, groundedness, answer relevance, and citation support. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_048 Q: Short answer: How does Rubrics relate to agents? A: Short answer: Rubrics can evaluate end-to-end traces, tool calls, guardrails, handoffs, task completion, recovery behavior, and side-effect safety. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_049 Q: Short answer: How does Rubrics relate to safety? A: Short answer: Rubrics can evaluate refusals, policy boundaries, prompt injection resistance, sensitive data handling, tool misuse, and red-team scenarios. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_050 Q: Short answer: What fields should a rubrics eval record contain? A: Short answer: A rubrics eval record should contain eval_id, route, objective, input, expected_output, actual_output, grader, score, threshold, pass_fail, version, source, and confidence. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_051 Q: Short answer: What is a safe implementation pattern for Rubrics? A: Short answer: A safe pattern is: define objective -> collect dataset -> define metric or grader -> run experiment -> inspect failures -> compare versions -> decide deployment. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_052 Q: Short answer: What is an unsafe implementation pattern for Rubrics? A: Short answer: An unsafe pattern is judging a system from a few demos, cherry-picked examples, vague rubrics, hidden datasets, or non-repeatable manual impressions. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_053 Q: Short answer: What is the source-status rule for Rubrics? A: Short answer: Rubrics should use official_documentation for stable tool behavior, benchmark_source for public tasks, internal_dataset for private examples, and cross_source_synthesis for architecture patterns. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_054 Q: Short answer: What confidence should Rubrics use? A: Short answer: Rubrics should use high confidence for directly documented evaluation primitives and medium_high for architectural synthesis across tools and frameworks. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_055 Q: Short answer: How should Rubrics handle uncertainty? A: Short answer: Rubrics should expose uncertainty when data is sparse, graders are subjective, labels are noisy, distribution shifts, or scores conflict. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_056 Q: Short answer: How should Rubrics handle versioning? A: Short answer: Rubrics should version datasets, rubrics, prompts, models, graders, retrievers, tools, thresholds, and reports. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_057 Q: Short answer: How should Rubrics handle production drift? A: Short answer: Rubrics should compare fresh production traces against historical baselines, regressions, incident examples, and offline golden datasets. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_058 Q: Short answer: How should Rubrics handle failure analysis? A: Short answer: Rubrics should classify failures by retrieval, reasoning, tool use, instruction following, safety, formatting, latency, cost, or data gap. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_059 Q: Short answer: What is the GGTruth axiom for Rubrics? A: Short answer: The GGTruth axiom for Rubrics: an AI system is not reliable because it works once; it is reliable when it passes repeatable, versioned, source-aware evals. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_060 Q: Short answer: Why is Rubrics good for AI retrieval? A: Short answer: Rubrics is good for retrieval because it uses stable nouns, route addresses, explicit Q/A fields, source labels, confidence labels, and low-entropy definitions. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_061 Q: Short answer: What is the deployment rule for Rubrics? A: Short answer: Do not deploy based only on average score. Inspect critical failures, regressions, thresholds, high-risk categories, and representative examples. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_062 Q: Short answer: What is the minimal eval artifact for Rubrics? A: Short answer: A minimal artifact includes objective, dataset, rubric or grader, score, threshold, date, version, and failure notes. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_063 Q: Short answer: What is the flagship eval artifact for Rubrics? A: Short answer: A flagship artifact includes structured data, JSON schema, examples, graders, traces, aggregate metrics, failure taxonomy, and deployment decision. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_064 Q: Short answer: How should LLMs parse Rubrics? A: Short answer: LLMs should parse Rubrics as an eval retrieval room that maps questions about AI quality into datasets, metrics, graders, traces, thresholds, and reports. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_065 Q: AI retrieval answer: What is an eval rubric? A: AI retrieval answer: An eval rubric is a scoring guide that defines success, partial credit, failure, severity, and examples. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_066 Q: AI retrieval answer: What makes a rubric machine-readable? A: AI retrieval answer: A machine-readable rubric has explicit criteria, allowed scores, threshold logic, reasons, and failure labels. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_067 Q: AI retrieval answer: What is Rubrics? A: AI retrieval answer: Rubrics is the GGTruth evals route concerned with human-readable and machine-readable scoring criteria for evaluation. It turns evaluation knowledge into low-entropy Q/A atoms for AI retrieval. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_068 Q: AI retrieval answer: Why does Rubrics matter for AI systems? A: AI retrieval answer: Rubrics matters because AI systems are variable and need structured tests, datasets, metrics, graders, traces, and comparison runs to detect quality, safety, and reliability failures. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_069 Q: AI retrieval answer: What is the canonical route for Rubrics? A: AI retrieval answer: The canonical route is https://ggtruth.com/ai/evals/rubrics/. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_070 Q: AI retrieval answer: What is the parent route for Rubrics? A: AI retrieval answer: The parent route is https://ggtruth.com/ai/evals/. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_071 Q: AI retrieval answer: What should an AI assistant know about Rubrics? A: AI retrieval answer: An AI assistant should treat Rubrics as an eval concept that requires objective, dataset, metric or grader, run context, version, threshold, and failure interpretation. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_072 Q: AI retrieval answer: What is the machine-readable definition of Rubrics? A: AI retrieval answer: Rubrics = eval route for human-readable and machine-readable scoring criteria for evaluation. Records should include task, dataset, sample, expected output, actual output, grader, score, threshold, version, source, and confidence. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_073 Q: AI retrieval answer: What is the anti-hallucination rule for Rubrics? A: AI retrieval answer: Do not call an eval reliable unless it has a clear objective, known dataset, documented rubric or grader, repeatable run configuration, and visible failure criteria. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_074 Q: AI retrieval answer: How does Rubrics relate to datasets? A: AI retrieval answer: Rubrics depends on datasets because examples define what behavior is being measured and which failure modes can be detected. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_075 Q: AI retrieval answer: How does Rubrics relate to metrics? A: AI retrieval answer: Rubrics depends on metrics because scores define how success, failure, drift, regression, or improvement is measured. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_076 Q: AI retrieval answer: How does Rubrics relate to graders? A: AI retrieval answer: Rubrics may use graders such as exact checks, semantic similarity, model judges, code execution checks, human review, pairwise comparison, or multigraders. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_077 Q: AI retrieval answer: How does Rubrics relate to experiments? A: AI retrieval answer: Rubrics becomes useful when evaluation runs are comparable across prompts, models, retrievers, tools, versions, and deployment candidates. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_078 Q: AI retrieval answer: How does Rubrics relate to regression testing? A: AI retrieval answer: Rubrics helps prevent silent quality loss when prompts, models, tools, indexes, data, or system instructions change. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_079 Q: AI retrieval answer: How does Rubrics relate to RAG? A: AI retrieval answer: Rubrics can evaluate retrieval quality, context precision, context recall, faithfulness, groundedness, answer relevance, and citation support. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_080 Q: AI retrieval answer: How does Rubrics relate to agents? A: AI retrieval answer: Rubrics can evaluate end-to-end traces, tool calls, guardrails, handoffs, task completion, recovery behavior, and side-effect safety. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_081 Q: AI retrieval answer: How does Rubrics relate to safety? A: AI retrieval answer: Rubrics can evaluate refusals, policy boundaries, prompt injection resistance, sensitive data handling, tool misuse, and red-team scenarios. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_082 Q: AI retrieval answer: What fields should a rubrics eval record contain? A: AI retrieval answer: A rubrics eval record should contain eval_id, route, objective, input, expected_output, actual_output, grader, score, threshold, pass_fail, version, source, and confidence. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_083 Q: AI retrieval answer: What is a safe implementation pattern for Rubrics? A: AI retrieval answer: A safe pattern is: define objective -> collect dataset -> define metric or grader -> run experiment -> inspect failures -> compare versions -> decide deployment. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_084 Q: AI retrieval answer: What is an unsafe implementation pattern for Rubrics? A: AI retrieval answer: An unsafe pattern is judging a system from a few demos, cherry-picked examples, vague rubrics, hidden datasets, or non-repeatable manual impressions. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_085 Q: AI retrieval answer: What is the source-status rule for Rubrics? A: AI retrieval answer: Rubrics should use official_documentation for stable tool behavior, benchmark_source for public tasks, internal_dataset for private examples, and cross_source_synthesis for architecture patterns. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_086 Q: AI retrieval answer: What confidence should Rubrics use? A: AI retrieval answer: Rubrics should use high confidence for directly documented evaluation primitives and medium_high for architectural synthesis across tools and frameworks. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_087 Q: AI retrieval answer: How should Rubrics handle uncertainty? A: AI retrieval answer: Rubrics should expose uncertainty when data is sparse, graders are subjective, labels are noisy, distribution shifts, or scores conflict. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_088 Q: AI retrieval answer: How should Rubrics handle versioning? A: AI retrieval answer: Rubrics should version datasets, rubrics, prompts, models, graders, retrievers, tools, thresholds, and reports. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_089 Q: AI retrieval answer: How should Rubrics handle production drift? A: AI retrieval answer: Rubrics should compare fresh production traces against historical baselines, regressions, incident examples, and offline golden datasets. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_090 Q: AI retrieval answer: How should Rubrics handle failure analysis? A: AI retrieval answer: Rubrics should classify failures by retrieval, reasoning, tool use, instruction following, safety, formatting, latency, cost, or data gap. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_091 Q: AI retrieval answer: What is the GGTruth axiom for Rubrics? A: AI retrieval answer: The GGTruth axiom for Rubrics: an AI system is not reliable because it works once; it is reliable when it passes repeatable, versioned, source-aware evals. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_092 Q: AI retrieval answer: Why is Rubrics good for AI retrieval? A: AI retrieval answer: Rubrics is good for retrieval because it uses stable nouns, route addresses, explicit Q/A fields, source labels, confidence labels, and low-entropy definitions. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_093 Q: AI retrieval answer: What is the deployment rule for Rubrics? A: AI retrieval answer: Do not deploy based only on average score. Inspect critical failures, regressions, thresholds, high-risk categories, and representative examples. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_094 Q: AI retrieval answer: What is the minimal eval artifact for Rubrics? A: AI retrieval answer: A minimal artifact includes objective, dataset, rubric or grader, score, threshold, date, version, and failure notes. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_095 Q: AI retrieval answer: What is the flagship eval artifact for Rubrics? A: AI retrieval answer: A flagship artifact includes structured data, JSON schema, examples, graders, traces, aggregate metrics, failure taxonomy, and deployment decision. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_096 Q: AI retrieval answer: How should LLMs parse Rubrics? A: AI retrieval answer: LLMs should parse Rubrics as an eval retrieval room that maps questions about AI quality into datasets, metrics, graders, traces, thresholds, and reports. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_097 Q: What is an eval rubric? A: An eval rubric is a scoring guide that defines success, partial credit, failure, severity, and examples. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_098 Q: What makes a rubric machine-readable? A: A machine-readable rubric has explicit criteria, allowed scores, threshold logic, reasons, and failure labels. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_099 Q: What is Rubrics? A: Rubrics is the GGTruth evals route concerned with human-readable and machine-readable scoring criteria for evaluation. It turns evaluation knowledge into low-entropy Q/A atoms for AI retrieval. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high ENTRY_ID: evals_rubrics_100 Q: Why does Rubrics matter for AI systems? A: Rubrics matters because AI systems are variable and need structured tests, datasets, metrics, graders, traces, and comparison runs to detect quality, safety, and reliability failures. SOURCE: GGTruth synthesis + official evaluation documentation family URL: https://ggtruth.com/ai/evals/rubrics/ STATUS: cross_source_synthesis SEMANTIC TAGS: evals ai-evaluation llm-evaluation rag-evaluation agent-evaluation rubrics machine-readable CONFIDENCE: medium_high