SAP Certified - SAP Generative AI Developer

Prompt Registry, Evaluation, and Grounding

Good generative AI delivery makes its reasoning testable. SAP's current learning journey includes Prompt Registry, evaluation, and grounding with trusted enterprise data. Together, these topics help a learner replace the vague goal of “better answers” with a controlled improvement loop.

Ground Responses in Trusted Enterprise Data

Grounding connects a response to information selected for a particular business purpose. It is especially important when an answer must reflect current company policies, product information, procedures, or other controlled knowledge. A model may generate fluent language without being able to demonstrate that it used the correct source. Grounding changes the design question from “can the model answer?” to “which approved evidence should shape the answer, and how will we show that it did?”

Start with the source collection. Identify the business owner, audience, access boundary, update rhythm, and authority level for each source. Remove duplicate or superseded documents where possible. A highly relevant but outdated document can be more harmful than no document because it creates a confident, traceable, but incorrect answer. If sources disagree, define the escalation behavior instead of letting the model silently choose one.

Design Retrieval and Context Carefully

Retrieval selects material that may help answer a specific request. Context is the material and instruction supplied to the model at generation time. Both require judgment. Too little context can leave the model unsupported. Too much unrelated context can distract it, raise processing cost, and make it harder to determine why a statement appeared. Use the user's question, document metadata, permissions, and relevance signals to choose the smallest useful evidence set.

Give the model a clear evidence rule. For example, it may summarize supplied material, cite a source title, distinguish a direct statement from an inference, and say when the evidence does not answer the question. The surrounding application should enforce access before context is assembled. A source should not become visible to a user merely because it was useful to a model. In scenario analysis, access control belongs before generation, not as a hopeful instruction after output is written.

SAP's current learning covers document grounding, vector embeddings, semantic search, retrieval-augmented generation, and vector knowledge bases in SAP HANA Cloud. These phrases describe a workflow, not a promise of truth. Embeddings help represent semantic similarity, a search step selects relevant information, and retrieved evidence is supplied to the model. The solution still needs source curation, authorization, evaluation, and a clear behavior when evidence conflicts or is missing. This is how a learner reduces hallucination and bias risk without claiming that any technique removes it completely.

Evaluate What Users Actually Need

Evaluation is the structured comparison of output against the outcome the solution promises. Define criteria that a reviewer can apply consistently. For a grounded assistant, criteria could include factual support, relevance to the request, completeness, useful wording, correct handling of uncertainty, and adherence to source and access boundaries. For a structured extraction task, criteria might include field accuracy, formatting, handling of missing values, and safe escalation of ambiguous cases.

Build a compact test set of routine requests, difficult requests, out-of-scope requests, and failure cases. Include requests whose answer is absent from the source collection. A system that calmly reports an evidence gap can be more trustworthy than one that fills the gap with polished speculation. Run the same tests when changing models, prompts, retrieval settings, source collections, or orchestration logic. Without a stable comparison set, “improvement” can be subjective and temporary.

Use Prompt Registry as an Audit Trail

Prompt Registry supports managed prompt work in SAP's learning journey. Think of a registry entry as a change record that connects a prompt to a purpose and an evaluation. Capture the prompt's name, intended users, permitted inputs, expected output, owner, version, test cases, and approval status. If an output changes after an update, a team can then identify whether the cause was prompt wording, model selection, retrieved context, or a downstream application change.

Keep the prompt itself focused on behavior and constraints. Keep technical secrets, broad access rules, and confidential source content out of a prompt that could be copied into an unsafe context. Reference only what the application is authorized to provide. This distinction is important: prompt management improves repeatability, but it is not a substitute for identity, authorization, data governance, or monitoring.

Diagnose a Weak Answer

Imagine a user asks for a travel expense policy and receives a friendly but unsupported answer. Work through a diagnosis. Confirm the user could access the policy. Inspect the retrieved material for currentness and relevance. Check whether the prompt required use of supplied evidence and an honest limitation. Compare the answer against an evaluation rubric. Then decide which change is warranted: better source curation, different retrieval, clearer instructions, a validation check, or an escalation path. Do not change every component at once, because that destroys the evidence needed to learn from the result.

This sequence also helps distinguish hallucination from a system design gap. A generated claim may be unsupported because no evidence was provided, because irrelevant evidence was selected, or because the output was not checked. The right response is to strengthen the relevant layer, not simply to request that the model “be more accurate.”

Readiness Checkpoint

• I can explain why grounding needs current, authorized, purpose-fit sources.
• I can distinguish retrieval, context construction, generation, and access control.
• I can define measurable evaluation criteria for a real user outcome.
• I can use managed prompt versions and a stable test set to investigate a regression.

Official Scope and Verification

Verification date: 2026-08-01. Sources: SAP generative AI hub learning journey, SAP course on prompts, Prompt Registry, and model evaluation, SAP course on document grounding, SAP artificial intelligence learning hub, and current SAP Generative AI Developer certification.