SAP Certified - SAP Generative AI Developer

Model Selection, Orchestration, and Prompt Development

A useful generative AI solution is designed as a sequence of decisions. SAP's current learning journey covers model management, orchestration, and prompt development. This lesson turns those topics into a repeatable way to move from a business request to a testable workflow.

Choose a Model for the Job to Be Done

Model selection begins with expected behavior, not a popularity contest. Write down the task type, language needs, input size, response format, quality threshold, latency tolerance, cost boundary, and data constraints. A task that extracts fields from a document is not identical to a task that drafts a customer explanation. A task that must answer from a controlled knowledge collection has a different design need than a task that creates a first draft for expert review.

Use an evaluation set before choosing. Collect representative inputs, define what a successful output contains, and identify failures that matter. Include edge cases such as incomplete requests, conflicting source material, sensitive information, and requests outside the assistant's scope. Compare candidate configurations against the same set. This keeps a team from selecting a model based on one impressive demonstration while ignoring accuracy, consistency, or safety in ordinary work.

Think of Orchestration as Controlled Work Flow

Orchestration coordinates the steps around a model interaction. A simple workflow might classify a user request, retrieve relevant approved information, assemble context, call a model with clear instructions, check the response, and return an answer or escalation. The orchestration layer is where a developer makes the path explicit. It identifies when external services are called, which information is supplied, and what happens when a dependency fails.

Do not treat every problem as a single prompt. If a process has multiple inputs, permissions, or quality checks, use stages that can be inspected and tested. A request for a policy answer may need an identity-aware access check before retrieval. A request to create a service ticket may need structured extraction and application validation after generation. Breaking the flow into steps improves observability and gives reviewers concrete places to apply controls.

Write Prompts as Interface Contracts

A prompt is an interface between application intent and model behavior. A robust prompt states the goal, audience, allowed evidence, output format, boundaries, and behavior when evidence is absent. It should make important constraints easy to find. For example: “Answer only from the supplied documents. If the documents do not answer the question, say that the answer is unavailable and identify what source is needed.” This is more testable than “be accurate.”

Examples can clarify format, but they do not replace a clear task definition. Keep examples representative and avoid including information that should not become part of ordinary output. Specify when an answer should ask a clarification question, when it should refuse an unsafe action, and when it should return structured fields for the calling application. A prompt that serves a human user may need plain language and source labels; a prompt that serves a system may need a constrained schema and validation before the response is accepted.

Iterate with Prompt Registry Discipline

SAP's learning journey includes Prompt Registry. Use the idea of a registry as a discipline even when studying without a live system: a prompt needs a meaningful name, purpose, owner, version, approved context, test cases, and a change reason. Treating prompts as disposable text makes it hard to reproduce an outcome or explain why behavior changed. Treating them as managed assets enables review and deliberate improvement.

When a response fails, label the failure precisely. It may be a task ambiguity, missing context, irrelevant retrieved information, a model limitation, an output-format problem, or a downstream validation gap. Change one variable at a time when possible. Re-run the same test cases. Record whether the change improved the intended behavior and whether it harmed another scenario. This method produces evidence instead of a collection of unexplained prompt variants.

SAP's current advanced learning scope also includes Cloud SDK for AI, few-shot and meta prompting, multimodal input, automated evaluation, and comparison of models for accuracy and cost. These are useful experiments, not shortcuts around a problem definition. Few-shot examples can clarify a desired transformation, while multimodal input can broaden what a workflow understands. Model comparison should use the same representative workload and include both answer quality and resource tradeoffs. A higher-cost response is not better if it cannot improve the stated success criteria, and a cheaper response is not suitable if it misses required evidence or structure.

Run a Scenario Walkthrough

Suppose a sales team wants a draft response to a customer question. First define the acceptable outcome: a concise draft that references approved product material and is reviewed before sending. Next determine whether the question needs current source grounding. Then describe the orchestration: receive request, identify product, retrieve permitted material, construct prompt, generate draft, check for missing citations or unsafe claims, and present the result to a human reviewer. The model generates language, but the workflow creates accountability.

Now test a counterexample. The requester asks for a commitment that is not in the approved material. A reliable design does not invent the commitment. It signals that the source is insufficient, asks for clarification, or routes the issue to an authorized owner. This is the type of reasoning that distinguishes a plausible demo from a dependable application.

Readiness Checkpoint

• I can choose evaluation criteria before choosing a model or prompt configuration.
• I can explain where orchestration adds access checks, retrieval, validation, and recovery behavior.
• I can write a prompt with a goal, source boundary, output format, and limitation behavior.
• I can explain why prompt versions need ownership, tests, and change records.

Official Scope and Verification

Verification date: 2026-08-01. Sources: SAP generative AI hub learning journey, SAP course on prompts and LLMs in generative AI hub, SAP artificial intelligence learning hub, and current SAP Generative AI Developer certification.