DSPy Explained: Programming, Not Prompting, Language Models

What DSPy is, how it differs from hand-writing prompts, and a basic example of defining a module and letting DSPy optimize it

Most LLM frameworks give you a nicer way to write prompt strings. DSPy takes a different position entirely: prompts shouldn’t be hand-written at all — they should be compiled, the same way you don’t hand-write assembly when you write Python.

The core idea

You define what a task needs to do in Python — inputs, outputs, and a signature describing their relationship — not the exact wording of the prompt that accomplishes it. DSPy’s compiler then generates and iteratively optimizes the actual prompt text against a metric you specify, using your training examples. When the underlying model changes, or the task shifts slightly, you re-optimize instead of manually rewriting prompt strings and hoping the new phrasing still works.

A basic example

import dspy

class AnswerQuestion(dspy.Signature):
    """Answer questions with short, factual responses."""
    question: str = dspy.InputField()
    answer: str = dspy.OutputField()

predictor = dspy.Predict(AnswerQuestion)
result = predictor(question="What is the capital of France?")
print(result.answer)

The Signature describes the task’s shape, not its wording. dspy.Predict (or more advanced modules like dspy.ChainOfThought) handles turning that into an actual prompt sent to the configured model — and DSPy’s optimizers can later tune that prompt automatically against a dataset of examples and a scoring function, without you touching the prompt text yourself.

Why this matters in practice

Hand-tuned prompts are brittle: a prompt tuned carefully against GPT-4 often needs re-tuning for Claude, or even for a new version of the same model family. Because DSPy treats the prompt as a compiled artifact rather than source code you maintain, switching models or improving accuracy becomes a re-optimization run rather than a manual rewrite — a meaningfully different maintenance story for any LLM application expected to survive more than one model generation.

When to reach for it

DSPy adds real complexity — you need training examples and a metric, which is more setup than string-formatting a prompt template. It’s worth that cost once you’re maintaining a prompt that keeps breaking as requirements or models shift, or building a pipeline with multiple chained LLM calls where manually tuning each step’s prompt in isolation stops working. For a one-off script calling an LLM once, a plain prompt string is still the right amount of engineering.