3.4.1

# DSPy 3.4.1

DSPy 3.4.1 makes GEPA's structured `InstructionProposer` the default instruction proposer and lets `dspy.RLM` build DSPy sub-agents from inside its REPL. It also adds Python 3.15 support, fixes a 3.4.0 import crash, and includes streaming, adapter, and interpreter fixes.

```bash
pip install --upgrade "dspy==3.4.1"
```

**Upgrade notices.** GEPA's default reflection request changes shape, so optimization results can differ from 3.4.0 even with the same seed. The reflection LM must now be a `dspy.LM` that can return JSON. Every RLM prompt now includes a sub-agent section, and `max_llm_calls` now also counts sub-agent LM calls. `dspy.ReActV2` signatures can no longer declare inputs named `history` or `tools`. Programs saved on 3.4.0 with a null `Score` level no longer load. See [API and Compatibility Changes](#api-and-compatibility-changes).

The [3.5 LM-interface cutoff](https://dspy.ai/community/normalized-lm-api-migration/#the-35-cutoff) announced in 3.4.0 still applies. Legacy `forward()`/`aforward()` custom LMs, the legacy-engine wrappers, custom `complete_legacy()` shortcuts, and OpenAI-style `messages=` calls are deprecated and will be removed in 3.5. `lm("hello")` stays supported.

## Highlights

### `InstructionProposer` Is GEPA's Default Proposer — @dbreunig

`dspy.GEPA` now builds `InstructionProposer()` when you don't pass `instruction_proposer=`. The old default formatted reflective examples into one raw prompt and pulled the new instruction out of backticks. The new one calls the reflection LM through `dspy.Predict` with a `JSONAdapter`. Reflective examples stay structured, so `dspy.History` inputs, tool trajectories, and `dspy.Type` values such as `dspy.Image` (including nested images) reach the reflection LM as structured content instead of text placeholders. GEPA's reflection instructions are unchanged.

You can configure the proposer directly:

```python
import dspy
from dspy.teleprompt.gepa import InstructionProposer

optimizer = dspy.GEPA(
    metric=metric,
    reflection_lm=dspy.LM("openai/gpt-5"),
    instruction_proposer=InstructionProposer(
        skills=["./skills/prompt-engineering"],
        additional_instructions="Write instructions in imperative voice.",
        max_chars=1500,
        truncate_history_outputs=True,
    ),
    auto="medium",
)
```

- `skills`: reference material for the reflection LM. Pass a path, a directory containing `SKILL.md`, an inline string, or a list of these. YAML frontmatter is parsed at construction.
- `additional_instructions`: extra guidance applied to every proposal.
- `base_instructions`: replaces the reflection prompt but keeps its input and output fields.
- `max_chars`: a hard character limit. A draft that is too long gets one compression attempt; DSPy never truncates an instruction.
- `truncate_history_outputs`: shortens long tool and REPL outputs inside history inputs before rendering, without changing the originals.
- `adapter`: the adapter used for proposal calls. The default is `JSONAdapter()`.

Custom `instruction_proposer` functions receive the same reflective dataset as before, and `MultiModalInstructionProposer` remains available.

This release also fixes reflection inputs that contain a value pydantic can't serialize, such as a `numpy.int64` from a pandas row. That value now renders as `str(value)` in place, instead of turning every example into a Python repr.

PRs: [#10555](https://github.com/stanfordnlp/dspy/pull/10555); follow-up fix [#10634](https://github.com/stanfordnlp/dspy/pull/10634) by @isaacbmiller

### RLM Can Build DSPy Sub-Agents — @michaelisaac-dev

Code that `dspy.RLM` writes can now `import dspy` and build sub-agents for subtasks that need structured inputs and outputs, tools, or their own recursion:

- `dspy.Predict` and `dspy.ChainOfThought`
- `dspy.ReActV2`, using the tools you gave the RLM
- a nested `dspy.RLM`, which runs on the same interpreter backend as its parent

Sub-agents are built and run on the host. Only JSON crosses the interpreter boundary.

There is no new API to adopt; this is on for every RLM:

```python
import dspy

rlm = dspy.RLM(
    "logs, question -> answer",
    sub_lm=dspy.LM("openai/gpt-4o-mini"),  # used by llm_query and sub-agents
    max_llm_calls=30,                       # one budget for both
)
```

Inside the REPL, the model can now write code like this:

```python
import dspy

triage = dspy.ChainOfThought("log_chunk -> errors")
errors = [triage(log_chunk=chunk).errors for chunk in chunks]
```

- **One budget.** `max_llm_calls` covers `llm_query`, `llm_query_batched`, and every sub-agent LM call. Each requested completion uses one slot, so `n` can't multiply it. Sub-agents use `sub_lm`, or `dspy.settings.lm` when it isn't set.
- **Restricted LM options.** LM calls made for sandboxed code accept only generation parameters, such as `temperature`, `max_tokens`, `response_format`, reasoning effort, tool settings, and caching options. Sandboxed code can't set `model`, `api_base`, `api_key`, or headers.
- **Interpreters.** The default `PythonInterpreter` and process-isolated interpreters such as `LocalInterpreter` host sub-agents. If an interpreter can't run the sub-agent shim and raises `CodeExecutionError` or `SyntaxError`, RLM logs a warning and runs that call without sub-agents. A RestrictedPython-based interpreter is one example.
- **In-process interpreters.** Interpreters that run code with `exec()` in the host process also host sub-agents. DSPy binds the facade in that interpreter's namespace and leaves the host's `dspy` module in place. These interpreters isolate nothing, so the call budget and LM-option restrictions only bind well-behaved code. Use them only for trusted code.

`dspy.Flex` uses the same facade for its bridged predictors. Flex now also works with in-process interpreters you choose, without replacing the host's `dspy` module.

PRs: [#10478](https://github.com/stanfordnlp/dspy/pull/10478); follow-up fixes [#10625](https://github.com/stanfordnlp/dspy/pull/10625) and [#10627](https://github.com/stanfordnlp/dspy/pull/10627) by @isaacbmiller

## API and Compatibility Changes

### GEPA Reflection Requests

- **Optimization results can differ from 3.4.0 even with the same seed.** The default reflection request is now a system message with structured input fields and JSON output.
- **The reflection LM must return JSON that the adapter can parse.** If the reflection model answers in plain text or a code fence, the proposal fails. GEPA records the failure in its log and keeps the current candidate, so a run can finish with the seed program unchanged. Use a reflection model that supports JSON or structured outputs.
- **`reflection_lm` must be a `dspy.LM` or another `dspy.BaseLM`.** In 3.4.0, GEPA called `reflection_lm(prompt)` directly, so a plain function happened to work. Now every proposal made with a plain function fails, and the program stays unoptimized. Wrap a custom endpoint in `dspy.LM`, using a custom `engine=` if needed.
- `dspy.teleprompt.gepa.gepa_utils` no longer re-exports `InstructionProposalSignature`. Import it from `gepa.strategies.instruction_proposal` instead.
- `pyyaml>=6.0.1` is a new dependency.

PRs: [#10555](https://github.com/stanfordnlp/dspy/pull/10555)

### RLM Sub-Agents

- **Every RLM prompt now includes a sub-agent section.** There is no flag to turn it off. Because RLM prompts differ from 3.4.0, LM caches and recorded responses from 3.4.0 won't match.
- **Sub-agent calls count toward `max_llm_calls`.** An RLM that uses sub-agents reaches the limit sooner; raise it if needed.
- **New reserved names.** An RLM input named `dspy`, or a tool named `dspy` or starting with `_dspy` or `__dspy`, now raises `ValueError`.
- **Interpreter authors:** each RLM call now runs the sub-agent shim through `execute()` before the first iteration. Scripted or fake interpreters used in tests receive that shim as their first `execute()` call. Recognize it by comparing against `dspy.primitives.facade.SHIM_SETUP`.
- **Known limitation:** an interpreter that raises anything other than `CodeExecutionError` or `SyntaxError` while running the shim aborts the RLM call. Raise `CodeExecutionError` for code your interpreter can't run. A follow-up will make RLM tolerate these interpreters when no sub-agent is used.

PRs: [#10478](https://github.com/stanfordnlp/dspy/pull/10478), [#10625](https://github.com/stanfordnlp/dspy/pull/10625)

### Flex Bridge Constants Moved to `dspy.primitives.facade`

**Breaking change for sandbox backend authors.** The shim constants and helper that `dspy.predict.flex.bridge` exposed now live in `dspy.primitives.facade`, which `dspy.Flex` and `dspy.RLM` share. The old module doesn't re-export them, so importing any of these names from `dspy.predict.flex.bridge` raises `ImportError`: `SHIM_SETUP`, `CONSTRUCT_TOOL`, `CALL_TOOL`, `BRIDGEABLE_KINDS`, `SIGNATURE_MARKER`, `TOOL_MARKER`, `prediction_to_fields`.

```python
# Before (3.4.0)
from dspy.predict.flex.bridge import BRIDGEABLE_KINDS, CALL_TOOL, CONSTRUCT_TOOL, SHIM_SETUP

# After (3.4.1)
from dspy.primitives.facade import BRIDGEABLE_KINDS, CALL_TOOL, CONSTRUCT_TOOL, SHIM_SETUP
```

The marker and tool-name constants keep their values. The private shim file moved from `dspy/predict/flex/_sandbox_shim.py` to `dspy/primitives/_facade_shim.py`, so `dspy.predict.flex._sandbox_shim` no longer imports. Read the shim through `SHIM_SETUP`. Code that only constructs and runs `dspy.Flex` programs is unaffected.

PR: [#10478](https://github.com/stanfordnlp/dspy/pull/10478)

### Null `Score` Level Descriptions Are Rejected

Each `Score` level description must be a string, an object, or an array. DSPy 3.4.0 accepted `null`; 3.4.1 rejects it before any request is made:

- Declaring `Score[None, "Disruptive", "Blocking"]` raises `ValueError: Score level descriptions cannot be null.` when the signature class is defined.
- `predictor.set_criteria(name, [None, ...])` raises `ValueError`.

**Programs saved on 3.4.0 with a null `Score` level no longer load.** `program.load(path)` raises `ValueError: Invalid criteria for '<name>': must match the declared decision type and options.` The message doesn't mention `null`. Whole-program saves (`save_program=True`) still load with `dspy.load()`, but raise the same error the first time the predictor runs or is saved.

To fix a saved JSON program, replace each `null` in the field's `criteria` list with a description. Alternatively, delete the `criteria` key so the field falls back to the levels declared in the signature:

```diff
 "fields": {
   "severity": {
-    "criteria": [null, "Disruptive", {"what": "Blocking"}]
+    "criteria": ["Minor", "Disruptive", {"what": "Blocking"}]
   }
 }
```

For a whole-program save, call `predictor.set_criteria(name, [...])` without `null` entries after `dspy.load()`, then save again.

PR: [#10538](https://github.com/stanfordnlp/dspy/pull/10538)

### Bundled Model Metadata Now Follows LiteLLM 1.105.0

DSPy's bundled model-metadata snapshot moved from LiteLLM 1.102.1 to 1.105.0. Upstream added 803 entries and removed 264. Of those, 234 had passed their provider deprecation dates and 30 were renamed or retired.

**Offline setups lose capability flags for some still-routable models.** DSPy uses the bundled snapshot only when it can't use LiteLLM's live map: offline, when the fetch fails, or with `LITELLM_LOCAL_MODEL_COST_MAP=true`. In those setups, models whose entries were removed no longer report tool-calling, structured-output, or reasoning support. Examples are `anthropic/claude-3-7-sonnet-20250219`, `gemini/gemini-2.0-flash`, `xai/grok-4`, and `groq/llama-3.3-70b-versatile`. DSPy changes these requests without a warning:

- `JSONAdapter` stops requesting structured outputs and falls back to JSON mode or prompt-only JSON.
- Adapters stop sending tools as native function calls.
- `dspy.Reasoning` no longer turns on native reasoning for models that lost the flag.

Online, 3.4.0 and 3.4.1 behave the same for these models, because LiteLLM's live map has dropped the same entries.

**Family fallback rules are now provider-scoped.** A capability rule that names its providers now fills missing flags only for those providers' entries. As a result, `gemini/deep-research-preview-04-2026`, `gemini/deep-research-max-preview-04-2026`, and `gemini/deep-research-pro-preview-12-2025` no longer inherit `supports_reasoning` from the OpenAI deep-research rule.

What to do:

- If you rely on structured outputs or native tool calls, move to current model IDs, and check `lm.supports_response_schema` and `lm.supports_function_calling` after upgrading.
- To keep a removed model working offline, point `LITELLM_MODEL_COST_MAP_URL` at a metadata file that still contains its entry, and unset `LITELLM_LOCAL_MODEL_COST_MAP`. The file replaces the whole map, so it must cover every model you use.

PRs: [#10517](https://github.com/stanfordnlp/dspy/pull/10517), [#10539](https://github.com/stanfordnlp/dspy/pull/10539), [#10598](https://github.com/stanfordnlp/dspy/pull/10598), [#10624](https://github.com/stanfordnlp/dspy/pull/10624)

### Python 3.15 and Dependencies

- DSPy now supports Python 3.15 (`requires-python = ">=3.10, <3.16"`).
- On Python 3.15, `litellm` and `gepa` are not installed, because both declare `requires-python <3.15`. LM calls run on the bundled native engine, and routes that need the LiteLLM fallback are unavailable there. `dspy.GEPA` raises `ImportError` on 3.15 until gepa supports it.
- `jsonschema>=4.0.0` is now a direct dependency.

PR: [#10521](https://github.com/stanfordnlp/dspy/pull/10521)

### Custom Adapters and `main_request`

History replay now passes `main_request=` to `format_user_message_content`, so each historical turn keeps the output-format reminder from the original request. A custom adapter that overrides `format_user_message_content` without a `main_request` parameter raises `TypeError` when it formats `dspy.History`. Add `main_request: bool = False` to your override. `TwoStepAdapter` has been updated.

PRs: [#10554](https://github.com/stanfordnlp/dspy/pull/10554), [#10628](https://github.com/stanfordnlp/dspy/pull/10628)

### ReActV2 Reserves the `history` and `tools` Input Names

`dspy.ReActV2` signatures can no longer declare input fields named `history` or `tools`; the constructor raises `ValueError`. ReActV2 supplies both itself: `history` is its structured run history, and `tools` is the tool list you pass to the constructor.

- **If your signature declares `history: dspy.History`**, for example a chat agent that carries earlier turns, remove the field and keep passing `history=` when you call the agent. ReActV2 already used that argument to seed the run, so behavior is unchanged:

  ```python
  # Before (3.4.0)
  agent = dspy.ReActV2("history: dspy.History, question -> answer", tools=[lookup_order])

  # After (3.4.1)
  agent = dspy.ReActV2("question -> answer", tools=[lookup_order])

  result = agent(question=question, history=history)  # the call is unchanged
  ```

- **If your signature uses `history` or `tools` for your own data**, for example `history: str` holding a ticket transcript, rename the field. In 3.4.0, ReActV2 replaced such fields with its own, and calls failed with a validation error or `TypeError`.
- Tools are fixed when the agent is constructed. Passing `tools=` at call time has no effect.

PR: [#10464](https://github.com/stanfordnlp/dspy/pull/10464)

### `X | Y` Annotations Render Like `typing.Union`

On Python 3.10–3.13, adapters now describe a field annotated `int | None` as `Union[int, NoneType]` instead of `UnionType[int, NoneType]`. This matches how `Optional[int]` already rendered and how Python 3.14 renders `int | None`. Parsing is unchanged, but prompts for signatures that use `|` unions differ from 3.4.0, so cached LM responses and recorded prompts from 3.4.0 won't match.

PR: [#10575](https://github.com/stanfordnlp/dspy/pull/10575)

### Additional Fixes

- `dspy.Tool` stores argument defaults in JSON form: an Enum becomes its value, `Decimal` a float, `datetime` and `date` ISO strings, `Path` a string, tuples and sets lists, and pydantic models and dataclasses dicts. Tool schemas with such defaults previously failed to serialize for native function calling. The tool still receives its original Python default when the argument is omitted. [#10574](https://github.com/stanfordnlp/dspy/pull/10574)
- Fix the 3.4.0 import crash `ImportError: cannot import name 'BaseModel' from partially initialized module 'openai._models'`, which could occur when DSPy lazily imported LiteLLM, for example through `dspy.LM(...).supports_function_calling`. [#10520](https://github.com/stanfordnlp/dspy/pull/10520)
- Streamed ChatAdapter and XMLAdapter chunks no longer include the whitespace around field markers, so the joined chunks equal the parsed field value. [#10553](https://github.com/stanfordnlp/dspy/pull/10553)
- Replayed `dspy.History` turns keep their output-format reminders, so each turn's prompt is a prefix of the next one for prompt caching. [#10554](https://github.com/stanfordnlp/dspy/pull/10554)
- A JSON value that contains one item pydantic can't serialize now renders as JSON with that item as `str(item)`, instead of the whole value becoming a Python repr. This applies to all adapters. [#10634](https://github.com/stanfordnlp/dspy/pull/10634)
- `BetterTogether` raises a `ValueError` that names the student predictors without an LM. 3.4.0 failed later with `AttributeError: 'NoneType' object has no attribute 'launch'`. Call `student.set_lm(lm)`; `dspy.configure(lm=...)` alone is not enough. [#10569](https://github.com/stanfordnlp/dspy/pull/10569)
- Calling a module's `forward` no longer walks the whole call stack to decide whether to warn. [#10534](https://github.com/stanfordnlp/dspy/pull/10534)

## Full PR List

### GEPA and Optimizers

- Make `InstructionProposer` GEPA's default instruction proposer by @dbreunig ([#10555](https://github.com/stanfordnlp/dspy/pull/10555)).
- Raise a clear error when `BetterTogether` student predictors have no LM by @adityashirsatrao007 ([#10569](https://github.com/stanfordnlp/dspy/pull/10569)).

### Agents, RLM, Flex, and Interpreters

- Reject ReActV2 input fields named `history` or `tools` by @chenlichao ([#10464](https://github.com/stanfordnlp/dspy/pull/10464)).
- Let RLM build DSPy sub-agents through a sandbox facade shared with Flex by @michaelisaac-dev ([#10478](https://github.com/stanfordnlp/dspy/pull/10478)).
- Fall back to no sub-agents when an interpreter rejects the facade shim with `SyntaxError` by @isaacbmiller ([#10625](https://github.com/stanfordnlp/dspy/pull/10625)).
- Host the facade in in-process interpreters without replacing the host's `dspy` by @isaacbmiller ([#10627](https://github.com/stanfordnlp/dspy/pull/10627)).

### Adapters, Streaming, and Types

- Keep boundary whitespace out of ChatAdapter stream chunks by @jayzuccarelli ([#10553](https://github.com/stanfordnlp/dspy/pull/10553)).
- Preserve output-format reminders in History replay by @isaacbmiller ([#10554](https://github.com/stanfordnlp/dspy/pull/10554)).
- Accept `main_request` in `TwoStepAdapter.format_user_message_content` by @isaacbmiller ([#10628](https://github.com/stanfordnlp/dspy/pull/10628)).
- Serialize non-JSON values one at a time in `serialize_for_json` by @isaacbmiller ([#10634](https://github.com/stanfordnlp/dspy/pull/10634)).
- Reject null `Score` level descriptions by @ilia-iliev ([#10538](https://github.com/stanfordnlp/dspy/pull/10538)).
- Correct return annotations that said `bool` but returned something else by @Aryan-Kochhar ([#10408](https://github.com/stanfordnlp/dspy/pull/10408)).
- Store tool argument defaults in JSON form by @jayhemnani9910 ([#10574](https://github.com/stanfordnlp/dspy/pull/10574)).
- Render PEP 604 unions like `typing.Union` in prompts by @kayzenline ([#10575](https://github.com/stanfordnlp/dspy/pull/10575)).

### Core, Performance, and Platform

- Fix lazy package initialization during submodule imports by @isaacbmiller ([#10520](https://github.com/stanfordnlp/dspy/pull/10520)).
- Avoid walking the stack in the module `forward` guard by @isaacbmiller ([#10534](https://github.com/stanfordnlp/dspy/pull/10534)).
- Prepare DSPy for Python 3.15 by @migurski ([#10521](https://github.com/stanfordnlp/dspy/pull/10521)).

### Model Metadata

- Refresh bundled model metadata to LiteLLM 1.103.1 and scope family fallback rules to their providers by @isaacbmiller ([#10517](https://github.com/stanfordnlp/dspy/pull/10517)).
- Update bundled model metadata to LiteLLM 1.104.0 by @isaacbmiller ([#10539](https://github.com/stanfordnlp/dspy/pull/10539)).
- Update bundled model metadata to LiteLLM 1.104.2 by @isaacbmiller ([#10598](https://github.com/stanfordnlp/dspy/pull/10598)).
- Refresh bundled model metadata from LiteLLM 1.105.0 by @isaacbmiller ([#10624](https://github.com/stanfordnlp/dspy/pull/10624)).

### Documentation

- Add a tutorial for Jev decision types (`Noul`, `Score`, `Choice`, `ReAnchor`) by @zamal-db ([#10510](https://github.com/stanfordnlp/dspy/pull/10510)).
- Fix broken notebook links by @thomasht86 ([#10513](https://github.com/stanfordnlp/dspy/pull/10513)).
- Fix broken cheatsheet snippets and typos by @zh-hanlabs ([#10514](https://github.com/stanfordnlp/dspy/pull/10514)).
- Fix the `from_dict_list` docstring parameter name by @simpleqt ([#10415](https://github.com/stanfordnlp/dspy/pull/10415)).
- Fix the `ngrams` docstring parameter name by @simpleqt ([#10417](https://github.com/stanfordnlp/dspy/pull/10417)).
- Fall back to Pillow's default font for social cards when DejaVu is missing by @isaacbmiller ([#10535](https://github.com/stanfordnlp/dspy/pull/10535)).
- Fix broken tutorial links by @atharva-mashalkar ([#10567](https://github.com/stanfordnlp/dspy/pull/10567)).
- Generate the missing Markdown files listed in `llms.txt` by @thomasht86 ([#10576](https://github.com/stanfordnlp/dspy/pull/10576)).
- Bump mkdocstrings to 1.0.7 for pymdown-extensions 12.2 by @isaacbmiller ([#10629](https://github.com/stanfordnlp/dspy/pull/10629)).
- Require agents to disclose PR origin by @isaacbmiller ([#10636](https://github.com/stanfordnlp/dspy/pull/10636)).

### Testing, CI, Dependencies, and Release Engineering

- Update main's version metadata after the 3.4.0 release ([#10496](https://github.com/stanfordnlp/dspy/pull/10496)).
- Support the latest LiteLLM and MCP test fixtures by @isaacbmiller ([#10593](https://github.com/stanfordnlp/dspy/pull/10593)).
- Deflake the LocalInterpreter timeout tests by @isaacbmiller ([#10630](https://github.com/stanfordnlp/dspy/pull/10630)).
- Update the GitHub Actions upload-artifact action ([#10541](https://github.com/stanfordnlp/dspy/pull/10541)).
- Update the documentation build's mkdocstrings-python dependency ([#10543](https://github.com/stanfordnlp/dspy/pull/10543)).
- Update the documentation build's zensical dependency ([#10544](https://github.com/stanfordnlp/dspy/pull/10544)).

## Contributors

Thank you to @adityashirsatrao007, @Aryan-Kochhar, @atharva-mashalkar, @chenlichao, @dbreunig, @ilia-iliev, @isaacbmiller, @jayhemnani9910, @jayzuccarelli, @kayzenline, @michaelisaac-dev, @migurski, @simpleqt, @thomasht86, @zamal-db, and @zh-hanlabs for contributing to this release. Dependency and version updates were automated by Dependabot and GitHub Actions.

**Full Changelog:** https://github.com/stanfordnlp/dspy/compare/3.4.0...3.4.1