fix(realtime): preserve transport audio formats during agent handoff#2961
fix(realtime): preserve transport audio formats during agent handoff#2961xodn348 wants to merge 1 commit intoopenai:mainfrom
Conversation
During agent handoff in _handle_tool_call, _get_updated_model_settings_from_agent
was called with starting_settings=None, causing session.update to reset
input_audio_format and output_audio_format back to pcm16 defaults.
This overwrote transport-configured audio formats (e.g. g711_ulaw for Twilio),
closing the Twilio session and sending noise over the phone.
Fix: pass starting_settings=self._model_config.get("initial_model_settings", None)
to mirror the behaviour of __aenter__, which correctly preserves audio formats
on initial connect.
Fixes openai#1632
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ff03a4d43c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| # reset to defaults during the handoff session update. | ||
| updated_settings = await self._get_updated_model_settings_from_agent( | ||
| starting_settings=None, | ||
| starting_settings=self._model_config.get("initial_model_settings", None), |
There was a problem hiding this comment.
Passing initial_model_settings into _get_updated_model_settings_from_agent on handoff causes those values to overwrite agent-derived fields, because that helper applies starting_settings after computing instructions, tools, and handoffs. If a caller sets any of those keys in initial_model_settings (they are valid RealtimeSessionModelSettings fields), a handoff will keep the original configuration instead of switching to the new agent’s behavior. Previously this path passed None, so the new agent settings were preserved.
Useful? React with 👍 / 👎.
|
Codex's review point sounds valid. I haven't investigated the right solution for this issue, but simply passing the config this way may not work well. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ff03a4d43c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| # reset to defaults during the handoff session update. | ||
| updated_settings = await self._get_updated_model_settings_from_agent( | ||
| starting_settings=None, | ||
| starting_settings=self._model_config.get("initial_model_settings", None), |
There was a problem hiding this comment.
Passing full initial_model_settings into handoff recomputation can stop the new agent from taking effect when users include instructions, tools, handoffs, or prompt there. Fresh evidence in the current tree: _get_updated_model_settings_from_agent computes agent-derived values and then applies starting_settings with update, so this new call path restores the original values on every handoff. In those configurations, the session.update after handoff keeps old behavior instead of switching to the target agent, so only transport-safe fields (like audio formats) should be preserved here.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ff03a4d43c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| # reset to defaults during the handoff session update. | ||
| updated_settings = await self._get_updated_model_settings_from_agent( | ||
| starting_settings=None, | ||
| starting_settings=self._model_config.get("initial_model_settings", None), |
There was a problem hiding this comment.
Passing self._model_config["initial_model_settings"] into the handoff path causes _get_updated_model_settings_from_agent to reapply any instructions, tools, handoffs, or prompt from startup config after the new agent is resolved. In sessions that set any of those keys in initial_model_settings, handoff no longer switches model behavior to the target agent and may keep using the original tool/handoff set. This regression is introduced by the new starting_settings=... call here; preserve only transport-safe fields (for example audio format) instead of the full settings dict.
Useful? React with 👍 / 👎.
2 participants
Summary
Fixes #1632 — Realtime agent handoff to Twilio closes the session and sends noise over the phone.
Root Cause
During a Realtime agent handoff,
_handle_tool_callcalled_get_updated_model_settings_from_agentwithstarting_settings=None. This causedsession.updateto resetinput_audio_formatandoutput_audio_formatback to theirpcm16defaults, overwriting the transport-configuredg711_ulawformat that Twilio requires.The asymmetry was:
__aenter__(initial connect): correctly passesstarting_settings=self._model_config.get("initial_model_settings", None)_handle_tool_call(handoff): was passingstarting_settings=None← bugFix
In
_handle_tool_call, passinitial_model_settingsasstarting_settingsto_get_updated_model_settings_from_agent, matching the behavior of__aenter__:This ensures transport-configured audio formats (e.g.
g711_ulawset viamodel_config["initial_model_settings"]) are preserved when updating session settings during handoff, instead of being reset topcm16defaults.Testing
All 215 realtime tests pass.