Specialized Models for Code, Speech, and Images

Codestral is the current reason to keep a specialized code model in the stack: fill-in-the-middle at IDE latency, not another chat endpoint. General models in the GPT-5.6 family, Claude, and Gemini still own planning, tools, and mixed context. No local bake-off, so no scoreboard. The only numbers below are vendor claims with a live official URL.

Route on shape, not on brand

A specialized model earns a second API key when the input and output stay the same shape all day. Cursor-middle completion, a finished recording, a prompt that must become pixels, a live voice session. If the next step depends on the last step’s judgment, stay on a general model.

Pick specialized when volume and latency dominate, and accept/reject is obvious. Skip it when the job is a refactor across files, a transcript that must be checked against a screenshot, or image generation that is only one tool call inside a larger agent.

Cheap is not the same as sufficient. A completer will not read a repository. A transcriber will not plan tool calls. An image model will not explain why a layout fails. Routing those jobs to a specialized endpoint just to save a few cents usually sends them back to the flagship model after a failed first pass.

Specialized models execute. They do not arbitrate. Adoption, retry, and rollback stay on a general model or on rules.

Call specialized first, general second. Codestral fills the cursor before a flagship model opens the repo. A transcript exists before a general model extracts action items. Pixels exist before a general model writes the caption. Reversing that order pays twice and throws away suffix, timestamps, and masks.

Code: FIM is the product

Codestral 25.08 shipped 2025-07-30. IDs: codestral-2508 and the alias codestral-latest. The model card lists FIM and code generation, a 128k context window, and $0.3 input / $0.9 output per million tokens.

Mistral’s published deltas are against the previous Codestral, not against GPT: +30% accepted completions, +10% code retained after suggestion, 50% fewer runaway generations. Cross-vendor HumanEval tables without an official URL are omitted.

FIM exists because a cursor has two sides. prompt is the prefix, suffix is what already sits after the caret. Chat models only continue forward. “Implement a Redis cache decorator” is a chat or general-model job. Keep FIM for the gap.

The call is documented at the FIM endpoint. Copied from that page; not run on this machine.

from mistralai.client import Mistral
import os

with Mistral(api_key=os.getenv("MISTRAL_API_KEY", "")) as mistral:
    res = mistral.fim.complete(
        model="codestral-latest",
        prompt="def",
        stream=False,
        suffix="return a+b",
    )
    print(res)

Failure that shows up immediately: omit suffix and the model continues into code that already exists, or emits a signature that fights the suffix. Same prefix, one call with suffix and one without — the difference is fill versus continue.

128k is a ceiling, not a daily autocomplete budget. Send nearby prefix and suffix, not the repository. Dumping a monorepo into Codestral is using a completer as a general model. The code generation page still centers FIM.

Pick Codestral for short, local edits in many languages at IDE latency. Skip it for API shape, unknown-stack tradeoffs, or reading a whole repository first.

Speech: two official paths

OpenAI does not ship one omni audio object. Current split, from speech to text and voice agents:

Finished files go to /v1/audio/transcriptions. Start with gpt-transcribe. Word timestamps, captions, English translation: whisper-1. The Whisper model page still names whisper-1, powered by open-source Whisper V2. Speaker labels: gpt-4o-transcribe-diarize only, not the default transcriber.

Live conversation is speech-to-speech on Realtime with gpt-realtime-2.1 (model page). Browsers use WebRTC; server-to-server uses WebSocket. Chain STT → text model → TTS only when the pipeline must keep an intermediate transcript. Barge-in and first-audio latency want Realtime, not three APIs glued together.

Hard limit from the speech-to-text guide: 25 MB per file. Formats: mp3, mp4, mpeg, mpga, m4a, wav, webm. Split on silence, not mid-sentence. whisper-1 prompts cap at 224 tokens — enough for a short glossary, not a catalog. Long term lists belong in a post-process text model, which the same guide describes.

from openai import OpenAI

client = OpenAI()
audio_file = open("audio.wav", "rb")
transcription = client.audio.transcriptions.create(
    model="gpt-transcribe",
    file=audio_file,
)
print(transcription.text)

Timestamps:

transcription = client.audio.transcriptions.create(
    file=open("speech.wav", "rb"),
    model="whisper-1",
    response_format="verbose_json",
    timestamp_granularities=["word"],
)
print(transcription.words)

Both blocks are from the official guide; not run here.

Do not chain the two transcribers as each other’s post-process. Word timestamps are a whisper-1 parameter. File streaming is documented for gpt-transcribe, not for whisper-1. Need both artifacts: two calls, or drop one requirement.

Visible failures: over 25 MB is a reject, not a quality drop. whisper-1 without verbose_json will not return words. Shipping a long-lived API key to the browser is the wrong Realtime auth layer; the documented path mints an ephemeral key on the server.

Pick file transcription for archives and subtitles. Pick Realtime for barge-in and tool use in voice. Skip swapping the two as if they were one product.

Images: gpt-image-2 in code, Midjourney in the studio

The current OpenAI image model is gpt-image-2, documented in image generation. One-shot generate/edit: Images API. Conversational edits: Responses API with a GPT-5-class mainline model (docs use gpt-5.6) and the hosted image_generation tool. Organization Verification may be required; a blocked account is not a “the model cannot draw” bug.

Vendor limits on that page: complex prompts can take up to two minutes; text placement is better than older models and still imperfect; character and brand consistency across shots is not guaranteed; layout control remains weak. No cross-product accuracy table.

from openai import OpenAI
import base64

client = OpenAI()
result = client.images.generate(
    model="gpt-image-2",
    prompt="A children's book drawing of a veterinarian using a stethoscope to listen to the heartbeat of a baby otter.",
)
image_bytes = base64.b64decode(result.data[0].b64_json)
with open("otter.png", "wb") as f:
    f.write(image_bytes)

From the official guide. Typical failure: error.code == "moderation_blocked" — change the prompt, do not retry blindly. An unknown image model ID is a hard reject.

Midjourney is a separate product. Default version is V8.2 as of 2026-07-24 (version docs, V8.2 note). Surfaces: website and Discord. There is no official REST bake-off against GPT Image.

Do not mix understanding with generation. The Claude vision guide states image understanding only — no generate, no edit. A general model can describe what a picture should look like. That is not an image API.

Pick gpt-image-2 when generation has to live in the same API bill as the rest of the app. Pick Midjourney when the work is stills, style, and human iteration. Skip asking a chat model to stand in for a generator.

Limits that actually bind

1. A number without an official URL is a rumor. Same-family deltas from a vendor post can be quoted. Unsourced cross-vendor accuracy cannot.
2. Copy current model IDs from the FIM, transcription, and Images pages. Demo names from memory go stale.
3. GPT-4o remains a 2024 multimodal milestone. The 2026-09 general line is GPT-5.6. New general-model work should follow 5.6.
4. One decision-maker per chain. Specialized calls can run in parallel. Retry and rollback stay on a general model or on rules.

Sanity check: one FIM round-trip, one file under 25 MB through gpt-transcribe, one gpt-image-2 generate. If those three fail, fix the calls before designing a toolchain.