Skip to main content

Generating code with an LLM

The OpenAPI document at https://feed.fm/api/v3/openapi.yaml is written to be read by AI coding tools as well as people. It needs no credentials and always matches production.

It also states what a client is obligated to do: the order of calls, when each playback report goes out, and when a skip is allowed. Those rules live in the document's prose rather than its schemas, spread across the introduction and the descriptions of individual operations.

The prompt below tells the model where the spec is and how to read it. Fill in the bracketed parts, paste it into your tool, and review the design it writes before you let it write code.

For a worked example of this approach, see the sample client, whose PROMPT.md and SPEC.md are the requirements and design it was built from.

Starter prompt​

Build [a TypeScript browser module / an Android service / a Python CLI / ...]
that plays music from the Feed.fm streaming API.

The API is described by the OpenAPI document at
https://feed.fm/api/v3/openapi.yaml. Fetch it and read it as follows before
designing anything:

1. Read all of info.description, the introduction at the top of the
   document. It is not background. It states rules a client must follow:
   authentication, client ids, the response envelope, throttling, station
   identity, the playback loop, and UI requirements. Licensing depends on
   the playback reports it describes.
2. Read the description of every operation you will call, in particular
   POST /session, POST /station, POST /play and every
   POST /play/{play_id}/... operation. These descriptions add rules that
   the introduction does not repeat.
3. Only then use the schemas for request and response shapes.

Where the prose and your own assumptions differ, follow the prose. Branch on
the `success` field in every response, never on the HTTP status alone.

The code should expose:
[the interface you want, e.g. connect(token, secret), findStation(name),
play(station), pause(), resume(), skip(), stop(), and events for song
started / paused / stopped / error]

Out of scope: [e.g. likes and dislikes, offline playback, crossfade]

Before writing any code, write a short design that shows which API call
happens at each point in playback: reserving a song, audio starting, while
it plays, pause, stop, skip, the song ending, and the station running out of
music. Quote the part of the spec each step relies on. Wait for me to
approve the design.

Write tests that assert on the HTTP requests the code sends, not only on its
events. A song that plays to the end without a POST /play/{play_id}/complete
request must fail a test. Test against https://feed.fm/api/v3 with the test
credentials listed in the spec. `counting` has songs that last only a few
seconds, which makes the reporting quick to check.

Checks every generated client should pass​

Tests a model writes for itself tend to check the client's own events, or that some matching request went out at some point. Neither catches a client that reports the wrong play, reports a song nobody heard, or skips without permission. Once the client runs, send the model the prompt below, adjusted to your platform and scope. Drop any check for a feature you left out.

Add a test suite to the client that checks its behavior against the Feed.fm
API rules. Keep the tests you already have, but these checks are the ones
that must pass.

1. Log every request the client sends to feed.fm and every playback state
   transition (audio start, pause, resume, stop, and end) as one JSON object per
   line. Include the time and play id; for requests also include the method, path,
   query/body fields client_id, station_id and seconds, response success/error
   code, and any returned duration_in_seconds and audio URL Expires value. Write
   a checker that reads the log and fails on any of these:

   - a start for a play id that no earlier POST /play or POST /station
     returned, or a play started twice
   - a POST /play between a POST /station search and the start of the play
     that search returned
   - two plays active at once, or the next song reserved before the current
     song's start, or more than one new play reserved between two starts
   - a started play that gets both complete and a granted skip, or that
     gets invalidate
   - a complete that arrives sooner after its start, less time paused, than
     90% of the song's duration_in_seconds
   - an elapse for a play that was never started, a gap of more than 30
     seconds between elapses while playing, no elapse on pause or stop, or
     elapse seconds that decrease
   - any POST /play to a station after it answered noMoreMusic
   - an invalidate for a started play, or for a play whose URL had expired
   - a request after the first POST /session without the same client_id

2. Run the client against https://feed.fm/api/v3 with the test credentials
   (the token and secret are the same string) and run the checker on each
   log:

   - counting: play at least three songs in a row
   - demo: play a full song with a pause partway through; stop partway
     through a song; switch stations partway through a song; restart and
     confirm the same client_id is reused and the saved station is found
     again by uuid
   - badgeo: the listener is told no music is available, and no POST /play
     is sent
   - words: with a new client_id, skip six songs in a row. The seventh
     song's start response has can_skip false: the skip control is
     disabled, and a skip request is refused with skipDenied (7) while the
     song keeps playing and no error is shown
   - twowords: with a new client_id, play both songs through. The request
     for a third answers noMoreMusic (9): the station stops cleanly, the
     second song is still completed, and POST /play is not called again.
     Then restart with the same client_id and play the station again. The
     first request answers noMoreMusic, and the listener is told the station
     has nothing to play

3. No test credential produces these failures on demand, so test them
   against [a fake server / a proxy that rewrites responses]:

   - playNotActive (12) in answer to a skip: the song keeps playing and no
     error is shown
   - audio that fails to load at a URL whose Expires is in the future:
     invalidate, reserve again, and stop after 3 failures in a row
   - an audio URL whose Expires has passed: call POST /play again for a
     fresh URL, with no invalidate
   - HTTP 200 with success: false from every endpoint the client calls
   - a 5xx, retried once, and a 4xx not retried
   - throttled (22), not retried immediately; confirm the client backs off and
     can resume requests later
   - a POST /play response that takes more than 4 seconds

4. Prove the suite works. Remove the start, elapse and complete calls one at
   a time, then the handling of a refused skip, and confirm some test fails
   each time. Put each call back afterwards and report what failed.

5. Confirm that the consumer secret is not in anything that ships to a
   device, that no numeric station id is persisted as stable identity across
   restarts, and that requests go straight to feed.fm with no proxy in between.

When the client works, go through the direct API section of the going-live checklist before you ship.