Developer Simon Edwardsson has released details of RollTab, a free iPhone and iPad application that uses a 125-million-parameter transformer to continue music played on a connected MIDI keyboard. The largest model generates about 108 notes per second on an iPhone 15, fast enough for the project's real-time performance target.
The work took 14 experiments over almost a year. Rather than process recorded audio, the system learns from MIDI events describing properties such as pitch, timing, duration and velocity. Early note-on and note-off representations sometimes left notes hanging or lost track of active state. A grammar-based alternative produced valid sequences but required several autoregressive steps for every musical note.
The final representation packages a complete note into one model step, with categorical fields for event type, pitch, time since the previous onset, duration and velocity. Chords use several notes with zero additional onset delay. The expensive transformer backbone runs once per note, while smaller output components predict the individual fields. Sustain-pedal behavior is folded into note duration during preprocessing instead of being represented as a separate event.
Training used a few hundred thousand MIDI files containing roughly 300 million note events. Edwardsson concentrated on piano-like and older public-domain classical material, removed problematic multi-track mixtures, filtered by musical coverage and deduplicated performances while accounting for transposition and tempo changes. Expanding the dataset to about five times its size made results worse, leading him to emphasize selection and cleaning over volume.
Three decoder-only transformer sizes were tested: about 33 million, 64 million and 125 million parameters. The largest generally performed best, though not by a wide margin. Scheduled sampling, in which later note fields sometimes train on the model's own earlier prediction, increased validation loss but improved the resulting continuations in the project's evaluations.
Listening tests and simple musical metrics were not sufficient to choose models consistently. Edwardsson used Gemini 3.5 Flash for paired comparisons, separately judging whether a continuation followed its prompt and whether it sounded good alone. Those preferences produced training pairs for direct preference optimization. After that stage, 69.05% of outputs from the best consensus setup were preferred to the base model in the same automated evaluation.
For deployment, the PyTorch model was exported to Core ML and quantized to eight-bit weights. Context is rebuilt from the most recent 384 notes as a session approaches its 512-note training limit.
RollTab remains experimental: short prompts are harder, generation can loop and first launch is slow while the device optimizes the model. Even so, the project shows a relatively small symbolic-music model performing interactive generation entirely on a mobile device.
