Automatic piano transcription has improved enormously. Neural networks trained on many hours of piano playing now hear which notes were played remarkably well on a clean recording. Turning those notes into readable sheet music is a separate, harder problem, and that is where most of the mistakes you will notice come from.

Two jobs, two kinds of mistake

Hearing the notes

The first job is working out which keys were pressed, when, and for how long. On a clear recording of one piano, this is the strong part. Mistakes here are mostly extra quiet notes (often an overtone heard as a note), missed notes in thick chords, and notes blurred together by the pedal.

Writing them down

The second job is deciding how a human would write those notes: where the beat is, where bar lines go, which hand plays each note, whether a note is a quaver or a dotted quaver. A performer never plays exactly on the grid, so the software has to guess what was meant. Most of the mistakes people notice are here: rhythms that look fussy, bar lines in the wrong place, a note on the wrong staff.

What trips it up

  • Other sounds. Singing, guitar and drums are written down as piano notes.
  • Heavy sustain pedal. Notes ring into each other and it is hard to tell where one ends.
  • Very free timing. Rubato, fermatas and pauses make the beat hard to follow.
  • Fast ornaments and runs. Trills and quick scales can come out with odd rhythms.
  • Crossed hands. A note in the middle of the keyboard may land on the wrong staff.
  • Three-four time and pickup bars. The bar lines can start one beat out.
  • Room echo and distance. A phone across the room hears the room as much as the piano.

Why we do not print an accuracy percentage

You will see services quote a single accuracy number. A number like that is usually measured on clean test recordings and counts how many notes were heard, not whether the sheet music reads well. It also hides where the mistakes are, which is what you need to know. We would rather show you.

So every score marks the notes the machine was unsure of in grey. Most of them are right, but they are the first place to look. And the example on our front page is a real result, unedited.

How to check a transcription in ten minutes

  1. Play the recording while following the score. Mark any bar that looks or sounds wrong.
  2. Look at the grey notes first.
  3. Check the first bar: if the bar lines are a beat out, everything after will look strange but the notes are fine.
  4. Check the left hand in thick passages, where notes are most often missed.
  5. Open the MusicXML in MuseScore to fix what you found. See how to open MusicXML in MuseScore.

Getting a better result in the first place

The recording matters more than anything else. Solo piano, close to the microphone, a quiet room and lighter pedal make a bigger difference than any setting. Read how to record piano on your phone.