Best way to convert synthesia videos into sheet music? (2024)

Best way to convert synthesia videos into sheet music?

#319998903/09/22 03:13 PM

Norway

When googling around for sheet music for my skill level, I very often find versions of songs or pieces that I like, but are presented as Synthesia tutorial videos where the uploader does not have the sheet music available. Many people like synthesia and applaud the uploader, however I prefer sheet music. "I'm trying to learn, not mimic!"

Frustrated about that I have on a few occasions entered all the notes into musescore manually. But I wonder if anyone has been successful with a more automated way to do it. From what I have found, there seems to be 2 different approaches that people use, but none of these are straight forward:

One approach is to use some overly advanced audio editing software which reads the audio as a frequency spectrogram, and tries to replicate the audio the best it can by placing the frequencies' corresponding notes into a midi. It works somewhat, can be "played" in a mini player and "sound ok", however the midi output is visually very cluttered and unusable as sheet music as it can't tell harmonics and percussion apart from the base note, so the effort to clean it up is not worth it compared to manually entering the notes.

The other approach seems to be a tool or script that reads the visible falling blocks or highlighted piano keys in the video itself, placing notes according to the position of the falling synthesia blocks. This seems to be a much more promising approach giving a cleaner and workable output, however the tools I have found are in Russian, not pre-compiled, and I am not that tech savvy, let alone into linux. There are forum threads around the web with old links to services that did the trick by supplying just a youtube link, however those services are now gone.

So, has anyone found a simple solution to this which they use regularly with success? I'd be very grateful!

Re: Best way to convert synthesia videos into sheet music?

Pianotron#320000703/09/22 05:02 PM

You will usually find the sheet music on musescore or a similar website. That said, could you link to the websites you mention? I could take a look.

Re: Best way to convert synthesia videos into sheet music?

Pianotron#320023203/10/22 04:44 PM

Norway

Here's the Russian script I mentioned (link to the files in the video description):

Indeed, I normally check Musescore or use google image search looking for something similar. Right now what I am trying to convert is a simplified version of Chopin's Nocturne Op.9 No.2:

As you can see in the description, there is no sheet music available, and if you search on musescore it is not there either. And I didn't find any other easy alternatives that I liked.

I am however having some good progress with some software called AnthemScore. It uses the spectrogram approach and needs an audio file, so I used this website to just rip the audio from the youtube video: https://loader.to/en81/youtube-wav-converter.html
Anthemscore does a remarkably good job, getting both the key signature and time signature right, with a sensible pick-up measure, and with the notes mostly being on the correct staff. And it gets the rhythm right where the notes sync up where they should be in the bar (and it doesn't throw in a bunch of tiny notes everywhere like other converters do). It misses a few notes here and there, and sometimes misjudges the duration of the notes.. (for example adding a 1/4 note followed by a 2/4 break instead of just holding a 3/4 note and such, which is easy to clean up). It looks great in the preview..

But the problem is when I export it to a midi and try to open the midi in MuseScore to clean up the mistakes it made. In Musescore it looks nothing like the preview in AnthemScore. It is not in the same bars (everything is shifted without a pickup measure), notes are mixed between the treble and bass clef staves with 100s of ledger lines, and it doesn't sound the same when played back. Anthemscore can't save any other formats that MuseScore can read.

Considering that midi is just a protocol and not really a sheet music format, I guess it loses information about what goes in left and right hand and such..

Re: Best way to convert synthesia videos into sheet music?

Pianotron#320024003/10/22 05:28 PM

North Queensland, Australia.

If the source is synthesia from a video ------ then computer video processing methods involving detecting which graphical keys are active ---- the time each one starts being active --- and duration, and when each one becomes de-activated again. The rest will be down to algorithms translating/mapping those notes to a music score format as best as it can.

Synthesia at least allows people to see what notes are played ----- which is at least easier than using our ears (audio only, with no graphical information at all) to extract all the notes. So from a transcribing or exact copying of every note point-of-view ------- the note information contained in the graphics is certainly all there.

Otherwise - if somebody is able to just play with their hands the keys ---- following the synthesia pattern, and record it to midi ---- then at least the midi notes can be converted to staff form. The looks might not be great - unless enough time is spent on correcting some attributes such as note duration ----- but at least it be viewable on a staff.

Spending time of that sort of conversion probably isn't worth it ----- as the algorithms probably won't be capable of getting the results that we want to finally see. Doing it as a computer program and processing exercise might be a nice challenge for anybody that wants to do it - for the experience and the challenge though.

Re: Best way to convert synthesia videos into sheet music?

Pianotron#352639306/11/24 09:47 AM

I understand your frustration with the lack of sheet music for Synthesia tutorials. Fortunately, there are more automated solutions that could help you out. Here are some methods you could try:

Using Automatic Transcription Software with AI:

PianoConvert is a highly effective option for converting YouTube piano videos (like Synthesia videos) into sheet music. You just need to provide the video link, and the software generates a relatively accurate sheet music.

Specialized Programs:

AnthemScore: This software uses AI to convert audio files into sheet music. While it's a paid tool, it is very efficient and can produce quite accurate results.
Sibelius or Finale: Although mainly used for manual notation, these programs can also help clean up and organize transcriptions obtained from MIDI files.
Online Services:

Notation Composer: Allows you to create and edit sheet music from MIDI files. If you can extract the MIDI file from your Synthesia video, this software could be very useful for getting clean sheet music.
Extracting MIDI from Synthesia Videos:

MIDI Visualizer Tools: While the exact tools you mentioned might not be available in English or pre-compiled, there are alternatives like MIDI Visualizer or Synthesia to MIDI, which can analyze Synthesia videos and extract MIDI files.
These solutions should help you convert Synthesia videos into sheet music without manually entering every note. Feel free to try these different methods to find the one that works best for you.

Re: Best way to convert synthesia videos into sheet music?

Pianotron#3526647Yesterday at 10:35 AM

Canada