Forums

Sega Master System / Mark III / Game Gear
SG-1000 / SC-3000 / SF-7000 / OMV
Home - Forums - Games - Scans - Maps - Cheats - Credits
Music - Videos - Development - Hacks - Translations - Homebrew

View topic - OVGen, SidWiz and similar programs

Reply to topic
Author Message
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 12477
  • Location: London
Reply with quote
OVGen, SidWiz and similar programs
Post Posted: Wed Oct 10, 2018 2:39 pm
I'm thinking of generating some YouTube videos from VGM packs. There was a program called OVGen which seems to have been used to make lots of videos, but has apparently disappeared; and there's SidWiz, which seems more active. I was wondering if someone has figured out a nice way to automate the process though:

* Take a VGM pack
* For each file, dump the channels to WAV files - seems possible using VGMPlay but requires some messing with the config file instead of commandline options
* Preferably, check if any channels are silent and exclude them - should be possible by checking for silence
* Render to a YouTube-compatible video file, ideally with the "album art" included somewhere - not sure how best to do that with SidWiz, it seems not to be command line driveable and it seems any album art stage would have to be an extra custom step. Are there other programs that may do better?
* Upload the videos to YouTube, and make a playlist - it seems there are commandline tools to use the API for that, not sure how many uploads I can make before they throttle me though
* Insert that playlist into our VGM pack page - more of a problem for me, but it's just HTTP operations really so curl-able

All of this looks doable, but I wondered if someone else has done if before and can offer some tips...
  View user's profile Send private message Visit poster's website
  • Joined: 15 Sep 2009
  • Posts: 373
Reply with quote
Post Posted: Wed Oct 10, 2018 3:46 pm
For generating the WAV files, you can use kode54's Multidumper.
IIRC someone also wrote a GUI for it.
  View user's profile Send private message
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 12477
  • Location: London
Reply with quote
Post Posted: Wed Oct 10, 2018 4:50 pm
I'm not a big fan of game_music_emu which is what multidumper uses - blip_buffer has a filtering effect which I dislike. I'll give it a try though, maybe it's got better in the last 10 years. I'm looking for things I can script, so it can churn through packs without interaction, so GUIs are actually bad for me.
  View user's profile Send private message Visit poster's website
  • Joined: 30 Jun 2016
  • Posts: 118
  • Location: Victoria, Australia
Reply with quote
Post Posted: Fri Oct 12, 2018 5:22 am
I've been interested in this particular sort of stuff for a few years now.

I've used SIDWiz in the past for printouts, and whilst I've not used OVGen, well, I've had contact with the author. He's a nice, chill guy.
Speaking with the OVGen author, he states that he's currently moving the application over to GitLab.

We do have a Discord channel, if you want to contact the people that work on SIDWiz and OVGen, but I won't openly advertise it in this post.

I have also contacted you/Maxim over the SMSPower! Discord channel. I've had a hand on producing videos by the batch for upload onto YouTube over the years, even if most of it was many years ago as former staff of TASVideos.

A lot of this stuff should be manageable, but not a lot of people haven't tried uploading this stuff on an automated industrial scale yet.
With regards to Album Art specifically, I feel that it shouldn't be used in a way that delays the music being played back. Opening bumpers tend to cheese off the audience.

Anyway! We have a Discord. I've poked you over Discord. You get the picture!
  View user's profile Send private message Visit poster's website
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 12477
  • Location: London
Reply with quote
Post Posted: Fri Oct 12, 2018 6:02 am
Well, I spent some time messing with the source to one version of SidWiz and got it running without user interaction, and able to customize the video background. It lacks some functionality I'd like so I'd like to try OVGen next, but that is currently in transition so I can't play with the source yet. I may play with the other parts next instead.
  View user's profile Send private message Visit poster's website
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 12477
  • Location: London
Reply with quote
Post Posted: Sun Oct 14, 2018 10:23 am
So I've hacked on it a bit more, I have made SidWiz drive FFMPEG to mix the audio tracks into the video file and I've added the album art in the background and currently I'm rendering to square videos as a result. I will also add some text with the tag contents and some site branding/link.

It's hard to avoid spending time to speed it up, too, as ultimately it could be so much faster by doing the work on the graphics card, but I don't really want to go into that rabbit hole. Encoding on the fly, it's a sub realtime but that's with only 10% of the time spent on rendering, so optimising it won't make much difference. When OVGen resurfaces I still want to try it, but I may well end up forking off from SidWiz and then pulling in the useful parts from OVGen.

I'm sure I'll need to experiment a bit for FM soundtracks, and multidumper is indeed producing the funny shaped waves from blip_buffer, but again I don't want to go down that rabbit hole if I can avoid it. The perfect being the enemy of the good, and all that.

I'll post some test videos some time soon, after I figure out how to use youtube-upload to do the rest.
  View user's profile Send private message Visit poster's website
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 12477
  • Location: London
Reply with quote
Post Posted: Mon Oct 15, 2018 12:16 am
Here's a few test videos:

https://youtu.be/jnZmzUoXsDI
https://youtu.be/InVLqJhTbPs
  View user's profile Send private message Visit poster's website
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 12477
  • Location: London
Reply with quote
Post Posted: Mon Oct 15, 2018 11:00 pm
A little more:

https://youtu.be/e3lbnCKnyBs

This was to test my automatic dropping of unused channels and also to go beyond the 9 channels supported in SidWiz as it was when I started (there are other versions without limits too). I'm close to being able to render and upload a whole pack with one command...
  View user's profile Send private message Visit poster's website
  • Joined: 05 Sep 2013
  • Posts: 2296
Reply with quote
Post Posted: Tue Oct 16, 2018 8:31 am
the last one has a very low volume (or is it a problem on my side?)
  View user's profile Send private message Visit poster's website
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 12477
  • Location: London
Reply with quote
Post Posted: Tue Oct 16, 2018 6:26 pm
And a playlist - which is the real goal, to have a way to listen to a VGM soundtrack on YouTube.

https://www.youtube.com/playlist?list=PLP0Z73pc4fdGOmgPLw-PlGnUohBQvKKcv

Amongst other changes, I've enabled antialiasing on the waveforms, and switched to Arial from Modern TwoSxtn ITC Std Light, which turns out to be pretty ugly when there's a lot of text and also hard to read when placed over an image. I'm facing some issues with the oscilloscope triggering algorithm, which falls down a lot with FM, and I still need to fill out the video titles and descriptions a bit better (populating from the GD3 tag and linking back to us, of course).

The BIOS one is indeed too quiet - it's because it's quite an old VGM, which means it doesn't disambiguate which FM chips it uses, which means the mixed result is extra quiet because it has a large number of silent channels mixed in. I implemented a check for this in the oscilloscope renderer, but I need to do the same for the audio itself. The correct thing to do is to measure the volume over a whole soundtrack, so songs stay the same volume relative to each other. The oscilloscope renderer is currently normalising the volume within one song.

YouTube has a feature where it normalises audio volume between videos - but as I discovered today, that only means reducing volume, never increasing it. So I need to fix it myself. I went through a related issue with the Winamp plugin some time ago, using Replay Gain to normalise volume between soundtracks - some games just are much more quiet, but you need a perceptual model to normalise them to.

Some links for me to pick up later:

https://github.com/karamanolev/NReplayGain for volume analysis
https://github.com/naudio/NAudio for high pass filtering, mixing and amplification
  View user's profile Send private message Visit poster's website
  • Joined: 16 May 2002
  • Posts: 1116
  • Location: italy
Reply with quote
Post Posted: Tue Oct 16, 2018 10:45 pm
Maxim wrote
The BIOS one is indeed too quiet - it's because it's quite an old VGM, which means it doesn't disambiguate which FM chips it uses, which means the mixed result is extra quiet because it has a large number of silent channels mixed in.
I addressed this issue some time ago, feel free to put it in a batch file or something.
  View user's profile Send private message Visit poster's website
  • Joined: 14 Sep 2018
  • Posts: 26
  • Location: Earth
Reply with quote
Post Posted: Wed Oct 17, 2018 12:24 am
I especially like how the sound channels are displayed, gives one ideas on how the music was composed plus its fun to listen to, GG Ristar had some nice tunes. Do you plan to do every game in the future? I ask because I'm thinking about Psychic World's music, would love to see a visual representation of the ice level theme sometime.. ^_^
  View user's profile Send private message
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 12477
  • Location: London
Reply with quote
Post Posted: Wed Oct 17, 2018 7:13 am
Yes, the plan is to generate this for all packs. Some will be more interesting to look at than others, but the goal is to make it easy to listen to a pack by playing a YouTube playlist, even if that is ludicrously higher data rates than the VGM pack itself.

Tom, I will give your tool a try and maybe fix up the packs on the site where this is an issue. I'm already processing the audio to remove unused channels, which means the unused chips are dropped, but it will save a little time to avoid them entirely.
  View user's profile Send private message Visit poster's website
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 12477
  • Location: London
Reply with quote
Post Posted: Wed Oct 17, 2018 11:25 pm
Well, I'm getting pulled into this now :) Source is at https://github.com/maxim-zhao/SidWiz
  View user's profile Send private message Visit poster's website
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 12477
  • Location: London
Reply with quote
Post Posted: Wed Oct 24, 2018 10:33 pm
Some more tests...

https://youtu.be/voh_E-oNNZI - Space Harrier BIOS looking and sounding much better. Edit: except I got the percussion instrument names backwards!
https://youtu.be/bPHY57vbTTg - Streets Of Rage, flashes rather a lot. I noticed I labelled the channels from 0...
https://youtu.be/AIspL0AmR1s - The Flash, arpeggiation is destined to look bad I guess :) and I hadn't noticed that it only used two tone channels.
https://youtu.be/Pqm0qWDN_-g - Ys (FM) - fixed the instrument names now
  View user's profile Send private message Visit poster's website
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 12477
  • Location: London
Reply with quote
Post Posted: Fri Oct 26, 2018 11:52 am
And now with proper automated uploads and description generation!

https://www.youtube.com/playlist?list=PLP0Z73pc4fdGaTfDnQ6rbmRznSexYG2uQ - Psychic World
https://www.youtube.com/playlist?list=PLP0Z73pc4fdGZGUUGQVWWgmYVXYMEPP9z - Streets of Rage (GG)
https://www.youtube.com/playlist?list=PLP0Z73pc4fdEzqaHIRtZ_N0YnzsebPnnh - Ys (PSG)
  View user's profile Send private message Visit poster's website
  • Joined: 16 Aug 2005
  • Posts: 79
  • Location: Brazil
Reply with quote
Post Posted: Fri Oct 26, 2018 3:31 pm
Awesome job; looking forward to see some automated Jeroen Tel uploads...
  View user's profile Send private message
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 12477
  • Location: London
Reply with quote
Post Posted: Fri Oct 26, 2018 4:09 pm
I added a few more, although I intend to make a new account for the site for this stuff. I also found out that YouTube has a limit of 50 videos per day per person - so I guess it's going to take me a while to work through the VGM packs.

https://www.youtube.com/playlist?list=PLP0Z73pc4fdFmM8PHJqyLnYLnEPvUQYDQ - The Flash
https://www.youtube.com/playlist?list=PLP0Z73pc4fdEzqaHIRtZ_N0YnzsebPnnh - Ys (FM), ended up in the same playlist as the PSG version due to name collision
https://www.youtube.com/playlist?list=PLP0Z73pc4fdF2fULSwI46lJG-81a-aAWf - Alex Kidd in Miracle World, where I hit the 50 videos limit for today

Requests are welcome. I'm particularly interested in ones that are interesting to look at - frequency sweeps seem most satisfying - and ones which are tricky/unusual. Also any suggestions on the formatting - colours, fonts, layout. I'm trying to make it look nice...
  View user's profile Send private message Visit poster's website
  • Joined: 16 Aug 2005
  • Posts: 79
  • Location: Brazil
Reply with quote
Post Posted: Mon Oct 29, 2018 3:43 pm
Thanks for The Flash; great pack.

Maxim wrote
Requests are welcome. I'm particularly interested in ones that are interesting to look at - frequency sweeps seem most satisfying - and ones which are tricky/unusual.

Maybe we can have some Double Dragon and Road Rash next...? Can't tell beforehand if they are interesting to look at, but they sure are interesting to hear.
  View user's profile Send private message
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 12477
  • Location: London
Reply with quote
Post Posted: Mon Oct 29, 2018 4:13 pm
I'd like to get some periodic noise in, so Road Rash should be good. I think if there are no more suggestions for the styling, I'll leave it as it is and start making "official" packs on a new channel.

One last thing - I am making square videos because I have square album art, and I don't like the options for extending it to 16:9. I then take away a space near the bottom for the text description. This looks OK on a portrait phone screen, but not so great on a computer or TV. I wonder if there's a better way we could lay it out that would look good at 16:9?
  View user's profile Send private message Visit poster's website
  • Joined: 16 Aug 2005
  • Posts: 79
  • Location: Brazil
Reply with quote
Post Posted: Mon Oct 29, 2018 5:35 pm
I suppose stretching it wouldn't yield the best results, indeed. Maybe multiple images? Two side by side, 5x3 assemblage? That's extra work, though, and may not always look nice... phone users would possibly be displeased, as well.
  View user's profile Send private message
  • Joined: 15 Sep 2009
  • Posts: 373
Reply with quote
Post Posted: Tue Oct 30, 2018 7:13 am
There is a tiny suggestion I have:
Could you reduce the volume of the OPLL drums in the visualization by half?

I know that's not what the sound chip outputs*, but I think it would improve the look of the melody channels' waveform. It would also make the melody channels' amplitude more similar to songs that don't use drums. (Compare Ys: Final Battle with Ys: Rest In Peace)

*With the same TL setting, the drum channels are always twice as loud as the melody channels on OPL chips.
  View user's profile Send private message
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 12477
  • Location: London
Reply with quote
Post Posted: Tue Oct 30, 2018 8:52 am
I deliberately made it so the relative volumes between tracks are maintained, because I didn't think it makes sense to misrepresent. Some percussion is still hard to see - hi-hat doesn't get as high amplitude as bass drum, for example.

I have been normalising the FM to 200% to compensate for this (but I missed that on the Ys tracks), but I would need to special case tracks with no percussion... My goal is make it all as automated as possible, so I invoke one command to go from a folder of VGMs to YouTube videos.

Audio levels are an issue for both visualisation and playback - I'm applying Replay Gain +3dB on the mixed audio in order to make it "YouTube loud", but on a single song basis so the relative loudness of songs is not maintained - but this perceptual loudness seems to give a good result anyway. For the visualisation I'm normalising to the peak amplitude of all channels, so the relative amplitude is maintained but again it's not maintained between songs.
  View user's profile Send private message Visit poster's website
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 12477
  • Location: London
Reply with quote
Post Posted: Thu Nov 01, 2018 4:35 pm
I've started uploading packs to a dedicated channel at https://www.youtube.com/channel/UCCsvqzh7JjNNheYTplGvhCQ/playlists . The 50 uploads per day limit means it's going to take a while...
  View user's profile Send private message Visit poster's website
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 12477
  • Location: London
Reply with quote
Post Posted: Fri Nov 02, 2018 3:22 pm
...and now I've hit a limit on the number of playlist I can create per day... :) Uploading without playlists now...
  View user's profile Send private message Visit poster's website
  • Joined: 05 Nov 2018
  • Posts: 1
  • Location: California, U.S.A.
Reply with quote
Post Posted: Mon Nov 05, 2018 4:58 am
SgtPepperArc360 forked the OVGen source before the transition to GitLab began. The GitHub repo is actually privatized, not deleted (which I thought it was at first).
  View user's profile Send private message
Reply to topic



Back to the top of this page

Back to SMS Power!