Forums

Sega Master System / Mark III / Game Gear
SG-1000 / SC-3000 / SF-7000 / OMV
Home - Forums - Games - Scans - Maps - Cheats - Credits
Music - Videos - Development - Hacks - Translations - Homebrew

View topic - VGM converter. Try to squeeze 800KB in 16KB.

Reply to topic
Author Message
  • Joined: 06 Apr 2011
  • Posts: 250
  • Location: Netherlands
Reply with quote
VGM converter. Try to squeeze 800KB in 16KB.
Post Posted: Sat Aug 31, 2019 11:07 am
Attached is a VGM I created for SN76489 and YM2413.
It's quite large and is not optimized.

My idea was to create a converter that would reduce the size by optimizing, reorganizing and compressing (lz4?) the data (Probably a 800KB to 16KB reduction will be a bit too optimistic).


Optimizing:
Remove all unnecessary data, waits to frames.

Reorganizing:
Make separate streams for SN and FM channels. Remove specific channel (SN) or register (FM) references to enable more redundant data (see compression).

Compressing:
Using LZ4 (or something alike) to find redundant data blocks.

I was inspired by this project: https://github.com/simondotm/vgm-packer
music0006.vgm (833.72 KB)

  View user's profile Send private message
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 14686
  • Location: London
Reply with quote
Post Posted: Sat Aug 31, 2019 2:01 pm
VGMTool should do a fair job of optimising the data itself.

Most music has a lot of repetition per channel. The PSG format uses a kind of LZ compression - but one where the lookback is in the compressed data stream, not the decompressed, to avoid needing to hold that in memory. I think a version which did that on a per channel basis (or even splitting volumes from notes for PSG), with some bitpacking too, could optimise a lot better due to removing the interleaving of the channels.

Real game music is effectively tracked - notes with durations, instruments and effects applied, so the raw data can be much smaller, typically a byte per note event. This takes advantage of the fact that you may only really have a few notes actually used, so you don't really need 10 bits of data per note to specify that. However encoding something like vibrato on VGM is quite data intensive. Add in a volume envelope which can also add 10+ "events" for a single note. Reversing that from the VGM data for better encoding is quite hard.

Applying generic compression relies on having plenty of memory to decompress into - which you don't really have on SMS.
  View user's profile Send private message Visit poster's website
  • Joined: 25 Dec 2005
  • Posts: 607
  • Location: São Paulo - Brazil
Reply with quote
Post Posted: Sat Aug 31, 2019 11:10 pm
It works fine on vgmplay! You should add GD3 to it.
  View user's profile Send private message
  • Joined: 25 Dec 2005
  • Posts: 607
  • Location: São Paulo - Brazil
Reply with quote
Post Posted: Sat Aug 31, 2019 11:11 pm
Just a minor point: it gets mute near the end, and has no loop.
  View user's profile Send private message
  • Joined: 06 Apr 2011
  • Posts: 250
  • Location: Netherlands
Reply with quote
Post Posted: Sun Sep 01, 2019 9:28 am
Maxim, Thanks for redirecting to VGMTool.

Now the result is a 62KB VGM file, which is quite a good result!
This makes me optimistic. For a re-player on the SMS we can even further optimize the size by combining the command bytes with the FM register and PSG channel data.

Edit:
Quick calculation on the VGM file with combining commands + registers/channel saves more than 17KB. So we are down to 45KB.


The other route using the tracker specific re-player is also an option and probably I will, as I already have a converter and re-player available (but using AY8910 and SCC).
  View user's profile Send private message
  • Joined: 06 Apr 2011
  • Posts: 250
  • Location: Netherlands
Reply with quote
Post Posted: Fri Sep 06, 2019 9:03 am
Current state:

VGM converted = 42KB
Tracker created data for re-player = 7KB

The VGM compression of repeated blocks is not that trivial as hoped.
  View user's profile Send private message
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 14686
  • Location: London
Reply with quote
Post Posted: Fri Sep 06, 2019 9:08 am
Certainly a tracker replayer will always win on size (but not CPU), but the advantage of a VGM player is it can take any source file.
  View user's profile Send private message Visit poster's website
  • Joined: 05 Sep 2013
  • Posts: 3759
  • Location: Stockholm, Sweden
Reply with quote
Post Posted: Fri Sep 06, 2019 2:29 pm
did you try your VGM on Calindro's PSGTool to see how it compresses?
  View user's profile Send private message Visit poster's website
  • Joined: 06 Apr 2011
  • Posts: 250
  • Location: Netherlands
Reply with quote
Post Posted: Sun Sep 08, 2019 11:11 am
sverx wrote
did you try your VGM on Calindro's PSGTool to see how it compresses?


Isn't that only for PSG vgm?
Anyway the PSGTool gives me this error:

Exception in thread "main" java.lang.ArithmeticException: / by zero
        at vgm.VGMInterpreter.run(Unknown Source)
  View user's profile Send private message
  • Joined: 05 Sep 2013
  • Posts: 3759
  • Location: Stockholm, Sweden
Reply with quote
Post Posted: Mon Sep 09, 2019 9:14 am
Zipper wrote
Isn't that only for PSG vgm?


Yes, it is. Sorry, once more I didn't read well enough :|
  View user's profile Send private message Visit poster's website
Reply to topic



Back to the top of this page

Back to SMS Power!