Forums

Sega Master System / Mark III / Game Gear
SG-1000 / SC-3000 / SF-7000 / OMV
Home - Forums - Games - Scans - Maps - Cheats - Credits
Music - Videos - Development - Hacks - Translations - Homebrew

View topic - New PSG tool retcon-audio (somewhat improved compression)

Reply to topic
Author Message
  • Joined: 06 Mar 2022
  • Posts: 596
  • Location: London, UK
Reply with quote
New PSG tool retcon-audio (somewhat improved compression)
Post Posted: Mon Mar 20, 2023 11:01 pm
Last edited by willbritton on Tue Mar 21, 2023 8:36 am; edited 1 time in total
Hi all,

Only 7 days to go until the 2023 coding challenge and if you're struggling to fit your assets into your target ROM size, I might have a little something to help you out :)

So long story short: I was looking for utilities to handle PSGs that I could build onto the Docker toolchain I'm working on, I didn't want to put either the JVM or Mono on the image, and reached out to sverx for some guidance, fully intending to use his psgcomp. We got to chatting, one thing led to another and I ended up attempting to write my own compression algorithm to rival the spicy compression ratios that Calindro somehow managed to achieve with psgtool.

It's been a bit of a labour of love (and when I really should be working on other things!) but I'd like to share it with the community in case it's of use.

Installing

It's written in Go and is by default statically linked with no runtime dependencies, and also cross-compiles very easily so you can almost certainly download directly to run on your favourite environment from the automatically produced release binaries.

Or build from source if you like (under MIT license i.e. permissive).

Usage

A few options for usage, but you can use as a more-or-less drop in replacement for psgcomp if you want to, i.e.


retcon-audio psg <infile> <outfile>



Headline benchmarks

It has the best compression ratio compared with psgcomp and psgtool in 7 out of the 11 test files I benchmarked with, with psgtool coming out top in the other 4.
It comes out worst in 1 of the 11 benchmarks, but not by much.

It compresses quicker than psgcomp in 8 out of 11 test files. I can't really compare it scientifically to psgtool since that also converts from VGM, but from casual observation it's somewhat slower than psgtool although relatively close for most of the files.

For one of the test files (Alex Kidd track 4) it's REALLY slow and I haven't figured out why. I'm getting some interesting results from the time utility - the real time is 12 seconds but the user and sys times are both small, so it might be some inefficiency in the way I'm handling the files or something.

Full benchmark results in this google doc.

Other notes

It doesn't convert from VGM so it needs to be used with something like vgm2psg. No immediate plans or need to make it do this functionality right now but maybe one day.

It also includes a fairly rudimentary debugging facility which kind of "plays" a PSG and prints what it's doing to the screen. It was useful while developing the compression algorithm and for comparing outputs between the three tools. There are a few options to control the debug output.

It reads both compressed and decompressed PSGs at the input and you can use it to decompress by simply running it with a compressed input and the --no-compress flag to force uncompressed output.

I've got some more left field ideas for even better compression algorithms but that will have to wait for some other day...

Thanks

Special thanks to sverx for his support, encouragement and inspiration.
Thanks also to Calindro for setting such a high bar with psgtool's compression in the first place, and without which I would have saved many hours of my life chasing down a few bytes xD
  View user's profile Send private message Visit poster's website
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 14687
  • Location: London
Reply with quote
Post Posted: Tue Mar 21, 2023 7:25 am
Hmm, I guess this may be useful for a certain FM project I’m working on recently (not in time for the competitions!) with some modifications.
  View user's profile Send private message Visit poster's website
  • Joined: 05 Sep 2013
  • Posts: 3761
  • Location: Stockholm, Sweden
Reply with quote
Post Posted: Tue Mar 21, 2023 8:33 am
Link to binary releases isn't working. I guess it's missing the final 's'.

Also, now I've got to release the fixed psgcomp too. I don't like at all that I'm losing the competition so badly ;)

Thanks for creating this very useful tool! Maxim will likely request a tweaked version later :rolleyes:
  View user's profile Send private message Visit poster's website
  • Joined: 06 Mar 2022
  • Posts: 596
  • Location: London, UK
Reply with quote
Post Posted: Tue Mar 21, 2023 8:41 am
sverx wrote
Link to binary releases isn't working. I guess it's missing the final 's'.

Oh weird, thanks for spotting, now corrected.

sverx wrote
Also, now I've got to release the fixed psgcomp too. I don't like at all that I'm losing the competition so badly ;)

I've come to the conclusion that everyone loses in the end when it comes to attempting to optimise PSG compression xD

Maxim wrote
Hmm, I guess this may be useful for a certain FM project I’m working on recently (not in time for the competitions!) with some modifications.

That wouldn't be converting directly from .fur would it? I'm sure I caught some chat about that on Discord. If so, I'm extremely interested in that...
  View user's profile Send private message Visit poster's website
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 14687
  • Location: London
Reply with quote
Post Posted: Tue Mar 21, 2023 10:50 am
No, converting from .vgm...
  View user's profile Send private message Visit poster's website
  • Joined: 06 Mar 2022
  • Posts: 596
  • Location: London, UK
Reply with quote
Post Posted: Tue Mar 21, 2023 10:33 pm
UPDATE: Calindro just dropped a new version of PSGTool.

Benchmarks updated, see attached screenshots.

Very close now!! Sorry sverx, you just got nudged out further :'(
before.png (117.97 KB)
before.png
after.png (117.38 KB)
after.png

  View user's profile Send private message Visit poster's website
  • Joined: 05 Sep 2013
  • Posts: 3761
  • Location: Stockholm, Sweden
Reply with quote
Post Posted: Wed Mar 22, 2023 12:58 pm
willbritton wrote
Sorry sverx, you just got nudged out further :'(


LOL! The funny part is that is not even my compressor's fault, it's the converter that needed a fix (you guys fixed that part already!).

Will post the fixed converter ASAP - in the while I hope everybody is using either Calindro's or willbritton's wonderful tools, they're much better than my own!
  View user's profile Send private message Visit poster's website
  • Joined: 05 Sep 2013
  • Posts: 3761
  • Location: Stockholm, Sweden
Reply with quote
Post Posted: Thu Mar 23, 2023 3:40 pm
OK I figured out how to use zlib in crosscompiling for Windows on Linux (I'm terrible with these things!) so I just committed the 'fixed' converter, which means probably my compressor will be again at the other guys' level too (or at least I hope so!)

edit: btw it's funny how one would expect the same VGM file to convert into the same compressed PSG and instead you get differences. It's probably because of the complexity of finding 'the sure best' compressed data...
  View user's profile Send private message Visit poster's website
Reply to topic



Back to the top of this page

Back to SMS Power!