Forums

Sega Master System / Mark III / Game Gear
SG-1000 / SC-3000 / SF-7000 / OMV
Home - Forums - Games - Scans - Maps - Cheats - Credits
Music - Videos - Development - Hacks - Translations - Homebrew

View topic - BMP2Tile new compressors (was: ZX7 support in BMP2Tile)

Reply to topic Goto page 1, 2  Next
Author Message
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 13291
  • Location: London
Reply with quote
BMP2Tile new compressors (was: ZX7 support in BMP2Tile)
Post Posted: Wed Jan 31, 2018 11:50 pm
Last edited by Maxim on Sun Feb 04, 2018 12:38 am; edited 2 times in total
I've finished up some work on the ZX7 compressor for BMP2Tile, using the latest compressor code with some customisations. The result is attached.

ZX7 generally compresses slightly less well than aPLib - but very close - and better than almost every other compressor in BMP2Tile. Its decompressor is, however, much smaller than aPLIb's - 117 vs 334 bytes - and faster - finishing 25% faster in my tests - and requires no RAM (apart from a little bit of stack).

To achieve this performance I made a slight modification to the compression parameters, limiting it to runs of 255 bytes. This means it is not 100% compatible with data compressed using the standard ZX7 compressor, if the source data has a great deal of redundancy (e.g. four sprites in a row repeated unchanged), but the compressed data is compatible with the standard decompressors. I could produce a "conformant" variant but so far I didn't find any source data requiring it.

The ZX7 licence terms require you to acknowledge you are using it in your end product. It also requires me to place my compressor addin under the BSD-3 licence which I will be doing at some point. Source is at https://github.com/maxim-zhao/bmp2tilecompressors .
gfxcomp_zx7.zip (7.46 KB)

  View user's profile Send private message Visit poster's website
  • Joined: 28 Jan 2017
  • Posts: 377
  • Location: Málaga, Spain
Reply with quote
Post Posted: Thu Feb 01, 2018 7:01 am
Great! This is my compression of choice (both zx7 and bm2tile) and its great....

But as i use z88dk think it could be nice if someone sends a sample Code with routines for sdcc.
  View user's profile Send private message
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 13291
  • Location: London
Reply with quote
Post Posted: Thu Feb 01, 2018 7:15 am
It is compatible with the z88dk decompressors, they are just a bit slower.
  View user's profile Send private message Visit poster's website
  • Joined: 05 Sep 2013
  • Posts: 2709
Reply with quote
Post Posted: Thu Feb 01, 2018 9:14 am
If there's real request for this I will support it in devkitSMS - but I need to fix the decompressor as it's not interrupt or VRAM safe the way it is currently - and that would make it slower.
  View user's profile Send private message Visit poster's website
  • Joined: 28 Jan 2017
  • Posts: 377
  • Location: Málaga, Spain
Reply with quote
Post Posted: Thu Feb 01, 2018 1:21 pm
Sure? It is massively used in silvervalley without a trouble at all. And It seems fast enough to everything i need.

I also tryed aplib and worked Well too. I Suppose It is simple a taste or Code size question....
  View user's profile Send private message
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 13291
  • Location: London
Reply with quote
Post Posted: Thu Feb 01, 2018 1:23 pm
What's the motivation for interrupt safety? I can only think of wanting to do the work while the screen is on and music is playing, which to be honest is a bit silly. The z88dk decompressors do this at some cost to speed.
  View user's profile Send private message Visit poster's website
  • Joined: 06 Apr 2011
  • Posts: 241
  • Location: Netherlands
Reply with quote
Post Posted: Thu Feb 01, 2018 1:43 pm
Maxim wrote
What's the motivation for interrupt safety? I can only think of wanting to do the work while the screen is on and music is playing, which to be honest is a bit silly. The z88dk decompressors do this at some cost to speed.


That's easy to work around. Just have a "no_vpd_update" flag check in your interrupt handler which is set when decompressing to the screen. I keep music always on when decompressing the next tileset or level.
  View user's profile Send private message
  • Joined: 28 Jan 2017
  • Posts: 377
  • Location: Málaga, Spain
Reply with quote
Post Posted: Thu Feb 01, 2018 1:53 pm
Last edited by eruiz00 on Thu Feb 01, 2018 1:55 pm; edited 1 time in total
In Silver valley the music does not stop while zx7 is working without notice at all
  View user's profile Send private message
  • Joined: 05 Sep 2013
  • Posts: 2709
Reply with quote
Post Posted: Thu Feb 01, 2018 1:54 pm
@eruiz00: I'm talking about Maxim's attached code, I don't know about z88dk code - that may be interrupt safe indeed.

@Maxim: in devkitSMS, under normal conditions, vBlank IRQ is enabled even when screen is off, thus it could happen between the first and the second write to the VDP address port. SMSlib code aims to complete interrupt safety and to VRAM safety (except for UNSAFE operations - OK, I admit aPlib decompressor still isn't strictly part of SMSlib)
  View user's profile Send private message Visit poster's website
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 13291
  • Location: London
Reply with quote
Post Posted: Fri Feb 02, 2018 8:56 am
OK, so to be safe we should always set the VRAM address/mode before writes and wrap the whole lot in a di/ei. This could be hidden behind a flag to make it optional.
  View user's profile Send private message Visit poster's website
  • Joined: 05 Sep 2013
  • Posts: 2709
Reply with quote
Post Posted: Fri Feb 02, 2018 9:11 am
just wrapping the writes into DI/EI pair would be sufficient, as reading from VDP status port (to acknowledge the interrupt) won't change the VDP access mode.
I had no problems so far with PSGaiden, where I write address just once, in the beginning of the whole stream...
  View user's profile Send private message Visit poster's website
  • Joined: 05 Sep 2013
  • Posts: 2709
Reply with quote
Post Posted: Fri Feb 02, 2018 10:52 am
uhmmm... I just tried the DLL you attached with my BMP2Tile (ver. 0.43) and it crashes right when I hit 'save', if I select zx7.
I'm on Windows 7 32bit - by chance did you compile the DLL as 64bit?
  View user's profile Send private message Visit poster's website
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 13291
  • Location: London
Reply with quote
Post Posted: Fri Feb 02, 2018 10:58 am
A lot of people might have the palette written on every VBlank, for example, which would cause trouble.

I'm moving on to other codecs, Exomizer is done, it uses a lot of RAM, beats aPLib on compression but loses on speed. I'm going to try LZ4 next.

The DLL is 32 bit, as it needs to match BMP2Tile. It is compiled using latest Visual Studio so maybe it's a runtime issue, are you on an up to date Windows?

Edit: it wouldn't get that far if it was unable to run the code, so maybe it's the graphics you're giving it triggering a bug. Can you send it to me?
  View user's profile Send private message Visit poster's website
  • Joined: 05 Sep 2013
  • Posts: 2709
Reply with quote
Post Posted: Fri Feb 02, 2018 12:13 pm
with default devkitSMS you can't run your own code for vertical ISR, thus you can't change palettes *while* you're loading data to VRAM, but you would have to wait for vBlank and change palettes, which is the same but won't happen *while* you're pushing data

as for your DLL, it fails pretty much with all my assets but just a few (I didn't initially find anyone working, I had to test on several to finally find the first working!)

for instance, it crashes while saving the tiles of this attached image (with duplicates removing and tile mirroring ON) - I'm on a fully patched Win 7 32bit box.
controllers.png (2.83 KB)
controllers test background image
controllers.png

  View user's profile Send private message Visit poster's website
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 13291
  • Location: London
Reply with quote
Post Posted: Fri Feb 02, 2018 11:20 pm
Well, aren't I silly. I had done a last pass over the code, which included fixing an intentional memory leak in the original, and I managed to add a double-free crash instead. And I accidentally kept using a previous build so I didn't notice.

The first post is now updated and I tested it!

Here's your controller image's size with various compressors...

Raw: 9792
PS: 7608
LZ4: 5414
Sonic: 4365
PS Gaiden: 3554
ZX7: 3263
aPLib: 3100
Exomizer v2: 2908

This is fairly representative of where the compressors sit. The decompression time starts to become the biggest factor to choose between them, I think - and ZX7 is about the speed of PS Gaiden, but with better compression and less memory use.
  View user's profile Send private message Visit poster's website
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 13291
  • Location: London
Reply with quote
Post Posted: Sun Feb 04, 2018 12:34 am
Last edited by Maxim on Thu Jul 19, 2018 9:04 am; edited 3 times in total
Well, the CPC guys seemed to like LZ4 but it's not very good compression (beating only plain RLE in my test) and not that much faster than ZX7. Maybe if I hack in an 8-bit run limit, it'd help with the speed - but not the compression.
gfxcomp_lz4.zip (11.49 KB)

  View user's profile Send private message Visit poster's website
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 13291
  • Location: London
Reply with quote
Post Posted: Sun Feb 04, 2018 12:37 am
And here's Exomizer. It's slow and uses a big RAM buffer, but seems to win on compression. Use it only if you really need the extra space, and don't mind spending ~2x longer compared to ZX7 or aPLib.

  View user's profile Send private message Visit poster's website
  • Joined: 28 Jan 2017
  • Posts: 377
  • Location: Málaga, Spain
Reply with quote
Post Posted: Sun Feb 04, 2018 1:14 am
Hi Maxim,

Wonderful work this you have doing about compression for graphics and binary content...

Although size space is not a trouble today, i found It could be more interesting a better compression and (realtime) decompression for psg music' as Its content is by far bigger than all other content types (at least on the average type of Game we often do)

... Congratulations :)
  View user's profile Send private message
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 13291
  • Location: London
Reply with quote
Post Posted: Sun Feb 04, 2018 8:20 am
PSG data is large because it's being stored raw, rather than driving a music engine. If you can transition to a real engine, it will help massively - most commercial games put the engine and all music data in 16KB with no compression.

The hard part with PSG compression is that there's no way to read back old data, which is the main way compression works.
  View user's profile Send private message Visit poster's website
  • Joined: 28 Jan 2017
  • Posts: 377
  • Location: Málaga, Spain
Reply with quote
Post Posted: Sun Feb 04, 2018 9:14 am
Sorry Maxim (dont want to talk in this thread about sound) .... Then.... What is doing the vgm to psg Converter in kidded when you open a psg from vgm2psg and resave that psg? It does a file size shrinking better than psgcomp, is It not this compression?
  View user's profile Send private message
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 13291
  • Location: London
Reply with quote
Post Posted: Sun Feb 04, 2018 12:23 pm
It is a sort of dictionary compression, where it can save space when some uncompressed data sequence is identical to something seen earlier in the compressed stream. Normally compression is able to refer to data seen earlier in the uncompressed stream, which is more likely to match and therefore compress well.

I'll split to a new thread if I do anything in this area :) but as I said, music engines are the way to go. They are just harder to compose for. I can't tell if there are suitable engines for VGM Music Maker (abandoned?) and Deflemask, which seem to be what people are mainly using. I reverse engineered one simple engine (from a Codemasters game) but I didn't find an easy way to write for it - it's very close to classic MOD format, though.
  View user's profile Send private message Visit poster's website
  • Joined: 05 Sep 2013
  • Posts: 2709
Reply with quote
Post Posted: Mon Feb 05, 2018 9:18 am
Maxim wrote
The first post is now updated and I tested it!


Thanks, it works perfectly now!
  View user's profile Send private message Visit poster's website
  • Joined: 05 Sep 2013
  • Posts: 2709
Reply with quote
Post Posted: Mon Feb 05, 2018 9:30 am
@eruiz: PSG 'format' comes from VGM, is it thus a logging of all the data sent to the PSG chip, not a music file. Since we haven't got enough RAM, we can't LZ-compress a PSG file and decompress it to RAM, so I had to somehow find a way to compress a file using references to the same ROM data - that's why it doesn't compress well enough, if compared to, say, a zipped PSG file.
PSGlib has indeed its downsides, but it's very light on CPU, supports SFX and you can create music on different trackers, given they can export music in VGM format. You can also use VGMs ripped from existing games, for instance as placeholder music in a beta version.
Alternatives... well, there's Mod2PSG2 - it has its tracker and its player, but its SFX support is incomplete (at least that's what I saw when I was searching for a music/SFX library and I couldn't find anything suitable for my needs, and that's where I decided to code PSGlib)
On the other hand, we could write a new library that plays modules from some tracker, or a library that plays custom modules and create converters from tracker modules to the custom module, and of course the data would be much smaller, but the library would be more complicated and would surely require more CPU power. And it would be lots of work (I know that for sure, I coded libXM7 on the NDS and it took me two years, even if it's C...).
  View user's profile Send private message Visit poster's website
  • Joined: 05 Sep 2013
  • Posts: 2709
Reply with quote
Post Posted: Mon Feb 05, 2018 12:16 pm
I've got a few more concerns about the attached asm decompressor.

First is: are we sure the following code works on real hardware?
out (c),l
out (c),h
in a,($be)

I mean - when the screen is off, is the VDP always fast enough to return the VRAM contents even with no wait between the second OUT and the IN instruction?

Second: is the unsupported opcode
sll e

safe to use? Each and every SMS/GG/MegaDrive's Z80 supports that consistently?

I'd especially like to hear Maxim's, Bock's and Calindro's opinions on these two concerns - if possible.

(excuse my triple-posting...)
  View user's profile Send private message Visit poster's website
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 13291
  • Location: London
Reply with quote
Post Posted: Mon Feb 05, 2018 1:07 pm
Setting the read address triggers the read into the buffer, I think the timing is OK provided the screen is off. The aPLib decompressor does it and has been used in homebrew for some time without any issues that I know of.

The Game Gear official documents specify a need for a delay but then qualify it with "this applies only to cases where the VDP displays a picture (the effective area)" and goes on to say no waits are needed when the screen is off or during VBlank.

The undocumented opcode is a consequence of the CPU design, I don't know of any Sega systems using knock off CPUs or any compatibility issues apart from the known issues with CMOS Z80s affecting two other opcodes. For those unfamiliar, the documented sla opcode shifts left and sets the least significant bit to 0; sll (sometimes called sl1) sets the LSB to 1, and differs in the opcode by a single bit. It might be interesting to find games using the undocumented opcodes (maybe by emulators monitoring them) to map out where they come into use.
  View user's profile Send private message Visit poster's website
  • Joined: 05 Sep 2013
  • Posts: 2709
Reply with quote
Post Posted: Mon Feb 05, 2018 3:10 pm
Maxim wrote
Setting the read address triggers the read into the buffer, I think the timing is OK provided the screen is off. The aPLib decompressor does it and has been used in homebrew for some time without any issues that I know of.


Dwedit's code -modified by you- that I'm using in devkitSMS's aPLib has
ei
pop af
ret

between the last OUT and the IN a,($be) instructions... you mean there's another implementation where the IN is the first instruction following the second OUT?
  View user's profile Send private message Visit poster's website
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 13291
  • Location: London
Reply with quote
Post Posted: Mon Feb 05, 2018 6:18 pm
Yes...

https://github.com/maxim-zhao/bmp2tilecompressors/tree/master/decompressors

https://github.com/z88dk/z88dk/blob/master/libsrc/_DEVELOPMENT/arch/sms/vram/z80...

The "VRAM ldir" is a bottleneck in most decompressors and thus has been fairly heavily optimised.
  View user's profile Send private message Visit poster's website
  • Joined: 05 Sep 2013
  • Posts: 2709
Reply with quote
Post Posted: Tue Feb 06, 2018 9:33 am
OK - I found out I/we already checked that last May, so it's verified we can IN the byte right after the second OUT that sets the address, IF screen isn't drawing (thus during vBlank or OFF)

I'm preparing the first ZX7 incarnation in devkitSMS... see you later :)
  View user's profile Send private message Visit poster's website
  • Joined: 05 Sep 2013
  • Posts: 2709
Reply with quote
Post Posted: Tue Feb 06, 2018 1:38 pm
I'm wrapping up the first ZX7 function:
UNSAFE_SMS_loadZX7compressedTiles (src, tilefrom)

as I see ZX7 can be used to compress tilemaps too, I'll prepare a function for that too, later.

I'm declaring UNSAFE functions only. I see that the time taken makes it unsuitable for VRAM safe loading, thus these functions are meant to load data while screen is OFF only. (for the curious ones: it takes approximately the same time [~ +1,5%] to load my test image tiles both using this ZX7 and the PSGaiden format, but the former loader requires the screen to be OFF whereas the latter requires a bit more of ROM space for the data and a 32 bytes decompression buffer in RAM - but the screen can be kept ON. Both can be useful/preferred so they'll both stay in the library.)

Now about the ZX7 licence: where can I read that?
  View user's profile Send private message Visit poster's website
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 13291
  • Location: London
Reply with quote
Post Posted: Tue Feb 06, 2018 5:40 pm
https://github.com/z88dk/z88dk/blob/master/libsrc/_DEVELOPMENT/compress/zx7/read...

Quote
LICENSE
=======

The optimal C compressor is available under the "BSD-3" license. In practice,
this is relevant only if you want to modify its source code and/or incorporate
the compressor within your own products. Otherwise, if you just execute it to
compress files, you can simply ignore these conditions.

The Z80 assembly decompressors can be used freely within your own ZX-Spectrum
programs (or any other Z80 platform), even for commercial releases. The only
condition is that you must indicate somehow in your documentation that you have
used "ZX7".


I'd like to experiment with deinterleaving the data before compression. It would necessarily slow down the decompression but it might be a big win for compression strength.

A screen on version would be feasible, the decompression time is faster than PS Gaiden in my tests. Given the limited maximum run length, I would allow an interrupt once per main loop rather than after every byte. VDP state assumptions would have to be relaxed too (set address on literals).
  View user's profile Send private message Visit poster's website
  • Joined: 05 Sep 2013
  • Posts: 2709
Reply with quote
Post Posted: Wed Feb 07, 2018 9:03 am
Maxim wrote
I'd like to experiment with deinterleaving the data before compression. It would necessarily slow down the decompression but it might be a big win for compression strength.


You mean de-interleaving the tilemap data? Would you offer TWO different ZX7 compressed formats for tilemap? :|

Maxim wrote
A screen on version would be feasible, the decompression time is faster than PS Gaiden in my tests.


I'll think about it. At the moment, I think that adding 26 cycles before each VRAM read and another 26 cycles after each VRAM write would be overkilling IMHO since we're doing a lot of backtracking in ZX7.

But of course since devkitSMS development is mainly request-driven nowadays, let's see if the users do request that feature.
  View user's profile Send private message Visit poster's website
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 13291
  • Location: London
Reply with quote
Post Posted: Wed Feb 07, 2018 9:24 am
I can easily test the benefits of deinterleaving, I'd present it as a different format (ZX7i perhaps). The tricky part is that the decompressor has to emit every fourth byte and then loop back three times, which is tricky.

The screen on version should be easy to add in behind an enable flag, so I can benchmark it.
  View user's profile Send private message Visit poster's website
  • Joined: 05 Sep 2013
  • Posts: 2709
Reply with quote
Post Posted: Wed Feb 07, 2018 9:36 am
I suspect you can't beat STM tilemap compression on the above 'controllers.png' test image :D
  View user's profile Send private message Visit poster's website
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 13291
  • Location: London
Reply with quote
Post Posted: Wed Feb 07, 2018 12:47 pm
In theory, tailoring the compression to the data type should be a big win. The question is whether some arbitrary compression algorithms can help, either in addition or instead, considering the CPU costs of both.

The PS Gaiden compression is a good example of something designed strictly for the data in question. I guess this is true for STM too.
  View user's profile Send private message Visit poster's website
  • Joined: 05 Sep 2013
  • Posts: 2709
Reply with quote
Post Posted: Wed Feb 07, 2018 4:03 pm
Yes, exactly. STM is based on RLE (regular and incremental) and on de-interleaving the bytes in the tilemap, which is good for the images that use lots of different tiles, as BMP2Tile will create and index them with an incremental number.
But this also means that won't perform better than ZX7 where the original image has repeating runs of few different tiles (RLE won't compress them) as for instance here, in the lower part of this image:
  View user's profile Send private message Visit poster's website
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 13291
  • Location: London
Reply with quote
Post Posted: Wed Feb 07, 2018 5:14 pm
Last edited by Maxim on Wed Feb 07, 2018 10:07 pm; edited 1 time in total
One other one I'd like to get to is looking at is the compressor from Micro Machines. It includes various versions of LZ but it also interestingly includes compression for reverse matches (same bytes, opposite order) and counting runs (eg 1 2 3 4 5 encoded as "count up from 1, four steps" - which I think is your "incremental RLE", which is a big win for image conversions). The problem there is that I don't have an existing compressor for it and the one I wrote is horrible :)
  View user's profile Send private message Visit poster's website
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 13291
  • Location: London
Reply with quote
Post Posted: Wed Feb 07, 2018 7:20 pm
So I'm trying the "screen on safe" thing now and I'm not sure why Emulicious is complaining. It's warning about bad VRAM access on this:

      out (c), l
      out (c), h
      push ix ; delay lots of cycles
      pop ix
      in a, (Port_VDPData)
      out (c), e
      out (c), d ; <-- error

...where c = $bf, so it's complaining that I'm setting the (write) address right after a read. This seems wrong to me, but I don't have a real system to benchmark on.

The official SMS docs say to allow 29 cycles between any in/out operations during active display, which we know is not entirely true (it should only hold for operations accessing VRAM). The GG docs say to allow 28 cycles between setting the address and reading; it's not clear if this is in addition to the cycles spent inside those out/in opcodes, as the actual operation takes place somewhere in the middle. Above I'm inserting 29 cycles with a push ix/pop ix. I don't think there should be any need for a wait before setting a new address, as it does not contend for VRAM. The GG docs are missing the page covering VRAM writes. I thought I knew this stuff... what's the real answer?
  View user's profile Send private message Visit poster's website
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 13291
  • Location: London
Reply with quote
Post Posted: Thu Feb 08, 2018 9:08 am
Benchmarking deinterleaving on the controllers test image above:

Tiles (zx7) = 3263 bytes
Tiles (zx7, after deinterleavig by 4) = 5513 bytes

Disappointing...

Tilemap (zx7) = 814 bytes
Tilemap (zx7, after deinterleaving by 2) = 567 bytes
Tilemap (STMcompr) = 466 bytes

So it is happier here (the deinterleaving gives us big runs in the high bytes section) but still not beating STM. I guess it's a winner :)

Note that I grabbed the release from GitHub which seems to be 1.0, before the rename and addition of the header byte. You might want to release an update.
  View user's profile Send private message Visit poster's website
  • Joined: 05 Sep 2013
  • Posts: 2709
Reply with quote
Post Posted: Thu Feb 08, 2018 9:54 am
1) MicroMachines compression would be a nice addition, as it would compress very well different kinds of data - but OK, we'll see if you can code a good version of it :p
In the while, I think ZX7, PSGaiden and STM can be used with good success.

2) 26 cycles after setting the address before reading the data from VRAM (and again before reading the next byte and so on) it's all the requirement AFAIK. But I guess we better ask Calindro, there might be something we're overlooking.

3) As expected ;) The new STM version (the one with the 1-byte header) has also the advantage that can be decompressed to a rectangular part of the screen directly, as the width of the image is in the header. So it may be used for nice effects too, not only for loading full-screen images (or full width, at least...).
(I'm going to release 1.1, I forgot it - thanks!)
  View user's profile Send private message Visit poster's website
  • Joined: 05 Sep 2013
  • Posts: 2709
Reply with quote
Post Posted: Thu Feb 08, 2018 2:21 pm
Maxim wrote
[...] controller image's size with various compressors...[...]
aPLib: 3100


mmm... I get the same size, but I then fail at decompressing the data to VRAM, using the old devkitSMS/aPLib code (2015) that had been tested at the time.
Did you recompile the plug-in for the new BMP2Tile 0.43? If not, can you try decompressing that file with your own code?
  View user's profile Send private message Visit poster's website
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 13291
  • Location: London
Reply with quote
Post Posted: Thu Feb 08, 2018 2:44 pm
I have been testing with my own test image, but I'll try this one.
  View user's profile Send private message Visit poster's website
  • Joined: 05 Sep 2013
  • Posts: 2709
Reply with quote
Post Posted: Thu Feb 08, 2018 3:53 pm
I tried downloading the aPlib plug-in again and re-converting the file: same result. So it's either an unnoticed bug in the compression plugin or an unnoticed bug in the decompression code, which comes from your code plus a couple DI/EI around the usual places.
So, let me know if you can decompress that fine - thanks.
  View user's profile Send private message Visit poster's website
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 13291
  • Location: London
Reply with quote
Post Posted: Thu Feb 08, 2018 4:24 pm
Probably the decompressor, the compressor is old code.
  View user's profile Send private message Visit poster's website
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 13291
  • Location: London
Reply with quote
Post Posted: Thu Feb 08, 2018 11:02 pm
Tested and works for me... There was a bug in an early version of my aPLib compressor, fixed in 2008 as seen at http://www.smspower.org/maxim/SMSSoftware/Compressors . It looks like the devkitSMS code is based on that, but there's more changes in recent versions so I suggest moving to those.
  View user's profile Send private message Visit poster's website
  • Joined: 17 Nov 2015
  • Posts: 97
  • Location: Canada
Reply with quote
Post Posted: Fri Feb 09, 2018 6:28 am
sverx wrote

I'll think about it. At the moment, I think that adding 26 cycles before each VRAM read and another 26 cycles after each VRAM write would be overkilling IMHO since we're doing a lot of backtracking in ZX7.


The safe version in z88dk is a screen-on version and is the version eruiz has been using while playing music.

https://github.com/z88dk/z88dk/blob/master/libsrc/_DEVELOPMENT/arch/sms/vram/z80...

This is also the code that was under scrutiny by Calindro & co when earlier versions were found to crash under Emulicious.

About SLL - I don't trust it so it's disabled by default in z88dk through a configuration bit:

https://github.com/z88dk/z88dk/blob/master/libsrc/_DEVELOPMENT/compress/zx7/z80/...

As Maxim noted, it's missing on cmos z80s (and later z80 derivatives) but it is known to work on nec nmos z80s and original zilog nmos z80s. There were many z80 second sources, not all of which used zilog's masks, so that's why we disable it by default.
  View user's profile Send private message Visit poster's website
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 13291
  • Location: London
Reply with quote
Post Posted: Fri Feb 09, 2018 7:03 am
That's not what I had thought... I guess we should avoid sll for Game Gear compatibility then. I think the screen on version needs to also assume the VRAM state is not stable - so it should set the write address on literals too, at some additional time cost. Screen on means slower by about 30%, but that seems acceptable.

Edit: searching again for documentation of the availability of the undocumented features, I can't find confirmation that sll e is unreliable. Master Systems (and older systems) made by Sega have genuine NMOS Z80s from Zilog or NEC (and they do apparently differ a bit), and some integrated ASIC version in CMOS for Game Gear. Compatibility with "compatible" chips (eg for Russian Spectrum clones) is less of a concern for us. Brazilian systems are maybe less reliable? I'd expect ZEXALL to show up the differences (although it doesn't test out (c), 0 for example), I'm surprised we haven't noticed.
  View user's profile Send private message Visit poster's website
  • Joined: 05 Sep 2013
  • Posts: 2709
Reply with quote
Post Posted: Fri Feb 09, 2018 8:56 am
@Maxim: ok, I'll take your most recent version, then. Where? :|

@AA: I also don't like using SLL, but I didn't know I could turn it into
sla e
inc e

because I just discovered INC opcode doesn't affect carry flag :( (and now I wonder how many bugs I've spread around...)
  View user's profile Send private message Visit poster's website
  • Joined: 05 Sep 2013
  • Posts: 2709
Reply with quote
Post Posted: Fri Feb 09, 2018 10:31 am
on a side note,
Maxim wrote
Brazilian systems are maybe less reliable?

if somebody have got some technical questions to ask TecToy, I currently have this chance (I will collect the questions, PM me!)
  View user's profile Send private message Visit poster's website
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 13291
  • Location: London
Reply with quote
Post Posted: Fri Feb 09, 2018 1:43 pm
I'll push to GitHub later today. I'm putting everything here: https://github.com/maxim-zhao/bmp2tilecompressors/tree/master/decompressors

Regarding CPU compatibility, I guess you would need to run a test program. My understanding is that the Game Gear CPU may not deal with the undocumented flags properly and therefore may fail ZEXALL...
  View user's profile Send private message Visit poster's website
  • Joined: 17 Nov 2015
  • Posts: 97
  • Location: Canada
Reply with quote
Post Posted: Fri Feb 09, 2018 3:48 pm
Maxim wrote

Edit: searching again for documentation of the availability of the undocumented features, I can't find confirmation that sll e is unreliable.


There's no real information on it. The cmos z80 claim comes from an old usenet post where someone detailed the problems they had when transferring code from an nmos z80 to a cmos z80.

For z80 derivatives, sll is definitely gone as that opcode slot was replaced with other instructions.
  View user's profile Send private message Visit poster's website
Reply to topic Goto page 1, 2  Next



Back to the top of this page

Back to SMS Power!