Forums

Sega Master System / Mark III / Game Gear
SG-1000 / SC-3000 / SF-7000 / OMV
Home - Forums - Games - Scans - Maps - Cheats - Credits
Music - Videos - Development - Hacks - Translations - Homebrew

View topic - Sdcc vs z88dk and 3 main banks

Reply to topic Goto page 1, 2  Next
Author Message
  • Joined: 28 Jan 2017
  • Posts: 233
  • Location: Málaga, Spain
Reply with quote
Sdcc vs z88dk and 3 main banks
Post Posted: Tue Aug 22, 2017 5:35 am
Hi! Have a trouble and a doubt. Could you give me an insight? Thanks!:

... so, this night, i cannot run my sdcc rom as main.c exceed the first 32kb. Does not run in emulicious. Sdcc talks about dont save sega header.


But if i compile with z88dk it runs. What it troubles me is that z88dk launch a warning about 366 bytes exceeding the bank0+bank1 space. It seems like it saves code in bank2, although the rom size remains the same (256kb)... z88dk talks about 6kb ram available, so i dont understand nothing at all. Also z88dk warns about not save the sega header.

See image, please.

Thanks!
20170822_073507.jpg (1.31 MB)
20170822_073507.jpg

  View user's profile Send private message
  • Joined: 17 Nov 2015
  • Posts: 91
  • Location: Canada
Reply with quote
Post Posted: Tue Aug 22, 2017 6:56 am
eruiz00 wrote

But if i compile with z88dk it runs. What it troubles me is that z88dk launch a warning about 366 bytes exceeding the bank0+bank1 space. It seems like it saves code in bank2, although the rom size remains the same (256kb)... z88dk talks about 6kb ram available, so i dont understand nothing at all. Also z88dk warns about not save the sega header.


Your memory space is usually:

0-32k fixed rom
32k-48k banked rom
48k-56k ram

So if your main program fits into the bottom 32k you can freely bankswitch data into the 32k-48k area.

The warning from z88dk "the main binary extends into slot 2 by 366 bytes" is saying that the main program is exceeding the 0-32k area and you're now running into the 32k-48k area that is normally used for banking.

z88dk will let you do this because maybe you just want to make a 48k rom cartridge. In this case, z88dk will put your main code into banks 0,1,2 so that it occupies the 0-48k area. However it will ignore any data you have assigned to bank 2 separately so that bank 2 will only contain your main code. The next message "Adding main banks 0,1,2 (16018 bytes free)" says that you can still add that many more bytes to the main code until bank 2 is filled up.

Because the headers (sega and sdsc) sit at the top of the 32k, if your code overruns that, the header will not be inserted so that your code is not overwritten.

The message about available ram space being 6663 bytes is talking about how much space is left in ram in the 48-56k area.

You probably don't want to have your program extend into bank 2 because that means some of the code will bank out when you put other banks into slot 2.

There is still a couple things you can try.

If you have any initialized data for ram (like "int a = 5;" where a variable is initialized to a non-zero value), a copy has to be stored as part of the main binary so that ram can be initialized at startup. z88dk can compress that to make your binary smaller. This part will come out as "foo_DATA.bin" after a compile. Try compressing that with "zx7 foo_DATA.bin". If you can save at least ~70 bytes then a compressed rom compile will shrink your main code. Add "-pragma-define:CRT_MODEL=2" to your last compile line (the one that makes the binary not the object files) and see what happens.

Another thing you can try is to place some code in ram. At the moment you have 6600 bytes free up there. You can do that by assigning it to an smc section (self-modifying code). What you're doing now is assigning data to sections BANK_03, BANK_04, etc. You can separate some of your c code into a separate file and place that into section "smc_user" in the same way you're doing these banks.

What will happen is that code will be placed in the DATA section. Then you'll be making a compressed compile with the pragma above. So this code will be stored in compressed form in the 0-32k area and it will be expanded into ram at startup.

You will lose compatibility with sdcc because sdcc cannot do these things.

The next thing you can do is start to think in dividing up your code to be run in banks just like you are dividing up your data. So the bottom 32k will be a fixed part and then you will have code that gets paged into the 32k-48k area just like your data. You may want to assign a bank to a level so that you bundle level-specific code with level-specific data (sound/graphics). Then as each level is started, your main code banks in the appropriate level's bank in 32k-48k and happily calls into that.
  View user's profile Send private message Visit poster's website
  • Joined: 05 Sep 2013
  • Posts: 1914
Reply with quote
Post Posted: Tue Aug 22, 2017 8:14 am
You should try to make your code fit into the first 32 KB - try checking if you've got some const data (LUTs for example) stored into your lower memory - if so, try moving it in a separate bank.
If this fails, we can try moving some code into a separate bank BUT that code should be carefully choosed as we can bank only code that doesn't need to bankswitch data, as doing that will make your code disappear from addressable space and your CPU will then just execute 'random' instructions :|
  View user's profile Send private message Visit poster's website
  • Joined: 28 Jan 2017
  • Posts: 233
  • Location: Málaga, Spain
Reply with quote
Post Posted: Tue Aug 22, 2017 12:33 pm
Ummmm...

Simply is that the game has reached to the same problem astroforce did. I need two boss code and some things to do, and have to fix code abuses done before. It will fit for sure, but i will have to work a bit more. This is part of the fun!!!!

On other part, the fact is i did a minigame (i wanted a minigame into the game from the first day i began width) and is a bit expensive (have an array of 375 bytes) and about 0.5kb of code and data associated. This game runs appart from main game so i think should be cappable to move it into bank14.

Si will try this night two possibilities: move the code and data into smc_user and into bank14. The possibility of have more code space is really appealing to me!!!!
  View user's profile Send private message
  • Joined: 05 Sep 2013
  • Posts: 1914
Reply with quote
Post Posted: Tue Aug 22, 2017 12:37 pm
eruiz00 wrote
The possibility of have more code space is really appealing to me!


If you place some code and the data it needs in the same bank, then it'll likely work. As soon as your banked code will try to access banked data in a different bank it'll crash - so beware! :)
  View user's profile Send private message Visit poster's website
  • Joined: 28 Jan 2017
  • Posts: 233
  • Location: Málaga, Spain
Reply with quote
Post Posted: Wed Aug 23, 2017 4:08 am
Ummm... in sdcc did not worrk...

I mean, compile without a single warning, and link too, but horrible msg saying size 264376/257564 bytes. Rom size is 256 kb and black screen.

Now trying with z88dk and smc_user (which should be nice, as i suposse i can upload functions and data up to 6kb)

...
  View user's profile Send private message
  • Joined: 28 Jan 2017
  • Posts: 233
  • Location: Málaga, Spain
Reply with quote
Post Posted: Wed Aug 23, 2017 6:43 am
Looking whats going on, cant understand why the minigame, which is a 7kb c file, which dense functions, seems to compile to less size that my boss functions.

I explain, the boss functions arent too big, but have switchs with 6-10 values. Maybe these switchs are expensive...
  View user's profile Send private message
  • Joined: 05 Sep 2013
  • Posts: 1914
Reply with quote
Post Posted: Wed Aug 23, 2017 8:09 am
eruiz00 wrote
the boss functions arent too big, but have switchs with 6-10 values. Maybe these switchs are expensive...


if those values are contiguous, the jump table shouldn't be so big, otherwise, they could become pretty big.

BTW, which parameters are you using in SDCC invocation? Can you try --opt-code-size eventually?
  View user's profile Send private message Visit poster's website
  • Joined: 28 Jan 2017
  • Posts: 233
  • Location: Málaga, Spain
Reply with quote
Post Posted: Wed Aug 23, 2017 8:46 am
I am yet with opt-code-size. Its only that i dont see correlation between code and size with large switchs.
  View user's profile Send private message
  • Joined: 05 Sep 2013
  • Posts: 1914
Reply with quote
Post Posted: Wed Aug 23, 2017 9:02 am
if the values you're using in your case statements are small and contiguous numbers (such as 0,1,2,3,4 for example) you'll get something like this:

;main.c:1708: switch (WaimanuStatus) {
   push   de
   ld   iy, #_WaimanuStatus
   ld   e, 0 (iy)
   ld   d, #0x00
   ld   hl, #00135$
   add   hl, de
   add   hl, de
;main.c:1709: case WS_NORMAL:
   pop   de
   jp   (hl)
00135$:
   jr   00101$
   jr   00106$
   jr   00116$
   jr   00108$
   jr   00115$


otherwise your jump tables will require more space (let's say (MAX-MIN+1)*2 bytes)

(edit on edit: I meant *2 bytes, obviously!)
  View user's profile Send private message Visit poster's website
  • Joined: 17 Nov 2015
  • Posts: 91
  • Location: Canada
Reply with quote
Post Posted: Fri Aug 25, 2017 1:28 am
eruiz00 wrote

Now trying with z88dk and smc_user (which should be nice, as i suposse i can upload functions and data up to 6kb)


You will save space but code doesn't always compress well so it may be less space than you're hoping for. Trying to pick code that might be best compressed is probably an art form :)

OTOH, if you have this:

Quote

try checking if you've got some const data (LUTs for example) stored into your lower memory


That is, const data in the main bank, try changing that to non-const data so that it gets placed in ram. Then the stored copy in your main bank will be compressed too.

To turn on a compressed compile you have to add "-pragma-define:CRT_MODEL=2" to your compile line (the one that makes the binary not the object files).

Adding "--opt-code-size" to the compile line making the binary will also save bytes.
  View user's profile Send private message Visit poster's website
  • Joined: 28 Jan 2017
  • Posts: 233
  • Location: Málaga, Spain
Reply with quote
Post Posted: Fri Aug 25, 2017 5:42 am
Sorry but i have been busy...

I am ripped parts of the game so i can finish it with sdcc.

I want to finish next week. After that, i will put the ripped code (the minigame) and will try the smc option, along with enemies code (npc.h).

For the luts, all the constant data is in its own dedicated banks (15 and 14) so no luck here.

But i think maybe can solve even without "tricks", as yesterday i had been optimizing things and it seems i am near of fit in.
  View user's profile Send private message
  • Joined: 05 Sep 2013
  • Posts: 1914
Reply with quote
Post Posted: Fri Aug 25, 2017 7:57 am
you can also examine the map file that the linker generates, in (the _CODE area) to see which functions are eating up space, comparing each ROM address with the next one, if you see that numbers are very far it means the function it's pretty big as in this example:
     000009F5  _readParameterLine                 main
     00000AE7  _loadLevel                         main
     00001264  _changeWaimanuStatus               main
     000012BC  _prepareBG                         main


as you can see, loadLevel() starts at 0AE7 and ends at 1264, which is quite impressive. In your case, you might just find some big functions that you didn't suspect were so big...
  View user's profile Send private message Visit poster's website
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 11824
  • Location: London
Reply with quote
Post Posted: Fri Aug 25, 2017 8:20 am
I think it would be really useful to make a tool to extract that info, it seems like a common issue with sdcc that code can bloat unexpectedly. Maybe the bytes to lines of code ratio might help too?
  View user's profile Send private message Visit poster's website
  • Joined: 05 Sep 2013
  • Posts: 1914
Reply with quote
Post Posted: Fri Aug 25, 2017 8:34 am
I actually never investigated this, but there might be SDCC options to create a map file with more info. Let me check...

edit: there seems to be none :|
  View user's profile Send private message Visit poster's website
  • Joined: 28 Jan 2017
  • Posts: 233
  • Location: Málaga, Spain
Reply with quote
Post Posted: Fri Aug 25, 2017 10:48 am
I actually use an excel table to extract function sizes. It works really well.
The final stage of the developing process is to review the bigger functions and do what it can be done.
  View user's profile Send private message
  • Joined: 28 Jan 2017
  • Posts: 233
  • Location: Málaga, Spain
Reply with quote
Post Posted: Fri Aug 25, 2017 10:49 am
In fact (call it the compiler, or the libraries it have) the difference of the code section between sdcc and z88dk in my actual game is about 1500 bytes.
  View user's profile Send private message
  • Joined: 05 Sep 2013
  • Posts: 1914
Reply with quote
Post Posted: Fri Aug 25, 2017 11:27 am
SMSlib is (still?) partly in C, whereas z88dk's implementation of it it's (mostly?) in handwritten ASM, and that could account for some of those bytes - I'm also curious to know exactly where they are, as I'm sure such info are very valuable to improve my devkit/library... so if you can share some info with us (or privately with me only, if you prefer) I'd be happy :)

For the remaining bytes, it's probably mostly the peep hole optimizer, at least I suspect this. A.A. might confirm/refute this :)
  View user's profile Send private message Visit poster's website
  • Joined: 28 Jan 2017
  • Posts: 233
  • Location: Málaga, Spain
Reply with quote
Post Posted: Fri Aug 25, 2017 12:45 pm
Tomorrow ill send you. I am out today.
  View user's profile Send private message
  • Joined: 17 Nov 2015
  • Posts: 91
  • Location: Canada
Reply with quote
Post Posted: Fri Aug 25, 2017 6:52 pm
sverx wrote
SMSlib is (still?) partly in C, whereas z88dk's implementation of it it's (mostly?) in handwritten ASM, and that could account for some of those bytes -

For the remaining bytes, it's probably mostly the peep hole optimizer, at least I suspect this. A.A. might confirm/refute this :)


I'm not sure what the size savings are in SMSlib and PSGlib. In the z88dk compile of Astroforce they come out to 737 bytes and 665 bytes respectively but that's just functions used by the game and it won't include the outi block.

In general, from just the c code, we tend to see about 5% code size reduction compared to sdcc when using ints or smaller. So 5% of a 32k program is about 1600 bytes so it's in the ballpark of what eruiz is seeing. The gains from library can only happen if the library is used (in this case probably just differences in size in SMSlib / PSGlib and the incidental stuff like integer multiply although at 16-bits and lower, sdcc's integer functions will be comparable to z88dk's).

The 5% reduction is coming from operating sdcc in --reserve-regs-iy mode and applying the peephole rules to work on that code.
  View user's profile Send private message Visit poster's website
  • Joined: 05 Sep 2013
  • Posts: 1914
Reply with quote
Post Posted: Mon Aug 28, 2017 9:03 am
Alcoholics Anonymous wrote
The 5% reduction is coming from operating sdcc in --reserve-regs-iy mode and applying the peephole rules to work on that code.


speaking of this: how is it that when I use --reserve-regs-iy the generated code is often bugged? Did you fixed that in z88dk and didn't port the fixes back to SDCC? :|
  View user's profile Send private message Visit poster's website
  • Joined: 17 Nov 2015
  • Posts: 91
  • Location: Canada
Reply with quote
Post Posted: Mon Aug 28, 2017 6:00 pm
Alcoholics Anonymous wrote

speaking of this: how is it that when I use --reserve-regs-iy the generated code is often bugged? Did you fixed that in z88dk and didn't port the fixes back to SDCC? :|


We didn't fix sdcc itself, we fixed the bugged code in post-processing (ie the peepholer):

https://github.com/z88dk/z88dk/blob/master/libsrc/_DEVELOPMENT/sdcc_peeph.2#L291

The relevant bugs are all logged into sdcc's bugtracker.
  View user's profile Send private message Visit poster's website
  • Joined: 28 Jan 2017
  • Posts: 233
  • Location: Málaga, Spain
Reply with quote
Z88DK error???
Post Posted: Sat Sep 16, 2017 1:44 pm
Ummm.... that code works in SDCC:

unsigned char numenemies=6;
enemy *enemies;

for(unsigned char a=1;a<numenemies;a++)
enemies[a].param=5;


Does not in Z88DK. In Z88DK only fills the item 0 of the array, even if a is beggining the loop from 1.

Tryed this:

unsigned char numenemies=6;
enemy *enemies;
enemy *e;

for(unsigned char a=1;a<numenemies;a++)
{
e=&enemies[a];
e->param=5;
}

Does not work. Only fills item 0.

This way works:

enemies[0].param=5;
enemies[1].param=5;
enemies[2].param=5;
enemies[3].param=5;
enemies[4].param=5;

Fails on today's z88dk nightly and on 12-8 nightly and 26-7 nightly. opt-code-size and opt-code-speed too.

Regards.
  View user's profile Send private message
  • Joined: 17 Nov 2015
  • Posts: 91
  • Location: Canada
Reply with quote
Post Posted: Sun Sep 17, 2017 1:16 am
I'm seeing correct code generated for:


typedef struct {
   unsigned char param;
} enemy;

unsigned char numenemies=6;

enemy enemies[6];  // also checked generation for pointer here
enemy *e;

void main(void)
{
   for(unsigned char a=1;a<numenemies;a++)
   {
      e=&enemies[a];
      e->param=5;
   }
   
   for(unsigned char a=1;a<numenemies;a++)
      enemies[a].param=5;
}


The loop given will only change elements 1-5 (not 0-4).

Could you provide the "enemy" struct so I can check with the right structure too?
  View user's profile Send private message Visit poster's website
  • Joined: 28 Jan 2017
  • Posts: 233
  • Location: Málaga, Spain
Reply with quote
Post Posted: Sun Sep 17, 2017 9:23 am
Sorry, the enemy struct is the same as the code i gave you some months ago:

typedef void (*MyInitEnemyFunction)( void *);
typedef unsigned char (*MyUpdateEnemyFunction)(void *);
typedef void (*MyKillEnemyFunction)(void *);
typedef void (*MyDamageEnemyFunction)(void *);

// Enemy definitions
typedef struct enemydef
{
unsigned char energy;
unsigned char damageable;
unsigned char killable;
unsigned char gravity;
unsigned char damagemovable;
unsigned char collideable;
unsigned char width;
unsigned char height;
MyInitEnemyFunction eif;
MyUpdateEnemyFunction euf;
MyKillEnemyFunction ekf;
MyDamageEnemyFunction edf;
}enemydef;

// Enemigos
typedef struct enemy
{
enemydef *ed;
signed char side;
signed char velx;
signed char vely;
unsigned int posx;
unsigned int posy;
unsigned char state;
unsigned char horizfree;
unsigned char vertfree;
unsigned char energy;
unsigned int frame;
signed char param;
signed char slot;
unsigned char base;
signed char sprite;
unsigned char killable;
unsigned char damageable;
}enemy;

The fact is im having trouble loading symbols into emulicious so, although i can see the memory addresses in .map files, y cannot (i dont know enough!) correlate the debugger data with the ansi c.

I can send the code if you need it.

The game funcs perfectly from begin to finish, excepting the loop which is at endingstage.c (the game ending). Have some similar loops in enemy.h, by example....

... but the fact, i am not sure, is that, maybe since two weeks ago, this error raised, as i splitted the compile process for separate .c files (instead one main.c and several .h files containing all the code). Some of the files are compiled with --opt-code-size and some other with --opt-code-speed. Dont know if this can produce the error...
  View user's profile Send private message
  • Joined: 17 Nov 2015
  • Posts: 91
  • Location: Canada
Reply with quote
Post Posted: Sun Sep 17, 2017 4:55 pm
With the structure in place, the code still looks correct with all combinations of opt-code-size, opt-code-speed, etc.

Quote

The fact is im having trouble loading symbols into emulicious so, although i can see the memory addresses in .map files, y cannot (i dont know enough!) correlate the debugger data with the ansi c.


I added an issue:
https://github.com/z88dk/z88dk/issues/387

If there's a way to import symbols into Emulicious maybe we can write a tool that will translate z88dk's map file to the correct format.

Quote

I can send the code if you need it.


PM me a link - the bug may only manifest in a specific situation.
  View user's profile Send private message Visit poster's website
  • Joined: 28 Jan 2017
  • Posts: 233
  • Location: Málaga, Spain
Reply with quote
Post Posted: Sun Sep 24, 2017 2:57 pm
About troubles compiling silvervalley on z88dk with -SO3....

Enemies do not appear on screen by a fault on enemyslot.c. if compile that file with SO 2 everything will func Well.

Error should be on enemyslotinsidescreen función or updateenemyslots función.

Tere are other error, but i am checking as i dont get the point where it fails. More info later.

Regards.
  View user's profile Send private message
  • Joined: 28 Jan 2017
  • Posts: 233
  • Location: Málaga, Spain
Reply with quote
Post Posted: Sun Sep 24, 2017 6:07 pm
Ummmk... Not.

The only troubles compiling full game with z88dk downloaded today are in some function in enemyslot.c.

Have compiled everything with so3 except enemyslot.c with so2 and played the full game and it runa flawlesly.

Regards.
  View user's profile Send private message
  • Joined: 17 Nov 2015
  • Posts: 91
  • Location: Canada
Reply with quote
Post Posted: Tue Sep 26, 2017 3:27 am
eruiz00 wrote

The only troubles compiling full game with z88dk downloaded today are in some function in enemyslot.c.


Thanks eruiz, the fix is committed. The affected files are sdcc_peeph.3 and sdcc_peeph_cs.3 in z88dk/libsrc/_DEVELOPMENT

It was trying to get rid of the "add a,0"


989   001C  21 00 00       ld   hl,_mapposx
990   001F  7E             ld   a, (hl)
991   0020  C6 00          add   a,0x00
992   0022  DD 77 FE       ld   (ix-2),a
993   0025  23             inc   hl
994   0026  7E             ld   a, (hl)
995   0027  CE 01          adc   a,0x01
996   0029  DD 77 FF       ld   (ix-1),a


and produced this:


987   001A  2A 00 00       ld   hl,(_mapposx)
988   001D  DD 75 FE       ld   (ix-2),l
989   0020  7C             ld   a,h
990   0021  CE 01          adc   a,0x01
991   0023  DD 77 FF       ld   (ix-1),a


but the "adc a,0x01" was grabbing the wrong carry flag state.
  View user's profile Send private message Visit poster's website
  • Joined: 28 Jan 2017
  • Posts: 233
  • Location: Málaga, Spain
Reply with quote
Post Posted: Tue Sep 26, 2017 12:08 pm
Great

Someday will download a z80 guide and understand something :)
  View user's profile Send private message
  • Joined: 07 Aug 2007
  • Posts: 187
  • Location: Yach, Germany
Reply with quote
Post Posted: Fri Sep 29, 2017 3:15 pm
Dear z88dk developers,

in revision #10002, some additional optimizations for addition were implemented in SDCC. The peephole for omitting the add a, #0 should no longer be necessary.

Philipp
  View user's profile Send private message Visit poster's website
  • Joined: 17 Nov 2015
  • Posts: 91
  • Location: Canada
Reply with quote
Post Posted: Sun Oct 01, 2017 9:52 pm
PkK wrote

in revision #10002, some additional optimizations for addition were implemented in SDCC. The peephole for omitting the add a, #0 should no longer be necessary.


Thanks Philip. We're still at 9958 but I'm trying to update to 10024 now. I'm seeing some bugs but I'm not sure yet whether it's our fault or not. I'm using some large test programs so it's hard to locate them.
  View user's profile Send private message Visit poster's website
  • Joined: 07 Aug 2007
  • Posts: 187
  • Location: Yach, Germany
Reply with quote
Post Posted: Mon Oct 02, 2017 8:36 am
Alcoholics Anonymous wrote
I'm using some large test programs so it's hard to locate them.


In SDCC, we have the regression tests to avoid (re)-introducing bugs (e.g,. "make test-ucz80" in sdcc/support/regression). Have you considered building the SDCC regression tests with zsdcc?

Philipp
  View user's profile Send private message Visit poster's website
  • Joined: 28 Jan 2017
  • Posts: 233
  • Location: Málaga, Spain
Reply with quote
APLIB
Post Posted: Sat Oct 07, 2017 7:19 am
Ummm.... changing to APLIB and i see big improvements on size.

...but have a trouble

the idea is to make aplib_depack and then loadtiles...

but i dont see how i can know how many tiles i have after depack, maybe i need a function like loadstmcomprcompressedtiles, to avoid having to save that value before compile time.

¿Any insight?
  View user's profile Send private message
  • Joined: 05 Sep 2013
  • Posts: 1914
Reply with quote
Post Posted: Sat Oct 07, 2017 11:41 am
depack to VRAM directly. It's slower, probably, but you don't need to know how many bytes you need to copy to VRAM. Of course anyway you should keep track of how you're using your tiles... but PSGaiden decompression function either doesn't give you info about how many tiles were decompressed.
  View user's profile Send private message Visit poster's website
  • Joined: 28 Jan 2017
  • Posts: 233
  • Location: Málaga, Spain
Reply with quote
Post Posted: Sat Oct 07, 2017 10:19 pm
Yeah the psgcompr files have changed to aplib.

And for the tilemaps (they are plain tilemaps) i am planning to compress them. In fact i save two full banks if compress them to zx7 (how i compress them to aplib? Is there a compresión tool?)

...but i have found a troubles with malloc. The compilers warns something about heap not intialized. I am using z88dk.

Any insight?

Thanks!
  View user's profile Send private message
  • Joined: 05 Sep 2013
  • Posts: 1914
Reply with quote
Post Posted: Sat Oct 07, 2017 10:44 pm
compress using appack.exe, it's here

I never ever used malloc in a SMS program, so I really can't help you here, sorry.
  View user's profile Send private message Visit poster's website
  • Joined: 17 Nov 2015
  • Posts: 91
  • Location: Canada
Reply with quote
Post Posted: Sun Oct 08, 2017 12:52 am
Last edited by Alcoholics Anonymous on Sun Oct 08, 2017 1:22 am; edited 2 times in total
Quote

...but i have found a troubles with malloc. The compilers warns something about heap not intialized. I am using z88dk.


The default heap size for sms is zero. This is controlled by a compile time crt variable "CLIB_MALLOC_HEAP_SIZE" (you can see the default settings for an sms compile in https://github.com/z88dk/z88dk/blob/master/libsrc/_DEVELOPMENT/target/sms/crt_co... and in particular line 50 shows that the default heap size is zero. The meaning of these variables is given here https://www.z88dk.org/wiki/doku.php?id=libnew:target_embedded#crt_configuration )

So you need to enable the heap by adding a pragma to your pragma file to change the default setting. I think you're using zpragma.inc:


#pragma output CLIB_MALLOC_HEAP_SIZE = -1


A value of -1 will create a heap that takes all the available memory in ram from the end of the bss section (your program's variables) to the bottom of the stack. You can also be more specific about heap size by giving it a certain amount of memory by assigning a positive value greater than 14 to that variable. Then a heap of that many bytes will be created.

However, I wouldn't use a heap. I'd just create a global array of the max size needed like this:


unsigned char buffer[MAXSIZE];


and decompress into that. Or if it's structured level data, create a struct that works for all levels and decompress into it. Then you don't have to pay for the malloc code.

If memory allocation is needed, an obstack is probably better than malloc for most uses as it has no overhead and is much faster. However you cannot free randomly allocated memory which quite often you don't need to do. Obstacks (object stacks) allocate and deallocate in stack order, with the advantage you can deallocate everything up to a specific object in one free.

Quote

And for the tilemaps (they are plain tilemaps) i am planning to compress them. In fact i save two full banks if compress them to zx7 (how i compress them to aplib? Is there a compresión tool?)


For zx7, as you know the compressor is included with z88dk so to compress:

zx7 -f data.bin // genereates data.bin.zx7

You don't have to change the binary to a .c file as is the normal workflow with sdcc; you can just include the binary directly in an asm file:

example.asm

SECTION BANK_05    ;; the bank you want it in

PUBLIC _my_stuff_compressed   ;; export name with leading underscore

_my_stuff_compressed:

   BINARY "data.bin.zx7"    ;; name of binary file to include


You can change section within a single asm file with a new SECTION directive so you could include lots of binaries for different banks in one file. Then just add the asm file to the compile line. Your C code can get the address of the binary with "extern unsigned char my_stuff_compressed[];" (the name without leading underscore).

In your program:

#include <compress/zx7.h>

and I recommend these decompressor functions:

// memory to memory decompression
extern unsigned char *dzx7_standard(void *src,void *dst);

// memory to vram decompression
extern unsigned int sms_dzx7_standard_vram(void *src,unsigned int dst);
extern unsigned int sms_dzx7_standard_vram_unsafe(void *src,unsigned int dst);

IIRC the return values indicate the next destination address. So if you need to find out how many bytes were decompressed you can take the result and subtract the initial destination address.

zx7 has the advantage that its decompressor functions are very small. dzx7_standard() is only about 70-80 bytes. If you use this then you may as well turn on the compressed rom mode compile too because the crt will then use the same function to decompress your stored data section in rom and the cost is essentially zero for doing this.

The vram functions are completely separate and the cost for them is higher. Although decompressing to vram is more direct, you may save more more code memory (not ram because of the array or heap) by decompressing into ram and then copying to vram because you're probably using those functions already.


For aplib, z88dk does not include the compressor but it does have the decompressor functions:

#include <compress/aplib.h>

and functions:

// memory to memory decompression
extern void aplib_depack(void *dst,void *src);

// memory to vram decompression
extern void sms_aplib_depack_vram(unsigned int dst,void *src);

The aplib decompression functions are bigger however; I seem to recall ~200 bytes?
  View user's profile Send private message Visit poster's website
  • Joined: 17 Nov 2015
  • Posts: 91
  • Location: Canada
Reply with quote
Post Posted: Sun Oct 08, 2017 1:13 am
PkK wrote

In SDCC, we have the regression tests to avoid (re)-introducing bugs (e.g,. "make test-ucz80" in sdcc/support/regression). Have you considered building the SDCC regression tests with zsdcc?


Yes... so many things to do though! If it's easy I may do it.
  View user's profile Send private message Visit poster's website
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 11824
  • Location: London
Reply with quote
Post Posted: Sun Oct 08, 2017 8:12 am
In my limited testing, focusing on Master System tile data, I found ZX7 not to be very competitive with aPLib. It would be good to get a better benchmark.
  View user's profile Send private message Visit poster's website
  • Joined: 28 Jan 2017
  • Posts: 233
  • Location: Málaga, Spain
Reply with quote
Post Posted: Sun Oct 08, 2017 8:45 am
Ummm.... first attempt on august was a failure for me...

But this time... YES! YES! YES! YES! YEEEEEEEESSSSSS!!!!!

All tilemaps changed from uncompressed .bin to .zx7
All images (tiles and sprites) changed from compressed .psgcompr to .bin.zx7

About 45kb saved!!!

Now i am going to try passing some code to data segment, and lets make more content!!!
  View user's profile Send private message
  • Joined: 05 Sep 2013
  • Posts: 1914
Reply with quote
Post Posted: Sun Oct 08, 2017 10:02 am
Even if I find this discussion very interesting on the technical level, I wonder if you really need to resort to using such strong and slow compression schemes. I mean, what's the size of your ROM? Is there some actual fence limiting you in having a bigger size or is it just you took it as a personal challenge? I mean, either way I don't blame you, it's just I am wondering someone actually cares nowadays about the final size...
  View user's profile Send private message Visit poster's website
  • Joined: 28 Jan 2017
  • Posts: 233
  • Location: Málaga, Spain
Reply with quote
Post Posted: Sun Oct 08, 2017 11:01 am
Consider it as a personal challenge... A formalism.

... but if i check goodsms x.xx, i see all roms are 1mb, 2mb, 4mb, ...

I suppose this is a hardware/mapper question, but i dont see 240mb roms!

If someday its possible to me to fabricate roms, i think i will need a fixed size rom.

by other hand, i wanted to do a ladder stage with a boss like contra3 in snes, a falling bridge like haunted castle one, maybe some in depth of sea ones (with oxygen, like newzealandstory), but no free space (2mb, 3mb or 4mb, thas a contract for me), so i am now excited!

The only you must see here is the search of fun, for me as developer, for others as players. And help on testing of tools/compilers, the community technical side of this work.
  View user's profile Send private message
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 11824
  • Location: London
Reply with quote
Post Posted: Sun Oct 08, 2017 1:45 pm
When developing in assembly, it's quite hard (but not impossible) to exceed 32KB of code, and then you can fairly easily put code in upper banks (with some care). The C compilers seem to produce rather large code, and do not deal with paged code well. So it seems to become a struggle to fit the code in.

For almost all games, the ROM space is mostly taken up with data, not code. You can make bigger games simply by having more data, more levels, more graphics, more music. You could also store all that decompressed (which may offer some advantages for speed) but we tend to feel bad about that for some reason (also, bigger chunks of data mean more wasted space too as we pack it into 16KB banks).
  View user's profile Send private message Visit poster's website
  • Joined: 28 Jan 2017
  • Posts: 233
  • Location: Málaga, Spain
Reply with quote
Post Posted: Sun Oct 08, 2017 1:57 pm
I am following AA's z88dk guide for better compression and, for code on data section, and it seems work well except...

I found zx7 decompress functions seems to fail in some lines on sprites (what a strange thing) as some lines appear empty or fullfiled, or wrong, while tilemaps seems to works ok.

for both (tiles and sprites) i use next function:

void LoadSprite(const unsigned char *psg,unsigned int base,unsigned char bank)
{
// Rom bank
changeBank(bank);

// zx7 decompress
sms_dzx7_standard_vram(psg,0x4000|(base<<5));

// Change to default bank
changeBank(FIXEDBANKSLOT);
}

Investigating and researching clues about this, which could be a fail of mine, of course.
Portapapeles-1_1.jpg (20.12 KB)
psgcompr
Portapapeles-1_1.jpg
Portapapeles-4_1.jpg (22.36 KB)
bin.zx7
Portapapeles-4_1.jpg
Portapapeles-2_1.jpg (9.8 KB)
psgcompr
Portapapeles-2_1.jpg
Portapapeles-3_1.jpg (8.83 KB)
bin.zx7
Portapapeles-3_1.jpg

  View user's profile Send private message
  • Joined: 05 Sep 2013
  • Posts: 1914
Reply with quote
Post Posted: Sun Oct 08, 2017 2:13 pm
check if they do disappear when you turn off Emulicious VDP constraints. If so, it means sms_dzx7_standard_vram isn't completely VRAM safe.

4Mb (megabit - also called '4 mega power') games are common, a few 8 Mb exists too. If your ROM size falls between two different 'exact power of 2' sizes, you don't need to worry as you just have to pad its size when burning to a ROM chip.
  View user's profile Send private message Visit poster's website
  • Joined: 28 Jan 2017
  • Posts: 233
  • Location: Málaga, Spain
Reply with quote
Post Posted: Sun Oct 08, 2017 2:57 pm
Last edited by eruiz00 on Sun Oct 08, 2017 4:29 pm; edited 1 time in total
Ummm...

It fails with constraints and without them. Also it fails in fusion.

Though it could fail because of the interrupt handler, which is the next function:

// Handler de interrupciones
void InterruptHandler(void)
{
   unsigned char oldbank=lastbank;
   
   // Update music
   if(musicbank)
   {
      changeBank(musicbank);
      PSGFrame();
   }

   // Update sounds
   if(PSGSFXGetStatus())
   {
      changeBank(soundbank);
      PSGSFXFrame();
   }

   SMS_mapROMBank(oldbank);

   numinterrupts++;
}

(note this function works perfect with psgcompr compressed tiles)

but not. It doesnt work even without these handler.[/code]
  View user's profile Send private message
  • Joined: 01 Feb 2014
  • Posts: 347
Reply with quote
Post Posted: Sun Oct 08, 2017 3:37 pm
OT: Could you please consider putting your code examples between code tags? Your posts would become much more readable.
  View user's profile Send private message
  • Joined: 28 Jan 2017
  • Posts: 233
  • Location: Málaga, Spain
Reply with quote
Post Posted: Sun Oct 08, 2017 5:49 pm
Lets see.... Done some testing on Game:

Image->bin->zx7->zx7decompresstovram->fails
Image->psgcompr->zx7->zx7decompresstoram->loadpsgcomprtiles->works

... So it Must be some error in zx7decompreesstovram function.

Note that zx7 tilemaps which are decompressed to a RAM array (like AA said) works perfect.

Regards
  View user's profile Send private message
  • Joined: 17 Nov 2015
  • Posts: 91
  • Location: Canada
Reply with quote
Post Posted: Sun Oct 08, 2017 6:12 pm
eruiz00 wrote
Lets see.... Done some testing on Game:
Image->bin->zx7->zx7decompresstovram->fails
Image->psgcompr->zx7->zx7decompresstoram->loadpsgcomprtiles->works
... So it Must be some error in zx7decompreesstovram function.


This is possible - the decompression to vram is something new that was written for the sms specifically.

Can you supply one of the problematic binary files?

I did do some limited testing with this program:
https://github.com/z88dk/z88dk/tree/master/libsrc/_DEVELOPMENT/EXAMPLES/sms/comp...

It checks aplib and zx7 decompression into ram and into vram. The vram check is done by decompressing into vram, then copying from vram to ram and then comparing the original bin to the copy in ram. We can do the same with your test case.

The original zx7 decompressor for memory to memory decompression that is known to work:
https://github.com/z88dk/z88dk/blob/master/libsrc/_DEVELOPMENT/compress/zx7/z80/...

The sms decompressor into vram that adapts that routine:
https://github.com/z88dk/z88dk/blob/master/libsrc/_DEVELOPMENT/compress/zx7/z80/...

And the couple of sms specific vram subroutines that function uses:
https://github.com/z88dk/z88dk/tree/master/libsrc/_DEVELOPMENT/arch/sms/vram/z80

I'll do a quick inspection now to see if I can spot anything wrong with the vram decompressor.

Edit: the vram to vram copy is using a special method to countdown the 16-bit counter in bc that maybe needs a second check ( https://github.com/z88dk/z88dk/blob/master/libsrc/_DEVELOPMENT/arch/sms/vram/z80... )
  View user's profile Send private message Visit poster's website
Reply to topic Goto page 1, 2  Next



Back to the top of this page

Back to SMS Power!