Forums

http://www.smspower.org/Development/ROMHeader mentions a $d checksum range bug. But http://www.smspower.org/Development/BIOSes only mentions the $2 checksum range bug.
What is the $d checksum range bug? Which BIOSes are affected?

Philipp

From the BIOSes page:

Value Lower region Upper region Total size
$a $0000..$1FEF n/a 8KB
$b $0000..$3FEF n/a 16KB
$c $0000..$7FEF n/a 32KB
$d $0000..$BFEF n/a 48KB
$e $0000..$7FEF $8000..$0FFFF 64KB
$f $0000..$7FEF $8000..$1FFFF 128KB
...

The ROM header can only be at $1ff0, $3ff0 or $7ff0 (never $bff0). Range $d is therefore buggy because it includes the header, and hence the value of the checksum itself, within the bytes which are summed.

A correct implementation of range $d would use a lower region of $0000..$7FEF and an upper region of $8000..$BFFF.

Do all BIOSes that check the checksum behave like that? If yes, it would be just a matter of calculating the checksum differently, and the 48K checksum would be useable.

Philipp

I know at least the SNES used a header format where both a checksum and a compliment checksum (to make both a total of 0, and thus both not counted) were stored.
Could that idea be used to get around that bug? (idea is calculate the rest of the data, then manually insert the checksum and then a compliment just outside the header, then they should both cancel, right?)

I thought about doing something like that - a similar technique is used by the Codemasters header.

Unfortunately the Sega checksum is a 16-bit sum of 8-bit bytes, so a simple complement would not be sufficient.

KingMike wrote

I know at least the SNES used a header format where both a checksum and a compliment checksum (to make both a total of 0, and thus both not counted) were stored.
Could that idea be used to get around that bug? (idea is calculate the rest of the data, then manually insert the checksum and then a compliment just outside the header, then they should both cancel, right?)

Looking at the equations, just one more byte outside the header should be enough. Since the lower byte of the checksum entry has no effect on the checksum test result, we would use the additional byte to get the lower byte of the checksum to 0. The upper byte is easier to handle, requiring no extra outside bytes.

Philipp

First idea:

Calculate the 16-bit checksum for the whole range excluding the two checksum bytes a (lower) and b (upper) and the extra byte c.
Let s be the lower byte of that checksum, let S be the upper byte of that checksum.

Let a := 0.
Let s' := 256 - s.

If s + s' + S < 256: Let b := S, c := s'.
Else if s + s' + S < 512 and S < 255 and s > 0: Let b := S + 1, c := s' - 1.
Else if S < 254 and s > 1: Let b := S + 2, c := s' - 2.
Else: It gets more complicated but we should get for only about 0.01 % of ROMS.

Philipp

P.S.: This should also allow using a header at 0x1ff0 or 0x3ff0 for large ROMs.

For homebrew it's generally preferred to minimise the checksum size (to minimise boot time).

PkK: you may want to set $c in the header and let the BIOS routine just check the first 32 KB and if you really want to check if the whole ROM is correct, run your own check later (against your own checksum you store somewhere - say in some free space within the first $200 bytes of the ROM)

sverx wrote

PkK: you may want to set $c in the header and let the BIOS routine just check the first 32 KB and if you really want to check if the whole ROM is correct, run your own check later (against your own checksum you store somewhere - say in some free space within the first $200 bytes of the ROM)

Personally, I'll probably go with a header at 0x1ff0 and check only 8KB. Since not all systems check the checksum, a custom check would have to be used where desired anyway.
Still, it seems 48KB is a usable as any other value.

Philipp

P.S.: I'm about to write a small program to put a valid header into a ROM file for use with my games.

It can be tricky to find unused space for the header, which is probably why the $7ff0 location is easiest to use. You can still checksum only 8KB though.

Maxim wrote

It can be tricky to find unused space for the header, which is probably why the $7ff0 location is easiest to use. You can still checksum only 8KB though.

I guess the easiest would have been for Sega to put the header at the start of the cartridge. I can see how for some ROM sizes 0x7ff0 is easier to handle. Since I'm writing Io as a 48KB game, however any of the three header locations is a good or bad as any other to me.

Philipp

P.S.: My program for inserting the checksum works, I can now play Io in an emulated SMS 1 in MAME.

Since a lot of games were 32KB, and the header locations suggest Sega were seriously considering smaller sizes, the intent was probably to add it at the end of the assembled image. The start of ROM is very valuable in terms of the Z80 reset vectors.

Maxim wrote

Since a lot of games were 32KB, and the header locations suggest Sega were seriously considering smaller sizes, the intent was probably to add it at the end of the assembled image.

Also, it seems to me that it's first checking at $7FEF, then at $3FEF, then at $1FEF which probably means again what Maxim said :)

Actually a smaller ROM would end up mirroring to higher locations so I'm not sure what use there is to check them all. No games ever used any other address.

maybe they were originally planning to have cartridges with both ROM and RAM within the first 32 KB... well, I'm of course just speculating, who might say what was in their mind? :)
(do by chance we have any contacts with technical staff working at SEGA at the time?)

I think the biggest reason for supporting a small ROM would be something like the SF-7000 which had a small (8KB) IPL to load from disk. The disk drive was somewhat planned, presumably the relative failure of the FDS and the changing economics of ROMs are what killed it off.

Author	Message
PkK Joined: 07 Aug 2007 Posts: 220 Location: Yach, Germany	$d checksum range bug? Posted: Wed May 31, 2017 8:24 am
	http://www.smspower.org/Development/ROMHeader mentions a $d checksum range bug. But http://www.smspower.org/Development/BIOSes only mentions the $2 checksum range bug. What is the $d checksum range bug? Which BIOSes are affected? Philipp

RetroSpark Joined: 08 Dec 2005 Posts: 488 Location: Melbourne, Australia	Posted: Wed May 31, 2017 12:36 pm
	From the BIOSes page: Value Lower region Upper region Total size $a $0000..$1FEF n/a 8KB $b $0000..$3FEF n/a 16KB $c $0000..$7FEF n/a 32KB $d $0000..$BFEF n/a 48KB $e $0000..$7FEF $8000..$0FFFF 64KB $f $0000..$7FEF $8000..$1FFFF 128KB ... The ROM header can only be at $1ff0, $3ff0 or $7ff0 (never $bff0). Range $d is therefore buggy because it includes the header, and hence the value of the checksum itself, within the bytes which are summed. A correct implementation of range $d would use a lower region of $0000..$7FEF and an upper region of $8000..$BFFF.

PkK Joined: 07 Aug 2007 Posts: 220 Location: Yach, Germany	Posted: Wed May 31, 2017 12:59 pm
	Do all BIOSes that check the checksum behave like that? If yes, it would be just a matter of calculating the checksum differently, and the 48K checksum would be useable. Philipp

KingMike Joined: 14 Oct 2008 Posts: 510	Posted: Wed May 31, 2017 2:49 pm
KingMike Joined: 14 Oct 2008 Posts: 510	I know at least the SNES used a header format where both a checksum and a compliment checksum (to make both a total of 0, and thus both not counted) were stored. Could that idea be used to get around that bug? (idea is calculate the rest of the data, then manually insert the checksum and then a compliment just outside the header, then they should both cancel, right?)

RetroSpark Joined: 08 Dec 2005 Posts: 488 Location: Melbourne, Australia	Posted: Wed May 31, 2017 2:56 pm
	I thought about doing something like that - a similar technique is used by the Codemasters header. Unfortunately the Sega checksum is a 16-bit sum of 8-bit bytes, so a simple complement would not be sufficient.

PkK Joined: 07 Aug 2007 Posts: 220 Location: Yach, Germany	Posted: Wed May 31, 2017 3:13 pm
	KingMike wrote I know at least the SNES used a header format where both a checksum and a compliment checksum (to make both a total of 0, and thus both not counted) were stored. Could that idea be used to get around that bug? (idea is calculate the rest of the data, then manually insert the checksum and then a compliment just outside the header, then they should both cancel, right?) Looking at the equations, just one more byte outside the header should be enough. Since the lower byte of the checksum entry has no effect on the checksum test result, we would use the additional byte to get the lower byte of the checksum to 0. The upper byte is easier to handle, requiring no extra outside bytes. Philipp

PkK Joined: 07 Aug 2007 Posts: 220 Location: Yach, Germany	Posted: Wed May 31, 2017 3:33 pm
	First idea: Calculate the 16-bit checksum for the whole range excluding the two checksum bytes a (lower) and b (upper) and the extra byte c. Let s be the lower byte of that checksum, let S be the upper byte of that checksum. Let a := 0. Let s' := 256 - s. If s + s' + S < 256: Let b := S, c := s'. Else if s + s' + S < 512 and S < 255 and s > 0: Let b := S + 1, c := s' - 1. Else if S < 254 and s > 1: Let b := S + 2, c := s' - 2. Else: It gets more complicated but we should get for only about 0.01 % of ROMS. Philipp P.S.: This should also allow using a header at 0x1ff0 or 0x3ff0 for large ROMs.

Maxim Site Admin Joined: 19 Oct 1999 Posts: 14740 Location: London	Posted: Wed May 31, 2017 6:44 pm
	For homebrew it's generally preferred to minimise the checksum size (to minimise boot time).

sverx Joined: 05 Sep 2013 Posts: 3827 Location: Stockholm, Sweden	Posted: Thu Jun 01, 2017 8:25 am
	PkK: you may want to set $c in the header and let the BIOS routine just check the first 32 KB and if you really want to check if the whole ROM is correct, run your own check later (against your own checksum you store somewhere - say in some free space within the first $200 bytes of the ROM)

PkK Joined: 07 Aug 2007 Posts: 220 Location: Yach, Germany	Posted: Thu Jun 01, 2017 9:08 am
	sverx wrote PkK: you may want to set $c in the header and let the BIOS routine just check the first 32 KB and if you really want to check if the whole ROM is correct, run your own check later (against your own checksum you store somewhere - say in some free space within the first $200 bytes of the ROM) Personally, I'll probably go with a header at 0x1ff0 and check only 8KB. Since not all systems check the checksum, a custom check would have to be used where desired anyway. Still, it seems 48KB is a usable as any other value. Philipp P.S.: I'm about to write a small program to put a valid header into a ROM file for use with my games.

Maxim Site Admin Joined: 19 Oct 1999 Posts: 14740 Location: London	Posted: Thu Jun 01, 2017 10:48 am
	It can be tricky to find unused space for the header, which is probably why the $7ff0 location is easiest to use. You can still checksum only 8KB though.

PkK Joined: 07 Aug 2007 Posts: 220 Location: Yach, Germany	Posted: Thu Jun 01, 2017 12:26 pm
	Maxim wrote It can be tricky to find unused space for the header, which is probably why the $7ff0 location is easiest to use. You can still checksum only 8KB though. I guess the easiest would have been for Sega to put the header at the start of the cartridge. I can see how for some ROM sizes 0x7ff0 is easier to handle. Since I'm writing Io as a 48KB game, however any of the three header locations is a good or bad as any other to me. Philipp P.S.: My program for inserting the checksum works, I can now play Io in an emulated SMS 1 in MAME.

Maxim Site Admin Joined: 19 Oct 1999 Posts: 14740 Location: London	Posted: Thu Jun 01, 2017 4:37 pm
	Since a lot of games were 32KB, and the header locations suggest Sega were seriously considering smaller sizes, the intent was probably to add it at the end of the assembled image. The start of ROM is very valuable in terms of the Z80 reset vectors.

sverx Joined: 05 Sep 2013 Posts: 3827 Location: Stockholm, Sweden	Posted: Thu Jun 01, 2017 10:27 pm
	Maxim wrote Since a lot of games were 32KB, and the header locations suggest Sega were seriously considering smaller sizes, the intent was probably to add it at the end of the assembled image. Also, it seems to me that it's first checking at $7FEF, then at $3FEF, then at $1FEF which probably means again what Maxim said :)

Maxim Site Admin Joined: 19 Oct 1999 Posts: 14740 Location: London	Posted: Thu Jun 01, 2017 10:48 pm
	Actually a smaller ROM would end up mirroring to higher locations so I'm not sure what use there is to check them all. No games ever used any other address.

sverx Joined: 05 Sep 2013 Posts: 3827 Location: Stockholm, Sweden	Posted: Fri Jun 02, 2017 8:55 am
	maybe they were originally planning to have cartridges with both ROM and RAM within the first 32 KB... well, I'm of course just speculating, who might say what was in their mind? :) (do by chance we have any contacts with technical staff working at SEGA at the time?)

Maxim Site Admin Joined: 19 Oct 1999 Posts: 14740 Location: London	Posted: Fri Jun 02, 2017 8:04 pm
	I think the biggest reason for supporting a small ROM would be something like the SF-7000 which had a small (8KB) IPL to load from disk. The disk drive was somewhat planned, presumably the relative failure of the FDS and the changing economics of ROMs are what killed it off.

Forums

View topic - $d checksum range bug?