Forums

Sega Master System / Mark III / Game Gear
SG-1000 / SC-3000 / SF-7000 / OMV
Home - Forums - Games - Scans - Maps - Cheats - Credits
Music - Videos - Development - Hacks - Translations - Homebrew

View topic - More Open Savestate stuff (semi-long)

Reply to topic
Author Message
vecna
  • Guest
Reply with quote
More Open Savestate stuff (semi-long)
Post Posted: Tue May 09, 2000 10:25 pm
Well, after finals my older brother and I hiked 23 miles of rocky, mountainous terrain to the highest point in Missouri (a somewhat anticlimatic 1700 some feet) in a period of 2 days, in quite some heat, carrying entirely too much weight (including 12 liters of water), but now that I'm back and moved in for the summer, I can get back to things. Particularly because I'm so sore I can't stand to do much aside from plant my butt at the computer and try to move as little as possible. :)

Anyhow, I have decided, since there seems to be some interest in an open savestate format, to go ahead and revise the format using a tagged-chunk format, and adding some of the things that Zoop addressed concerns about.

This is a *draft*, I have not created the FM sound chunk yet since I don't emulate it yet (althought I've begun looking into digital FM softsynthesis... will be uh, interesting). But the general structure is there. Also I have not yet decide on the RLE compression format -- I've decided that RLE compression is the best way to go because it's simple and fast, meaning you don't need anything advanced like ZLIB to support the format, although ZLIB is about as portable as they come.

What I basically am looking for is overall comments on the structure and specific comments on 'you need to include this data' or things of that nature. After I get some initial feedback, I will go ahead and rewrite the CHASMS savestate structure to support this, and once things are stabilized, I will include example code for state saving, loading, RLE compression and decompression, and checksumming. Without further ado, here goes:

The header is always first, and does not work the same way as the rest
of the tagged structures. It will always be 18 bytes long.

Immediately following, however, is a series of any number of tagged segments.
The format for each tag is the same: first, a 4-byte identifying code,
followed by an unsigned integer specifying the length of the chunk. This way,
an older emulator needs know nothing of a tag in order to skip it if is not
recognized. Following that is whatever raw data is dictated by that chunk.

I propose that if a modification needs to be made to a chunk format, that it,
well, not be. Instead, put new data into another chunk, or change the chunk
identifier, so an emulator expecting a tag that it understands won't get
something different. Either way, if you're going to add chunks, please
send suggested changes to me so they can be included in this document for
all to use. I *really* don't want to see propietary savestate tags.

---------------------- Header ---------------------
'SMSSTATE\0' - Header 9 bytes
Version number (2) 1 byte
Extensions 1 byte -- currently undefinied (future use)
ROM Checksum 4 bytes
Machine 1 byte (0=SMS, 1=GG, 2=SG-1000, 3=SC-3000
Language 1 byte (0=Japanese 1=US/European)
Mode 1 byte (0=NTSC 1=PAL)

TAGS:

=============================
TAG: Z80A (0x )
LENGTH: 32 bytes
=============================
Description: This is assumed to be all the values needed for Z80 emulation,
and all machines supported by the format are assumed to use the same cpu.
If your CPU emulator does not use all these values, please try to fill
in a value that will not break other cpu emulators.
=============================
REG_AF 2 bytes
REG_BC 2 bytes
REG_DE 2 bytes
REG_HL 2 bytes
REG_IX 2 bytes
REG_IY 2 bytes
REG_PC 2 bytes
REG_SP 2 bytes
REG_AF2 2 bytes
REG_BC2 2 bytes
REG_DE2 2 bytes
REG_HL2 2 bytes
REG_IFF1 1 byte
REG_IFF2 1 byte
REG_IR 2 bytes
REG_IM 1 byte
REG_IRQVECTOR 1 byte
REG_IRQLINE 1 byte
ICount 1 byte

=============================
TAG: URAM (0x )
LENGTH: Varies (per machine, per RLE compression)
=============================
Description: Contains a dump of the machine's User RAM section.
Remember that the Length field must contain the total number of following
bytes in the *file* to skip or load, not the size of the actual chunk of RAM.
The actual data is RLE compressed, and it's uncompressed size varies per
Machine:
0, 1: 8192 bytes
2, 3: 4096 bytes

=============================
TAG: VRAM (0x )
LENGTH: Varies (per RLE compression)
=============================
Description: Video RAM dump. RLE compressed. 16kb uncompressed.

=============================
TAG: SRAM (0x )
LENGTH: Varies (per cart, per RLE compression)
=============================
First a unsigned 2-byte word containing how much Save RAM is contained in
this tag, followed immediately by the RLE compressed save data. This length
may be up to 32k (two pages), but it may be smaller than one page. It
could be zero, but rather than save a SRAM chunk with zero bytes, one should
simply not save a SRAM chunk if it has not been accessed, or does not exist.

=============================
TAG: CRAM (0x )
LENGTH: Varies (per Machine; 32 or 64 bytes)
=============================
Uncompressed, binary dump of CRAM registers. On all Machines except Game
Gear, this is 32 bytes. On Game Gear it is 64 bytes.

=============================
TAG: VDP0 (0x )
LENGTH: 21 bytes
=============================
VDP_Regs 11 bytes
VDP_Ofs 2 bytes
VDP_Status 1 byte
VDP_Wait 1 byte
VDP_Latch 1 byte (first byte of command word)
VDP_ReadBuffer 1 byte (Read ahead)
VDP_CurrentLine 2 bytes
VDP_HLineCounter 1 byte
VDP_WriteMode 1 byte (0 if VRAM, 1 if CRAM)

=============================
TAG: PSG0 (0x )
LENGTH: 14 bytes
=============================
PSG_FreqTable 8 bytes (4 words)
PSG_VolTable 4 bytes
PSG_NoiseType 1 byte
PSG_Latch 1 byte

=============================
TAG: BANK (0x )
LENGTH: 4 bytes
=============================
$FFFC 1 byte
$FFFD 1 byte
$FFFE 1 byte
$FFFF 1 byte

IMPORTANT EMULATION NOTE: Remember that you should update your paging
setup to reflect the settings of these values, but you should NOT alter
User RAM or shadow RAM to reflect these. If this is not clear, look at
the example code at the end of this document.

=============================
TAG: PORT (0x )
LENGTH:
=============================
Port 3F 1 byte
Port DC 1 byte
Port DD 1 byte
Port DF 1 byte

=============================
TAG: GGPT (0x )
LENGTH: 7 bytes
=============================
Port 0 1 byte
Port 1 1 byte
Port 2 1 byte
Port 3 1 byte
Port 4 1 byte
Port 5 1 byte
Port 6 1 byte

(FM sound data TAG -- need to look into this)
Any additional tags for specific hardware or what not can be added. I don't know for sure what other tags need to be supported. I suppose since the SC/SGs have been tossed into the mix, I should maybe look into supporting these in CHASMS so I have an idea what I'm talking about.

What on earth does port DF do? Is this related to the lightun? Are there any other standard ports that need to be saved, for either SMS or GG?

The little (0x ) parts next to the TAG markers are where I intend to list integer values for these 4-character tags. Since the tags are not null-terminated, the easiest way to see what a tag is would be to read it in as a 4-byte int and use a switch statement.

Anyhow, I'd definitely appreciate any comments you guys have. Also I'd appreciate suggestions for RLE. I used to use code that used byte $FF as a run-marker, but on the SMS $FF may be one of the more commonly used bytes, so.. Idunno. We'll see.
Hrm.. I hope the formatting on this turns out alright..

- vecna
 
  • Joined: 18 Sep 1999
  • Posts: 498
  • Location: Portland, Oregon USA
Reply with quote
Post Posted: Tue May 09, 2000 10:54 pm
Quote
> I propose that if a modification needs to be made to a chunk format, that it,
> well, not be. Instead, put new data into another chunk, or change the chunk
> identifier, so an emulator expecting a tag that it understands won't get
> something different. Either way, if you're going to add chunks, please
> send suggested changes to me so they can be included in this document for
> all to use. I *really* don't want to see propietary savestate tags.

This may be a bit premature, as I've only given your proposed specification a preliminary examination. However I can say with absolute certainty that my emulator will need to define its own tags, and I would very much like to make the save state as compatible as possible with this "open format".

As you may or may not know, my emulator actually emulates a super-set of the SMS/GG architectures. Mostly, the extensions are for debug purposes. For example, I've defined uses for ports 0xA0 through 0xAF (which I've been assured are not used by any commercial SMS and GG software) as access points to various configuration registers in the CPU and VDP. The states of these registers will need to be preserved in a save state file. I am currently working on documenting the architecture used in my emulator. A new version will appear on my web-page in a few months.

I sincerely hope that one consideration when designing this format is the ability to create "forward compatiblity" with systems that are compatible with the SMS/GG but may also enhance their basic architectures. In other words, the specification should be flexible enough to grow beyond its original conceived use. The idea of simply ignoring unknown tags is a great start in this direction as it promotes forward compatibility.

Please contact me about questions/problems.

Eric Quinn
  View user's profile Send private message Visit poster's website
vecna
  • Guest
Reply with quote
Post Posted: Wed May 10, 2000 12:15 am
Quote
> As you may or may not know, my emulator actually emulates a super-set of the SMS/GG architectures. Mostly, the extensions are for debug purposes. For example, I've defined uses for ports 0xA0 through 0xAF (which I've been assured are not used by any commercial SMS and GG software) as access points to various configuration registers in the CPU and VDP. The states of these registers will need to be preserved in a save state file.

Ah, I see... I hadn't thought of that, but that's really a prime example of why the tagged chunk format is a good idea. The only real trick then, is that the standard chunks that _are_ defined need to be kept intact. So if any changes to those need to be made, they need to be brought to my attention soon-ish. (Zoop?) But in this case, you could easily create custom tag for your emulator, and all other emulators should just ignore it - as long as you kept the standard tags intact.

- vecna
 
Evan Clarke
  • Guest
Reply with quote
Post Posted: Wed May 10, 2000 9:14 am
Sounds very much like an adaptation of the TIF (Tagged Image File). Very good idea. Forward compatablity.

Standards will be the thing that will really need to be inforced, especially with the format id of the tag. You might need to set up a 'panel' to hear requests and grant tag id's.

Just my two cents...
 
  • Joined: 24 Jun 1999
  • Posts: 1732
  • Location: Paris, France
Reply with quote
Post Posted: Wed May 10, 2000 9:34 am
Quote
> ---------------------- Header ---------------------
> 'SMSSTATE\0' - Header 9 bytes
[..]
Quote
> Machine 1 byte (0=SMS, 1=GG, 2=SG-1000, 3=SC3000

So a Game Gear save state file will begin with "SMSSTATE" ?
What about "S8BSTATE:SMS ", "S8BSTATE:GG ", "S8BSTATE:SG1000", etc..
  View user's profile Send private message Visit poster's website
Quinntesson
  • Guest
Reply with quote
Post Posted: Fri May 12, 2000 3:39 pm
Quote
> Also I have not yet decide on the RLE compression format -- I've decided that RLE compression is the best way to go because it's simple and fast, meaning you don't need anything advanced like ZLIB to support the format, although ZLIB is about as portable as they come.

Is compression really necessary (or useful)? Standard HD size is around 15 GB and savestates are pretty small anyway.

Also, has anyone considered expanding beyond the Sega 8-bits? What about complete flexibility for use with other consoles? I am, of course, thinking of Genesis and 32X particularly.
 
vecna
  • Guest
Reply with quote
Post Posted: Fri May 12, 2000 4:26 pm
Quote
> Is compression really necessary (or useful)? Standard HD size is around 15 GB and savestates are pretty small anyway.

I am personally perfectly happy to not use any compression whatsoever (the average savestate w/out compression is only ~24k, more if it has saveRAM), but it seems that most other emu authors would like some compression.

Quote
> Also, has anyone considered expanding beyond the Sega 8-bits? What about complete flexibility for use with other consoles? I am, of course, thinking of Genesis and 32X particularly.

I had considered the possibility, but I tend to think that those systems are sufficiently different enought that they warrant their own format, even if it's similarly based. For instance, right now, all the systems the fall under the 'Sega 8 bit' category ALL use a Z80 cpu, with the same VDP chip with 16k of RAM, same PSG sound chip, etc. There are other differences of course, but they all share that certain set of similarities.

- vecna
 
  • Joined: 28 Sep 1999
  • Posts: 1197
Reply with quote
Post Posted: Fri May 12, 2000 9:23 pm

Quote
> Is compression really necessary (or useful)? Standard HD size is around 15 GB and savestates are pretty small anyway.

I'd guess most SMS emulators won't generate a lot of data, but
you never know what kind of extra features could be added to
a regular save state. Take for instance the huge ~400k save
files that Magic Engine makes, that has screenshots and other
weird stuff in each state. Compression would be very useful
in that kind of situation.

Quote
> Also, has anyone considered expanding beyond the Sega 8-bits? What about complete flexibility for use with other consoles? I am, of course, thinking of Genesis and 32X particularly.

An 'S16-DEV' forum would be nice. :) Since all the 8-bit
systems are very similar, it's easy to lump them together, and
most emulators support more consoles than just the SMS. On
the other hand, there aren't many programs that support
the Genesis and anything else, except for MESS, Ages, and Vegas.
Not that expanding the save state format to support other Sega
machines is a bad idea, though, but not terribly practical
at the moment. (IMHO)



  View user's profile Send private message Visit poster's website
  • Joined: 24 Jun 1999
  • Posts: 1732
  • Location: Paris, France
Reply with quote
Post Posted: Fri May 12, 2000 9:24 pm
About RLE, it's a nice idea but there should be a byte to enable/disable it. RLE would be a pain for savegame hackers, cheats and stuff.
Then we'll write an utility to convert to RLE/non RLE.

Quote
> ROM Checksum 4 bytes

What kind of checksum?

Quote
> =============================
> TAG: Z80A (0x )
> LENGTH: 32 bytes
> =============================
> ICount 1 byte

2 bytes would be better for that.

Quote
> Uncompressed, binary dump of CRAM registers. On all Machines except Game
> Gear, this is 32 bytes. On Game Gear it is 64 bytes.

It's not 32 bytes on all machines expect GG. It's 32 byte only on the SMS.

Quote
> =============================
> TAG: VDP0 (0x )
> LENGTH: 21 bytes
> =============================
> VDP_ReadBuffer 1 byte (Read ahead)

Not absolutly needed I think.

Quote
> =============================
> TAG: BANK (0x )
> LENGTH: 4 bytes
> =============================
> $FFFC 1 byte
> $FFFD 1 byte
> $FFFE 1 byte
> $FFFF 1 byte

It depends on the mapper.
Mapper #0 should be defined as the default, common mapper.
So rename BANK to MAPPER, add a byte to choose mapper and then those bytes will
follow when mapper = 0.

; =============================
Quote
> TAG: PORT (0x )
> LENGTH:
> =============================
> Port 3F 1 byte
> Port DC 1 byte
> Port DD 1 byte
> Port DF 1 byte

Not really updatable at my opinion.
Maybe a succession of PORT NUMBER / PORT DATA would be better.

Quote
> =============================
> TAG: GGPT (0x )
> LENGTH: 7 bytes
> =============================
> Port 0 1 byte
> Port 1 1 byte
> Port 2 1 byte
> Port 3 1 byte
> Port 4 1 byte
> Port 5 1 byte
> Port 6 1 byte

Move intos ports.

Quote
> What on earth does port DF do? Is this related to the lightun? Are there any other standard ports that need to be saved, for either SMS or GG?

I don't know what 0xDF do.

Quote
>IMPORTANT EMULATION NOTE: Remember that you should update your paging
>setup to reflect the settings of these values, but you should NOT alter
>User RAM or shadow RAM to reflect these. If this is not clear, look at
>the example code at the end of this document.

Where is the example code? :)
  View user's profile Send private message Visit poster's website
Reply to topic



Back to the top of this page

Back to SMS Power!