|
ForumsSega Master System / Mark III / Game GearSG-1000 / SC-3000 / SF-7000 / OMV |
Home - Forums - Games - Scans - Maps - Cheats - Credits Music - Videos - Development - Hacks - Translations - Homebrew |
Author | Message |
---|---|
|
Low-level Dumping SF-7000 Floppy Disks (Sega Super Control Station)
Posted: Thu Jul 14, 2022 10:04 am
|
Been purchasing everything I could over the past few decades, and I've finally started dumping SF-7000 Floppy Disks "the right way", using a custom Pauline controller. I've got 100 floppies so far (about 3 GB of flux data) and with about 150 to go. That include meticulously inspecting and cleaning disk surface etc.
Will post more information about the process and result. We will need developers to: - Pick a more suited file formats for compact release of dumps + work on adding them to emulators. Adding support for raw stream dumps would be great too. - Reverse engineer common file systems so we can extract files along with dumps, and create tools to facilitate identifying data (checksum per file, notes about data correctness). |
|
|
Posted: Thu Jul 14, 2022 12:36 pm |
I have nothing useful to add, but just wanted to say awesome work as always. | |
|
Posted: Thu Jul 14, 2022 3:57 pm |
I disassembled an SF7 pirate disk some years ago, it had no file system because it was loading game ROMs into RAM directly. I think the IPL similarly doesn’t have any kind of file system; I presume Disk Basic will have the beginnings of one.
Is there any evidence yet of disks not formatted in the standard way (according to the IPL setup of the FDC)? Or do I need to write some code for the flux parser? |
|
|
Posted: Thu Jul 14, 2022 5:36 pm |
It probably has some sort of mechanism for giving a title and finding where the data is located, that's a primitive file system what we ought to write an extraction tool for (easy and probably tweaked on a case by case basis).
Correct, and many things in SF-7000 are based or forked from Disk Basic.
Yes. First photo is a typical disk. Second photo is Burglar Bill by Michael Boyds. I just finished indexing and typing all labels, I have 283 disks (many are dev stuff) + a dozen of empty ones. We need to figure out what's the best way to release those data (either way we will release the streams but it would be good to find the right format that suits the stuff we have). |
|
|
Posted: Thu Jul 14, 2022 6:56 pm |
Ought to mention, disks include other booting simili operating systems, I've got disks labelled "CP/M", "Segados", "G BASIC", "D BASIC" not sure what they all are yet. | |
|
Posted: Thu Jul 14, 2022 7:02 pm |
It seems popular to lean on archive.org for this sort of thing, but certainly better to also have usable disk images where possible. I don’t know very well what is done in other parts of emulation for exotic disk images - apparently for many, pirated versions cracked to work as normal are the general case and original disks often aren’t even dumped. | |
|
Posted: Thu Jul 14, 2022 7:39 pm |
CP/M is probably this. It's basically the father of DOS. |
|
|
Posted: Fri Jul 15, 2022 12:16 am |
Not would be a primitive data base for Z80 cpu´s? I would say that it´s the father of MS Access. edit: In fact is a Control Program father of DOS. The app was called "dBase" |
|
|
Posted: Fri Jul 15, 2022 6:46 am |
No, it's an 8-bit era operating system, very common at the time. | |
|
Posted: Fri Jul 15, 2022 7:05 am |
Yes, I had CP/M on my Amstrad CPC.
I think @ICEknight can help us with the disks. In Retrolel they're discussing how to preserve properly this kind of raw images. |
|
|
Posted: Fri Jul 15, 2022 9:32 am |
I mean that you is correct. My edit is for confirm my mistake. I had a apple II e i had a board in with a z80 for CP/M apps like as Visicalc. So you is correct is a kind of father of DOS that had some apps wrote for it. dBase for example. |
|
|
Posted: Fri Jul 15, 2022 11:39 am |
no worries. Though I never used CP/M myself I remember in the early DOS days that there was much info about how that was built on CP/M legacy. | |
|
Posted: Fri Jul 15, 2022 11:46 am |
How much a disk dump takes, in the raw stream format you get out of your hardware and program? We surely have to preserve this but I think your point is to create/use some disk image format where all the data is decoded/corrected already, right? |
|
|
Posted: Fri Jul 15, 2022 11:52 am |
It depends how long we read a track. Right now I'm reading 820 ms per track (about 4 rotations) at 50 Mhz, the average dump for one side takes 11 MB. For unformatted side I reduced the dumping length to 200 ms per track just to keep a data reference of the fact they were unformatted. Some disks are redumped multiple times after cleaning. Right now I have 4.32 GB for 158 disks (316 sides) so average 14 MB/side taking account of redumps increasing the average and unformatted lowering the average. The most common 640-sectors format, omitting errors and assuming valid data, and read via IPL functions can be binarized as 163840 bytes (this is what our .SF7 files where). I believe 80%+ could be safely released under that format. I don't understand the technicalities well enough yet to tell if a slightly better format (e.g. DSK encoding sector location) can be enough for 100% or if we need an even lower-level format. |
|
|
Posted: Fri Jul 15, 2022 12:03 pm |
I would go for the lowest hanging fruits first. If 80%+ of those disks can fit an existing format and be released in that way, let's do that first.
Then we probably need to investigate on the remaining ones, how many different schemes we need to accomodate them all - if that's at all possible of course. Hard to tell without knowing much at the moment but it's something that will come when we'll have more info. |
|
|
Posted: Sat Jul 16, 2022 1:53 am |
I've only heard recently of an effort to get original disk dumps for Apple II. That's certainly a noble effort, hopefully 40 years later isn't too late. I believe for some such systems as Commodore 64, the pirated "cracktos" are even celebrated as a part of history. (I can some of the appeal in cracks for emulation, as it surely is possible to load faster, I would say as someone who has not used the original hardware. As I understand, the official 1541 disk drive was like it's own computer with own firmware and emulating copy protection would probably entail low-level emulation of the disk drive firmware. Which I imagine would also mean accurate load time emulation.) |
|
|
Posted: Sat Jul 16, 2022 11:00 am |
I am super interested in contributing!
I already played around with the disk basic's fat https://github.com/fabiodl/sctape/blob/main/scfloppy.py and would love to start working on the flux data |
|
|
Low-level Dumping SF-7000 Floppy Disks (Sega Super Control Station)
Posted: Sun Jul 17, 2022 3:43 pm
|
Interesting stuff.
Preserving disks down to the magnetic flux level is uber cool, but I feel it’s overkill, as this level of information doesn’t make it past the FDC to the computer. It does have some practical use, such as being able to identify corrupt sectors from the sector info and therefore bad dumps, no doubt. I think we can do better than just a raw stream of digitized FM-encoding data. An XML schema that defined tags for the markers, gaps, sector info, sector data, etc… would allow the entire disk to be preserved, while allowing simpler access to the sector data with just parsing. About the 20% of oddball disks, wouldn’t they just be system disks with their own IPL in Track 0 Sector 1 that is copied to RAM and then executed to load the rest of the disk data into RAM? That would mean the IPL on the disk would be in the standard format, and could be disassembled to see how the FDC is configured and how the rest of the disk is formatted in theory. |
|
|
Posted: Sun Jul 17, 2022 4:05 pm |
That seems plausible, yes, so you could have metadata to describe the rest of the disk format, perhaps just in terms of the FDC parameters used to read it.
I imagine some copy protections may want to have corrupt sectors that trigger certain FDC behaviour or even some borderline data that triggers some non-deterministic behaviour in the FDC. |
|
|
Posted: Sun Jul 17, 2022 7:31 pm |
That's enough of a justification. Disks are not flawless to read, some hard, some downright corrupted. Also consider that some copy protection may be using intentionally corrupted MFM sectors and I would assume the software would intentionally verify that the controller can't read them. If we don't record that information in theory the dump is incorrect and software needs to be cracked. Also consider the typical user disks with files that happens to have corrupted sectors. If we release a "packed" version of this I think it would be appropriate and correct that we release it with information that some parts couldn't be read (with maybe a chance they eventually it'll be read).
Of course and we should. There are loads of formats (HxCFloppyEmulator only has nearly 30 formats) so we should be investigating what's best.
Pretty sure all the fancy disks are booting indeed, you are right Track 0 Sector 1 is "regular". https://www.smspower.org/forums/files/img_7281_577.jpeg is one of those disk. Not sure what you are suggesting here, interested. When I'm done with my first pass of dumping I'll open a work group to solve those problems. |
|
|
Low-level Dumping SF-7000 Floppy Disks (Sega Super Control Station)
Posted: Mon Jul 18, 2022 1:51 pm
|
I think I convinced myself it wasn't overkill by the time I finished the paragraph and typed "no doubt". :)
You are totally right. See you in the work group. |
|
|
Posted: Fri Aug 12, 2022 6:29 am |
Very interesting work!
i had just got and repaired an SF-7000 (making the missing I/O cassette cartridge from scratch) and used a gotek with flashfloppy as a drive, since the replacement belt for the 3" is not yet arrived). The only disk image i can get working was the Basic "dsk" found on the net and dumped with CPCDiskXP. I also get booting a compilation "SF7" disk adding the Basic sector 0, but it doesn't work after the initial menu. So, a TOSEC compilation of Sega disks will be very appreciated, and i vote for the HFE or DSK format, so the gotek's user could play with them and write real floppies if they want. |
|
|
Posted: Fri Aug 12, 2022 7:44 am |
SF7 files can be converted to DSK format using the program found here | |
|
Posted: Wed Aug 31, 2022 10:18 am |
Useful links to compare the available formats.
https://stardot.org.uk/forums/viewtopic.php?t=19676 https://www.msx.org/wiki/Emulation_related_file_formats I will update the post if I find more. I have yet to find the "this is it" one. However, since the source is Pauline, maybe hfe could be a natural choice. |
|
|
Posted: Wed Aug 31, 2022 10:25 am |
We need a format which can carry the fact that some sectors are damaged and unreadable by the controller. I don’t know if DSK can do that (maybe it does, i haven’t checked).
We are going to have to release dumps with known damaged sectors, so the damage is properly encoded in the dump, rather than wipe out damaged sectors and lose track of it. |
|
|
Posted: Wed Aug 31, 2022 11:31 pm |
I totally agree. I believe DSK can represent damaged sectors. It is also based on the NEC765, so it is a good match (essentially, for each sector it reports the status bits the NEC765 would report). The weak bits can be represented with an extension, as reported at the end of this page https://www.cpcwiki.eu/index.php/Format:DSK_disk_image_file_format I believe the "unwritable sector" protection scheme is the only one not covered by the format. |
|
|
Posted: Fri Sep 02, 2022 1:21 pm |
But are there disks with protection schemes? I only have a dozen made with "c" command of the sf7000 utility, sourcing from sf7 files found on the web, but none of them seems protected (I was thinking to write them via rs232 and DSKO$ basic command too! 😄) so curious in which disks are protected |
|
|
Posted: Sun Sep 04, 2022 2:54 am |
I had an SF-7000 in the 80's, so I'm really interested in playing with the dumped disk images. I assume they're in a format that Meka can handle? | |
|
Posted: Sun Sep 04, 2022 7:50 pm |
Not yet. This is what will be working on. The 163840 bytes .SF7 format we used on our first SF-7000 releases is not a good format for all types of releases. |
|
|
Posted: Sun Sep 04, 2022 8:27 pm |
Some of the disks you mentioned seem really interesting (CP/M, SegaDOS, G-Basic...)! In the meanwhile, couldn't you post some of theese in HFE format, to start studing them? |
|
|
Posted: Wed Nov 02, 2022 3:17 pm |
My friend Nanard did some work on this and settled on adding support for the Extended DSK format as a new format for SF-7000 release.
See Burglar Bill which had an odd layout incompatible with earlier .SF7 format. |
|
|
Posted: Wed Nov 02, 2022 3:58 pm |
Good news! i hope Extended DSK format is supported by Flashfloppy, i'd like to test some disks from your huge collection with my SF-7000 and SF-7017! | |
|
Posted: Thu Nov 10, 2022 12:33 am |
This is a very sensible choice! Did you already open a work group? |
|