Forums

The translation is under way!

I know it's supposed to say Highschool Kimengumi, but for some reason I made the tiles to big and I can barely fit just the highschool bit.

I made this layout quickly, i'll make the text nicer later.

The problem I'm having is that the japanese text use less tiles for text, so i'm having to shift data around alot and I might be overwriting useful data.

I had to create a data compression program to go with the tile and map data decompression routines this game uses, the good thing is I can use it in my own productions now :)

Can you publish the information about the compression scheme? I ripped the Z80 code for tile and tilemap decompression from Phantasy Star, and coded a compressor in my Bitmap to Tile convertor; if you can document the format, I might find time to add that too, and we can all use them in our projects. (Maybe this one can compress better, or is faster to decompress.)

The Phantasy Star tile compression format also allows you to tell if the compressed data is valid or not because it must decompress to four identical-length bitplanes that make a whole number of tiles. This allows you to search a file to test if it uses that compression, although false positives are common. As such, a PC-based decompressor is handy and useful for hacking other games using the format.

I think I know what you're talking about searching for four identical-length bitplanes thing.

For example.
Compressed data = 04 01 00 04 02 00 04 03 00 04 04 00

Decompresses to = 01 02 03 04 4 bytes
01 02 03 04 4 bytes
01 02 03 04 4 bytes
01 02 03 04 4 bytes

The compression used in high school kimengumi is basically the same except
there's only one zero terminating byte at the end of the last "column" instead of four
for each.

A counter is used to determine when to reset to the begining of the next "column" of data.
The counter is stored as two bytes at the begining of the compressed data.

For example.(same data)
Compressed data = (04 00) 04 01 04 02 04 03 04 04 00
|
counter

The counter is 8 times the number of tiles used. because 1 tile has 8 "rows" to scan.

(for anyone not knowing about "rows and columns" then read maxim's document on phantasy star tile decoder)

Here's the actual decompression routine.

;////////////////////////////////////////////////////
;example routine call
;
; ld bc,$8000 ;
; ld hl,($8000) ; the two bytes at ($8000) divided by 8 deterime the number of tiles
; ld de,$6020 ;
; call output_tiles ;

;----------------------------------------------------------

output_tiles:

call decode_tiles ;start decoding

xor a ;
ld ($c141),a ;clear both
ld ($c142),a ;RAM pointers
ld ($c143),a ;
ld ($c144),a ;
ret ;

;--------------------------------------------------------------

decode_tiles:

ld ($c141),hl ;load first two
ld ($c143),hl ; bytes into two
         ;RAM pointers

ld h,b ;HL = BC
ld l,c ;
inc hl ;hl points to after
inc hl ;the first two bytes

determine_decoding_style:

ld a,(hl) ;if next byte is zero
or a ;then exit decoding
ret z ;

bit 7,a    ;if bit 7 = 0
jr nz,decode_style_2 ;then jump

decode_style_1 (RLE)

ld b,a ;
inc hl ;
- call output_data ;decode style 1
djnz -    ;
inc hl ;point to next byte to determine decoding
jp determine_decoding_style

decode_style_2: (non RLE)

and $7f ; res bit 7 and load into counter
ld b,a ;
inc hl ;
- call output_data ;
inc hl ;
djnz -    ;
jp determine_decoding_style ;

;///////////////////////////////////////

output_data:

ld a,e ;
out ($bf),a ;de = Vram addr
ld a,d ;
out ($bf),a ;

ld a,(hl) ;(hl) = data
out ($be),a ;

push hl ;
ld hl,($c141) ;dec ($c141)
dec hl ;and if zero
ld a,h ;
or l ;
jr z,set_next_column
ld ($c141),hl ;

inc de ;
inc de ;add 4 to de
inc de ;
inc de ;
pop hl ;
ret ;

;///////////////////////////////////
;sets de to the next column
;
;

set_next_column:

ld hl,($c143) ;restore the counter
ld ($c141),hl ;

;multiply hl by 4
xor a ; clear carry
rl l ;
rl h ; HL x 2
xor a ; clear carry
rl l ;
rl h ; HL x 2
         ; eg. $03a8 x 4 = $0ea0

inc de ;
inc de ;
inc de ;
inc de ; de = HL + de before routine
         ; eg. DE-$6020 + HL-$ea0 = DE-$6EC0

;subtract
ex de,hl ;
xor a ; clear carry
sbc hl,de ;
inc hl ; eg (DE-$6ec0 - HL-$ea0) + 1 = DE-$6021
ex de,hl ;
pop hl ;
ret ;

I did a test to see which compression scheme would compress the most.

I compressed a $57F byte file with both the compression schemes from phantasy star and high school kimengumi.

Outcome

File compresses to $27F using Phantasy star compression.

File compresses to $276 using Highschool Kimengumi compression.

Reason

1.
Well the Phantasy Star compression uses four zero terminating bytes while
the Highschool Kimengumi compression uses two bytes for a counter and one
zero terminating byte.

So you save one byte WOW!!!!!!

2.
The second reason is that the Highschool Kimengumi compression does not need to stop compressing when it reaches the end of the column.

Example (1 tile in the best case scenario)

01 01 01 01
01 01 01 01
01 01 01 01
01 01 01 01
01 01 01 01
01 01 01 01
01 01 01 01
01 01 01 01

Using Phantasy Star compression = 08 01 00 08 01 00 08 01 00 08 01 00
Using Highschool Kimengumi compression =(08 00) 20 01 00

OK, well every byte counts :) Did you try it on "real" data? The similarity means I'll probably be able to add it to BMP2Tile easily enough.

My PS compressor might need some work. It doesn't always compress the data from Phantasy Star as well as the existing compressed data...

Is the same compression used for tilemap data? In PS, the tilemap algorithm uses 2-interleaving instead of 4-interleaving and outputs to RAM instead of VRAM, compared to the tile decompressor.

Quote

OK, well every byte counts :) Did you try it on "real" data? The similarity means I'll probably be able to add it to BMP2Tile easily enough.

My PS compressor might need some work. It doesn't always compress the data from Phantasy Star as well as the existing compressed data...

What I've been doing is taking the compressed data out of the game, decompressing it, then compressing it again to see if my program compresses to the same size. Sometimes it does sometimes it doesn't so I'm still fixing it.

Quote

Is the same compression used for tilemap data? In PS, the tilemap algorithm uses 2-interleaving instead of 4-interleaving and outputs to RAM instead of VRAM, compared to the tile decompressor.

That's exactly the same.

Author	Message
Nick Joined: 20 Dec 2004 Posts: 26 Location: Wellington, New Zealand	Translating Highschool Kimengumi Posted: Wed Feb 09, 2005 6:53 am
	The translation is under way! I know it's supposed to say Highschool Kimengumi, but for some reason I made the tiles to big and I can barely fit just the highschool bit. I made this layout quickly, i'll make the text nicer later. The problem I'm having is that the japanese text use less tiles for text, so i'm having to shift data around alot and I might be overwriting useful data. I had to create a data compression program to go with the tile and map data decompression routines this game uses, the good thing is I can use it in my own productions now :)

Maxim Site Admin Joined: 19 Oct 1999 Posts: 14745 Location: London	Posted: Wed Feb 09, 2005 10:50 am
	Can you publish the information about the compression scheme? I ripped the Z80 code for tile and tilemap decompression from Phantasy Star, and coded a compressor in my Bitmap to Tile convertor; if you can document the format, I might find time to add that too, and we can all use them in our projects. (Maybe this one can compress better, or is faster to decompress.) The Phantasy Star tile compression format also allows you to tell if the compressed data is valid or not because it must decompress to four identical-length bitplanes that make a whole number of tiles. This allows you to search a file to test if it uses that compression, although false positives are common. As such, a PC-based decompressor is handy and useful for hacking other games using the format.

Nick Joined: 20 Dec 2004 Posts: 26 Location: Wellington, New Zealand	Posted: Wed Feb 09, 2005 4:14 pm
	I think I know what you're talking about searching for four identical-length bitplanes thing. For example. Compressed data = 04 01 00 04 02 00 04 03 00 04 04 00 Decompresses to = 01 02 03 04 4 bytes 01 02 03 04 4 bytes 01 02 03 04 4 bytes 01 02 03 04 4 bytes The compression used in high school kimengumi is basically the same except there's only one zero terminating byte at the end of the last "column" instead of four for each. A counter is used to determine when to reset to the begining of the next "column" of data. The counter is stored as two bytes at the begining of the compressed data. For example.(same data) Compressed data = (04 00) 04 01 04 02 04 03 04 04 00 \| counter The counter is 8 times the number of tiles used. because 1 tile has 8 "rows" to scan. (for anyone not knowing about "rows and columns" then read maxim's document on phantasy star tile decoder) Here's the actual decompression routine. ;//////////////////////////////////////////////////// ;example routine call ; ; ld bc,$8000 ; ; ld hl,($8000) ; the two bytes at ($8000) divided by 8 deterime the number of tiles ; ld de,$6020 ; ; call output_tiles ; ;---------------------------------------------------------- output_tiles: call decode_tiles ;start decoding xor a ; ld ($c141),a ;clear both ld ($c142),a ;RAM pointers ld ($c143),a ; ld ($c144),a ; ret ; ;-------------------------------------------------------------- decode_tiles: ld ($c141),hl ;load first two ld ($c143),hl ; bytes into two ;RAM pointers ld h,b ;HL = BC ld l,c ; inc hl ;hl points to after inc hl ;the first two bytes determine_decoding_style: ld a,(hl) ;if next byte is zero or a ;then exit decoding ret z ; bit 7,a ;if bit 7 = 0 jr nz,decode_style_2 ;then jump decode_style_1 (RLE) ld b,a ; inc hl ; - call output_data ;decode style 1 djnz - ; inc hl ;point to next byte to determine decoding jp determine_decoding_style decode_style_2: (non RLE) and $7f ; res bit 7 and load into counter ld b,a ; inc hl ; - call output_data ; inc hl ; djnz - ; jp determine_decoding_style ; ;/////////////////////////////////////// output_data: ld a,e ; out ($bf),a ;de = Vram addr ld a,d ; out ($bf),a ; ld a,(hl) ;(hl) = data out ($be),a ; push hl ; ld hl,($c141) ;dec ($c141) dec hl ;and if zero ld a,h ; or l ; jr z,set_next_column ld ($c141),hl ; inc de ; inc de ;add 4 to de inc de ; inc de ; pop hl ; ret ; ;/////////////////////////////////// ;sets de to the next column ; ; set_next_column: ld hl,($c143) ;restore the counter ld ($c141),hl ; ;multiply hl by 4 xor a ; clear carry rl l ; rl h ; HL x 2 xor a ; clear carry rl l ; rl h ; HL x 2 ; eg. $03a8 x 4 = $0ea0 inc de ; inc de ; inc de ; inc de ; de = HL + de before routine ; eg. DE-$6020 + HL-$ea0 = DE-$6EC0 ;subtract ex de,hl ; xor a ; clear carry sbc hl,de ; inc hl ; eg (DE-$6ec0 - HL-$ea0) + 1 = DE-$6021 ex de,hl ; pop hl ; ret ;

Nick Joined: 20 Dec 2004 Posts: 26 Location: Wellington, New Zealand	Posted: Thu Feb 10, 2005 3:07 am
	I did a test to see which compression scheme would compress the most. I compressed a $57F byte file with both the compression schemes from phantasy star and high school kimengumi. Outcome File compresses to $27F using Phantasy star compression. File compresses to $276 using Highschool Kimengumi compression. Reason 1. Well the Phantasy Star compression uses four zero terminating bytes while the Highschool Kimengumi compression uses two bytes for a counter and one zero terminating byte. So you save one byte WOW!!!!!! 2. The second reason is that the Highschool Kimengumi compression does not need to stop compressing when it reaches the end of the column. Example (1 tile in the best case scenario) 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 Using Phantasy Star compression = 08 01 00 08 01 00 08 01 00 08 01 00 Using Highschool Kimengumi compression =(08 00) 20 01 00

Maxim Site Admin Joined: 19 Oct 1999 Posts: 14745 Location: London	Posted: Thu Feb 10, 2005 9:39 am
	OK, well every byte counts :) Did you try it on "real" data? The similarity means I'll probably be able to add it to BMP2Tile easily enough. My PS compressor might need some work. It doesn't always compress the data from Phantasy Star as well as the existing compressed data... Is the same compression used for tilemap data? In PS, the tilemap algorithm uses 2-interleaving instead of 4-interleaving and outputs to RAM instead of VRAM, compared to the tile decompressor.

Nick Joined: 20 Dec 2004 Posts: 26 Location: Wellington, New Zealand	Posted: Thu Feb 10, 2005 10:52 am
	Quote OK, well every byte counts :) Did you try it on "real" data? The similarity means I'll probably be able to add it to BMP2Tile easily enough. My PS compressor might need some work. It doesn't always compress the data from Phantasy Star as well as the existing compressed data... What I've been doing is taking the compressed data out of the game, decompressing it, then compressing it again to see if my program compresses to the same size. Sometimes it does sometimes it doesn't so I'm still fixing it. Quote Is the same compression used for tilemap data? In PS, the tilemap algorithm uses 2-interleaving instead of 4-interleaving and outputs to RAM instead of VRAM, compared to the tile decompressor. That's exactly the same.

Forums

View topic - Translating Highschool Kimengumi