Forums

v6.2! ( http://sfx.gligli.free.fr/smsdev/sms_video_player_v6.2.7z )

What's new:

Made player code smaller to allow for longer videos.
Fixed a player tile corruption bug.
Encoder reindexing process speedup.
Proper 1MB Sonic CD video plus a full version.

gligli wrote

v6.2! ( http://sfx.gligli.free.fr/smsdev/sms_video_player_v6.2.7z )

What's new:
[*]Fixed a player tile corruption bug.

This one?

test bug.jpg (108.66 KB)

test bug 2.jpg (122.4 KB)

Looks like it yep, just recompile the ROM with the new asm and it should be good.

Thanks, I encoded four versions of this;

Average tiles per frame;

4 (1.55MB):

Emulicious: Fine

MEKA: Has some tile bug that doesn't change between frames. Audio issues.

10 (1.85MB):

Emulicious: Fine.

MEKA: Audio Issues , but no tile bug (or at least very minor and not same every frame)

50 (2.76MB):

Emulicious: Tile bug , frame rate collapses about a minute in and it falls apart.

MEKA: Tile bug, typical audio issues, but doesn't fall apart, crashes at end of video.

100 (3.48MB):

Emulicious: First few seconds black, tile bug, falls apart about a minute in.

MEKA: First few seconds are black (looking at pngs first 51 frames are black, should only be 3) typical skipping audio issues, tile bug, doesn't fall apart, doesn't crash.

I'm guessing that the black frames at the beginning of the 100 tile/frame video could be caused by my PC freezing during the encoding process. Some of the other issues are probably emulator related (but it's not possible to test on real hardware).

I wonder if it's possible to test in SMS mode on a Mega Drive with a Mega EverDrive as it supports larger ROMs and SMS ROMs.

Recompiling with 6.2 asm fixed the tile bug and the falling apart issue in Emulicious, it now runs fine except for the first few seconds being black. Hopefully re-encoding will fix that (and I can bump up the quality a bit with having 600KB spare).

That will probably take 12 hours on my weak CPU (hopefully less with the new version speed improvements) ...

Ah yeah, I had the full crash too in emulicious after the tile corruptions.
One more thing that's fixed in this version that I didn't mention is audio/video sync. Although I don't think the bug I fixed could lead to obvious desynchronisation.

With one keyframe every 100 frames, the new version shaves 1.5 hours from 1MB Sonic CD encoding.

Yeah, it's a nice speed up.

I thought I had got this encoding almost perfect;

Frame count: 1190
Average tiles per frame: 150
Max tiles per frame: 208
KMeans restarts: 8
Spatial / temporal smoothing: 1

Size: 3.97 MB (4,167,380 bytes)

But about 75 seconds in it corrupts. It doesn't crash though. I don't know if it's because it was too large, issues with my PC during the encoding process, or an unknown bug with the software.

Edit: It plays OK in SMS video tiler, so I guess that the issue is post tiling..

corrupt 1.jpg (92.36 KB)

corrupt 2.jpg (137.48 KB)

corrupt 3.jpg (174.6 KB)

Can you send me the data files? (I found another spot in the code where there could be a similar bug as I fixed)

Sure, data.bin & index.bin;

tiled.rar (2.94 MB)

Ok, I found what's wrong, but I can't do anything about it: It's not possible to encode a video with more than 2MB of tiles.

This one has about 2.5MB of tiles so the end of it is bad.
If 100 avg tiles per frame worked, I guess it was just below 2MB and close to the max quality for this video.

You'll probably have to encode the video again, sorry about that...

By the way, I'd like to try to encode the Bad Apple 3D video if you don't mind.

OK, no worries, I'll re-encode it overnight.

The Bad Apple video is here if you want to have a go at it. The files that I originally used aren't really useful now as I only had part of the video and discarded too many frames.

As there was a 2MB limit on total tile size, I decided to increase quality by increasing KMeans to 16 from 8. There is a noticeable improvement in quality, and size increased a bit too (from 3.48MB to 3.61MB), although as the first encoding messed up the first few seconds that could explain some of the size difference. Still, this significantly increased encoding time.

A minor bug also occurs in the last few seconds, maybe increasing KMeans pushed me past the 2MB limit, but the video ended before it became a major issue. I've encoded this video so many times already that I can't be bothered to re-encode in order to try to fix the minor bug in the last few seconds.

Also, I gave up on trying to fit it in to 1MB, I think that the video is just too long.

Zillion anime opening (creditless) ... I recommend Emulicious.

Edit: I've said it before, but I have to say again, this is incredibly impressive!

zillion.rar (2.67 MB)

I think the limit is 2MB tiles, due to using 16 bits to specify a bank number (7 bits) and a tile index within it (9 bits). The frame data and index are not included in this limit, but but I'm not sure how the data is split between files. It would be trivially possible to extend past this 2MB limit at the cost of increased index size.

This could even be done without increasing the index size, as there's an unused byte per index entry in practice.
The problem will be that additional ops will be needed in TUDoMiddle to load the carry with the additional bit, and that will alter the PlaySample timing and TilesUploadSlow / TilesUploadFast sequencing.

That's true, it messes with the timing and you have to redo everything... I've been writing pcmenc player cores recently and I've almost memorised all the opcode timings now, it's a constant grind...

Bad Apple 3D video: http://sfx.gligli.free.fr/smsdev/bad_apple_3d_sms.7z

I had to reencode it many times to get something that looks nice but I'm quite happy with the result.
It's 1400 frames, which seems to be the maximum that fits. I even had to rewrite a little bit of the code to make it fit.

Now I think my next goal is to play it on the hardware, and for this I'm going to try to build a custom cartridge with a Teensy 3.2 microcontroller to do the mapping and a 4MB flash chip.

Nice work, 1400 frames is probably just about enough to fit the second half in another ROM. I might take a look at that unless you're planning on doing it.

I'll be following your progress with the 4MB cartridge, I'm interested in building one myself.

Nope, go ahead!
The zillion video is pretty cool too, with the light phaser and all. Not an easy encode as there's a lot of scenes.

I don't plan to make a proper pcb for it, so the wiring may be a little hackish though.

I needed to refresh my memory ahead of getting around to Bad Apple part 2.

The Good, the Bad, and the Ugly final showdown scene:

GBU Video.rar (2.25 MB)

Thank you for all these videos guys, these is great!

I've 2 requests, if someone could do these videos I would be so happy :)

Hello, I'm working on this again for the 2017 coding competition (trying new algorithms to improve quality / size ratio) and I have a question about the Z80 R register.
I thought about using it as a clock source for audio samples and from my understanding of the way the 8KB XRAM chip works, I wonder if it may be possible to alter it without risking losing RAM contents? Has something like that been tried already?

Refresh is tied to Z80 refresh output and there's no way you can stop it from happening. R register contents have no effect on how refresh works on SMS.
EDIT: Seems the chips actually use address pins for refresh row in non power-down refresh mode. I will see which refresh method is being used, I'll report back soon.
EDIT2:
Seems only power-down refresh is used, !RFSH is only active when !CE is not active, so external address input is not used by the chip. You can mess with R register freely.
EDIT3: Nnnnnnope, self refresh requires very long pulses to work and those don't happen on SMS. There's periodic !CE pulses with !RFSH pulses between them. R register is not safe to touch.

Actually I want to use the fact that it's incremented each instruction as a clock source, but also need to reset it at times.
The XRAM chip datasheet says it goes into self refresh when the /RFSH input is high but I wonder if it can switch back an forth between self refresh and external refresh at will.
Edit:
Yay, thank you! Sample playback will be way easier if I can get this trick to work fine :)

Unfortunately it is not the case, I looked at timings more closely and self refresh requires 40µs pulse to work, and that's not happening in SMS. External refresh is being used instead, so you cannot mess with R unfortunately...

Wait, so you mean there would just need to completely disable refreshes (ie /RFSH always high) from the Z80 to actually get the chip to self refresh?
I don't think it is a problem if I take it into account and roughly count instructions to reset R before it reaches 128.

Z80 will always pulse !RFSH during M1 cycle and R register is incremented and put an the bus, XRAM will then use the content on the bus as refresh address. If you continuously spammed R with 0 then only the first row gets refreshed in the XRAM and rest of the contents get corrupt over time. Now the good news is that refresh cycles happen very often (every instruction) and typical time to refresh all rows is 16ms (which is just around one frame) and that means all rows get refreshed multiple times per frame, so I don't think occasional resetting of R will cause any ill effects.

Self refresh mode which does not use externally supplied refresh address requires !RFSH to go low for 40µs which is around 2/3 of a scanline, and that never happens on SMS.

Ok, I had misunderstood the way refresh works.
An occasional reset is better than nothing, and worst case I can still use R without setting it, it would just cost more ops per audio sample.
In any case, I will try it on my SMS and see what I can get to work, thanks again!

I was not completely satisfied by my first encoding of Bad Apple 3D, the colors were pretty washed out and of course there was only half of the video.

I had a few ideas in mind to make a better version:
- Use the two SMS hardware palettes, to up the max colors on screen.
- Use horizontal and vertical mirrors on the tiles to improve size/quality ratio.
- Have a global tile repository. Prior versions had a per keyframe repository which made tiles a lot less reusable.
- Make it compatible with 60Hz hardware.

So, first I had to rewrite the encoder because the encoding pipeline would be different.

v6 was: load -> keyframe tiling -> frame tiling -> dithering -> reindex -> save.
Tiling was done on truecolor tiles, which wouldn't now work if the goal was to make tiles reusable no matter the palette.

I came up with this pipeline for v7: load -> dithering -> make unique -> global tiling -> frame tiling -> smoothing -> reindex -> save.
- I used a much better dithering algorithm from http://bisqwit.iki.fi/story/howto/dither/jy/ which improved color even with a single palette. This process also tries to find the best keyframes palettes.
- "make unique" is just a simple algorithm that merges strictly similar tiles to make tiling faster.
- As tiling is now done on colorless 4bit tiles, KMeans wouldn't work (color indices are not continuous, so eg. indices 1 and 2 are the same distance as indices 1 and 15). In statistics, we would say we have 64 nominal factors per tile. KModes is what I used then as it's a bit like KMeans except for nominal data.
Global tiling uses KModes to reduce the overall number of tiles to a fixed number and frame tiling uses DCT (discrete cosine tranform) and KModes to find the best matching tiles for the frame, accounting for palettes and H/V mirrors.
Especially global tiling is really time consuming as there is often more than 1 million tiles to reduce so I used SIMD optimisation to speed it up.
- Smoothing uses DCT to tag tiles that are roughly the same as in preceding frame, so that the tilemap can only be updated partially each video frame.
- Save compresses tiles indexes and tilemaps into byte command streams (basically encoding index differences for tiles indexes and dictionnary with skips and repeats for TileMap).

I then wrote the player naively and it was 2x too slow, even without sound. The main optimisations I made were:
- Use jump tables to handle tiles indexes and tilemaps byte command streams. They are cheap on the Z80 and aligning jump tables and jump offsets on 256 bytes and storing only high address bytes makes them super fast (eg. code table).
- Use 256 bytes aligned lookup tables, so eg you load H with the upper byte of the LUT address, and L with the byte to be looked up, the load is then a simple LD A, (HL).
- Remove as many conditional branches as possible and use the main jump table instead. The best of those was to make the encoder handle the switch between fast and slow VRAM tiles upload depending on VBlank or active display by counting the Z80 cycles and the corresponding scanlines at encode time.
- Systematically abuse the stack as a general purpose data pointer and consequently use no interrupts.

Sound was another huge problem. Counting cycles and adding PlaySample macros every X CPU cycles wasn't possible with jump tables as you don't have a fixed execution flow anymore.
My first idea was to use the Z80 R register to keep track of time but it didn't work well (it's incremented each op, but ops timings varies widely on the Z80) and was not cheap in CPU time.
I finally came up with a macro that was able to handle "partial" samples. That is, use some kind of fixed point math for the sample index so you can use it in any function and just pass any cycle count as parameter (code). You end up calling it more often than a simple PlaySample but it's very good at keeping a low sample jitter no matter the code.

Anyway I think the end result is better than v6 and probably more maintainable too. In the end I didn't do 60Hz compatibility but at least it's doable now as tilemaps updates are done in both active display and VBlank.

The full source code is available online at: https://github.com/gligli/tiler and encoder binaries are included in http://www.smspower.org/uploads/Homebrew/BadApple3DFullLength-SMS-7.00.zip

Phew, that was a lot of text ^^

Thanks man!

Using macros with step / skew to time sample updates is incredibly creative.

How many unique tiles in total were used? I have to imagine the new tile streaming approach introduced quite a bit of meta data not to mention cpu cost plus you have more details nametable data (flips and palette) and yet you fit twice as much video on ROM!

Your optimizations are smart, the solution is total coder porn. I love it!

Actually, the step / skew stuff is just bare bones direct digital synthesis. I knew this because I wrote a lot of sound synthesizer code for previous projects.

About 30000 tiles I think. Overall, there is about 1MB of tiles, 1MB of PCM data and the rest for tilemaps and tiles indexes.
Tiles indexes are reduced to about 1 byte per tile with the encoding. VBlank metadata is just a few bytes per video frame, otoh some bytes are wasted to avoid crossing ROM bank boundaries (for decoding speed).
Tilemaps are usually about 500-600 bytes each. That's a lot but the tilemaps are pretty noisy in my case, hard to compress.

Hehe, at first the Z80 may look like a shitty prehistoric CPU, it really shows its strength once you start abusing it :)

Excellent work Gligli!

It's unbelievable what the Z80 can do in the right hands!

Could you please explain step by step how to user the encoder?
I read the readme and created the 256x192 bmps from my video, but I don't know how to create keyframes and what sample rate/bits to use for the wav

When I start the encoding, tiler stops responding using all my cpu time and nothing happens even if I try to encode 10 frames.

It should just work. You need to manually create keyframe markers for longer videos, in order to capture palette changes. You do this by looking for big changes in the images and create a file with the same name as the changed image, but a .kf extension. The file contents don't matter. The first image is a keyframe by default.

The other options are a bit trial and error to decide what values get you the best quality for your target file size.

Thanks Maxim, I finally succeeded!

Resurrecting old topic!

I'm just giving a try to the encoder, and it looks like I've the same problem as Bollux78. The tiler looks hanged, and it don't do anything at all. I've created the images, the keyframes, I've the sound in 16 bit wavs...

Bollux78, how did you solve that?

After some parameter tweaking on the avg tiles, etc, I'ven been able to have it finishing (with a crash), and having the animation truncated early

Hi,

I don't remember the details, but it was like Maxim said, renaming keyframes with big pallete changes to .kf files so the encoder knows how to proceed.

I was able at the end to make it work with right parameters, and some .kf. The problem is that these videos are not working on a real system, are they?

They should work on everdrive (with the last update) and on Ichigo's PCB.

Oh, then teh problem should be in the everdrive, I've an old one...

"Old" everdrive can only handle 8Mbits, new one (X7) can "nearly" handle 32Mbits (from memory, the os eats 128ko/8 banks at the end, so only 247 pages on 256 are available).

Some of my pcbs (which are not "everdrive", but flashable) can run a 32Mbits rom completely.

Ah, I know about the size limit, but i did a fairly small video (600kb) and it doesn't work on my masterdrive...

Depending on the version of your Everdrive OS you need to pad your romfile.
You can test it in Emulicious.

Thanks! I'll try!

Everdrive OS (at least the 1MB version) rounds ROM sizes down to the next lowest multiple of 64KB, if greater then 64KB. This should only affect the end of the video though.

I only get a blank screen, but the rom works in most emulators...

I can test it on RH if you want.

Here it is, also if anyone more wants to test it...

flyintro10seg.zip (439.98 KB)

Here it is ;)

IMG_20191219_083942.jpg (164.79 KB)

IMG_20191219_084256.jpg (126.85 KB)

IMG_20191219_084258_1.jpg (178.37 KB)

It looks amazing! :-D

I'll need to get one of your carts for development, it's not the first time the everdrive starts meedling with the results of my roms... Would it be possible? how do you flash them?

I've made my own little dumper/writer ^^
I've planned to sell some (source will be on github) as soon as i can compile the software for windows (for the moment it's only for macos, some trouble with the toolchain on windows)

Author	Message
gligli Joined: 14 Sep 2016 Posts: 82 Location: Vienne, France	Posted: Sat Oct 08, 2016 9:15 pm
	v6.2! ( http://sfx.gligli.free.fr/smsdev/sms_video_player_v6.2.7z ) What's new: Made player code smaller to allow for longer videos. Fixed a player tile corruption bug. Encoder reindexing process speedup. Proper 1MB Sonic CD video plus a full version.

BKK Joined: 01 May 2011 Posts: 478	Posted: Sat Oct 08, 2016 10:06 pm
BKK Joined: 01 May 2011 Posts: 478	gligli wrote v6.2! ( http://sfx.gligli.free.fr/smsdev/sms_video_player_v6.2.7z ) What's new: [*]Fixed a player tile corruption bug. This one? test bug.jpg (108.66 KB) test bug 2.jpg (122.4 KB)

gligli Joined: 14 Sep 2016 Posts: 82 Location: Vienne, France	Posted: Sat Oct 08, 2016 10:11 pm
	Looks like it yep, just recompile the ROM with the new asm and it should be good.

BKK Joined: 01 May 2011 Posts: 478	Posted: Sat Oct 08, 2016 11:03 pm
BKK Joined: 01 May 2011 Posts: 478	Thanks, I encoded four versions of this; Average tiles per frame; 4 (1.55MB): Emulicious: Fine MEKA: Has some tile bug that doesn't change between frames. Audio issues. 10 (1.85MB): Emulicious: Fine. MEKA: Audio Issues , but no tile bug (or at least very minor and not same every frame) 50 (2.76MB): Emulicious: Tile bug , frame rate collapses about a minute in and it falls apart. MEKA: Tile bug, typical audio issues, but doesn't fall apart, crashes at end of video. 100 (3.48MB): Emulicious: First few seconds black, tile bug, falls apart about a minute in. MEKA: First few seconds are black (looking at pngs first 51 frames are black, should only be 3) typical skipping audio issues, tile bug, doesn't fall apart, doesn't crash. I'm guessing that the black frames at the beginning of the 100 tile/frame video could be caused by my PC freezing during the encoding process. Some of the other issues are probably emulator related (but it's not possible to test on real hardware). I wonder if it's possible to test in SMS mode on a Mega Drive with a Mega EverDrive as it supports larger ROMs and SMS ROMs.

BKK Joined: 01 May 2011 Posts: 478	Posted: Sat Oct 08, 2016 11:35 pm
BKK Joined: 01 May 2011 Posts: 478	Recompiling with 6.2 asm fixed the tile bug and the falling apart issue in Emulicious, it now runs fine except for the first few seconds being black. Hopefully re-encoding will fix that (and I can bump up the quality a bit with having 600KB spare). That will probably take 12 hours on my weak CPU (hopefully less with the new version speed improvements) ...

gligli Joined: 14 Sep 2016 Posts: 82 Location: Vienne, France	Posted: Sun Oct 09, 2016 9:30 am
	Ah yeah, I had the full crash too in emulicious after the tile corruptions. One more thing that's fixed in this version that I didn't mention is audio/video sync. Although I don't think the bug I fixed could lead to obvious desynchronisation. With one keyframe every 100 frames, the new version shaves 1.5 hours from 1MB Sonic CD encoding.

BKK Joined: 01 May 2011 Posts: 478	Posted: Sun Oct 09, 2016 2:09 pm
BKK Joined: 01 May 2011 Posts: 478	Yeah, it's a nice speed up. I thought I had got this encoding almost perfect; Frame count: 1190 Average tiles per frame: 150 Max tiles per frame: 208 KMeans restarts: 8 Spatial / temporal smoothing: 1 Size: 3.97 MB (4,167,380 bytes) But about 75 seconds in it corrupts. It doesn't crash though. I don't know if it's because it was too large, issues with my PC during the encoding process, or an unknown bug with the software. Edit: It plays OK in SMS video tiler, so I guess that the issue is post tiling.. corrupt 1.jpg (92.36 KB) corrupt 2.jpg (137.48 KB) corrupt 3.jpg (174.6 KB)

gligli Joined: 14 Sep 2016 Posts: 82 Location: Vienne, France	Posted: Sun Oct 09, 2016 2:49 pm
	Can you send me the data files? (I found another spot in the code where there could be a similar bug as I fixed)

BKK Joined: 01 May 2011 Posts: 478	Posted: Sun Oct 09, 2016 2:53 pm
BKK Joined: 01 May 2011 Posts: 478	Sure, data.bin & index.bin; tiled.rar (2.94 MB)

gligli Joined: 14 Sep 2016 Posts: 82 Location: Vienne, France	Posted: Sun Oct 09, 2016 3:36 pm
	Ok, I found what's wrong, but I can't do anything about it: It's not possible to encode a video with more than 2MB of tiles. This one has about 2.5MB of tiles so the end of it is bad. If 100 avg tiles per frame worked, I guess it was just below 2MB and close to the max quality for this video. You'll probably have to encode the video again, sorry about that... By the way, I'd like to try to encode the Bad Apple 3D video if you don't mind.

BKK Joined: 01 May 2011 Posts: 478	Posted: Sun Oct 09, 2016 3:43 pm
BKK Joined: 01 May 2011 Posts: 478	OK, no worries, I'll re-encode it overnight. The Bad Apple video is here if you want to have a go at it. The files that I originally used aren't really useful now as I only had part of the video and discarded too many frames.

BKK Joined: 01 May 2011 Posts: 478	Posted: Mon Oct 10, 2016 11:16 pm
BKK Joined: 01 May 2011 Posts: 478	As there was a 2MB limit on total tile size, I decided to increase quality by increasing KMeans to 16 from 8. There is a noticeable improvement in quality, and size increased a bit too (from 3.48MB to 3.61MB), although as the first encoding messed up the first few seconds that could explain some of the size difference. Still, this significantly increased encoding time. A minor bug also occurs in the last few seconds, maybe increasing KMeans pushed me past the 2MB limit, but the video ended before it became a major issue. I've encoded this video so many times already that I can't be bothered to re-encode in order to try to fix the minor bug in the last few seconds. Also, I gave up on trying to fit it in to 1MB, I think that the video is just too long. Zillion anime opening (creditless) ... I recommend Emulicious. Edit: I've said it before, but I have to say again, this is incredibly impressive! zillion.rar (2.67 MB)

Maxim Site Admin Joined: 19 Oct 1999 Posts: 14959 Location: London	Posted: Tue Oct 11, 2016 7:36 am
	I think the limit is 2MB tiles, due to using 16 bits to specify a bank number (7 bits) and a tile index within it (9 bits). The frame data and index are not included in this limit, but but I'm not sure how the data is split between files. It would be trivially possible to extend past this 2MB limit at the cost of increased index size.

gligli Joined: 14 Sep 2016 Posts: 82 Location: Vienne, France	Posted: Tue Oct 11, 2016 9:08 am
	This could even be done without increasing the index size, as there's an unused byte per index entry in practice. The problem will be that additional ops will be needed in TUDoMiddle to load the carry with the additional bit, and that will alter the PlaySample timing and TilesUploadSlow / TilesUploadFast sequencing.

Maxim Site Admin Joined: 19 Oct 1999 Posts: 14959 Location: London	Posted: Tue Oct 11, 2016 9:40 am
	That's true, it messes with the timing and you have to redo everything... I've been writing pcmenc player cores recently and I've almost memorised all the opcode timings now, it's a constant grind...

gligli Joined: 14 Sep 2016 Posts: 82 Location: Vienne, France	Posted: Wed Oct 12, 2016 11:33 am
	Bad Apple 3D video: http://sfx.gligli.free.fr/smsdev/bad_apple_3d_sms.7z I had to reencode it many times to get something that looks nice but I'm quite happy with the result. It's 1400 frames, which seems to be the maximum that fits. I even had to rewrite a little bit of the code to make it fit. Now I think my next goal is to play it on the hardware, and for this I'm going to try to build a custom cartridge with a Teensy 3.2 microcontroller to do the mapping and a 4MB flash chip.

BKK Joined: 01 May 2011 Posts: 478	Posted: Wed Oct 12, 2016 2:28 pm
BKK Joined: 01 May 2011 Posts: 478	Nice work, 1400 frames is probably just about enough to fit the second half in another ROM. I might take a look at that unless you're planning on doing it. I'll be following your progress with the 4MB cartridge, I'm interested in building one myself.

gligli Joined: 14 Sep 2016 Posts: 82 Location: Vienne, France	Posted: Wed Oct 12, 2016 3:21 pm
	Nope, go ahead! The zillion video is pretty cool too, with the light phaser and all. Not an easy encode as there's a lot of scenes. I don't plan to make a proper pcb for it, so the wiring may be a little hackish though.

BKK Joined: 01 May 2011 Posts: 478	Posted: Sun Nov 27, 2016 4:41 pm
BKK Joined: 01 May 2011 Posts: 478	I needed to refresh my memory ahead of getting around to Bad Apple part 2. The Good, the Bad, and the Ugly final showdown scene: GBU Video.rar (2.25 MB)

Revo Guest	Posted: Mon Nov 28, 2016 6:18 pm
Revo Guest	Thank you for all these videos guys, these is great! I've 2 requests, if someone could do these videos I would be so happy :)

gligli Joined: 14 Sep 2016 Posts: 82 Location: Vienne, France	Posted: Wed Mar 01, 2017 4:47 pm
	Hello, I'm working on this again for the 2017 coding competition (trying new algorithms to improve quality / size ratio) and I have a question about the Z80 R register. I thought about using it as a clock source for audio samples and from my understanding of the way the 8KB XRAM chip works, I wonder if it may be possible to alter it without risking losing RAM contents? Has something like that been tried already?

TmEE Joined: 31 Oct 2007 Posts: 853 Location: Estonia, Rapla city	Posted: Wed Mar 01, 2017 4:52 pm Last edited by TmEE on Wed Mar 01, 2017 5:20 pm; edited 2 times in total
	Refresh is tied to Z80 refresh output and there's no way you can stop it from happening. R register contents have no effect on how refresh works on SMS. EDIT: Seems the chips actually use address pins for refresh row in non power-down refresh mode. I will see which refresh method is being used, I'll report back soon. EDIT2: Seems only power-down refresh is used, !RFSH is only active when !CE is not active, so external address input is not used by the chip. You can mess with R register freely. EDIT3: Nnnnnnope, self refresh requires very long pulses to work and those don't happen on SMS. There's periodic !CE pulses with !RFSH pulses between them. R register is not safe to touch.

gligli Joined: 14 Sep 2016 Posts: 82 Location: Vienne, France	Posted: Wed Mar 01, 2017 4:58 pm
	Actually I want to use the fact that it's incremented each instruction as a clock source, but also need to reset it at times. The XRAM chip datasheet says it goes into self refresh when the /RFSH input is high but I wonder if it can switch back an forth between self refresh and external refresh at will. Edit: Yay, thank you! Sample playback will be way easier if I can get this trick to work fine :)

TmEE Joined: 31 Oct 2007 Posts: 853 Location: Estonia, Rapla city	Posted: Wed Mar 01, 2017 5:21 pm
	Unfortunately it is not the case, I looked at timings more closely and self refresh requires 40µs pulse to work, and that's not happening in SMS. External refresh is being used instead, so you cannot mess with R unfortunately...

gligli Joined: 14 Sep 2016 Posts: 82 Location: Vienne, France	Posted: Wed Mar 01, 2017 5:29 pm
	Wait, so you mean there would just need to completely disable refreshes (ie /RFSH always high) from the Z80 to actually get the chip to self refresh? I don't think it is a problem if I take it into account and roughly count instructions to reset R before it reaches 128.

TmEE Joined: 31 Oct 2007 Posts: 853 Location: Estonia, Rapla city	Posted: Wed Mar 01, 2017 5:40 pm
	Z80 will always pulse !RFSH during M1 cycle and R register is incremented and put an the bus, XRAM will then use the content on the bus as refresh address. If you continuously spammed R with 0 then only the first row gets refreshed in the XRAM and rest of the contents get corrupt over time. Now the good news is that refresh cycles happen very often (every instruction) and typical time to refresh all rows is 16ms (which is just around one frame) and that means all rows get refreshed multiple times per frame, so I don't think occasional resetting of R will cause any ill effects. Self refresh mode which does not use externally supplied refresh address requires !RFSH to go low for 40µs which is around 2/3 of a scanline, and that never happens on SMS.

gligli Joined: 14 Sep 2016 Posts: 82 Location: Vienne, France	Posted: Wed Mar 01, 2017 5:52 pm
	Ok, I had misunderstood the way refresh works. An occasional reset is better than nothing, and worst case I can still use R without setting it, it would just cost more ops per audio sample. In any case, I will try it on my SMS and see what I can get to work, thanks again!

gligli Joined: 14 Sep 2016 Posts: 82 Location: Vienne, France	Posted: Thu Mar 30, 2017 1:12 pm
	I was not completely satisfied by my first encoding of Bad Apple 3D, the colors were pretty washed out and of course there was only half of the video. I had a few ideas in mind to make a better version: - Use the two SMS hardware palettes, to up the max colors on screen. - Use horizontal and vertical mirrors on the tiles to improve size/quality ratio. - Have a global tile repository. Prior versions had a per keyframe repository which made tiles a lot less reusable. - Make it compatible with 60Hz hardware. So, first I had to rewrite the encoder because the encoding pipeline would be different. v6 was: load -> keyframe tiling -> frame tiling -> dithering -> reindex -> save. Tiling was done on truecolor tiles, which wouldn't now work if the goal was to make tiles reusable no matter the palette. I came up with this pipeline for v7: load -> dithering -> make unique -> global tiling -> frame tiling -> smoothing -> reindex -> save. - I used a much better dithering algorithm from http://bisqwit.iki.fi/story/howto/dither/jy/ which improved color even with a single palette. This process also tries to find the best keyframes palettes. - "make unique" is just a simple algorithm that merges strictly similar tiles to make tiling faster. - As tiling is now done on colorless 4bit tiles, KMeans wouldn't work (color indices are not continuous, so eg. indices 1 and 2 are the same distance as indices 1 and 15). In statistics, we would say we have 64 nominal factors per tile. KModes is what I used then as it's a bit like KMeans except for nominal data. Global tiling uses KModes to reduce the overall number of tiles to a fixed number and frame tiling uses DCT (discrete cosine tranform) and KModes to find the best matching tiles for the frame, accounting for palettes and H/V mirrors. Especially global tiling is really time consuming as there is often more than 1 million tiles to reduce so I used SIMD optimisation to speed it up. - Smoothing uses DCT to tag tiles that are roughly the same as in preceding frame, so that the tilemap can only be updated partially each video frame. - Save compresses tiles indexes and tilemaps into byte command streams (basically encoding index differences for tiles indexes and dictionnary with skips and repeats for TileMap). I then wrote the player naively and it was 2x too slow, even without sound. The main optimisations I made were: - Use jump tables to handle tiles indexes and tilemaps byte command streams. They are cheap on the Z80 and aligning jump tables and jump offsets on 256 bytes and storing only high address bytes makes them super fast (eg. code table). - Use 256 bytes aligned lookup tables, so eg you load H with the upper byte of the LUT address, and L with the byte to be looked up, the load is then a simple LD A, (HL). - Remove as many conditional branches as possible and use the main jump table instead. The best of those was to make the encoder handle the switch between fast and slow VRAM tiles upload depending on VBlank or active display by counting the Z80 cycles and the corresponding scanlines at encode time. - Systematically abuse the stack as a general purpose data pointer and consequently use no interrupts. Sound was another huge problem. Counting cycles and adding PlaySample macros every X CPU cycles wasn't possible with jump tables as you don't have a fixed execution flow anymore. My first idea was to use the Z80 R register to keep track of time but it didn't work well (it's incremented each op, but ops timings varies widely on the Z80) and was not cheap in CPU time. I finally came up with a macro that was able to handle "partial" samples. That is, use some kind of fixed point math for the sample index so you can use it in any function and just pass any cycle count as parameter (code). You end up calling it more often than a simple PlaySample but it's very good at keeping a low sample jitter no matter the code. Anyway I think the end result is better than v6 and probably more maintainable too. In the end I didn't do 60Hz compatibility but at least it's doable now as tilemaps updates are done in both active display and VBlank. The full source code is available online at: https://github.com/gligli/tiler and encoder binaries are included in http://www.smspower.org/uploads/Homebrew/BadApple3DFullLength-SMS-7.00.zip Phew, that was a lot of text ^^

psidum Joined: 01 Jan 2014 Posts: 331	Posted: Fri Mar 31, 2017 3:04 am
psidum Joined: 01 Jan 2014 Posts: 331	Thanks man! Using macros with step / skew to time sample updates is incredibly creative. How many unique tiles in total were used? I have to imagine the new tile streaming approach introduced quite a bit of meta data not to mention cpu cost plus you have more details nametable data (flips and palette) and yet you fit twice as much video on ROM! Your optimizations are smart, the solution is total coder porn. I love it!

gligli Joined: 14 Sep 2016 Posts: 82 Location: Vienne, France	Posted: Fri Mar 31, 2017 8:49 am
	Actually, the step / skew stuff is just bare bones direct digital synthesis. I knew this because I wrote a lot of sound synthesizer code for previous projects. About 30000 tiles I think. Overall, there is about 1MB of tiles, 1MB of PCM data and the rest for tilemaps and tiles indexes. Tiles indexes are reduced to about 1 byte per tile with the encoding. VBlank metadata is just a few bytes per video frame, otoh some bytes are wasted to avoid crossing ROM bank boundaries (for decoding speed). Tilemaps are usually about 500-600 bytes each. That's a lot but the tilemaps are pretty noisy in my case, hard to compress. Hehe, at first the Z80 may look like a shitty prehistoric CPU, it really shows its strength once you start abusing it :)

bollux78 Joined: 17 Apr 2017 Posts: 10	Posted: Thu Aug 10, 2017 9:11 pm
bollux78 Joined: 17 Apr 2017 Posts: 10	Excellent work Gligli! It's unbelievable what the Z80 can do in the right hands! Could you please explain step by step how to user the encoder? I read the readme and created the 256x192 bmps from my video, but I don't know how to create keyframes and what sample rate/bits to use for the wav When I start the encoding, tiler stops responding using all my cpu time and nothing happens even if I try to encode 10 frames.

Maxim Site Admin Joined: 19 Oct 1999 Posts: 14959 Location: London	Posted: Fri Aug 11, 2017 8:01 am
	It should just work. You need to manually create keyframe markers for longer videos, in order to capture palette changes. You do this by looking for big changes in the images and create a file with the same name as the changed image, but a .kf extension. The file contents don't matter. The first image is a keyframe by default. The other options are a bit trial and error to decide what values get you the best quality for your target file size.

bollux78 Joined: 17 Apr 2017 Posts: 10	Posted: Wed Aug 16, 2017 2:51 am
bollux78 Joined: 17 Apr 2017 Posts: 10	Thanks Maxim, I finally succeeded!

kusfo Joined: 29 Mar 2012 Posts: 900 Location: Spain	Posted: Tue Dec 10, 2019 5:19 pm
kusfo Joined: 29 Mar 2012 Posts: 900 Location: Spain	Resurrecting old topic! I'm just giving a try to the encoder, and it looks like I've the same problem as Bollux78. The tiler looks hanged, and it don't do anything at all. I've created the images, the keyframes, I've the sound in 16 bit wavs... Bollux78, how did you solve that?

kusfo Joined: 29 Mar 2012 Posts: 900 Location: Spain	Posted: Tue Dec 10, 2019 10:22 pm
kusfo Joined: 29 Mar 2012 Posts: 900 Location: Spain	After some parameter tweaking on the avg tiles, etc, I'ven been able to have it finishing (with a crash), and having the animation truncated early

bollux78 Joined: 17 Apr 2017 Posts: 10	Posted: Wed Dec 18, 2019 11:15 am
bollux78 Joined: 17 Apr 2017 Posts: 10	Hi, I don't remember the details, but it was like Maxim said, renaming keyframes with big pallete changes to .kf files so the encoder knows how to proceed.

kusfo Joined: 29 Mar 2012 Posts: 900 Location: Spain	Posted: Wed Dec 18, 2019 11:28 am
kusfo Joined: 29 Mar 2012 Posts: 900 Location: Spain	I was able at the end to make it work with right parameters, and some .kf. The problem is that these videos are not working on a real system, are they?

Revo Guest	Posted: Wed Dec 18, 2019 12:52 pm
Revo Guest	They should work on everdrive (with the last update) and on Ichigo's PCB.

kusfo Joined: 29 Mar 2012 Posts: 900 Location: Spain	Posted: Wed Dec 18, 2019 1:19 pm
kusfo Joined: 29 Mar 2012 Posts: 900 Location: Spain	Oh, then teh problem should be in the everdrive, I've an old one...

ichigobankai Joined: 04 Jul 2010 Posts: 546 Location: Angers, France	Posted: Wed Dec 18, 2019 1:36 pm
	"Old" everdrive can only handle 8Mbits, new one (X7) can "nearly" handle 32Mbits (from memory, the os eats 128ko/8 banks at the end, so only 247 pages on 256 are available). Some of my pcbs (which are not "everdrive", but flashable) can run a 32Mbits rom completely.

kusfo Joined: 29 Mar 2012 Posts: 900 Location: Spain	Posted: Wed Dec 18, 2019 1:50 pm
kusfo Joined: 29 Mar 2012 Posts: 900 Location: Spain	Ah, I know about the size limit, but i did a fairly small video (600kb) and it doesn't work on my masterdrive...

Calindro Joined: 14 Apr 2013 Posts: 637	Posted: Wed Dec 18, 2019 1:53 pm
Calindro Joined: 14 Apr 2013 Posts: 637	Depending on the version of your Everdrive OS you need to pad your romfile. You can test it in Emulicious.

kusfo Joined: 29 Mar 2012 Posts: 900 Location: Spain	Posted: Wed Dec 18, 2019 2:30 pm
kusfo Joined: 29 Mar 2012 Posts: 900 Location: Spain	Thanks! I'll try!

Maxim Site Admin Joined: 19 Oct 1999 Posts: 14959 Location: London	Posted: Wed Dec 18, 2019 7:16 pm
	Everdrive OS (at least the 1MB version) rounds ROM sizes down to the next lowest multiple of 64KB, if greater then 64KB. This should only affect the end of the video though.

kusfo Joined: 29 Mar 2012 Posts: 900 Location: Spain	Posted: Wed Dec 18, 2019 7:19 pm
kusfo Joined: 29 Mar 2012 Posts: 900 Location: Spain	I only get a blank screen, but the rom works in most emulators...

ichigobankai Joined: 04 Jul 2010 Posts: 546 Location: Angers, France	Posted: Wed Dec 18, 2019 7:32 pm
	I can test it on RH if you want.

kusfo Joined: 29 Mar 2012 Posts: 900 Location: Spain	Posted: Wed Dec 18, 2019 10:33 pm
kusfo Joined: 29 Mar 2012 Posts: 900 Location: Spain	Here it is, also if anyone more wants to test it... flyintro10seg.zip (439.98 KB)

ichigobankai Joined: 04 Jul 2010 Posts: 546 Location: Angers, France	Posted: Thu Dec 19, 2019 8:07 am
	Here it is ;) IMG_20191219_083942.jpg (164.79 KB) IMG_20191219_084256.jpg (126.85 KB) IMG_20191219_084258_1.jpg (178.37 KB)

kusfo Joined: 29 Mar 2012 Posts: 900 Location: Spain	Posted: Thu Dec 19, 2019 8:17 am
kusfo Joined: 29 Mar 2012 Posts: 900 Location: Spain	It looks amazing! :-D I'll need to get one of your carts for development, it's not the first time the everdrive starts meedling with the results of my roms... Would it be possible? how do you flash them?

ichigobankai Joined: 04 Jul 2010 Posts: 546 Location: Angers, France	Posted: Thu Dec 19, 2019 8:21 am
	I've made my own little dumper/writer ^^ I've planned to sell some (source will be on github) as soon as i can compile the software for windows (for the moment it's only for macos, some trouble with the toolchain on windows)

Forums

View topic - Fullscreen video playback on Sega Master System