Forums

Sega Master System / Mark III / Game Gear
SG-1000 / SC-3000 / SF-7000 / OMV
Home - Forums - Games - Scans - Maps - Cheats - Credits
Music - Videos - Development - Hacks - Translations - Homebrew

View topic - Fullscreen video playback on Sega Master System

Reply to topic Goto page Previous  1, 2, 3
Author Message
  • Joined: 14 Sep 2016
  • Posts: 71
  • Location: Lyon, France
Reply with quote
Post Posted: Sat Oct 08, 2016 9:15 pm
v6.2! ( http://sfx.gligli.free.fr/smsdev/sms_video_player_v6.2.7z )

What's new:
  • Made player code smaller to allow for longer videos.
  • Fixed a player tile corruption bug.
  • Encoder reindexing process speedup.
  • Proper 1MB Sonic CD video plus a full version.
  View user's profile Send private message
  • Joined: 01 May 2011
  • Posts: 371
Reply with quote
Post Posted: Sat Oct 08, 2016 10:06 pm
gligli wrote
v6.2! ( http://sfx.gligli.free.fr/smsdev/sms_video_player_v6.2.7z )

What's new:
[*]Fixed a player tile corruption bug.


This one?
test bug.jpg (108.66 KB)
test bug.jpg
test bug 2.jpg (122.4 KB)
test bug 2.jpg

  View user's profile Send private message
  • Joined: 14 Sep 2016
  • Posts: 71
  • Location: Lyon, France
Reply with quote
Post Posted: Sat Oct 08, 2016 10:11 pm
Looks like it yep, just recompile the ROM with the new asm and it should be good.
  View user's profile Send private message
  • Joined: 01 May 2011
  • Posts: 371
Reply with quote
Post Posted: Sat Oct 08, 2016 11:03 pm
Thanks, I encoded four versions of this;

Average tiles per frame;

4 (1.55MB):

Emulicious: Fine

MEKA: Has some tile bug that doesn't change between frames. Audio issues.

10 (1.85MB):

Emulicious: Fine.

MEKA: Audio Issues , but no tile bug (or at least very minor and not same every frame)

50 (2.76MB):

Emulicious: Tile bug , frame rate collapses about a minute in and it falls apart.

MEKA: Tile bug, typical audio issues, but doesn't fall apart, crashes at end of video.

100 (3.48MB):

Emulicious: First few seconds black, tile bug, falls apart about a minute in.

MEKA: First few seconds are black (looking at pngs first 51 frames are black, should only be 3) typical skipping audio issues, tile bug, doesn't fall apart, doesn't crash.

I'm guessing that the black frames at the beginning of the 100 tile/frame video could be caused by my PC freezing during the encoding process. Some of the other issues are probably emulator related (but it's not possible to test on real hardware).

I wonder if it's possible to test in SMS mode on a Mega Drive with a Mega EverDrive as it supports larger ROMs and SMS ROMs.
  View user's profile Send private message
  • Joined: 01 May 2011
  • Posts: 371
Reply with quote
Post Posted: Sat Oct 08, 2016 11:35 pm
Recompiling with 6.2 asm fixed the tile bug and the falling apart issue in Emulicious, it now runs fine except for the first few seconds being black. Hopefully re-encoding will fix that (and I can bump up the quality a bit with having 600KB spare).

That will probably take 12 hours on my weak CPU (hopefully less with the new version speed improvements) ...
  View user's profile Send private message
  • Joined: 14 Sep 2016
  • Posts: 71
  • Location: Lyon, France
Reply with quote
Post Posted: Sun Oct 09, 2016 9:30 am
Ah yeah, I had the full crash too in emulicious after the tile corruptions.
One more thing that's fixed in this version that I didn't mention is audio/video sync. Although I don't think the bug I fixed could lead to obvious desynchronisation.

With one keyframe every 100 frames, the new version shaves 1.5 hours from 1MB Sonic CD encoding.
  View user's profile Send private message
  • Joined: 01 May 2011
  • Posts: 371
Reply with quote
Post Posted: Sun Oct 09, 2016 2:09 pm
Yeah, it's a nice speed up.

I thought I had got this encoding almost perfect;

Frame count: 1190
Average tiles per frame: 150
Max tiles per frame: 208
KMeans restarts: 8
Spatial / temporal smoothing: 1

Size: 3.97 MB (4,167,380 bytes)

But about 75 seconds in it corrupts. It doesn't crash though. I don't know if it's because it was too large, issues with my PC during the encoding process, or an unknown bug with the software.

Edit: It plays OK in SMS video tiler, so I guess that the issue is post tiling..
corrupt 1.jpg (92.36 KB)
corrupt 1.jpg
corrupt 2.jpg (137.48 KB)
corrupt 2.jpg
corrupt 3.jpg (174.6 KB)
corrupt 3.jpg

  View user's profile Send private message
  • Joined: 14 Sep 2016
  • Posts: 71
  • Location: Lyon, France
Reply with quote
Post Posted: Sun Oct 09, 2016 2:49 pm
Can you send me the data files? (I found another spot in the code where there could be a similar bug as I fixed)
  View user's profile Send private message
  • Joined: 01 May 2011
  • Posts: 371
Reply with quote
Post Posted: Sun Oct 09, 2016 2:53 pm
Sure, data.bin & index.bin;
tiled.rar (2.94 MB)

  View user's profile Send private message
  • Joined: 14 Sep 2016
  • Posts: 71
  • Location: Lyon, France
Reply with quote
Post Posted: Sun Oct 09, 2016 3:36 pm
Ok, I found what's wrong, but I can't do anything about it: It's not possible to encode a video with more than 2MB of tiles.

This one has about 2.5MB of tiles so the end of it is bad.
If 100 avg tiles per frame worked, I guess it was just below 2MB and close to the max quality for this video.

You'll probably have to encode the video again, sorry about that...

By the way, I'd like to try to encode the Bad Apple 3D video if you don't mind.
  View user's profile Send private message
  • Joined: 01 May 2011
  • Posts: 371
Reply with quote
Post Posted: Sun Oct 09, 2016 3:43 pm
OK, no worries, I'll re-encode it overnight.

The Bad Apple video is here if you want to have a go at it. The files that I originally used aren't really useful now as I only had part of the video and discarded too many frames.
  View user's profile Send private message
  • Joined: 01 May 2011
  • Posts: 371
Reply with quote
Post Posted: Mon Oct 10, 2016 11:16 pm
As there was a 2MB limit on total tile size, I decided to increase quality by increasing KMeans to 16 from 8. There is a noticeable improvement in quality, and size increased a bit too (from 3.48MB to 3.61MB), although as the first encoding messed up the first few seconds that could explain some of the size difference. Still, this significantly increased encoding time.

A minor bug also occurs in the last few seconds, maybe increasing KMeans pushed me past the 2MB limit, but the video ended before it became a major issue. I've encoded this video so many times already that I can't be bothered to re-encode in order to try to fix the minor bug in the last few seconds.

Also, I gave up on trying to fit it in to 1MB, I think that the video is just too long.

Zillion anime opening (creditless) ... I recommend Emulicious.

Edit: I've said it before, but I have to say again, this is incredibly impressive!
zillion.rar (2.67 MB)

  View user's profile Send private message
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 11875
  • Location: London
Reply with quote
Post Posted: Tue Oct 11, 2016 7:36 am
I think the limit is 2MB tiles, due to using 16 bits to specify a bank number (7 bits) and a tile index within it (9 bits). The frame data and index are not included in this limit, but but I'm not sure how the data is split between files. It would be trivially possible to extend past this 2MB limit at the cost of increased index size.
  View user's profile Send private message Visit poster's website
  • Joined: 14 Sep 2016
  • Posts: 71
  • Location: Lyon, France
Reply with quote
Post Posted: Tue Oct 11, 2016 9:08 am
This could even be done without increasing the index size, as there's an unused byte per index entry in practice.
The problem will be that additional ops will be needed in TUDoMiddle to load the carry with the additional bit, and that will alter the PlaySample timing and TilesUploadSlow / TilesUploadFast sequencing.
  View user's profile Send private message
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 11875
  • Location: London
Reply with quote
Post Posted: Tue Oct 11, 2016 9:40 am
That's true, it messes with the timing and you have to redo everything... I've been writing pcmenc player cores recently and I've almost memorised all the opcode timings now, it's a constant grind...
  View user's profile Send private message Visit poster's website
  • Joined: 14 Sep 2016
  • Posts: 71
  • Location: Lyon, France
Reply with quote
Post Posted: Wed Oct 12, 2016 11:33 am
Bad Apple 3D video: http://sfx.gligli.free.fr/smsdev/bad_apple_3d_sms.7z

I had to reencode it many times to get something that looks nice but I'm quite happy with the result.
It's 1400 frames, which seems to be the maximum that fits. I even had to rewrite a little bit of the code to make it fit.

Now I think my next goal is to play it on the hardware, and for this I'm going to try to build a custom cartridge with a Teensy 3.2 microcontroller to do the mapping and a 4MB flash chip.
  View user's profile Send private message
  • Joined: 01 May 2011
  • Posts: 371
Reply with quote
Post Posted: Wed Oct 12, 2016 2:28 pm
Nice work, 1400 frames is probably just about enough to fit the second half in another ROM. I might take a look at that unless you're planning on doing it.

I'll be following your progress with the 4MB cartridge, I'm interested in building one myself.
  View user's profile Send private message
  • Joined: 14 Sep 2016
  • Posts: 71
  • Location: Lyon, France
Reply with quote
Post Posted: Wed Oct 12, 2016 3:21 pm
Nope, go ahead!
The zillion video is pretty cool too, with the light phaser and all. Not an easy encode as there's a lot of scenes.

I don't plan to make a proper pcb for it, so the wiring may be a little hackish though.
  View user's profile Send private message
  • Joined: 01 May 2011
  • Posts: 371
Reply with quote
Post Posted: Sun Nov 27, 2016 4:41 pm
I needed to refresh my memory ahead of getting around to Bad Apple part 2.

The Good, the Bad, and the Ugly final showdown scene:
GBU Video.rar (2.25 MB)

  View user's profile Send private message
  • Joined: 26 Jan 2014
  • Posts: 414
Reply with quote
Post Posted: Mon Nov 28, 2016 6:18 pm
Thank you for all these videos guys, these is great!

I've 2 requests, if someone could do these videos I would be so happy :)
https://www.youtube.com/watch?v=ZZ5LpwO-An4
https://www.youtube.com/watch?v=1wnE4vF9CQ4
  View user's profile Send private message Visit poster's website
  • Joined: 14 Sep 2016
  • Posts: 71
  • Location: Lyon, France
Reply with quote
Post Posted: Wed Mar 01, 2017 4:47 pm
Hello, I'm working on this again for the 2017 coding competition (trying new algorithms to improve quality / size ratio) and I have a question about the Z80 R register.
I thought about using it as a clock source for audio samples and from my understanding of the way the 8KB XRAM chip works, I wonder if it may be possible to alter it without risking losing RAM contents? Has something like that been tried already?
  View user's profile Send private message
  • Joined: 31 Oct 2007
  • Posts: 808
  • Location: Estonia, Rapla city
Reply with quote
Post Posted: Wed Mar 01, 2017 4:52 pm
Last edited by TmEE on Wed Mar 01, 2017 5:20 pm; edited 2 times in total
Refresh is tied to Z80 refresh output and there's no way you can stop it from happening. R register contents have no effect on how refresh works on SMS.
EDIT: Seems the chips actually use address pins for refresh row in non power-down refresh mode. I will see which refresh method is being used, I'll report back soon.
EDIT2:
Seems only power-down refresh is used, !RFSH is only active when !CE is not active, so external address input is not used by the chip. You can mess with R register freely.
EDIT3: Nnnnnnope, self refresh requires very long pulses to work and those don't happen on SMS. There's periodic !CE pulses with !RFSH pulses between them. R register is not safe to touch.
  View user's profile Send private message Visit poster's website
  • Joined: 14 Sep 2016
  • Posts: 71
  • Location: Lyon, France
Reply with quote
Post Posted: Wed Mar 01, 2017 4:58 pm
Actually I want to use the fact that it's incremented each instruction as a clock source, but also need to reset it at times.
The XRAM chip datasheet says it goes into self refresh when the /RFSH input is high but I wonder if it can switch back an forth between self refresh and external refresh at will.
Edit:
Yay, thank you! Sample playback will be way easier if I can get this trick to work fine :)
  View user's profile Send private message
  • Joined: 31 Oct 2007
  • Posts: 808
  • Location: Estonia, Rapla city
Reply with quote
Post Posted: Wed Mar 01, 2017 5:21 pm
Unfortunately it is not the case, I looked at timings more closely and self refresh requires 40µs pulse to work, and that's not happening in SMS. External refresh is being used instead, so you cannot mess with R unfortunately...
  View user's profile Send private message Visit poster's website
  • Joined: 14 Sep 2016
  • Posts: 71
  • Location: Lyon, France
Reply with quote
Post Posted: Wed Mar 01, 2017 5:29 pm
Wait, so you mean there would just need to completely disable refreshes (ie /RFSH always high) from the Z80 to actually get the chip to self refresh?
I don't think it is a problem if I take it into account and roughly count instructions to reset R before it reaches 128.
  View user's profile Send private message
  • Joined: 31 Oct 2007
  • Posts: 808
  • Location: Estonia, Rapla city
Reply with quote
Post Posted: Wed Mar 01, 2017 5:40 pm
Z80 will always pulse !RFSH during M1 cycle and R register is incremented and put an the bus, XRAM will then use the content on the bus as refresh address. If you continuously spammed R with 0 then only the first row gets refreshed in the XRAM and rest of the contents get corrupt over time. Now the good news is that refresh cycles happen very often (every instruction) and typical time to refresh all rows is 16ms (which is just around one frame) and that means all rows get refreshed multiple times per frame, so I don't think occasional resetting of R will cause any ill effects.

Self refresh mode which does not use externally supplied refresh address requires !RFSH to go low for 40µs which is around 2/3 of a scanline, and that never happens on SMS.
  View user's profile Send private message Visit poster's website
  • Joined: 14 Sep 2016
  • Posts: 71
  • Location: Lyon, France
Reply with quote
Post Posted: Wed Mar 01, 2017 5:52 pm
Ok, I had misunderstood the way refresh works.
An occasional reset is better than nothing, and worst case I can still use R without setting it, it would just cost more ops per audio sample.
In any case, I will try it on my SMS and see what I can get to work, thanks again!
  View user's profile Send private message
  • Joined: 14 Sep 2016
  • Posts: 71
  • Location: Lyon, France
Reply with quote
Post Posted: Thu Mar 30, 2017 1:12 pm
I was not completely satisfied by my first encoding of Bad Apple 3D, the colors were pretty washed out and of course there was only half of the video.

I had a few ideas in mind to make a better version:
- Use the two SMS hardware palettes, to up the max colors on screen.
- Use horizontal and vertical mirrors on the tiles to improve size/quality ratio.
- Have a global tile repository. Prior versions had a per keyframe repository which made tiles a lot less reusable.
- Make it compatible with 60Hz hardware.

So, first I had to rewrite the encoder because the encoding pipeline would be different.

v6 was: load -> keyframe tiling -> frame tiling -> dithering -> reindex -> save.
Tiling was done on truecolor tiles, which wouldn't now work if the goal was to make tiles reusable no matter the palette.

I came up with this pipeline for v7: load -> dithering -> make unique -> global tiling -> frame tiling -> smoothing -> reindex -> save.
- I used a much better dithering algorithm from http://bisqwit.iki.fi/story/howto/dither/jy/ which improved color even with a single palette. This process also tries to find the best keyframes palettes.
- "make unique" is just a simple algorithm that merges strictly similar tiles to make tiling faster.
- As tiling is now done on colorless 4bit tiles, KMeans wouldn't work (color indices are not continuous, so eg. indices 1 and 2 are the same distance as indices 1 and 15). In statistics, we would say we have 64 nominal factors per tile. KModes is what I used then as it's a bit like KMeans except for nominal data.
Global tiling uses KModes to reduce the overall number of tiles to a fixed number and frame tiling uses DCT (discrete cosine tranform) and KModes to find the best matching tiles for the frame, accounting for palettes and H/V mirrors.
Especially global tiling is really time consuming as there is often more than 1 million tiles to reduce so I used SIMD optimisation to speed it up.
- Smoothing uses DCT to tag tiles that are roughly the same as in preceding frame, so that the tilemap can only be updated partially each video frame.
- Save compresses tiles indexes and tilemaps into byte command streams (basically encoding index differences for tiles indexes and dictionnary with skips and repeats for TileMap).

I then wrote the player naively and it was 2x too slow, even without sound. The main optimisations I made were:
- Use jump tables to handle tiles indexes and tilemaps byte command streams. They are cheap on the Z80 and aligning jump tables and jump offsets on 256 bytes and storing only high address bytes makes them super fast (eg. code table).
- Use 256 bytes aligned lookup tables, so eg you load H with the upper byte of the LUT address, and L with the byte to be looked up, the load is then a simple LD A, (HL).
- Remove as many conditional branches as possible and use the main jump table instead. The best of those was to make the encoder handle the switch between fast and slow VRAM tiles upload depending on VBlank or active display by counting the Z80 cycles and the corresponding scanlines at encode time.
- Systematically abuse the stack as a general purpose data pointer and consequently use no interrupts.

Sound was another huge problem. Counting cycles and adding PlaySample macros every X CPU cycles wasn't possible with jump tables as you don't have a fixed execution flow anymore.
My first idea was to use the Z80 R register to keep track of time but it didn't work well (it's incremented each op, but ops timings varies widely on the Z80) and was not cheap in CPU time.
I finally came up with a macro that was able to handle "partial" samples. That is, use some kind of fixed point math for the sample index so you can use it in any function and just pass any cycle count as parameter (code). You end up calling it more often than a simple PlaySample but it's very good at keeping a low sample jitter no matter the code.

Anyway I think the end result is better than v6 and probably more maintainable too. In the end I didn't do 60Hz compatibility but at least it's doable now as tilemaps updates are done in both active display and VBlank.

The full source code is available online at: https://github.com/gligli/tiler and encoder binaries are included in http://www.smspower.org/uploads/Homebrew/BadApple3DFullLength-SMS-7.00.zip

Phew, that was a lot of text ^^
  View user's profile Send private message
  • Joined: 01 Jan 2014
  • Posts: 294
Reply with quote
Post Posted: Fri Mar 31, 2017 3:04 am
Thanks man!

Using macros with step / skew to time sample updates is incredibly creative.

How many unique tiles in total were used? I have to imagine the new tile streaming approach introduced quite a bit of meta data not to mention cpu cost plus you have more details nametable data (flips and palette) and yet you fit twice as much video on ROM!

Your optimizations are smart, the solution is total coder porn. I love it!
  View user's profile Send private message
  • Joined: 14 Sep 2016
  • Posts: 71
  • Location: Lyon, France
Reply with quote
Post Posted: Fri Mar 31, 2017 8:49 am
Actually, the step / skew stuff is just bare bones direct digital synthesis. I knew this because I wrote a lot of sound synthesizer code for previous projects.

About 30000 tiles I think. Overall, there is about 1MB of tiles, 1MB of PCM data and the rest for tilemaps and tiles indexes.
Tiles indexes are reduced to about 1 byte per tile with the encoding. VBlank metadata is just a few bytes per video frame, otoh some bytes are wasted to avoid crossing ROM bank boundaries (for decoding speed).
Tilemaps are usually about 500-600 bytes each. That's a lot but the tilemaps are pretty noisy in my case, hard to compress.

Hehe, at first the Z80 may look like a shitty prehistoric CPU, it really shows its strength once you start abusing it :)
  View user's profile Send private message
  • Joined: 17 Apr 2017
  • Posts: 2
Reply with quote
Post Posted: Thu Aug 10, 2017 9:11 pm
Excellent work Gligli!

It's unbelievable what the Z80 can do in the right hands!

Could you please explain step by step how to user the encoder?
I read the readme and created the 256x192 bmps from my video, but I don't know how to create keyframes and what sample rate/bits to use for the wav

When I start the encoding, tiler stops responding using all my cpu time and nothing happens even if I try to encode 10 frames.
  View user's profile Send private message
  • Site Admin
  • Joined: 19 Oct 1999
  • Posts: 11875
  • Location: London
Reply with quote
Post Posted: Fri Aug 11, 2017 8:01 am
It should just work. You need to manually create keyframe markers for longer videos, in order to capture palette changes. You do this by looking for big changes in the images and create a file with the same name as the changed image, but a .kf extension. The file contents don't matter. The first image is a keyframe by default.

The other options are a bit trial and error to decide what values get you the best quality for your target file size.
  View user's profile Send private message Visit poster's website
  • Joined: 17 Apr 2017
  • Posts: 2
Reply with quote
Post Posted: Wed Aug 16, 2017 2:51 am
Thanks Maxim, I finally succeeded!
  View user's profile Send private message
Reply to topic Goto page Previous  1, 2, 3



Back to the top of this page

Back to SMS Power!