All graphics on the Master System are built up from 8×8 pixel tiles.
Each pixel is a palette index from 0 to 15, i.e. 4 bits.
So a whole tile is a stack of bits:
The tile data is in a planar format, split by tile row. That means that the first byte contains the least significant bit, bit 0, of each pixel in the top row of the tile. The second byte contains bit 1 of each pixel, the third bit 2, and the fourth bit 3. Thus the top eight pixels are represented by the first four bytes of data, split by “bitplane”. The process is repeated for consecutive rows of the tile, producing 32 bytes total.
In order to convert images to this format, it is sensible to pre-process it before inclusion in the ROM. You can make your own tools to do this, or (if you use Windows) you can use BMP2Tile. This will take a range of common image formats, such as PNG, and convert it to data in a format the SMS understands.
How many tiles can we have?
In the most typical VRAM layout, 14KB of the total 16KB is available for tiles; that is enough space for 448 tiles. (With some tricks you can get space for a few more.)
Don’t forget the palette
All of your tile data must be generated with attention to the palette that will be applied to it. The tile data does not contain the palette information. Try to use a program that offers close control over the palette, and won’t re-order it.
Tile data takes up a relatively large amount of space - after all, to fill 14KB of VRAM will take 14KB of data. However, it is also typically quite compressible. There are a few compression schemes available through BMP2Tile, or you can make your own.
Game Gear differences
There aren’t any - it works exactly the same way.