::|CONTENTS
- When you should not use XM
- When you should not use MOD
- The math behind pattern packing
- What everyone SHOULD know
- How the hell do I pack samples
With the release of
munch.py, this article might look useless. But that's fundamentally for
compressing samples and
ruining the file structure. It does
some pattern optimisation, but not much.
This article mostly focuses on pattern optimisation.
When you should not use XM
This is a note to you XM fanatics out there, if you're ever entering a size-limited compo. Note that it's usually OK if you fail ONE (except for this first one...)
DO NOT USE LOTS OF MOSTLY EMPTY CHANNELS. That stuff would fit better even in .s3m. .xm packing requires at least one byte per cell, whereas both .s3m and .it only require a minimum of one byte per row.
Case in point: michu's 8k "rose.xm" from Winter Chip 6 has 24 channels and packs down to 4535 bytes
when simply resaved in Schism as an .it file. After modifying the pattern data, it then reaches 4407 bytes. My (iamgreaser's) record for munching that is 2498 bytes (it uses IT214 sample compression, but this only saves one byte).
If you're not using instrument features heavily (note: NOT JUST A SIMPLE SLIDEDOWN), .it has a sample mode which doesn't require you to use instruments. Apparently BoobieSqueezer can also help in this case if you're doing this. Otherwise, if you're using instruments, bear in mind that .it instruments are
huge (547 bytes each) compared with .xm's which are roughly 260 bytes (this includes a single sample - .it's sample headers are 80 bytes).
If you insist on having large samples, IT214/IT215 sample compression would be worth trying -- it
really helps.
If you recycle samples lots for many different instruments, .it only requires you to define the sample once, unless you are screwing around with different frequencies / the vibrato-related crap nobody uses.
If your patterns are quite redundant and/or empty, .it's pattern packing handles this very well.
But yeah, basically, if you're not taking serious advantage of instruments, then .it is usually a lot better-suited for the job.
With that said, if Strobe writes an article on XM optimisation (hint hint), some of the tricks could well work here - just note that pattern cells in XM usually take (notcompletelyfull?+note?+ins?+vol?+efftype?+effparam?) bytes each, and so they might not work well. Plus, Fast Tracker 2 is insane, and so pretty much any seemingly illogical trick will not work.
When you should not use MOD
When your pattern data isn't completely packed to the brim and completely unredundant.
.mod has no packing, so don't use it in size-limited compos unless you feel like being silly.
I'm not going to go into details about every format, I'm just noting that .mod has no packing. But if you're feeling like a masochist, then
go ahead.
The math behind pattern packing
If you want to know how it's actually stored at the byte level, you can't go wrong with
ITTECH.TXT.
What's important is that every cell has a mask denoting where there's a new note, instrument, (volume or voleffect), effect. It also denotes where you use the last n/i/v/e used in that track. e.g.
C-5 01 / ^^^ / D-5 01 / ^^^:
81 03 3C 01 00 (5)
81 01 FE 00 (4)
81 21 3E 00 (4)
81 01 FE 00 (4)
Total: 17 bytes.
This could be optimised to C-5 01 / ^^^ 01 / D-5 01 / ^^^ 01:
81 03 3C 01 00 (5)
81 21 FE 00 (4)
01 3E 00 (3)
01 FE 00 (3)
Total: 15 bytes - 2 bytes saved.
Note that the 00 byte at the end is the "end of row" marker.
Of course, other than what affects the mask, this is all you need to know:
- Cost of pattern (note, no header for an empty 64-row pattern): 12 bytes (8-byte header + 4-byte pointer).
- Cost of row: 1 byte.
- Cost of nonempty cell: 1 byte.
- Cost of mask change: 1 byte.
- Cost of new note / instrument / (volume or voleffect): 1 byte.
- Cost of new effect: 2 bytes.
What everyone SHOULD know
This isn't like most pattern formats. Don't be afraid to split your drum beat into lots of different channels - this should hopefully save masks and instrument values.
Effects take up 2 bytes. So doing, e.g. G20, G00, G00 will almost always result in using up more bytes than necessary, and essentially never less. Change it to G20, G20, G20.
Voleffects take up only one byte. These can be very good to use if they fit. Also, avoid using e.g. G5 G0 G0.
On the note of voleffects, this is the map of Gx values (from 1 to 9 respectively; as read in ITTECH.TXT): 1, 4, 8, 16, 32, 64, 96, 128, 255
If you have any channels which only use a single volume value, consider setting the channel volume (NOT WITH Mxx BUT IN THE SONG SETTINGS) to that volume and drop the value.
Filling in the gaps for empty instruments / samples / patterns is a very good idea. Filling in the empties will save 4 bytes for each empty.
Finally, SBx, SEx, Axx and shoving stuff into a single pattern can REALLY help. Every pattern has an 8-byte header and a 4-byte pointer to the pattern.
How the hell do I pack samples
In short: Save it in, e.g. Schism, if you've got a bunch of extra crap stored in the file, e.g. MIDI (modplug/IT) or timestamps (IT). Get IT 2.14p3. Save it in that with IT214 mode.
In short, paying money to Jeffrey Lim (or pirating if you're on that side of the law): Get IT 2.15, and do the same thing but with IT215 packing. I hope it doesn't compress EVERY sample as IT215 as a lot of samples work better with just IT214 (but straight squares/saws work better with IT215 usually).
UPDATE:
Impulse Tracker is now Open Source. For reals. If you're willing to pilfer Turbo Assembler from somewhere, you can build your own IT 2.15. Still, munch tends to pack better for some reason... even though it seems to follow a vaguely similar concept to IT's packer (work out the bit widths then optimise the widths accordingly).
In short, paying nothing but destroying your file structure: Get
munch.py. It is advised that you munch it several times as the algorithm used for packing the stuff in is quite frankly a bit crap (it's an O(n^2) greedy algorithm which has been optimised and at each stage gets faster but worse for some weird reason).
In short, rolling your own and just about losing your marbles: Search for "itsex.c". Just remember that the data is split into sections of 0x8000
bytes (NOT 0x8000 samples - for 16-bit samples, split into 0x4000 samples),
before compression.
