Milk and Compressors

Late last December, Decipher and I had plans to complete a highly massive 64k intro for The Gathering. Around January, we started on tool/framework development, and had a pretty good start. We made one mistake, however: We assumed that once we had the technology, the creative aspect of the intro would be trivial. Because of this, we developed highly capable tools and such, but in the end had nothing to show. In other words, we failed :) . On a side note, I was also busy preparing to move to Norway to work for a fantastic startup company founded by former demoscene group Outracks and finish up my current semester at WIT, and Decipher was tied up in work projects at Nokia. Excuses, excuses ;) .

Needing a backup project to avoid arriving at TG empty-handed and deal with the stress I had encountered, I decided in the last three weeks to produce a 1k intro. Problem was, the version of Crinkler 1k that we had on-hand was an older test version that wasn't very compatible yet (and do to mistakes we had made the previous year, we didn't want to request a new version unless we absolutely had to). So, I wanted to produce a prototype of Juicer; a 1k compressor that Decipher and I had planned to produce many times before but never got around to finishing.

So, [re-]development of Juicer began. It started with producing some sort of dumbed-down version, as a proof of concept (and to refresh my very dated data compression experience). For this, I chose to develop a packer for Commodore 64, simply because the executable format is much simpler (no header besides a 16-bit load address, as opposed to the Windows PE header, which is a real pain), and I just love coding for that system. In about five hours or so composed of researching compression techniques, coding, and testing, I had a working lz77-based C64 intro packer. The compression wasn't very good, but the decompressor (including autostart stub) occupied only 87 bytes, without optimization. Unfortunately, I don't have the original packer, as my beloved flash drive was lost in the move, so I can only show an unworking version that I have left over here:

Go ahead and laugh :) . But yes, this was a success (at one point). Also allowed me to write a nice little class I call bitbuf (bitwise data buffer). Self-explanatory ;) .

Next was to do it in Windows. If you read my previous posts, you'll know I've had plenty of trial-and-error experience with PE hacking, so this part was nothing new :) . Once I had the systems (and virtual machines) to test on, I had a working, handmade PE header in assembly in a few hours. The compressor was much harder, though.

The first thing I tried was based on the C64 lz77 scheme, but with some lzma-based modifications. I had decent results on ASCII text (namely, size-optimized GLSL code), but it didn't work very well on binary data. So, I read up on what Crinkler was doing. As described in the Crinkler manual, this blog post, and a few other sources, Crinkler is doing a (very strange, mind you) 33-bit arithmetic encoder with order-8 context modelling. These buzzwords are too complex to explain here (and that's what Google's for, you know), so I won't :) . But basically, I delved into these methods myself....and I can't yet share my results until the project's finished, so let's move on a bit ;) .

Long story short, three weeks simply isn't enough (at least for me, hehe) to beat such an efficient compressor as Crinkler 1k with a decent decompressor size. Not to mention I lost two days from TG stuck at the airport in Boston, but that's another story.

So, we had nothing to show at The Gathering, except perhaps a nice effect I planned to squeeze in the 1k. Which I was going to save until we did a proper packer, until Duckers/Outracks called us 30 minutes before the intro compo saying, "If you don't release something, we have to drop the compo because of the number of entries." So, with 30 minutes to do an intro, you do what you can with what you have :) .

And, what we did was this; Milk by The Compo Xaviours (a group name coined by the group PlayPsyCo from last year's TG, when they saved the same compo with another entry):

It's the 1k content I had planned, with Decipher's C framework and a quick tune with my 4k synth. The visuals are kindof cool, so I'd like to break them down for you. Here is the code:

Vertex shader:

varying float t2;

void main(){

t2=gl_Color.x*2222222;

gl_Position=ftransform();

}

Fragment shader:

varying float t2;

float t=t2-fract(t2*.4),d,c=cos(t*.6)*4;

vec3 r(vec3 x,float t){

for(float v=0.;v<3.;v++)t*=.8,x=vec3(cos(t)*x.x+sin(t)*x.z,cos(t)*x.z-sin(t)*x.x,x.y);

return x;

}

void main(){

vec3 x,a,z=vec3(0),y=r(normalize(vec3((gl_FragCoord.xy-vec2(320,180))/300,1)),c);

for(float v=0.;v<3.;v+=.06){

x=abs(a=r(abs(z-r(

vec3(sin(t*.4),0,3.5+sin(t)+t*.05)// Camera

,c))

-vec3(clamp(t-20,0.,1.5))// Mirror displacement

,

max(t-10,0.)*1.2// Mirror rotation

));

d=min(max(max(x.x,x.y),x.z)-1,8.-length(x));

for(float v=0.;v<3.;v++){

x=abs(fract(a*pow(3.,v))-.5);// Subdivision

d=max(d,mix(

.17-min(min(max(x.x,x.y),max(x.x,x.z)),max(x.y,x.z)),// Cubes

length(x)-.6,// Spheres

.5-cos(t*.07)*.5

));

}

gl_FragColor=mix(vec4(1.1,1,1.2,1),vec4(1,1.1,1.2,1),y.y+.5)*v*.3;

if(d<=0)break;

z+=y*max(d,.0001);

}

}

..unobfuscated, that is. Basically, it's the popular sphere tracing technique, but with some neat little tricks. Let's break some of it down. We start with this:

This image really only has two objects; a cube and a background sphere. These just look like this:

Then, I intersect the two volumes with a repetitive cutout cube. This looks like this:

Simple enough. Now, do this with multiple iterations, where you decrease the size of each cutout and increase the frequency, and you have the original image. The next trick is the cube deformation, seen here:

This is MUCH simpler than it looks, actually. Let's remove the "sponginess," for clarity:

In this shot, it's actually quite obvious what it is that I'm doing. I'm tracing just the cube and sphere normally, except for one thing: mirroring. Imagine 3 mirrors, set on each axis, and all facing the upper-right corner. This is the entire trick. Everything you see is just a mirror of the upper-right corner (1st domain). So, this deformation is just a matter of rotating the cube. That's all :) . Camera operations still work normally, so it looks quite nice.

The next trick, if you look at the original screenshot of the prod (with the four spherical things), is that the cube separates into four more cube-like structures. This is actually the same trick as before; the mirrors. All I do is move the cube outwards diagonally, and the mirrors follow. The demoscene really is all smoke and mirrors, anyways ;) .

My next trick explains why the objects in the main screenshot are spherical. This is simple, too. As the intro progresses, the cube cutouts actually transform into spherical cutouts. And, well, that's actually enough explanation :) . Instead of cutting out cubes, we cut out spheres. Blend from cube to sphere, and you have the main timeline for the intro.

Finally, we have the "multiple camera angles." Actually, we don't :) . This is the simplest part of the intro, and one of the most rewarding. Instead of cutting multiple cameras, we just create one that's very dynamic. After that, just cut time instead. The result looks like multiple angles, when in reality it's multiple instances of time.

So, that's it for the intro. The 1k version uses some more nasty hacks to get it smaller, but the posted code (obfuscated, of course) is exactly what made it into the actual prod. Had it been 1k, I think I'd've gladly used the Youth Uprising label, but it was just too insignificant as a 2.5k I think :) .

Speaking of YUP, I'd like to mention, we have a new member, Gunda. He released his first production at The Gathering in the demo compo, Nitesco. A cool firstie; he did all code and design, while I did the music and framework, and helped out here and there. Look forward to his next releases - he's bound to progress quite fast (as he did during the making of this prod).

And finally, I released a 4k intro starter kit and seminar at TG. The kit can be found here and the seminar video is here. Won't say anything more about that; you can just check the links :) .

Until next time :) .