We decided to watch the new movies hoping that would serve as a good digest for Evangelion. Suddenly, I found myself getting excited for Eva again. When I was a teenager I was inspired by the Christian-laden mythology and the psychobabble. After watching Evangelion 1.11 I’m more impressed with how they rebuilt the movie’s imagery. After watching so much anime it hadn’t really hit me just how far toon-shaded rendering had come. I could easily tell that vehicles, buildings were redone as full-on 3D models but they were well integrated with the hand-drawn characters. What solidified that mind-blowing-away-ness was watching The Rebuild of Evangelion featurette. It went through how they reconstructed Tokyo-3′s buildings as 3D models and the layers of effects they put over it to achieve the images in the film.

And I thought, “I’m an engineer. I’m an artist. I can do that.”

Just to give you an idea of what’s inspiring me, here’s some screencaps from Evangelion. First up is Tokyo-3. Those skyscrapers are 3D and the houses in front are painted:

Below is one of Tokyo-3′s buildings as it’s rising out of the Geofront. The staircase for the overhead walkway, the rail guard, and trees are painted. The building is created in 3D and toon-shaded. You can see this scene in Rebuild of Evangelion as they go through the different processes of making this scene. It starts out with some crude drawings of a tree with the 3D elements behind it and then gets refined and various filters and effects are used to create the sunset scene.

I’m not sure how much CG is in this shot (I think the Angel in the upper right is CG but most of this shot looks painted), but it is quite an epic view of Tokyo-3:

As an artist, I’ve shied away from doing 3D and mixing it with my hand drawn work. I have this mental block that says, “It’s cool, but it’s cheating.” I’ve seen plenty of brilliant artists on DeviantArt who have the skills to create amazing art without having to resort to it. I vowed to myself unconsciously that I would never cheat, but now that I’ve seen the Rebuild of Evangelion it’s removed that block from my mind. It’s not cheating to me anymore. I decided I wanted to make cinematic, cel-shaded looking artwork and with that I started exploring my old friend, Blender.

Blender has gone through a great deal of changes since I last used it back in January of 2008. For instance the entire UI has changed. I had long forgotten everything I knew about modeling and rendering with Blender so it wasn’t an issue. I downloaded the latest stable build and got to work learning the ins-and-outs of this new beast.

Without further ado, this is the final rendering at 1920×1080:

I did this as a request for a Deviant Art member. It wasn’t enough for me to just draw his character. I didn’t want another free floating character without a backdrop. I wanted a cinematic look. I wanted a detailed background, toon-rendered, and bigger than life. I drew his character from the position of a camera looking down at her and to reinforce that perspective I decided to create a city to match. I deliberated on how I would approach this for a few days and then decided to go into Blender and just start modeling rudimentary block buildings and came out with this:

I superimposed my finished version of the girl, her name is Superhawke, over top of the first rendering and it served as a crude prototype. From there I built a better city and pulled the camera back to make it appear that she was higher up and floating in the sky above 50 story skyscrapers. The buildings are still fairly low-poly. I used some of Evangelion’s rebuild of Tokyo-3 as a benchmark for various details. I put simple air-conditioning units and water towers on the rooftops to give them more life. I placed some crude trees which were there to show scale and color so I could paint over them in Photoshop later.

This is the final version of that city as it was rendered:

While I was constructing this city I had begun doing various rendering tests to get down the toon shading.  The first test I did was just to use Blender’s edge rendering with the toon shaded materials. What happens is that some of the lines don’t show up and if you render it to small the lines are thick. I’d have to go into Photoshop and up the levels until the lines were dark enough and I didn’t want to do that since the color would be ruined. The thick lines, bleck. Do not want. I wanted the right colors and sharp, thin lines where I expected them to be. To get thin lines what you need to do is to render out at least 200% larger than your final resolution (which is 1920×1080 for me). In Photoshop I could then resize the image and get my crisp lines. Still, some of the thinner lines don’t show up. I decided the best way would be if I could separate out the line rendering and shading so I could manipulate the lines to darken them and composite everything together.

Well, Blender has a means of doing that when it finishes the rendering, it’s called (wait for it) the compositor. You can basically add nodes and string them together to manipulate the rendered images and layer them together. You can try and combine nodes to create all sorts of effects. To do this you need different render layers. So, one layer had my AC Units with the shadeless materials so I could get the the black lines on a white background. The only rendering options I set on this layer was to do edges. The edge rendering was set to 100. The second layer was my toon shaded version and it had all of the default rendering options checked. To get darker lines I used the RGB curve on the line render layer and increased the levels until the lines showed up and then Blender composites the toon-shaded layer to it using an Overlay mix node.

Here is a line shading test with shadows:

Here’s the blender compositor node setup I used:

click to enlarge

This is the final rendering with both the toon render layer and the lines:

So far so good. I figured out a decent way of doing the rendering. Here’s the problem: with my city scene doing the compositing in Blender is going to be a pain in the ass. Basically it means duplicating the geometry and there’s already a lot of geometry for the city. This is what my blender file looks like from the camera’s POV:

Doubling that geo to do both layers would tax my machine. I attempted it anyway and it only caused Blender to up and crash during the rendering phase after all that trouble. So it was back to the alternative way: render out the individual scenes for line rendering and the toon shading and composite them in Photoshop. Here’s a grayscale version of the buildings with the lines as composited in Photoshop:

Once I had all of this going there were a few other tests I wanted to try. I wanted to see if I could reflect the sky in some of the buildings but building a skybox, while that was easy, didn’t provide good results. It’s something I still need to explore. I thought of environment maps, but I’m still shaky on how they work in Blender. I even played with ambient occlusion and got this rendering:

I thought of compositing that with my final image, but the AO just doesn’t fit. I wanted cel-shaded coloring and it would make it look too shaded.

One more note, I also tried a version of Blender 2.5 with Freestyle integrated. Freestyle is a Google Summer of Code project to do toon rendering. I thought it would be a boon and solve all my problems. Certainly there are scripts in there that do an amazing job. I found one with the right line thickness and decent enough shading, but alas, it didn’t work out. Everytime I tried to render my city at 200% the resolution it would crash Blender. I tried the parametric line rendering but it didn’t work either. Of course this version of Blender 2.5 is an experimental one, so it’ll be a bit before it’s working well. I thought of going back to an older version of Blender with Freestyle as well, but honestly, it’s a different UI, and I’m not interested in learning it. So this approach was the best way and something I intend to explore further.

As for my friend…we watched Evangelion 1.11 and 2.22 and it left him brimming with questions and no resolution so I picked up the Evangelion TV series and we rewatched that. All I can say is, he’s seen it all now except for End of Evangelion which we’ll watch sooner or later. Those last two episodes of the TV show…painful, utterly painful to watch for me nowadays. Of course, Eva’s probably not best marathoned straight through. Back when I was in high school we did that to a friend. When he watched the last episode he turned to us and said, “I’m never watching anime again.”

…it’s a partially reflective UV texture mapped sphere with the image of the moon textured on it. And that one behind it is the place the Battlestar Galactica is going next season. And there’s a big fucking crate, because what wouldn’t a “Hello World” raytraced image be without a big fucking crate. Before that I had a nice ranchy image of Ino from Guilty Gear XX, but I figured, who would want to see that when you could be looking at a crate?

So what does this image prove?

a. I’m a big nerd and have too much free time — and I do, cause I’m on vacation (or “vacation?”) which means I get to do whatever the hell I want, and yes, I choose to work on my raytracer, because I enjoy it. Put that in your pipe and smoke it;

b. I got rudimentary UV texture maps working for spheres and triangle meshes;

c. I can load arbitrary triangle meshes, so obviously I would load a cube because there’s nothing more interesting than a cube;

d. The moon is actually partially reflective and would in reality reflect a red and blue infinite checkerboard plane that stretches far and wide across the universe;

e. All of the above.

My raytracer’s actually come to the point where it takes a couple of minutes to raytrace a triangle mesh. Such as this image:

So what’s next on the agenda? Well…I fixed some of the horkiness of the raytracer, but it’s still horked in other ways. There are still issues with the reflection, refraction doesn’t work and so on and so forth…a lot of little quibbles with the mathematics that I need to iron out. It’ll get there. If you’re curious how long it took to add texture mapping and mesh loading, on my part not too long… a couple of hours each. For mesh loading, I had an OBJ loader cooked up for our old game engine Bushido. The tricky part was to understand how to find ray-triangle intersections — and to find a good one at that. If you’re looking for a good resource for barycentric ray-triangle intersection testing then google search for “Fast, Minimum Storage Ray/Triangle Intersection” by Tomas Moeller and Ben Trumbore. They also wrote a book called Real-Time Rendering which I need to get my hands on. As one senior engineer where I work told me, it’s the new bible for computer graphics.

For texture mapping. I had also done a TGA loader for Bushido, so I imported that and made it much more portable. There are also great UV coordinate finding algorithms online for spheres. The 3D mesh UV coordinate algorithm is just based off of the barycentric intersection test. Nice and easy and it gives me much more flexibility in what I can load into my raytracer.

For even further down the road I’m looking at implementing a kd-tree to speed up my raytracing especially with all these triangle meshes. I also need to consider anti-aliasing. I should also think about a transformation pipeline — that’s right, I got away without one for as long as I could! Multiple lights might be fun. Soft shadows. I’d really like to get into the procedural texturing.

Above all…having a GUI interface to set up a raytracer scene would really help. One of my goals is to keep my raytracer code portable, so it’s not dependent on OGL or DirectX. That way I can implement a GUI in the future so I can get a wireframe view of everything before I hit the big ole render button.

More to come…but dinner beckons, and then Rapture.

I believe the issue I’m facing here is a z-depth one. I’m noticing that a lot of my calculations are the inverse of what I expect. I don’t believe I’m supposed to see the shadow like that, and definitely I should not be seeing the reflection of those two smaller spheres. Now that I have a fairly decent raytracer working it’s time to go back and correct all the little details.

With the help of Foley, et al. I fixed my illumination code. The specular highlight you see here works more correctly then in the previous image. I’m not so hot about the washed out look of the sphere, but that’s mainly due to the white ambient light I’m using. I also have shadows. I’m not 100% sure if the shadows are correct, but it’s a start.

Three spheres, one checkerboard infinite plane…I have a very simple scene list now and I can add spheres and planes at will. Maybe oneday even arbitrary 3D models. How could would that be?

I’m excited.

There’s so much I can do now. The next major refactoring/addition will be to create a recursive raytracing algorithm which will provide me with refraction and reflection. Chrome spheres here I comes!

At work we’ve got a little bit of downtime coming up, and one of the senior engineering guys, Bill, and myself started talking about raytracing. I don’t recall exactly how we began discussing the topic — something to do with Pixar, Ed Catmull, and him trying to remember the name of the guy that invented raytracing, and then there was a short discussing I initiated about this guy: Ron Fedkiw. Do yourself a favor and look at the videos on his site. Be prepared to change your pants and/or underpants (if you are the kind that wears underpants) during the viewing of these animations.

Our discussions on computer graphics rekindled my interest to write a raytracer. During my undergrad at the University of Southern California I took a graphics course and the last assignment was to put one together. I only got mine partially working, and had no more time during the semester to go back to the raytracer. So, I gave it up. Plus, through my over-zealous use of object oriented programming, OpenGL, and MFC, it was a horrible mess of partially functioning shit. Moral of that story: keep it simple, stupid.

Last night and all of today, I’ve been working on remedying that epic failure in my college career. Raytracing and all that it entails is an expansive subject. At it’s core, raytracing is firing (out) rays into a 3D scene and intersecting objects to return their color values. You have a picture plane (defined as a grid of pixels), and you use some vector mathematics to fire rays into a 3D scene. A ray is a point in space along with a direction. In this case, you use a point camera, and fire rays from the point through each pixel in the grid into the 3D scene. You test to see if the rays hit things in the scene…spheres, boxes, planes, what have you, and you return the color value and write it to the grid. We can get into discussing things like local illumination models, shadow casting, radiosity, super-sampling, and a whole slew of mathematically challenging topics related to image-based rendering, which I haven’t touched in a couple of years, but we won’t. Time for me to dust that portion of my brain off anyway.

The “Hello World” of raytracing is a checkerboard floor with any grouping of chrome spheres in which everything is reflecting everything else and casting shadows all around. That’s my first major goal, and this time, being a bit wiser, I’ll try and keep it as simple as I can be.

The work-in-progress of my version 0.1 raytracer:

Click on it to see it in it’s full 1920×1200 pixel glory.

Features so far: well it works, don’t it? It’s fairly fast. I can raytrace an image of any size. It has a simple illumination model and I can do plane and sphere intersections. All the bread and butter.

How did I go about building this program with my busy work schedule and only having scant hours available on weekends and weekdays? I broke it down into milestones that I thought were doable in a couple of hour chunks.

The first milestone was just to get the project setup. In that case, if I sudden got incredibly lazy, I could have at least the basic project setup in Visual Studio for me to continue on later. This included some math code I jacked from the Cloud game engine, Bushido and a TGA exporter I wrote a long while back. My idea is that this program will draw everything into a pixel buffer (huge ass array of RGBA values), and then spit it out as an image that I can view in Photoshop.

The second milestone was just to build a raycaster. This is the first step in a raytracer…just being able to fire rays into the scene and hit things. I kept true to my word and kept it simple, there’s no complicated scene manager or lighting models, etc. It’s currently all hardcoded. I wrote ray-plane and ray-sphere intersection tests.

The next milestone was to do a simple illumination model. Since I started writing my code willy-nilly, hardcoding things here and there, and such, you can see the lights don’t match. The purple and white plane has a light source of it’s own and the sphere’s light isn’t the same, but that’s fine, I’m messing around with phong shading, and right now it works, and it produces a pretty damn cool (and asstastic) result.

I though this would take me at least a couple of days to do…but it’s 1 am, Sunday morning, and after only at least, I’d say, 10 hours of coding between Friday and Saturday, I’ve got quite a lot of what I wanted to do.

My next milestone is to make everything a little more organized so I can do some recursive raytracing — this will allow me to get the refections, refraction, and shadows, that I’ve been dying to see in my own raytraced imagine.

Once I have that, I think I will have the start of a scene list, which I can expand to include ray-triangle intersection and generic 3D models I can import into the raytracing engine. What I’d love to do is to raytrace a Dyadin level or a the islands in Cloud. That would be pretty damn schweet.

And from there…well the sky’s the limit, and I’d love to start learning the more complicated mathematics behind all the cool stuff you can do. I have a book on procedural texturing and modeling, that would be awesome to infuse into my raytracer. You’ll be seeing more of this on the Courne Supremacy.

Oh, and I saw the Courne Ultimatum. I suggest you do the same, it’s an excellent movie.