Videos and executable demo of Brigade 2 GUI and animation test

Some videos of the GUI that I developed for Brigade 2, showing real-time changing of materials with simultaneous animation and physics simulation. The GUI is still a work in progress, but being able to tweak any material on the fly is so much more easy to get the look right.

480p (320x240 render res, 4 spp, max depth 4)

http://www.youtube.com/watch?v=4Q_L_At_mAk

480p (640x480 render res, 4 spp, max depth 4)

http://www.youtube.com/watch?v=65t2_FKd4Ds

The Ogre (model from here) is just simply rotating for now. The mesh consists of 51k triangles of which the BVH is dynamically updated every frame (as are the BVHs of the car and the stack of blocks which are physics driven). 

The executable demo is available at 

http://code.google.com/p/tokap-the-once-known-as-pong/downloads/list

(all CUDA architectures supported)

Further experiments will include: 

- skeletal animation

- first person camera and gun

- cam following the vehicle

- architecture 

GUI test in Brigade 2

I've been working on a IMGUI (immediate mode GUI) for Brigade 2 based on SDL, which is working pretty well and is quite responsive when the framerate is more than 5 fps. There are sliders for specularity, transparency and RGB values of the diffuse color of the material. Every material in the scene can be picked and updates in real-time + you can navigate the scene, control the vehicle and let a physics simulation run, all simultaneously while chaning materials. I will upload an executable demo and some videos of the GUI in action tomorrow.

Screenshot of the GUI while tweaking a glossy Ogre (almost 50k triangles, model from Javor Kalojanov's real-time raytraced animation demo):

szczyglo74 (CG artist/architect) also provided me with a modern half-open architectural structure (Altana model from Januszowka) which is great to test indirect lighting and which I will turn into a demo soon (screen rendered on 8600M GT): 

Paper + video of "Real-time bidirectional path tracing via rasterization"

Available here: http://www.square-enix.com/jp/info/library/  (thanks to selfshadow)

It looks pretty damn good and the technique seems to be a very likely candidate for implementation in next-gen games because it fully exploits the rasterization hardware. One of the limitations (as mentioned in the paper) is that it doesn't work well for highly glossy and perfectly specular surfaces (sharp reflections) for which ray tracing is proposed as an alternative.

Real-time bidirectional path tracing via rasterization

This is the title of an upcoming I3D 2012 paper by Yusuke Tokuyoshi and Shinji Ogaki, see

http://graphics.ics.uci.edu/I3D2012/papers.php

The title reminds me of "High-quality global illumination rendering using rasterization" by Toshiya Hachisuka from 2005, which described a technique to obtain photorealistic images on a typical 2005 GPU (like the Radeon 9700) in mere seconds, extremely impressive for that time. Shinji Ogaki is also a co-author on the Progressive Photon Mapping paper by Hachisuka and Jensen, so this new paper is definitely going to be interesting.

If the paper lives up to the title, this could be quite interesting. Both researchers work at Square Enix and there seems to be a connection with the recently unveiled photorealistic Luminous engine which uses high quality offline baked lightmaps (see this page for more details). A paper about the rasterization-based lightmap baking in Luminous can be found here and the real-time bidirectional PT technique probably works very similarly (i.e. ray bundles computed with rasterization by parallel visibilty tests):

Quote from the "Fast global illumination baking via ray bundles" paper (describing the tech behind the Luminous engine):

7 high-quality light maps are rendered in 181 seconds with NVIDIA GeForce GTX 580. The resolution of ray-bundle is 2048x2048 pixels, and 10000 directions are sampled. The performance of our renderer is over 200 M rays per second on a commodity GPU.

Assuming everything scales linearly, this means that it would take about 16 milliseconds (60 fps) on a GTX 580 to compute a GI lightmap with ray bundles of 512x512 pixels and 100 ray bundle directions (= 100 directional samples) which should still yield great quality real-time global illumination. This tech could potentially be used for making real-time photorealistic games on current GPUs. It doesn't work however for objects with highly glossy and perfectly specular materials.

Executable for Brigade 2 vehicle physics test available

An executable demo of the scene in the previous post is available at

http://code.google.com/p/tokap-the-once-known-as-pong/downloads/list

I've recompiled the cuda binaries on my laptop for every possible CUDA GPU architecture (compute capability 1.0, 1.1, 1.2, 1.3, 2.0 and 2.1). Since the binaries were not natively compiled on their respective architecture and use a fixed maxregistercount of 63, higher end GPUs like the GTX 560 and up are not used to their fullest potential (a member of the Beyond3D forum reports only 35% GPU load on a GTX 460 with this demo). But at least it works on all CUDA architectures :-)

UPDATE: Roeny (one of Brigade's developers) has posted an explanation of why the GPU is not fully loaded here: http://forum.beyond3d.com/showpost.php?p=1608008&postcount=86

UPDATE 2: szczyglo74 has posted a cool video on Youtube of the 480p demo on a GTX 590 + GTX 460 (3 GPUs in total!)

Brigade 2 Glossy test 480p and 720p videos on GTS 450

Some videos of the scene in the previous post:

480p Watch on Youtube

720p HD Watch on Youtube

Stats:

- rendered on just one GTS 450, no CPUs involved (hybrid CPU/GPU rendering is actually slower than pure GPU rendering (due to synching) besides being more complex)

- 640x480 render resolution (first video)

- 1280x720 HD render resolution (second video)

- extremely fast convergence: great image quality with only 4 samples per pixel per frame (when the camera is non-stationary) which is sufficient for scenes with skydome lighting

- max path depth 4

- floor, blue boxes, vehicle wheels and chassis are partially glossy to create a plastic look

- containers, mirror ball and flat mirror are perfectly specular

- orange ball is perfectly refractive in first video

- walls are perfectly diffuse (Lambertian)

- almost 3800 triangles (not that many but it's a lot less restrictive than the spheres and boxes from TOKAP and actually not that much slower thanks to BVH acceleration) 

The framerate would be a lot smoother on a GTX 580, or better yet a pair of them :-)

Glossy test in Brigade 2

Some screenshots of a new Brigade 2 powered demo that I'm working on (rendered with a 8600M GT at 640x360). The floor is textured and slightly glossy and almost every object in the scene is fully or partially specular e.g. the blue boxes in the stack and the wheels and body of the vehicle now have a plastic look. The scene looks great in motion. Videos on a GTS 450 will follow tomorrow. 

480p and 720p videos of Brigade 2 physics and UI tests on the GTS 450

The following videos demonstrate some Bullet physics tests that I've been working on and which were rendered in real-time with the Brigade 2 path tracer on a GeForce GTS 450. All the rendering is pure GPU path tracing with CUDA, the CPU doesn't take part in the rendering process except for recalculating the dynamic BVH of the car and the boxes every frame. All the light in the scene is coming from a skydome. The code for the car physics is essentially the same as what was used in the "Unbiased Stunt Racer" demo. The physics simulation can be paused and resumed and the image converges (extremely fast) when nothing moves (physics simulation paused + stationary camera). There is also a short demo of a work-in-progress material UI (only diffuse to glossy/specular for now, the final version will have sliders for transparency, specularity, diffuse color and emissive color).

480p physics test (640x480 screen resolution, 320x240 render resolution):

480p UI test (640x480 screen resolution, 320x240 render resolution):

480p UI test (640x480 screen and render resolution) watch on youtube in 480p:

720p UI test (1280x720 screen resolution and render resolution!) watch on Youtube in 720p:

Physics tests in Brigade 2

Jeroen van Schijndel, one of the developers of the Brigade path tracer, has posted a cool video showing off the physics inside Brigade:

Video: http://www.youtube.com/watch?v=DkLhPQ-NeBU

During the last several days, I have been working on implementing vehicle physics in a new simple demo that uses the Brigade 2 path tracer. The scene in the demo is only lit by a sky texture and consists of a vehicle with mass, a few perfectly specular containers and a stack of boxes (more than 2800 triangles in total). The maximum path depth is 4 to make the reflections more interesting.

The following screenshots and videos were all rendered in real-time on a mobile GPU, the 8600M GT, which is extremely low-end by today's standards and is about 30 times slower than a GTX 580. Nevertheless, despite its very low "CUDA core" count and clocks, it can still produce some pretty amazing footage, path-traced in real-time at 4 samples per pixel and a max path depth of 4 (allowing for reflections of reflections). Better videos with much higher framerate (captured on a GTS 450) will follow soon.

Somewhat better quality, but lower framerate

4 spp, 480x360

Glossy floor, 4 spp

24 spp, 480x360:

Some ideas that I want to work on next:

- frame averaging, which greatly reduces the noise at the expense of slight blurring

- a camera that follows the vehicle automatically

- a first-person gun which can shoot projectiles with physical mass

- (low-poly) animated enemies (with cheap-to-update BVH)

- investigate the use of bump mapping to further increase realism (Dade's Sfera demo has shown that bump mapped surfaces converge almost as fast as non-bump mapped ones)

- more interesting scenes, using a greater variety of materials (Brigade supports diffuse, glass, glossy and perfectly specular) and textures. I'm still aiming to replicate this path traced animation in real-time

The Brigade 2 path tracer is just great fun to play with and the speed and quality in a well-lit environment (on a high end GPU) are truly amazing. Seeing a real-time animated character with perfectly photoreal lighting while orbiting the camera around it is so damn rewarding...

Sfera, the first real-time path traced game based on OpenCL!

This is very cool: there is a new game called Sfera, it's using only spheres and is rendered with real-time path tracing on the GPU and CPU using OpenCL. The developer of the game, Dade (David Bucciarelli), is a LuxRender developer who is the first to make a working implementation of OpenCL-based unbiased path tracing on ATI cards. He created SmallptGPU, SmallLuxGPU and LuxRays.

Youtube video of the Sfera game: http://www.youtube.com/watch?v=Dh9uWYaiP3s

The demo features HDR image based lighting, bump mapping, multiple physically accurate materials, Bullet physics and multi-GPU support. I think it is absolutely awesome to see more developers using the power of the GPU (I'm glad that this time OpenCL and GPUs from ATI are used) to create amazing looking simple games with real-time photorealistic lighting. This game is a very nice proof-of-concept to show that real-time photorealism is very close. I hope this is the beginning of a paradigm shift for game graphics from rasterization to real-time ray tracing. The code is also open source and can be found at http://code.google.com/p/sfera/ (includes executables for Windows and Linux).

More info about the game Sfera can be found at http://www.luxrender.net/forum/viewtopic.php?f=17&t=7539

Btw, thanks Dade for mentioning Tokap and my blog on Sfera's project page ;-)

Vehicle test in Brigade 2

Another test with the Brigade 2 path tracer: this time I've added a user-controllable vehicle (included in Brigade) to the scene from the previous post. The truck consists of 1953 triangles and uses a dynamic BVH which is updated (BVH refitting as there are only translations and rotations) every frame when the truck moves.

There is also some variation in the materials (all objects are no longer diffuse): the horse has a glass material applied to it (no Beer's law, although the engine supports it), the green sphere is glossy and the truck's trailer is perfectly specular. The following video and screenshots were rendered at 320x240 with 12 spp per frame on a low-end GPU (GeForce GTS 450):

Video: http://www.youtube.com/watch?v=8nwotFaB0X4

The shadows and refraction are awesome:

640x480 render resolution:

Some ideas for upcoming tests:

- recycle the vehicle physics code from the Unbiased Stunt Racer demo

- move to an outdoor environment with skydome lighting. From early trials I've found out that the convergence in such a setting is extremely fast and 8 spp are often enough

- make a real-time Bullet physics demo of a collapsing structure, e.g. something like this or this

Bullet physics test 1 in Brigade 2

I've made a very simple real-time, interactive Bullet physics simulation of a bouncing light-emitting cube, rendered using the Brigade 2 path tracer with 8 spp per frame on a low-end GTS 450 (render resolution is 320x240 to keep the framerate acceptable). The simulation can be paused and resumed and the image converges when nothing moves. The scene contains 12733 triangles.

Video:

http://www.youtube.com/watch?v=stahUUH8yIE

Real-time path traced outdoor scene with Brigade 2 engine

Jacco Bikker has just posted a gorgeous looking new video of a WIP game oozing with atmosphere, which is running on the real-time path tracer Brigade 2:

Youtube: http://www.youtube.com/watch?v=AZz9zNgDxlM

A downloadable direct feed version (strongly recommended) can be found here: http://igad.nhtv.nl/~bikker/files/sections.avi

The video description says it all:

"This is some early footage from the student game "It's About Time", created by students of the IGAD program of the NHTV university of applied sciences, Breda, The Netherlands. This game uses the Brigade 2 path tracer for rendering. Although this results in a noisy image, the lighting is very realistic. A proof-of-concept demo of the game will be ready in January 2012. After that, the team intends to continue working on the project until the end of the academic year."

Some screengrabs from the video:

Note the ambient occlusion under the cylindrical stones and the greenish color bleeding on them

Sun + Sky lighting:

The lighting in the video is of the same quality as a precomputed lightmap, but is completely real-time. Obviously there is still some noise, most visible in indirectly lit parts of the scene, but I personally think it adds a special touch to the image, while at the same time attesting to the "real-timeness" of the lighting. The Brigade engine will truly shine when moving objects and animated characters will be added, which will both receive and radiate the same "perfect" physically based lighting as their environments. Moreover, this scene is just showing diffuse materials, but Brigade 2 can handle materials such as glass (with physically based light attenuation according to Beer's law), glossy (Blinn shader) and perfectly specular effortlessly, as proven in the videos in this post.

In the image below, notice the indirect lighting with brownish color bleeding under the wooden roof.

Today's games use a variety of tricks to approximate indirect lighting, such as high quality precomputed lightmaps (e.g. RAGE from id Software, Mirror's Edge from D.I.C.E.), instant radiosity (e.g. Geomeric's Enlighten technology in Battlefield 3, Lightsprint in Rockstar's L.A. Noire), cascaded light propagation volumes (LPV) with screen space GI (Crysis 2) and point based color bleeding (Danger Planet tech demo from Fantasy Labs, based on Michael Bunnell's surfel technique which was also used in Pixar's Up and in Pirates of the Caribbean). Each of these techniques has its own drawbacks:

- full or partial precomputation of lighting,

- limited scene destructability,

- real-time hard shadows superimposed over high quality, prebaked soft shadows resulting in an ugly shadow mix (RAGE and Mirror's Edge are nice examples)

- limited number of bounces

- crudely approximated, diffuse only indirect lighting

- no influence from the color of dynamic objects on the environment (e.g. a red wall will cast a reddish glow on a nearby white barrel, but a red explosive barrel near a white wall will do nothing)

- time consuming artist intervention (e.g. manual light probe placement in areas of interest)

While it's great to see that some games are finally implementing real-time GI approximations, they are still hacks that start to break when mirrors, glass and highly glossy objects are added to the scene. Real-time path tracing solves all these issues in one breath. While the performance on one current high-end GPU is still not sufficient, very nice looking real-time path traced prototypes (with animation) are already possible, giving a glimpse of where game graphics are headed when GPU clusters in the cloud come into play.

Right now, I'm busy working on some Brigade 2 powered real-time demo's (with Bullet physics) which I hope to show very soon.

Temporary solution for Ompf forum

Since the ompf.org forum (the one and only forum for ray tracing aficionados) has been down for over a week, and only the administrator can bring it back online, Jacco Bikker has set up a server with a temporary replacement for the forum (see comments in the previous post) and asked me to point the readers of this blog who frequented ompf (and who are -like me- craving for a daily dose of ray tracing news with accompanying withdrawal symptoms) to the new address at:

http://igad2.nhtv.nl/ompf2

Enjoy!

UPDATE: there's an interesting discussion and technical details on the improvements in the Brigade path tracer in this topic

New videos of Brigade!

The developers behind the Brigade path tracer (Jacco Bikker and Jeroen van Schijndel) have released two very impressive videos on Youtube and the jump in quality and performance is quite huge. The scene in the video runs smoothly on just one GTX 470 at 8spp and 640x360 render resolution (1280x720 display resolution) with very little noise:

http://www.youtube.com/watch?v=YQNUtzCpEZ4

http://www.youtube.com/watch?v=0U0hDb7dDFs

This latest version of the Brigade path tracer contains at least two major improvements compared to previous versions: multiple importance sampling and a Blinn shader for glossy materials (e.g. the floor) which greatly enhances the realism of the scene (and seems to converge very fast). The kernel now runs on the GPU only.

Very impressive what they have achieved so far. With an extra GPU, the noise should almost vanish and become imperceptible after playing a while. And with Nvidia's Kepler and AMD's compute-focused GCN (HD7000) on the horizon, this path tracer is going to become very interesting (when the code will be ported to OpenCL in AMD's case). Something like real-time photorealistic chess rendered on the GPU is very close now. Jacco also mentions in the comments under the video that a new game is in the works with support for animated objects (including skeletal animation).

The mere thought of having truly real-time, photoreal global illumination in games is the most exciting thing that has happened in computer graphics over the past ten years. The last time I was this excited was with the introduction of real-time lighting and shadowing with normal mapping in Doom 3, first shown to the world on the MacWorld Expo 2001:

I can't wait to get my hands dirty and experiment with the new Brigade code.

CentiLeo paper available

A paper about the out-of-core GPU ray tracer CentiLeo, entitled "Out-of-core GPU ray tracing of complex scenes" is available at ACM here: http://dl.acm.org/citation.cfm?id=2037826.2037854 (behind a paywall).

If you don't have an ACM subscription, you can still read a free version of the paper by downloading the file "Supplemental files" under the tab "Source Materials" on the same page.

The paper contains an interesting analysis of the overhead of the out-of-core GPU ray tracing algorithm compared to the in-core Aila/Laine GPU ray tracing algorithm.

Photorealistic animation rendered with Octane Render from within 3ds Max UPDATE: new video of Octane/3ds Max integration

Check it out: http://octanerender.com/forum/viewtopic.php?f=7&t=9692

Details about the scene and system used are in the video description (average rendertime 2.5 min at 720p HD resolution with only 1 GTX580).

Besides the fact that it looks so unbelievably real, the most interesting part about this animation is that it was rendered from within 3ds Max, using an in-development plug-in version of Octane Render that is completely integrated with the Max software. This will enable users to edit, move, add and delete geometry and lights in the scene and see the results instantly rendered in the viewport with Octane Render's photorealistic quality. It also eliminates the lengthy per-frame export times (in some cases longer than the actual rendertime of the frame itself) which will do wonders for animation rendering. I think this integration is going to become Octane's new killer feature which will take insanely fast photorealistic rendering on the GPU to the next level.

UPDATE: a new video appeared on the Octane Render forum, showing fully integrated Octane in the 3ds Max environment: http://octanerender.com/forum/viewtopic.php?f=7&t=9692

Update 2 on Exposure Render

There is now a stable release of Exposure Render, the open source direct volume renderer with physically based lighting on the GPU: http://code.google.com/p/exposure-render/

It now even runs on a laptop with a 8600M GT (and all the example datasets fit in the GPU memory). The photoreal lighting makes this renderer stand out when compared to other volume renderers. And it converges surprisingly fast. Some pictures (rendered for only 20 seconds on the ultraslow 8600M GT):

OTOY partners with Autodesk to stream real-time path traced graphics from the cloud

Awesome news!!! In addition to the partnerships with Refractive Software (makers of Octane Render) and the development team behind the Brigade path tracer (which I blogged about here), OTOY now announced a partnership with Autodesk to integrate Octane Render into the viewport of Autodesk's 3D Studio Max 2012 and stream real-time path traced visuals for animations, movies and games from remote GPU clusters in the cloud to the client. Real-time GPU path tracing is big business, and the best part is that it's going to bring game graphics up to ultrarealistic levels in the not too distant future.

VentureBeat article (includes an image of Octane's subsurface scattering implementation): http://venturebeat.com/2011/10/20/otoy-teams-up-with-autodesk-to-create-cloud-rendering-platform/

Yahoo biz (Business Wire): http://finance.yahoo.com/news/OTOY-and-Autodesk-to-Create-bw-395764800.html?x=0&.v=1

Some quotes from the article and official press release:

Real Time Path Tracing in the Viewport - A New Level of Photorealism

OTOY's 3D rendering tools, including Octane Render™ and Brigade™, are the premier rendering solutions for next generation 'path traced' games and films.

Path tracing significantly reduces the cost and complexity of high quality rendering by automatically generating effects traditionally handled through manual post processing – including spectral rainbows, lens flares, unbiased motion blur and depth of field.

“A year ago, path tracing was considered too expensive to be used even in high-end Hollywood blockbusters. Today, thanks to advances in GPU rendering, OTOY is bringing real time path-tracing technology to millions of artists by leveraging GPU hardware that costs only a few hundred dollars. This is a game-changer on any budget,” said Jules Urbach, CEO of OTOY.

“Autodesk is the leader in 3D design software for film and video game production. We are incredibly excited about our partnership and proud to be bringing their industry leading tools to an ever-expanding market through our cloud solutions,” said Alissa Grainger, President of OTOY.

One of the images in the VentureBeat article looks like the Kajiya scene (which can now be rendered at more than 10 fps with the Brigade engine, including caustics) in an IBL environment:

Very cool!

Real-time path tracing of animated meshes on the GPU (2): OGRE

Another test in my never relenting quest for real-time photorealistic graphics. This time I was inspired by one of the first animations rendered with unbiased Monte Carlo path tracing. The animation was made by Daniel Martinez Lara from Pepeland in 1999 and can be seen here:

http://www.youtube.com/watch?v=VNyknZ2zrsM

It’s one of the first animations that uses Arnold, the Monte Carlo path tracing based production renderer developed by Marcos Fajardo, that is currently taking Hollywood VFX by storm: it was used in e.g. Cloudy with a Chance of Meatballs, 2012, Alice in Wonderland and the Assassin’s Creed: Brotherhood and Revelations CG trailers and is giving PRMan and mental ray a run for their money (probably making them obsolete soon, mainly because of ease of use and huge artist time savings). The animation shows a very lifelike clay figure coming to life. Despite the simplicity of the scene, the whole looks very believable thanks to physically accurate global illumination and materials and an extensive use of depth-of-field and camera shake.

In an attempt to reproduce that particular scene, I’ve used the animated Ogre model from a ray tracing demo developed by Javor Kalojanov which can be found at http://javor.tech.officelive.com/tmp.aspx. The Ogre model (which was created by William Vaughan) consists of 50,855 triangles and was also used in the excellent paper “Two-level grids for ray tracing on GPUs” by Javor Kalojanov and Philipp Slusallek (really great people btw, whom I've had the pleasure to meet in person recently. The conversations I've had with them inspired me to finally try triangles as primitive for my real-time path tracing experiments (instead of just spheres), which led to this Ogre demo. To my surprise, triangle meshes are not that much slower to intersect compared to spheres. I think this is due to the fact that the cost of primitive intersection is becoming increasingly smaller compared to the cost of shading).

The following videos show an animated Ogre path traced in real-time with real-time, per frame update of the acceleration structure of the Ogre’s 50k triangle mesh (watch in 480p):

http://www.youtube.com/watch?v=qPFjBuVEGec

http://www.youtube.com/watch?v=z7IjCiazdJ0

http://www.youtube.com/watch?v=u0a0mPR8jdM

Path tracing is performed entirely on the GPU, in this case a GTS 450 (a low-end GPU by today’s standards). The framerate of the walk animation is about 4 fps max on my card but should be around 15-20 fps on a GTX 580. The image converges extremely fast to a very high quality result (in about 1-2 seconds). The movement of the Ogre (translation, rotation, animation) is actually much more fluid in real life without Fraps, the overhead of the video capturing software almost halfs performance.

The images below were each rendered at 20 samples per pixel in under 2 seconds on a GTS 450 (it would take less then 0.5 seconds on a GTX 580):

Note the very subtle color bleeding from the ground plane onto the Ogre:

With a glossy floor:

If you’re interested in trying this scene out yourself, send me an e-mail at sam [dot] lapere [at] live [dot] be. A CUDA enabled GPU is required (minimum compute capability 1.1).

I’m planning to build a (very) simple game with this tech. The possibilities are really endless. We're on the cusp of having truly photorealistic games.