Chet and I are almost done with our incoming book, Filthy Rich Clients. Last week, I decided to add a fun demo to the book to show how to implement a bloom effect.
Blooms are widely spread in modern video games to spread out light sources. It works usually very well in games with a particular mood, like The Legend of Zelda: Twilight Princess, ICO or Shadow of the Colossus. This effect is also often overused in First Person Shooters (FPS), in which you can sometimes see rusty or wooden surfaces shine like a sun. The picture below shows what the bloom effect looks like in Far Cry. Notice how the sun light spreads out.
A bloom is quite easy to implement. You must start with a bright-pass filter that makes all of the dark pixels black, thus retaining only the bright pixels. The result is then blurred several times and all the generated images are added to the original image, using a composite like Screen or Add.
Because my implementation applies four Gaussian blurs on the image, the result is a bit slow. No matter how hard you optimize the code, and I did, you won’t get far in pure software. On my MacBook Pro, the Java 2D version of the bloom runs in about 120ms.
I then decided to implement the effect using OpenGL’s pixel shaders, or fragment programs. With the help of Chris Campbell, I have been able to rewrite the effect using JOGL and the OpenGL Shading Language (GLSL). It required a lot more code, due to the nature of OpenGL itself —a rather low-level API— but the result was worth it. Using JOGL, my bloom implementation runs in about 4ms.
I have been wanting to implement a pixel shader based image filter for quite a long time and I am glad I finally did it. I discovered GLSL and I must confess I am tempted to use it for any future image processing task as it makes things a lot easier in some situations. As a comparison, let us take a look at the implementation of the bright pass filter in Java and then in GLSL.
The Java version is pretty straightforward but requires some fiddling with the bits:
private void brightPass(int[] pixels, int width, int height) {
int threshold = (int) (brightnessThreshold * 255);
int r;
int g;
int b;
int luminance;
int[] luminanceData = new int[3 * 256];
// pre-computations for conversion from RGB to YCC
for (int i = 0; i < luminanceData.length; i += 3) {
luminanceData[i ] = (int) (i * 0.2125f);
luminanceData[i + 1] = (int) (i * 0.7154f);
luminanceData[i + 2] = (int) (i * 0.0721f);
}
int index = 0;
for (int y = 0; y < height; y++) {
for (int x = 0; x < width; x++) {
int pixel = pixels[index];
// unpack the pixel's components
r = pixel >> 16 & 0xFF;
g = pixel >> 8 & 0xFF;
b = pixel & 0xFF;
// compute the luminance
luminance = luminanceData[r * 3] + luminanceData[g * 3 + 1] +
luminanceData[b * 3 + 2];
// apply the treshold to select the brightest pixels
luminance = Math.max(0, luminance - threshold);
int sign = (int) Math.signum(luminance);
// pack the components in a single pixel
pixels[index] = 0xFF000000 | (r * sign) < < 16 |
(g * sign) << 8 | (b * sign);
index++;
}
}
}
This Java version takes about 0.3ms on my machine to process a 512×256 image so it was not the performance bottleneck. I implemented it as a shader for educational purpose, and to practice before writing the Gaussian blur shader. Besides, looping through all of the pixels, unpacking the color components and packing the filtered results is quite tedious. Take a look at the GLSL implementation to see what I mean:
uniform sampler2D baseImage;
uniform float brightPassThreshold;
void main(void) {
vec3 luminanceVector = vec3(0.2125, 0.7154, 0.0721);
vec4 sample = texture2D(baseImage, gl_TexCoord[0].st);
float luminance = dot(luminanceVector, sample.rgb);
luminance = max(0.0, luminance - brightPassThreshold);
sample.rgb *= sign(luminance);
sample.a = 1.0;
gl_FragColor = sample;
}
GLSL is very close to C and supports conditionnal expressions, loops, variables assignments, etc. Even if you know nothing about GLSL, you can see some interesting things in this code snippet. The data type include floating point vectors and matrices, making it really easy to work with pixels.
For instance, a pixel can be stored as a 4 components vector (alpha, red, green and blue). This lets you do crazy computations very easily. In the example above, the instruction sample.rgb *= sign(luminance) multiplies the red, green and blue components of a pixel by 0 or 1. Similarly the luminance —the Y component in the YCC color model— is the result of the dot product between the pixel’s RGB components vector and the luminance vector.
Unfortunately, as I already said, OpenGL requires to write quite a lot of code to setup and apply a fragment program on an image. I am seriously thinking about writing a high-level API to simplify this and come closer to a Java version of Apple’s Core Image for instance. Or maybe I’ll just ask Chris until he gives up and write it for me :-) He’s the OpenGL mastermind after all.
Bloom works best in high-constrast scenes, which is often the case of video games. Yet, you can successfully use this effect in other situations; mostly on logos and splash screens. The picture below shows Aerith splash screen without and with a bloom:
And if you think this is silly, take a look at the next two screenshots. Those show what a Swing application using Substance, Kirill’s look and feel, might look like with bloom on. Considering the number of visual effects this look and feel provides, I thought this might be a perfect fit ;-)
Last but not least, don’t worry about finding other useless demos in our book, this is just a bonus that shows how to combine several graphics techniques explained in other chapters.
Please more “useless” demos like that ;-)
I wrote about shaders a while back, I think Java can have a serious effect in this domain:
http://jroller.com/page/vprise?entry=writing_3d_shaders
Saa-weeeet. Glad to see you used my tips for good and not evil. Sometime in the not-so-distant future these effects should come for free simply by enabling the OpenGL-based Java 2D pipeline (some details omitted of course). Until then, we can make this stuff easier in JOGL; now all I need is time (and all you need is cake).
Wow ! That looks great, now can you make the bloom animate and move around the components ;)
Just when Java/OpenGL seem to be working together so nicely its ironic that Vista OpenGL support is next to useless. Its almost as if Microsoft and ATI/Nvidia have conspired to kill off OpenGL in Windows land…
I’m really looking forward to reading Filthy Rich Clients now, hurry up and finish it please ;)
Rob
You KNOW where I’m going to use this…. front most carousel component bloom here we come!
bugfaceuk: Send me an email if you want the source code of these effects :)
nice, where is the webstart link? ;)
When the book is out :p
Looking great. Is the source code based on JDK 6.0? I’m wondering why the bloom around the text (in the last two screenshots) is so focused around specific parts of the text? Say, with “enabled unselected” checkbox it seems to concentrate around “abl” in the first word instead of spreading around the entire text.
The source code works on J2SE 5.0. As for the effect on text, this can be fixed by tuning the smoothness down to 1. Smoothness actually controls the amount of blur I apply prior to the bright-pass filter. Thus, with a high smoothness factor, the “abl” has more focus because it contains more pixels (due to the b and l letters) and spread out more. Drop me an email, I’ll send you the demo so you can see for yourself.
Go on bugfaceuk, send an email and then publish a “bloomy” carousel before the book is out ;^)
Solidarité linguistique entre français, tu dirais pas plus upcoming que incoming ?
Tout dépend du sens que tu souhaites donner :) Upcoming est en effet le sens commun mais incoming est souvent utilisé pour parler par exemple d’une attaque (on dira ainsi “incoming missile”). Et donc j’ai choisi incoming ^^
One little detail. Looking at your code, isn’t the line:
luminance = luminanceData[r] + luminanceData[g + 1] + luminanceData[b + 2];
supposed to be:
luminance = luminanceData[r*3] + luminanceData[g*3 + 1] + luminanceData[b*3 + 2];
Dean, ooops thanks for catching that bug ^^ Optimisations are always dangerous :p
c’est sûr que mettre du bloom sur substance fait très missile dans la face rom :)
Is it possible to have a look at the source code of these effects?
When the book is out :)
Romain, glad to be of help. In fact I would be happy to look at any other source code you might have…you know, maybe for a book or something.
I know, I know. Shameless. But I couldn’t resist. :-)
> When the book is out :)
Can’t wait for it :)
I’m wondering how the composing could be done. If you have something like forms… in my (maybee false) opinion the compositing and the bloom calculation has to be done based on a “shot” of the form and the “bloom”-layer would be placed on an higher layer.
Please correct me if i’m wrong, but wouldn’t this make the form unusable cause of mouse-event issues?
We explored a similar problem by making a demo for our diploma (you remember my e-mail about the colorpicker)
(In my case assume the blooming effect is to be used in realtime over the hole screen and not only at certain places)
Klemens
This can be done without any problem by using a RepaintManager and a specific content pane for the frame.
Ehm… could you give an additional short hint on how this could be done? (I have no idea on how to use the RepaintManager as I have never done it before)
Maybee there’s another blog entry that explains that by source-samples?
The greatest answer would be a simple portion of source-code with the basic setup on how to place something ontop of the real content without overiding/disabling the mouse-sensitive form-elements
thanks in advance
klemens
Klemens: There’s a chapter on this in the book :p More seriously though, a RepaintManager allows you to catch all the repaint events. What you would do is create a class called BloomPanel. This class would override paint() and paint all its children in a BufferedImage, apply the bloom on the BufferedImage and paint that image on screen. The RepaintManager would be used to tell the BloomPanel to re-apply the effect. You can take a look at SwingX’s JXPanel and RepaintManagerX to get a nice example. Or read the book when it’s out ;-)
And within my own repaintmanager i would then paint the bloomed stuff rather than the normal component-stuff right?
Wow, I am really looking forward to the book now. Untill then, I found a method to create the bloom effect without writing any code.
1) Wear contact lenses for the whole day. 2) Play “Perfect Dark Zero” the whole evening. 3) Forget to take out the lenses. 4) The next day, start any application and… you got the bloom effect all over even outside the screen.
Sorry bout that, but if you get Chris to do the Core Image thingy can you convice him to do some Core Animation stuff as well. I still got some ‘But Java is slow, isn’t it’ colleagues.
Klemens: Indeed, but it might be tricky with the clip.
Vorgi: Nice tip indeed :)
BTW, I’ll post the source code on this web site as soon as I can find some time to clean it up a bit.
WOW!! Simply mind blowing!!
Salut !
Je vous ai envoyé deux email successivement à votre compte de mac.com, et de filthyrichclient.org, mais jusqu’à present je n’ai pas reçu de repense, s’il vous plait je l’attend il y a plus d’une semaine !
Et je ne répondrai pas. Je ne suis pas là pour faire tes devoirs.