Torturing image ops and Swing

hendrich at informatik.uni-hamburg.de hendrich at informatik.uni-hamburg.de
Thu Aug 3 13:59:00 UTC 2006


Hello all,

anyone interested in torturing our Swing, awt.image and javax.image operations
a bit? Long text with a question buried at the end. Sorry, but I had  
to get this
off my soul; feeling much better already :-)



I have just uploaded a new, completely re-written version of my image  
viewer to
http://tams-www.informatik.uni-hamburg.de/personal/hendrich/niffler/index.html

Unlike the previous version with its deeply-nested popup-menu, the new version
uses a more traditional user-interface. It also adds image histogram,  
EXIF support,
and basic image editing based on get/setRGB, get/setRaster, ConvolveOp, and
ShortLookupOp (ByteLookupOp would be faster, but is broken on the JDK/Linux).


Download niffler-exif.jar (or the sources and build it yourself; you  
will need the
metadata-extractor library from www.drewnoakes.com for building the  
exif stuff).

jamvm -Xmx300m -Xms100m niffler.Niffler

1. Select menu > File > Open image directory...  and select a directory with
    some images. This is the first challenge, because JFileChooser is still
    about as user-unfriendly as possible. (I somewhat fear that many first-time
    classpath users might give up after trying to use JFileChooser. This is
    unfortunate, because the rest of Swing works pretty well these days.)

2. By default, Niffler also includes subdirectories in its search.  
Don't select
    your home directory or this can take a long time (about one minute with
    jamvm+cvs in my home directory, with 93.000+ files. For some reason,
    cacao 0.96 crashes after listing about 15.000 files. Smaller  
directories work.)

3. Loading small to medium-sized images works fine. Navigation works fine
    (type 'space' or 'n' for next image, 'p' for previous image, or  
use the menu).
    Zooming works fine ('f' for zoom-fit, 'g' for original-size,  
etc.). Mouse dragging
    works fine. The navigation tree, thumbnails preview, and histogram all work
    as they should (a little slow, perhaps, but jamvm is an  
interpreter, after all).
    Nitpicking: the splitpane dividers look bad.

4a. Loading typical digicam images with 4+ Mpixels is somewhat slower,  
but still
    almost acceptable. jamvm needs about 5 seconds for a 3000x2000 JPEG for
    image loading, plus about 7 seconds for calculating the histogram  
(when enabled).
    JDK 1.5 needs about 1.5 seconds for loading plus 0.3 secs for the  
histogram.

4b. Cacao calculates the histogram much faster (almost as fast as the  
JDK), but
      unfortunately it leaks memory and crashes after loading a few  
4+Mpixel images
      (with -Xmx300m and top reporting about 320M RSS actually used.)

      It seems that cacao 0.96 never garbage-collects image data?

4c. I didn't test with gcj yet, neither did I try jcvm.

---

For the following, use smaller images (800x600 or so) to avoid frustration.

5. Select Tools > Sharpen > Laplace 3x3. Simple convolution filter implemented
     with ConvolveOp and applied to a BufferedImage TYPE_INT_RGB.

    java.lang.ArrayIndexOutOfBoundsException: 3
    at java.awt.image.ColorModel.getComponentSize(ColorModel.java:200)
    at java.awt.image.ColorModel.coerceData(ColorModel.java:641)
    at java.awt.image.DirectColorModel.coerceData(DirectColorModel.java:405)
    at java.awt.image.BufferedImage.coerceData(BufferedImage.java:288)
    at java.awt.image.ConvolveOp.filter(ConvolveOp.java:126)

    This worked a week ago, but very slowly. Try the Tools > Edges > Mexican
    Hat 13x13 filter, if you don't believe me. The JDK seems to include some
    optimizations for such (separable?) kernels.

    OK, lets try something else:

6. Select Tools > Negative Image. Obvious implementation based on
     LookupOp. Works. Performance is ok (1 sec for 800x600). The result is
     a BufferedImage.TYPE_INT_RGB.

     BUT repainting suddenly takes 3 seconds for each paintComponent,
     and the application is pretty much dead. For comparison, a repaint of the
     BufferedImage before the filtering took about 10 msec.

     For 3000x2000 images, each repaint takes 40 seconds on my system.
     Any ideas about what I am doing wrong here are HIGHLY appreciated.

     Load a new image. Repainting time is back to the millisecond range.

7. Perhaps LookupOp and ConvolveOp are bad? Select Tools > Rotate
     image left (or right). Implemented 'by hand' via getRGB and setRGB.
     Much slower than LookupOp, about 4 seconds on my system at 800x600.

     But again, repainting suddenly takes many seconds.

8. What about ImageIO instead of java.awt.Toolkit?  Just select
     Edit > Load images via ImageIO.

     Loading a 800x600 JPEG takes about 200 msecs with Toolkit, and about
     7 seconds with Toolkit. Loading a 3000x2000 JPEG takes 200+ seconds.

     The imageio GIF reader is much faster (4 seconds at 3000x2000), but now
     the conversion to BufferedImage.TYPE_INT_RGB takes 90+ seconds...

     Images returned by the PNG reader render as transparent.

9. Select Help > Commands...  A simple JTextArea in a JScrollPanel, but
     with about 700 lines of text. Try scrolling. Painfully slow. The vertical
     scrollbar only scrolls down on click event (but dragging works).

     I also get a lot of these:

   (.:2449): GdkPixbuf-CRITICAL **: gdk_pixbuf_new: assertion `height  
 > 0' failed
   java.lang.InternalError: GdkPixbuf: gdk_pixbuf_new: assertion  
`height > 0' failed
    at gnu.java.awt.peer.gtk.GtkVolatileImage.nativeCopyArea(Native Method)
    at  
gnu.java.awt.peer.gtk.GtkVolatileImage.copyArea(GtkVolatileImage.java:74)
    at  
gnu.java.awt.peer.gtk.VolatileImageGraphics.copyAreaImpl(VolatileImageGraphics.java:67)
    at  
gnu.java.awt.peer.gtk.CairoGraphics2D.copyArea(CairoGraphics2D.java:1148)
    at javax.swing.JViewport.paintBlit(JViewport.java:916)
    at javax.swing.JViewport.paint(JViewport.java:515)

10. Neither JOptionPane (Help > About) nor JToolTip include support for
     HTML formatting (e.g. the histogram tooltip). Audrius told me  
that the HTML
     parser part already works for my examples, but the parser isn't used...

---

Overall, I am quite happy about the current state of Swing. JFileChooser
obviously needs a lot of work, and JTextArea is unuseable when it holds
more than a few lines of text. It would also be nice to have HTML support
on JOptionPane and JToolTip, but this is hardly mission-critical.

However, what should I do about the repaint performance issue? Is there
an inherent reason why images of type BufferedImage.TYPE_INT_RGB
render so slowly? (I also tried INT_ARGB, but this doesn't help, needs
more memory, and needs postprocessing for the ConvolveOp filters).
Any obvious workaround?

Thanks in advance,
   Norman






































More information about the Classpath mailing list