xmlgraphics-fop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeremias Maerki <...@jeremias-maerki.ch>
Subject Re: Images in FOP 0.92beta
Date Fri, 14 Jul 2006 20:26:57 GMT
That was worth more than 2 cents. Thanks, Thomas. I didn't really care
too much about left-over references at first, but in a long-running
service they add up unnecessarily even if it's only a Map.Entry, a
String and a Reference instance per entry.

At first, I'd have preferred to avoid an extra thread if possible so I
just added a local ReferenceQueue and used poll() to do house-keeping
whenever a user agent signs off. I assume you don't have a
non-too-frequently called method you could do on-demand house-keeping in,
so the thread is probably ok. And given that we have Batik in memory
anyway FOP could co-use that thread. But since I'd like to avoid
dependencies on Batik directly if possible, can we move CleanerThread to
XML Graphics Commons and rename it to ReferenceCleanerThread to give it
a more speaking name? In the beginning, this means we will have two
threads doing the same thing but it is ultimately cleaner design in the
long run (when Batik starts using Commons).

The SoftReferenceCache is indeed a little odd, especially the method
names. I think I'll skip that one for now.

Some other interesting things I observed while playing around for those
interested (ATM, I'm still doing the house-keeping without the thread
but I might rewrite):

When using weak references (as the current code does but with the fixed
behaviour) FOP takes around 35 sec on my machine to produce that 182
image PDF. Heap usage is usually around 12MB with peaks to 26MB. The
house-keeping after the user agent retires removes around 178 references.

Switching to soft references which is actually the recommended type for
caches, the heap usage goes up to the 64MB maximum and pretty much stay
there. The whole thing takes 29-30 sec average. The house-keeping after
the user agent retires removes between 161 and 170 references. So this
means the VM actually keeps more references around, only freeing as many
as it needs not to run into memory problems. And it runs faster this way.

I learned a few things today. :-)

On 14.07.2006 14:35:06 thomas.deweese wrote:
> Hi all,
> 
>     Just a small comment on HashMaps with weak values:
> 
> Jeremias Maerki <dev@jeremias-maerki.ch> wrote on 07/13/2006 04:43:07 PM:
> 
> > Ok, so I changed the WeakHashMap to a HashMap and wrapped the values in
> > WeakReferences. Tadaaa! A PDF with 182 JPEG images with a total size of
> > 258 MB is suddenly produced without exceptions using the VM's default
> > heap settings, never going beyond 26MB heap usage. *g*
> 
>    There is a potential problem with this approach that Batik ran into.
> Unless you go a little further those weak values accumulate in the map. 
> In your case this probably isn't a big deal, but for Batik where there
> are potentially of thousands (or tens of thousands, think mouse move 
> events) 
> of entries, these 'dead' entries start to add up.
> 
>    As a result Batik has batik.util.CleanerThread.  This class has
> inner classes that subclass the various SoftReference classes with an 
> additional method 'public void cleared()'.  This method is called by
> the CleanerThread when the object the soft reference is point at is
> cleared from memory (it uses the ReferenceQueue part of soft references).
> 
>    This gives you the hook you need to then de-register the entry from
> the has table.  This is actually an incredibly useful 'addition' to
> the standard soft reference classes (for example I will often use
> it to check if classes I think should go to GC really do go to GC).
> 
>    I should also mention that Batik has a class called 
> 'SoftReferenceCache'
> which is a thread safe implementation of exactly what you just 
> implemented. 
> The interface may seem a little odd but it is designed to ensure that
> only one party ever has to decode a resource even if multiple threads
> request it "at the same time".
> 
>    Anyway just thought I would add my 2 cents...



Jeremias Maerki


Mime
View raw message