[OpenSPIM] 64-bit Fiji Memory Limit Issue

[Johannes Schindelin]

Dear Thomas,

please note that while it is convenient for you to take this problem to 
a private mail exchange with me, it is really inconvenient for me: I 
have nothing to gain from a private mail exchange solving this issue -- 
it is taking my time, to solve a problem that is not mine, in a plugin 
that is not mine, and the warm and fuzzy feeling of solving a single 
problem for a single person when nobody else knows what I managed to 
accomplish does not further my career nor does it pay my bills -- while 
a public discussion not only offers the possibility that other people 
than you might benefit from my advice (which would in turn help me), but 
also offers the possibility that other people might help you, relieving 
me.

I really wish that more people were trying to help the OpenSPIM 
community thrive as a publicly visible community. There are some 
laudable examples on the OpenSPIM mailing list recently, and I am very 
thankful for that.

On 2015-05-11 16:11, Roberts, Thomas wrote:

> I am still encountering this memory issue, unfortunately.
> 
> I downloaded the latest version of the OpenSPIM software that you sent
> a link to a few weeks ago
> (http://openspim.org/downloads/OpenSPIM-20150427.zip).

Please note that it is not obvious from your screenshots what amount of 
memory Fiji *actually* reserves. Two Fiji instances running side by 
side, for example, will not have the same amount of memory available 
because Java works by reserving the memory at start-up, and that memory 
is then not available to a second Fiji instance. The ImageJ launcher 
will detect that less memory is available than it was asked to ask for 
and fall back to try with a lesser amount. That is my guess regarding 
what happens on your computer.

By clicking on the status bar, you will see how much memory Fiji's Java 
VM thinks is available. Clicking on the status bar will also trigger a 
so-called "garbage collection", i.e. it will ask the Java VM to 
re-organize its memory consumption. Therefore, sporadically clicking on 
the status bar while the plugin is running might even work around the 
memory issue.

> However, I still can't open large images in Fiji. I am trying to run
> through the Bead Based Registration tutorial on OpenSPIM.org, but I
> encounter the Memory Limit issue - I've attached some PNGs showing the
> error messages.
> 
> I tried your suggestion of opening the memory monitor, but it doesn't
> really tell me much - just that the program spikes and stops once it
> reaches the 494MB memory limit (see image1.png).
> 
> I still don't understand why Fiji thinks it has 5000MB of memory
> available, but crashes at 500MB (see image2.png).

You really scared me when you said that it crashed. You see, "to crash" 
means one very specific thing, and that thing only, when you talk about 
computer programs: it means that the entire application just stopped to 
run, all its windows vanished, without any error message nor a chance to 
store so-far unsaved data.

That is distinctly not what happened. What you saw is an 
OutOfMemoryException which stopped the plugin but nothing else.

> Also, I have tried running the registration tutorial on my office
> computer, and this works, but the machine is very slow for such large
> datasets.

It might make sense to open large images in that Fiji instance first, 
until it runs out of memory, in order to determine whether the problem 
lies with the Java VM in general, or with the plugin you are trying to 
run. I am getting the impression that something is wrong with your Fiji 
installation rather than with the SPIM Registration plugin.

Ciao,
Johannes