[OpenSPIM] OpenSPIM 64-bit [0000129]

[Nikul]

> In the concrete case of writing data to a RAID as fast as possible, I
> would suggest writing *raw* data, though. You can always stitch back the
> metadata. The only metadata you have to store at the same time as the
> frames you are writing out are the metadata that *do* change with the
> frame, i.e. z position and time. All the other metadata can be written out
> at a much, much slower pace, some of them actually need to be written only
> once per experiment.

That's a good idea! I will try that out as well.

> P.S.: It might make sense to experiment with a low-impact compression
> scheme such as LZO (e.g. via https://github.com/Karmasphere/lzo-java)
> because it can reduce a large chunk of the redundancies at the cost of CPU
> but at the benefit of the network or processor bus. If you find time to
> experiment with this, I would be very much interested in hearing how you
> fared!

We might look at some sort of compression later, but for now I am going
to concentrate on trying to maximize throughput.

---------------------------------------------
Nikul Ukani

Doctoral Candidate,
Dept. of Electrical Engineering,
Columbia University.
http://www.bionet.ee.columbia.edu/


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://openspim.org/pipermail/openspim/attachments/20140205/9050395d/attachment-0002.html>