Saving large number of images and circular buffer overflow

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Saving large number of images and circular buffer overflow

Nico Stuurman-2
Hi all,

It has come to my attention that several people are running into issues
saving large numbers of images acquired at high speed. Basically,
running a modern sCMOS camera at full speed generates images at a rate
of ~ 100 (fps) * 2000 (width) * 2000 (height) * 2 (bytes per pixel ) =
800 MB/sec.  Luckily, modern NVMe drive can easily write at the speeds,
so it comes as a surprise to some that MM does not always keep up.

Back when sCMOS cameras just became available, Arthur Edelstein put a
lot of effort into making MM keep up with them, and he managed to set up
a system that could keep up (writing to an array of SSDs, which back in
the day was the only way to get high enough disk speed).  During further
development of MM, that feature has not been forcefully maintained (in
part because our lab does not use it), so it does not surprise me that
saving speed apparently has gone down over time.

To test pure saving speed, you can use the demo configuration, set the
demo camera to 2048 x 2048, 2 bytes per pixel, and set the FastImage
property to "1" (that way, the cpu does not need to generate a new image
every time.  I used a modified version of the burstAcquisition script
(see below) to test saving speed.  On my laptop with NVMe drive (speeds
 > 2 GB/s) I failed to save a sequence when the exposure time is set to
10 ms (100 fps), but doe succeed with exposures at 20 ms (50 fps).  That
suggests that performance is not far off, but not enough for people who
want this 100 fps full frame.

I will not have the bandwidth to work on this more, but it would be
great if whomever needs this could put in the effort to make saving work
again at higher speeds (the nice thing is that the saving code is very
modular, so even writing to other file formats is feasible by writing
the applicable code).

The script I used for testing is here:


/**
  * Example of running sequence acquisitions (a.k.a. burst acquisitions).
  */

nrFrames = 20000;
exposureMs = mmc.getExposure();
location = "C:\\Users\\NicoLocal\\tmp\\MMtemp\\t1";

// Create a Datastore for the images to be stored in on disk
store = mm.data().createMultipageTIFFDatastore(location, false, false);

// Start collecting images.
// Arguments are the number of images to collect, the amount of time to wait
// between images, and whether or not to halt the acquisition if the
// sequence buffer overflows.
mmc.startSequenceAcquisition(nrFrames, 0, true);
// Set up a Coords.CoordsBuilder for applying coordinates to each image.
builder = mm.data().getCoordsBuilder().z(0).channel(0).stagePosition(0);
int curFrame = 0;
while (mmc.getRemainingImageCount() > 0 ||
mmc.isSequenceRunning(mmc.getCameraDevice())) {
    if (mmc.getRemainingImageCount() > 0) {
       tagged = mmc.popNextTaggedImage();
       // Convert to an Image at the desired timepoint.
       image = mm.data().convertTaggedImage(tagged,
          builder.time(curFrame).build(), null);
       store.putImage(image);
       curFrame++;
    }
    else {
       // Wait for another image to arrive.
       mmc.sleep(Math.min(.5 * exposureMs, 20));
    }
}

mmc.stopSequenceAcquisition();

mm.scripter().message("Stored " + curFrame + " images.");





_______________________________________________
micro-manager-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/micro-manager-general