LiveView during stage movement

[Stefan Schoenleitner]

Hi,

I'm using Koen's GPhoto DeviceAdapter with a Canon EOS camera and my own
custom DeviceAdapter driving a XYZ stage.
Now that LiveView is working, I was wondering if it can be avoided that
during stage movement, LiveView stops.

I designed my stage driver so that positioning functions like
SetPositionsteps() do not block.
These functions just send the movement commands to the stage.
Polling code is called from Busy() so that as long as the stage has not
yet reached its position, Busy() returns true.
Since the polling code in Busy() only takes a short time, I was thinking
that micro-manager had enough time to do something else (like updating
LiveView) in between.

However, before I start to implement my polling code somewhere else
(i.e. in a separate polling thread), I wanted to ask whether LiveView is
supposed to work at all during stage movement.
IMHO working LiveView during stage movement is also necessary for
auto-focusing, right ?

I was also wondering whether different DeviceAdapters run as parallel
threads.
If this is the case (which I assume), then changing my DeviceAdapter
wouldn't have any effect on the camera device, correct ?

regards,
Stefan

------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure
contains a definitive record of customers, application performance,
security threats, fraudulent activity, and more. Splunk takes this
data and makes sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-novd2d
_______________________________________________
micro-manager-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/micro-manager-general

[Nico Stuurman-4]

Hi Stefan,

> I was wondering if it can be avoided that during stage movement, LiveView stops.
>
> I designed my stage driver so that positioning functions like SetPositionsteps() do not block.
> These functions just send the movement commands to the stage. Polling code is called from Busy() so that as long as the stage has not yet reached its position, Busy() returns true.

> I wanted to ask whether LiveView is supposed to work at all during stage movement?

How do you move your stage? The stage control plugin uses the function "gui_.setRelativeXYPosition()".  The implementation moves the stage, and then calls "core_.waitForDevice()", hence live mode stops.  You can easily test whether this is the issue by opening the Beanshell script panel and typing the command:

core_.setRelativeXYPosition("XYStageName", 100, 100);

while live mode is running.  

There is something to be said for changing the Stage Control plugin and call the core function directly and not wait for the stage to finish moving.


> I was also wondering whether different DeviceAdapters run as parallel threads.

No, they do not, unless they create their own threads, they will run on whatever thread they are called from.  The Micro-Manager Core currently does manage threads in any way (in fact, it is certainly not thread safe).


Best,

Nico


------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure
contains a definitive record of customers, application performance,
security threats, fraudulent activity, and more. Splunk takes this
data and makes sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-novd2d
_______________________________________________
micro-manager-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/micro-manager-general

[Stefan Schoenleitner]

Hi Nico,

On 11/19/2011 07:55 PM, Nico Stuurman wrote:
>> I wanted to ask whether LiveView is supposed to work at all during stage movement?
>
> How do you move your stage? The stage control plugin uses the function "gui_.setRelativeXYPosition()".
> The implementation moves the stage, and then calls
"core_.waitForDevice()", hence live mode stops.

yes, I was using the stage control plugin for my tests.
Am I right by assuming that all code parts that call stage positioning
functions also somehow call the Busy() callback (i.e. through
core_.waitForDevice()) ?

Does this mean that *every* positioning action always stops LiveView ?
If so, isn't this a problem for things like software autofocus ?
Apart from that, I would think that actually seeing the stage movement
through LiveView is a nice-to-have feature as it gives visual movement
feedback to the user.


> You can easily test whether this is the issue by opening the Beanshell
script panel and typing the command:
>
> core_.setRelativeXYPosition("XYStageName", 100, 100);
>
> while live mode is running.  

Thanks, I'll try that.

> There is something to be said for changing the Stage Control plugin and call the core function directly and not wait for the stage to finish moving.

Well, I think that waiting until the stage has reached its position is
something really rudimentary.
However, just because the the stage is busy, it shouldn't mean that
micro-manager can't do anything else in between.

If one device being busy really stalls all other devices, I would say
that this is doesn't speak for the design.
However, you might have your reasons to do it this way, I don't know.


Since I made the assumption that Busy() would not stall other parts of
micro-manager, I might have implemented my DeviceAdapter the wrong way.
At the moment calling any positioning function just sends the
positioning command to the stage.
Then, in Busy(), I poll the stage to see whether it has reached its
position and stopped moving.
As long as it has not, I return "true".

So, if Busy() really blocks everything else, I only see the following
options:

1) The positioning functions just send the movement commands to the
stage, but after that we don't care what happens afterwards.
But just hoping that everything went well is not a safe concept in my
opinion.

2) All positioning functions are blocking.
I did this in the beginning and quickly became aware that this also
freezes the GUI. So this is not really a good idea.

In my eyes, both options are no real solution.

>
>
>> I was also wondering whether different DeviceAdapters run as parallel threads.
>
> No, they do not, unless they create their own threads, they will run on whatever thread they are called from.
> The Micro-Manager Core currently does manage threads in any way (in
fact, it is certainly not thread safe).

Good to know.
I have multiple threads in my DeviceAdapter and I was already wondering,
whether calls to core functions are actually thread safe.

Wouldn't it be better if micro-manager would be multi-threaded ?
Although it would require the core design to be thread-safe, I guess it
would make a lot easier (i.e. writing drivers, plugins, etc.).
Is this something that is on the micro-manager TODO list ?

Regards,
Stefan

>
>
> Best,
>
> Nico
>
>
> ------------------------------------------------------------------------------
> All the data continuously generated in your IT infrastructure
> contains a definitive record of customers, application performance,
> security threats, fraudulent activity, and more. Splunk takes this
> data and makes sense of it. IT sense. And common sense.
> http://p.sf.net/sfu/splunk-novd2d
> _______________________________________________
> micro-manager-general mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/micro-manager-general

------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure
contains a definitive record of customers, application performance,
security threats, fraudulent activity, and more. Splunk takes this
data and makes sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-novd2d
_______________________________________________
micro-manager-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/micro-manager-general

[Nico Stuurman-4]

Hi Stefan,

>>> I wanted to ask whether LiveView is supposed to work at all during stage movement?
>>
>> How do you move your stage? The stage control plugin uses the function "gui_.setRelativeXYPosition()".
>> The implementation moves the stage, and then calls "core_.waitForDevice()", hence live mode stops.
>
> yes, I was using the stage control plugin for my tests.

> Am I right by assuming that all code parts that call stage positioning functions also somehow call the Busy() callback (i.e. through
> core_.waitForDevice())?

No, that is completely up to the UI code.  

> Does this mean that *every* positioning action always stops LiveView ?

No, it does not.  A call to core_.setXYPosition() - as opposed to gui_.setXYPosition() - should not stop live mode.

> If so, isn't this a problem for things like software autofocus ?

Since the answer is no, this is not a problem.

> Apart from that, I would think that actually seeing the stage movement through LiveView is a nice-to-have feature as it gives visual movement feedback to the user.

The reason that I had the stage control plugin call the waitForDevice is that it is very easy to send multiple commands to move to the stage, likely causing problems for the device.  What may make sense is to first wait until the stage is no longer busy, and then send the command to move (which would cause the live display to stutter when you click the arrows twice in short succession).  Alternatively, the plugin could ignore movement commands until the stage is done moving.  Or, the stage control plugin could start its own thread on which it sends the commands for the stage to move (after waiting for it to become non-busy).  Not easy, since this needs data to be send from one thread to another.  As you see, I took the easy way out by moving the stage and then waiting for it to become non-busy.  This is all an UI issue, not an issue with the underlying structure.  Let me know how you would like the plugin to behave.

> You can easily test whether this is the issue by opening the Beanshell script panel and typing the command:
>>
>> core_.setRelativeXYPosition("XYStageName", 100, 100);
>>
>> while live mode is running.  
>
> Thanks, I'll try that.
>
>> There is something to be said for changing the Stage Control plugin and call the core function directly and not wait for the stage to finish moving.
>
> Well, I think that waiting until the stage has reached its position is something really rudimentary.
> However, just because the the stage is busy, it shouldn't mean that micro-manager can't do anything else in between.
>
> If one device being busy really stalls all other devices, I would say that this is doesn't speak for the design.
> However, you might have your reasons to do it this way, I don't know.

Hopefully explained clearly above.  The "lock-up" is caused by the plugin calling the waitForDevice() function that keeps on polling your stage's Busy() function.  The plugin runs on the EDT, as does the live window update.

> Since I made the assumption that Busy() would not stall other parts of micro-manager, I might have implemented my DeviceAdapter the wrong way.

Your assumption is correct, so I don't see your point.  It seems that you don't like the stall caused by the Stage Control Plugin.  As discussed above, there are various options, none of which I like very much.

> At the moment calling any positioning function just sends the positioning command to the stage. Then, in Busy(), I poll the stage to see whether it has reached its position and stopped moving. As long as it has not, I return "true".

Good.  That is how things should be.

> So, if Busy() really blocks everything else, I only see the following options:

Again, this is not true.  Setting Busy() to false will only block when the UI code calls the waitForDevice() function.  In some cases, it really should do so.  Since your assumption does not appear to be valid, I will not go into the points you raised.

>>> I was also wondering whether different DeviceAdapters run as parallel threads.
>>
>> No, they do not, unless they create their own threads, they will run on whatever thread they are called from.
>> The Micro-Manager Core currently does manage threads in any way (in fact, it is certainly not thread safe).
>
> Good to know. I have multiple threads in my DeviceAdapter and I was already wondering, whether calls to core functions are actually thread safe.
>
> Wouldn't it be better if micro-manager would be multi-threaded ? Although it would require the core design to be thread-safe, I guess it would make a lot easier (i.e. writing drivers, plugins, etc.).

> Is this something that is on the micro-manager TODO list ?

Yes.  However, we have been discussing this for a long time now and realize that it is a difficult task.  There is no ETA for this.

Best,

Nico


------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure
contains a definitive record of customers, application performance,
security threats, fraudulent activity, and more. Splunk takes this
data and makes sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-novd2d
_______________________________________________
micro-manager-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/micro-manager-general

[Stefan Schoenleitner]

Hi Nico,

On 11/19/2011 11:53 PM, Nico Stuurman wrote:
>> Does this mean that *every* positioning action always stops LiveView ?
>
> No, it does not.  A call to core_.setXYPosition() - as opposed to gui_.setXYPosition() - should not stop live mode.

thank you for explaining.
I had a look at the micro-manager Java code, it is now clearer to me.

> What may make sense is to first wait until the stage is no longer
busy, and then send the command to move
> (which would cause the live display to stutter when you click the
arrows twice in short succession).

Yes, this is how it is working for me right now (since my Busy()
implementation returns true as long as the stage is moving).

> Alternatively, the plugin could ignore movement commands until the
stage is done moving.

Hmm, yes, that's the easy way, but it does not provide visual feedback
to the user whether the stage is busy or not.

> Or, the stage control plugin could start its own thread on which it
sends the commands for the stage to move
> (after waiting for it to become non-busy).  Not easy, since this needs
data to be send from one thread to another.
> As you see, I took the easy way out by moving the stage and then
waiting for it to become non-busy.
> This is all an UI issue, not an issue with the underlying structure.
> Let me know how you would like the plugin to behave.

I believe from a longterm perspective it would make sense if the stage
control plugin would use the threaded model.

>From a user perspective, I think that it would be nice to have visual
feedback from the stage control plugin.
So if you press a button, it should stay pressed as long as the stage is
moving, but without interrupting live view.
As you mentioned, in the current implementation, whenever a button is
pressed, it sends the command and then blocks through waitForDevice().
Since the "button pressed" UI function does not return, this also blocks
live view.

In contrast to that, the "button pressed" function should send the
command, start a thread that calls waitForDevice() and return immediately.
The tricky part is that whenever the button is pressed, it should "stay
pressed" as long as the device is not ready which also means that the
button can't be pressed again.
As soon as in the thread waitForDevice() returns, the thread should make
the button click-able again.
This way it is ensured that no movement command is sent to the stage, if
it is still busy.
If the UI objects for the stage control plugin would run as
"synchronized" code, it might be possible to just set the button state
out of the thread.
I guess that in Java, this is something that can be easily done without
too much work involved.

Maybe you have a better solution, but I think the above would also work
without being too complicated to implement.
Let me know what you think.


Besides, if you are changing the stage control plugin, you could also
add buttons for stage calibration.
I know that at the moment the XYStage can be calibrated through the "XY
List" dialog, but for the ZStage there is no way to calibrate it through
the UI.
(ATM you need to call SetOrigin through beanshell.)

So if you believe that it makes sense, maybe you can add these buttons
for both stage types.

>> You can easily test whether this is the issue by opening the Beanshell script panel and typing the command:
>>>
>>> core_.setRelativeXYPosition("XYStageName", 100, 100);
>>>
>>> while live mode is running.

I tested it and it's exactly like you said.
If no waiting code is involved, live view is not interrupted.

Regards,
Stefan

------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure
contains a definitive record of customers, application performance,
security threats, fraudulent activity, and more. Splunk takes this
data and makes sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-novd2d
_______________________________________________
micro-manager-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/micro-manager-general

[Nico Stuurman-4]

On Nov 21, 2011, at 1:52 AM, Stefan Schönleitner wrote:

>> Alternatively, the plugin could ignore movement commands until the stage is done moving.
>
> Hmm, yes, that's the easy way, but it does not provide visual feedback to the user whether the stage is busy or not.

I have implemented this approach (it should be in the current nightly build).  Works as expected (as far as I can tell).

>> Or, the stage control plugin could start its own thread on which it sends the commands for the stage to move
>> (after waiting for it to become non-busy).  Not easy, since this needs data to be send from one thread to another.
>> Let me know how you would like the plugin to behave.
>
> I believe from a longterm perspective it would make sense if the stage control plugin would use the threaded model.
>
>> From a user perspective, I think that it would be nice to have visual feedback from the stage control plugin. So if you press a button, it should stay pressed as long as the stage is moving, but without interrupting live view. The "button pressed" function should send the command, start a thread that calls waitForDevice() and return immediately. The tricky part is that whenever the button is pressed, it should "stay pressed" as long as the device is not ready which also means that the button can't be pressed again. As soon as in the thread waitForDevice() returns, the thread should make the button click-able again. This way it is ensured that no movement command is sent to the stage, if it is still busy. If the UI objects for the stage control plugin would run as "synchronized" code, it might be possible to just set the button state out of the thread. I guess that in Java, this is something that can be easily done without too much work involved.

>> Maybe you have a better solution, but I think the above would also work without being too complicated to implement. Let me know what you think.

I am happy with the current implementation and have absolutely no time to work on this.  I'll consider patches though;)

Best,

Nico
------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure
contains a definitive record of customers, application performance,
security threats, fraudulent activity, and more. Splunk takes this
data and makes sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-novd2d
_______________________________________________
micro-manager-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/micro-manager-general