What's the correct way to wait in a DeviceAdapter ?

[Stefan Schoenleitner]

Hi,

in my DeviceAdapter I implemented several functions that need to wait
for en event to occur.

For instance I have a function that performs stage calibration.
The function sends several commands to the stage and uses stage polling
commands to determine whether some of these commands have finished.
Once they are finished, more commands may have to be executed until
the stage calibration is finally finished.

Another example would be stage positioning.
In SetPositionSteps() I tell the stage to move to the the given coordinates.
After that I use stage polling commands to determine whether the
position could be reached within a given time frame.

Thus all my positioning functions are blocking.
Once they return, the stage is either finished with the requested action
or an error code is returned (which is just like other DeviceAdapters do
as well).

However, as I do not wan't to "hammer" the stage with polling commands,
I added short sleeps to the polling loops:

        CDeviceUtils::SleepMs(100);


So far the effect of that is exactly what I wanted to avoid:
The stage does what it should do, but at the time the stage moves not
only the micro-manager application, but also the complete Linux GUI
freezes and is non responsive.

How can I change this ?
What is the "correct way" to poll a device ?

I grepped over the other DeviceAdapter implementations and found nearly
300 references to SleepMs().
So I don't think that calling SleepMs() is something one shouldn't do,
right ?

Regards,
Markus

------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure contains a
definitive record of customers, application performance, security
threats, fraudulent activity and more. Splunk takes this data and makes
sense of it. Business sense. IT sense. Common sense.
http://p.sf.net/sfu/splunk-d2d-oct
_______________________________________________
micro-manager-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/micro-manager-general

[Stefan Schoenleitner]

Hi,

On 10/19/2011 10:39 AM, Stefan Schoenleitner wrote:
> So far the effect of that is exactly what I wanted to avoid:
> The stage does what it should do, but at the time the stage moves not
> only the micro-manager application, but also the complete Linux GUI
> freezes and is non responsive.

I think we found the culprit.
We're performing the calibration in the "calibration" property callback
function.
Since this is a GUI function, it obviously stalls the execution of all
GUI code.

So we have to move the call to the calibration function to somewhere else.
Any ideas where an appropriate place for that might be ?
Initialize() is already called at that time.
My best guess is to move the call to any place that requires a
calibrated stage and ensure that the calibration is only called once.

Or is it possible to have a "calibrate stage" button somewhere ?

cheers,
Stefan


P.S: In case you were puzzled by the different name: my brother was
asking the last question from my account ;)

------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure contains a
definitive record of customers, application performance, security
threats, fraudulent activity and more. Splunk takes this data and makes
sense of it. Business sense. IT sense. Common sense.
http://p.sf.net/sfu/splunk-d2d-oct
_______________________________________________
micro-manager-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/micro-manager-general

[Nico Stuurman]

On Oct 19, 2011, at 3:31 AM, Stefan Schoenleitner wrote:

> Or is it possible to have a "calibrate stage" button somewhere ?

There is the "Set Origin" button in the Stage-Positin List Dialog that will call the Calibrate function in your device adapter.

Best,

Nico


------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure contains a
definitive record of customers, application performance, security
threats, fraudulent activity and more. Splunk takes this data and makes
sense of it. Business sense. IT sense. Common sense.
http://p.sf.net/sfu/splunk-d2d-oct
_______________________________________________
micro-manager-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/micro-manager-general

[Nico Stuurman]

On Oct 19, 2011, at 1:39 AM, Stefan Schoenleitner wrote:

> in my DeviceAdapter I implemented several functions that need to wait
> for en event to occur.

> In SetPositionSteps() I tell the stage to move to the the given coordinates.
> After that I use stage polling commands to determine whether the
> position could be reached within a given time frame.
>
> Thus all my positioning functions are blocking.
> Once they return, the stage is either finished with the requested action
> or an error code is returned (which is just like other DeviceAdapters do
> as well).
>
> However, as I do not wan't to "hammer" the stage with polling commands,
> I added short sleeps to the polling loops:
>
> CDeviceUtils::SleepMs(100);
>
>
> So far the effect of that is exactly what I wanted to avoid:
> The stage does what it should do, but at the time the stage moves not
> only the micro-manager application, but also the complete Linux GUI
> freezes and is non responsive.
>
> How can I change this ?
> What is the "correct way" to poll a device ?

If possible, return immediately after sending out the command from the SetPositionSteps function.  Ask the stage if it was done moving in your Busy function.  This will let the Micro-Manager UI/Core do the polling for you and the UI will remain responsive.

> I grepped over the other DeviceAdapter implementations and found nearly
> 300 references to SleepMs().
> So I don't think that calling SleepMs() is something one shouldn't do,
> right ?

While writing a device adapter, it is often the simplest solution to block until the device finishes executing the command.  However, this approach will slow down the application as a whole.  We are regularly talking about ways to keep device adapters simple, yet avoid commands like these blocking the application.  We have not come up with a simple, reliable solution yet.


Best,

Nico


------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure contains a
definitive record of customers, application performance, security
threats, fraudulent activity and more. Splunk takes this data and makes
sense of it. Business sense. IT sense. Common sense.
http://p.sf.net/sfu/splunk-d2d-oct
_______________________________________________
micro-manager-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/micro-manager-general

[Stefan Schoenleitner]

Hi,

On 10/19/2011 07:02 PM, Nico Stuurman wrote:
>> What is the "correct way" to poll a device ?
>
> If possible, return immediately after sending out the command from the SetPositionSteps function.  Ask the stage if it was done moving in your Busy function.
> This will let the Micro-Manager UI/Core do the polling for you and the
UI will remain responsive.

based on your above response I started to implement positioning like you
suggested.
Hence in the SetPositionSteps I'm sending out the movement command
whereas in the Busy() function I ask the stage whether it is finished yet.
Thus as long as the stage is moving, Busy() returns true to MM.

Unfortunately, there is a problem with that:
-----------------------------------------------------------------
LOG(2011-10-23T15:21:48.345046, 12273, 140173961660160:): Error occured.
Device XYStage. wait timed out after 5000 ms.
-----------------------------------------------------------------

Apparently there is a timeout if consecutive calls to Busy() return true
for more than 5 seconds ?

Since the stage is not fast enough to move from end end to another or
perform calibration within that time frame, I run into these timeouts
most of the time.

Is there a way I can change the 5000ms timeout ?
Since my implementation already checks for timeouts, I'm also
considering turning off the Busy() timeout completely. Is that possible ?

Regards,
Stefan

------------------------------------------------------------------------------
The demand for IT networking professionals continues to grow, and the
demand for specialized networking skills is growing even more rapidly.
Take a complimentary Learning@Cisco Self-Assessment and learn
about Cisco certifications, training, and career opportunities.
http://p.sf.net/sfu/cisco-dev2dev
_______________________________________________
micro-manager-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/micro-manager-general

[Nico Stuurman-4]

On Oct 23, 2011, at 6:34 AM, Stefan Schoenleitner wrote:

> Hence in the SetPositionSteps I'm sending out the movement command
> whereas in the Busy() function I ask the stage whether it is finished yet.
> Thus as long as the stage is moving, Busy() returns true to MM.
>
> Unfortunately, there is a problem with that apparently there is a timeout if consecutive calls to Busy() return true
> for more than 5 seconds ?

> Is there a way I can change the 5000ms timeout ?

Set the property "Core" "TimeoutMs" to the value you desire.  You can do so in the Device/Property Browser (under the Tools menu), or on startup of Micro-Manager by adding it to the "Startup' Preset in the "System" Group.

> Since my implementation already checks for timeouts, I'm also
> considering turning off the Busy() timeout completely. Is that possible ?

No.  

Best,

Nico


------------------------------------------------------------------------------
The demand for IT networking professionals continues to grow, and the
demand for specialized networking skills is growing even more rapidly.
Take a complimentary Learning@Cisco Self-Assessment and learn
about Cisco certifications, training, and career opportunities.
http://p.sf.net/sfu/cisco-dev2dev
_______________________________________________
micro-manager-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/micro-manager-general

[Stefan Schoenleitner]

Hi,

On 10/24/2011 12:41 AM, Nico Stuurman wrote:
> On Oct 23, 2011, at 6:34 AM, Stefan Schoenleitner wrote:
>> Is there a way I can change the 5000ms timeout ?
>
> Set the property "Core" "TimeoutMs" to the value you desire.
> You can do so in the Device/Property Browser (under the Tools menu),
or on startup of Micro-Manager
> by adding it to the "Startup' Preset in the "System" Group.
>
>> Since my implementation already checks for timeouts, I'm also
>> considering turning off the Busy() timeout completely. Is that possible ?
>
> No.  

thank you, Nico !

Regards,
Stefan

------------------------------------------------------------------------------
The demand for IT networking professionals continues to grow, and the
demand for specialized networking skills is growing even more rapidly.
Take a complimentary Learning@Cisco Self-Assessment and learn
about Cisco certifications, training, and career opportunities.
http://p.sf.net/sfu/cisco-dev2dev
_______________________________________________
micro-manager-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/micro-manager-general

[Stefan Schoenleitner]

Hi,

On 10/24/2011 12:41 AM, Nico Stuurman wrote:
> On Oct 23, 2011, at 6:34 AM, Stefan Schoenleitner wrote:
>
>> Hence in the SetPositionSteps I'm sending out the movement command
>> whereas in the Busy() function I ask the stage whether it is finished yet.
>> Thus as long as the stage is moving, Busy() returns true to MM.

I'm currently working on the stage calibration implementation.
As you explained, the "SetOrigin" button can be used to start the device
calibration function.
Since all waiting should be done in Busy(), I implemented a calibration
routine that just starts the calibration and returns DEVICE_OK.

Then, in Busy(), there is a state machine that performs the rest of the
calibration.
The important thing to remember here is that as long as the calibration
(and thus also stage movement) is in progress, Busy() always returns true.

Once the calibration has completed, a call to Busy() will finally return
false and the stage can receive the next command.

However, when starting the calibration with the "SetOrigin" button, it
*sometimes* happens that the SetOrigin() function is called *before*
calibration has completed, even though Busy() is still returning true at
that time.

For my understanding, subsequent functions like SetOrigin() should only
be called if the stage is not Busy() anymore, right ?

cheers,
Stefan

------------------------------------------------------------------------------
The demand for IT networking professionals continues to grow, and the
demand for specialized networking skills is growing even more rapidly.
Take a complimentary Learning@Cisco Self-Assessment and learn
about Cisco certifications, training, and career opportunities.
http://p.sf.net/sfu/cisco-dev2dev
_______________________________________________
micro-manager-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/micro-manager-general

[Nico Stuurman-4]

On Oct 26, 2011, at 8:47 AM, Stefan Schoenleitner wrote:

I'm currently working on the stage calibration implementation. As you explained, the "SetOrigin" button can be used to start the device calibration function.
Since all waiting should be done in Busy(), I implemented a calibration routine that just starts the calibration and returns DEVICE_OK.

Then, in Busy(), there is a state machine that performs the rest of the calibration. The important thing to remember here is that as long as the calibration (and thus also stage movement) is in progress, Busy() always returns true.

Once the calibration has completed, a call to Busy() will finally return false and the stage can receive the next command.

However, when starting the calibration with the "SetOrigin" button, it *sometimes* happens that the SetOrigin() function is called *before* calibration has completed, even though Busy() is still returning true at that time.

For my understanding, subsequent functions like SetOrigin() should only be called if the stage is not Busy() anymore, right ?

The UI code that runs the calibration can be found here:

https://valelab.ucsf.edu/svn/micromanager2/trunk/mmstudio/src/org/micromanager/PositionListDlg.java

It looks like that code has a 60 seconds time out, so if the procedure takes more than 60 seconds, then it will time out and continue.  I'd be happy to make that longer, especially since the user has the option to interrupt the procedure anyways.

Best,

Nico

------------------------------------------------------------------------------
The demand for IT networking professionals continues to grow, and the
demand for specialized networking skills is growing even more rapidly.
Take a complimentary Learning@Cisco Self-Assessment and learn
about Cisco certifications, training, and career opportunities.
http://p.sf.net/sfu/cisco-dev2dev
_______________________________________________
micro-manager-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/micro-manager-general

[Stefan Schoenleitner]

Hi,

On 10/27/2011 06:40 AM, Nico Stuurman wrote:

> On Oct 26, 2011, at 8:47 AM, Stefan Schoenleitner wrote:
>> For my understanding, subsequent functions like SetOrigin() should only be called if the stage is not Busy() anymore, right ?
>
> The UI code that runs the calibration can be found here:
>
> https://valelab.ucsf.edu/svn/micromanager2/trunk/mmstudio/src/org/micromanager/PositionListDlg.java
>
> It looks like that code has a 60 seconds time out, so if the procedure takes more than 60 seconds,
> then it will time out and continue.
> I'd be happy to make that longer, especially since the user has the

option to interrupt the procedure anyways.

I changed the timeout and the problems are gone.
Thank you, Nico, for pointing this out.
I think either increasing this timeout or implementing it as
user-configurable parameter would be a good idea.


Regards,
Stefan

------------------------------------------------------------------------------
The demand for IT networking professionals continues to grow, and the
demand for specialized networking skills is growing even more rapidly.
Take a complimentary Learning@Cisco Self-Assessment and learn
about Cisco certifications, training, and career opportunities.
http://p.sf.net/sfu/cisco-dev2dev
_______________________________________________
micro-manager-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/micro-manager-general

[Stefan Schoenleitner]

Hi,

On 10/27/2011 06:40 AM, Nico Stuurman wrote:
> The UI code that runs the calibration can be found here:
>
> https://valelab.ucsf.edu/svn/micromanager2/trunk/mmstudio/src/org/micromanager/PositionListDlg.java
>
> It looks like that code has a 60 seconds time out, so if the procedure takes more than 60 seconds, then it will time out and continue.
> I'd be happy to make that longer, especially since the user has the
option to interrupt the procedure anyways.

I saw that you committed this change to the SVN.
However, it seems to me that you increased the wrong timeout limit.

When the "SetOrigin" button is pressed, the first thing that is called
in the DeviceAdapter is "Home()".
Once Home() (and thus also the calibration) is completed, SetOrigin() is
called.

In MMCore/MMCore.cpp:1409 it says that Home() "Calibrates and homes the
XY stage" whereas setOriginXY is used to "zero the current XY position".
Thus the calibration code clearly belongs to Home() and not to
SetOriginXY().

As far as I can see, in PositionListDlg.java:922, calibrate(), two
threads are started:

        StopCalThread
        CalThread

In the StopCalThread thread you changed the timeout from 60 to 600 seconds.
However, in the CalThread, the timeout is still 60 seconds.

I added debugging code to the waiting loops of both threads and
discovered that the thread responsible for the calibration waiting (i.e.
the waiting until Home() has completed) is not StopCalThread, but CalThread.

And since the timeout in there is still 60 seconds, the calibration is
still aborted early.

For my understanding, CalThread initially calls Home() (first call in
run()), whereas StopCalThread() only calls Home() if the user decides to
resume the calibration.
So both timeouts should be increased actually.

I attached a patch that fixes the problem.

Regards,
Stefan

------------------------------------------------------------------------------
The demand for IT networking professionals continues to grow, and the
demand for specialized networking skills is growing even more rapidly.
Take a complimentary Learning@Cisco Self-Assessment and learn
about Cisco certifications, training, and career opportunities.
http://p.sf.net/sfu/cisco-dev2dev
_______________________________________________
micro-manager-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/micro-manager-general

[Nico Stuurman]

On Oct 28, 2011, at 2:40 PM, Stefan Schoenleitner wrote:

>> https://valelab.ucsf.edu/svn/micromanager2/trunk/mmstudio/src/org/micromanager/PositionListDlg.java
>>
>> It looks like that code has a 60 seconds time out, so if the procedure takes more than 60 seconds, then it will time out and continue.
>
> I saw that you committed this change to the SVN.
> However, it seems to me that you increased the wrong timeout limit.

> In the StopCalThread thread you changed the timeout from 60 to 600 seconds.
> However, in the CalThread, the timeout is still 60 seconds.
>
> I attached a patch that fixes the problem.

I committed your patch.  Thanks!


Nico



------------------------------------------------------------------------------
The demand for IT networking professionals continues to grow, and the
demand for specialized networking skills is growing even more rapidly.
Take a complimentary Learning@Cisco Self-Assessment and learn
about Cisco certifications, training, and career opportunities.
http://p.sf.net/sfu/cisco-dev2dev
_______________________________________________
micro-manager-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/micro-manager-general