tcp/ip network layer for µManager

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

tcp/ip network layer for µManager

Karl Bellve-3
I was wondering if anyone has written or has an ideas about making µManager TCP/IP network capable?

My idea is to write a network µManager device adapter that can be used to manipulate other device adapters, call µManager API  and perhaps return images.

Just thinking...


Cheers

Karl Bellvé
Biomedical Imaging Group
Molecular Medicine
University of Massachusetts Medical School



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
micro-manager-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/micro-manager-general
Reply | Threaded
Open this post in threaded view
|

Re: tcp/ip network layer for µManager

Nico Stuurman-2
Hi Karl,

> I was wondering if anyone has written or has an ideas about making
> µManager TCP/IP network capable?
>
> My idea is to write a network µManager device adapter that can be used
> to manipulate other device adapters, call µManager API  and perhaps
> return images.
There are several ideas that have been floated over time:
1. Design a protocol so that the Micro-Manager api (either the Java
level one and/or the C++ level one) can be serialized so that it can be
streamed through an arbitrary port.  Several people have done some
things towards an implementation (for instance, Jon Daniels added a Java
RMI interface to the diSPIM plugin), but nothing is supported at an
application level yet.
2.  Build a TCP/IP adapter similar to the serial port adapter.  That
should make it easier to talk to devices that can be controlled through
TCP/IP (such as the Zeiss microscope bodies).

Best,

Nico



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
micro-manager-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/micro-manager-general
Reply | Threaded
Open this post in threaded view
|

Re: tcp/ip network layer for µManager

Peterbauer Thomas
On 2017-03-24 18:52, Nico Stuurman wrote:

>
>> I was wondering if anyone has written or has an ideas about making
>> µManager TCP/IP network capable?
>>
>>
> There are several ideas that have been floated over time:
> 1. Design a protocol so that the Micro-Manager api (either the Java
> level one and/or the C++ level one) can be serialized so that it can be
> streamed through an arbitrary port.  Several people have done some
> things towards an implementation (for instance, Jon Daniels added a Java
> RMI interface to the diSPIM plugin), but nothing is supported at an
> application level yet.
> 2.  Build a TCP/IP adapter similar to the serial port adapter.  That
> should make it easier to talk to devices that can be controlled through
> TCP/IP (such as the Zeiss microscope bodies).

Ad 2.: there is already an example in the current MM device list: the
Olympus MT20 adapter communicates to the Olympus IX2 real time
controller through TCP/IP.
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
micro-manager-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/micro-manager-general
Reply | Threaded
Open this post in threaded view
|

Re: tcp/ip network layer for µManager

Mark Tsuchida-3
Hello,

On Fri, Mar 24, 2017 at 11:53 AM, Peterbauer Thomas <[hidden email]> wrote:
On 2017-03-24 18:52, Nico Stuurman wrote:
>> I was wondering if anyone has written or has an ideas about making
>> µManager TCP/IP network capable?
>>
> There are several ideas that have been floated over time:
> 1. Design a protocol so that the Micro-Manager api (either the Java
> level one and/or the C++ level one) can be serialized so that it can be
> streamed through an arbitrary port.  Several people have done some
> things towards an implementation (for instance, Jon Daniels added a Java
> RMI interface to the diSPIM plugin), but nothing is supported at an
> application level yet.
> 2.  Build a TCP/IP adapter similar to the serial port adapter.  That
> should make it easier to talk to devices that can be controlled through
> TCP/IP (such as the Zeiss microscope bodies).

Ad 2.: there is already an example in the current MM device list: the
Olympus MT20 adapter communicates to the Olympus IX2 real time
controller through TCP/IP.

 And there are other examples of such devices I have encountered (not in our codebase), so generalizing it into a serial-like API seems like a good idea. We might be getting into this at Open Imaging soon as a customer-sponsored project, although it is not decided yet.

Regarding the other, higher-level idea (Nico's No. 1), I think TCP/IP is too low level and using a remote procedure call API (like RMI, Thrift, or (although not exactly an RPC) Protocol Buffers) and/or HTTP (or something like ZeroMQ) makes more sense -- although the technical challenge will be what to do for the image transport if high bandwidth is desired.

Best,
Mark 

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
micro-manager-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/micro-manager-general
Reply | Threaded
Open this post in threaded view
|

Re: tcp/ip network layer for µManager

Karl Bellve-3


On Fri, Mar 24, 2017 at 3:45 PM Mark Tsuchida <[hidden email]> wrote:
Hello,

On Fri, Mar 24, 2017 at 11:53 AM, Peterbauer Thomas <[hidden email]> wrote:
On 2017-03-24 18:52, Nico Stuurman wrote:
>> I was wondering if anyone has written or has an ideas about making
>> µManager TCP/IP network capable?
>>
> There are several ideas that have been floated over time:
> 1. Design a protocol so that the Micro-Manager api (either the Java
> level one and/or the C++ level one) can be serialized so that it can be
> streamed through an arbitrary port.  Several people have done some
> things towards an implementation (for instance, Jon Daniels added a Java
> RMI interface to the diSPIM plugin), but nothing is supported at an
> application level yet.
> 2.  Build a TCP/IP adapter similar to the serial port adapter.  That
> should make it easier to talk to devices that can be controlled through
> TCP/IP (such as the Zeiss microscope bodies).

Ad 2.: there is already an example in the current MM device list: the
Olympus MT20 adapter communicates to the Olympus IX2 real time
controller through TCP/IP.

 And there are other examples of such devices I have encountered (not in our codebase), so generalizing it into a serial-like API seems like a good idea. We might be getting into this at Open Imaging soon as a customer-sponsored project, although it is not decided yet.

Regarding the other, higher-level idea (Nico's No. 1), I think TCP/IP is too low level and using a remote procedure call API (like RMI, Thrift, or (although not exactly an RPC) Protocol Buffers) and/or HTTP (or something like ZeroMQ) makes more sense -- although the technical challenge will be what to do for the image transport if high bandwidth is desired.

Best,
Mark 

I was sort of using TCP/IP in the general sense. I don't think I would want to be programming sockets... :D 

I have been thinking about it today and I think doing it as at a device level, rather than at the serial port level, which sits below devices, makes more sense to me.  Normally, each uM device adapter (DA)is written specifically to use a communication layer like serial port, direct communication, or maybe even a network protocol. Each DA would have to be adapted to use the network, and I don't think that gets you anywhere. Maybe it lets you run µManager remotely from its devices? Way too much work. :D 

I would rather control uManager remotely and the devices run locally to uManager.

I would just use the µM Core/GUI API system that already exists, i.e. set/get <string> <string>. Don't kill me on that getting that correct... :D Its been awhile since I have done any programming, much less µM programming. :D 

The network Device Adapter would just send/ receive a string set, and maybe verify it, check for buffer overflows, and then pass it to uManager API, and wait for a response, and return it across the network. The network Device Adapter would be very generic, and maybe have an option to set password, port, etc.

If I decide to do this, my timeline for doing this is in the next few weeks. Let me know if this is an issue for the customer project.

Cheers 

Karl Bellvé
Biomedical Imaging Group
Molecular Medicine
University of Massachusetts Medical School


 

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
micro-manager-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/micro-manager-general
Reply | Threaded
Open this post in threaded view
|

Re: tcp/ip network layer for µManager

JonD
Administrator
Karl Bellve-3 wrote
> Regarding the other, higher-level idea (Nico's No. 1), I think TCP/IP is
> too low level and using a remote procedure call API (like RMI, Thrift, or
> (although not exactly an RPC) Protocol Buffers) and/or HTTP (or something
> like ZeroMQ) makes more sense -- although the technical challenge will be
> what to do for the image transport if high bandwidth is desired.

I would rather control uManager remotely and the devices run locally to
uManager.

I would just use the µM Core/GUI API system that already exists, i.e.
set/get <string> <string>. Don't kill me on that getting that correct... :D
Its been awhile since I have done any programming, much less µM
programming. :D

The network Device Adapter would just send/ receive a string set, and maybe
verify it, check for buffer overflows, and then pass it to uManager API,
and wait for a response, and return it across the network. The network
Device Adapter would be very generic, and maybe have an option to set
password, port, etc.
Perhaps relevant: recently Matthijs Dorst announced that he had implemented a basic REST server for controlling MM, including changing CMMCore properties and retrieving semi-live images.  Perhaps this would be a good starting point for further efforts.  See Matthijs' note at http://micro-manager.3463995.n2.nabble.com/Communication-with-other-software-td7588086.html#a7588156.

Disclaimer: I haven't looked into the code at all, but just wanted to connect this thread with a related earlier one.

Jon

-------------------------------------------
Jon Daniels
Applied Scientific Instrumentation
29391 West Enid Rd, Eugene, OR 97402
Phone: (541) 461-8181 x118
-------------------------------------------
Reply | Threaded
Open this post in threaded view
|

Re: tcp/ip network layer for µManager

Egor Zindy
In reply to this post by Karl Bellve-3
Hi Karl,

On 24 March 2017 at 14:29, Karl Bellve <[hidden email]> wrote:
>
> I was wondering if anyone has written or has an ideas about making µManager TCP/IP network capable?
>

Ideas yes, but very little time to act on them unfortunately...

Not on the "imaging side" but on the "control side", an MQTT broker
embedded in micromanager could provide a way to interact with all
micromanager properties (groups?), as well as reflect property changes
on listening hardware. In that way, you could build interactive
software or hardware modules (buttons / sliders / dials / displays)
which link to specific micromanager properties as well as each other
(modify one, all the others listening to the topic get updated).

A broker (or a listener if an external broker is available) could be
implemented at the Java level with moquette
https://github.com/andsel/moquette or as a device adapter (but I
hadn't really thought about that option).

I would personally use that to have microscope, stage and LED controls
decoupled from any one particular manufacturer and made available on a
single touchpad or keypad. Something akin to the Olympus touchpad but
able to control *everything MM can* and be designed independently from
MM (e.g. a QT / Yocto or Android app hosted on a cheap tablet).

This would deal with the proliferation of (expensive) keypads on our
benches, some of which covered in buttons which are seldom or never
used. Check out this random image for example
http://www.fluorescencemicroscopes.com/wp-content/themes/CompoundMicroscopes/images/products/olympus-model-IX83-6.jpg
which shows three different controllers but doesn't include a keypad
for the LED intensities...

Anyway, I don't mean this to sound like a rant so I'll stop right here :-)

Cheers,
Egor

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
micro-manager-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/micro-manager-general
Reply | Threaded
Open this post in threaded view
|

Re: tcp/ip network layer for µManager

Karl Bellve-3


Thanks everyone for the great feedback and ideas.

I have some reading to do. I will probably to write a uManager device adapter that would send/receive string commands to control GUI/Core/Device API.

What I should have included, but didn't was that in the end I want uManager to be controlled by a KNIME workflow for intelligent image acquisition for another project. I am in communication with a KNIME/KNIP developer.

For right now, simple things like snap image, move stage, move Z position is what I hope to implement via API communication. 

Retrieving images across the network isn't a priority since it would be rate limiting but saving to a local file system can work without impacting uManager too much, and then move the files to a location accessible by KNIME.


Cheers 

Karl Bellve
Biomedical Imaging Group
Molecular Medicine
University of Massachusetts Medical School



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
micro-manager-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/micro-manager-general
Reply | Threaded
Open this post in threaded view
|

Re: tcp/ip network layer for µManager

Karl Bellve-3
In reply to this post by JonD


On Sat, Mar 25, 2017 at 4:35 PM JonD <[hidden email]> wrote:
Karl Bellve-3 wrote
 <snip>
> maybe
> verify it, check for buffer overflows, and then pass it to uManager API,
> and wait for a response, and return it across the network. The network
> Device Adapter would be very generic, and maybe have an option to set
> password, port, etc.

Perhaps relevant: recently Matthijs Dorst announced that he had implemented
a basic REST server for controlling MM, including changing CMMCore
properties and retrieving semi-live images.  Perhaps this would be a good
starting point for further efforts.  See Matthijs' note at
http://micro-manager.3463995.n2.nabble.com/Communication-with-other-software-td7588086.html#a7588156.

Thanks Jon,

This project is very interesting. Exploring it further.

Cheers

Karl Bellvé
Biomedical Imaging Group
Molecular Medicine
University of Massachusetts Medical School 

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
micro-manager-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/micro-manager-general
Reply | Threaded
Open this post in threaded view
|

Re: tcp/ip network layer for µManager

Christian Carsten Sachs
In reply to this post by Karl Bellve-3
Hello,

concerning the topic of remote control of Micro-Manager, I'd like to add
my two cents about a project I'm working on.

I've been developing a message based interaction mechanism for MMCore
based on Akka (an actor parallelism/RPC framework for Scala/Java) and
Scala, which allows to interact with basically all functionality of MMCore.

Two ways of remote access are possible, either within Scala via remoting
based on the Akka framework, or via a JSON bridge.
The approach is that a set of instructions and/or questions are sent to
a MMCore connection layer, allowing for commands to be executed in batches.

E.g.

microscope ? TellAndAsk(MoveTo(AbsoluteXY(1238.0, 4234.43)) ::
MoveTo(AbsoluteZ(1238.0)) ::
SnapImage(),
LastImage() :: CurrentProperties() :: NoQ()
)

Could be used to first instruct Micro-Manager to move the stage to a
certain position and snap an image; and ask to get the snapped image as
well as all current properties.

Furthermore, a transparent JavaScript bridge was developed, which allows
for a comparable syntax from within web-frontends:

microscopeActor(o.TellAndAsk([o.MoveTo(o.RelativeXY(50, 0))],
[o.AbsoluteXY()]))
.success(function(data) {
console.log("New position");
console.log(data[0]);
});

This connection layers have been developed to connect an experimental
control framework for joint microscope-control/image-analysis with
MMCore, which additionally uses a web-frontend to allow for complete
remote control of the platform developed.

I've found this "queued messages" transport to be fast and versatile, as
batches of commands can be transferred without too much RPC overhead.
(And HTTP-based JPEG image transport tends to be fast enough to reach
multiple frames per second, allowing for near real-time control of the
microscope via a web frontend)

The platform is planned to be published later this year, if interest in
the aforementioned RPC mechanisms exists, it might be possible to
release them ahead.

Best regards,
Christian Sachs

On 24.03.2017 15:29, Karl Bellve wrote:

> I was wondering if anyone has written or has an ideas about making
> µManager TCP/IP network capable?
>
> My idea is to write a network µManager device adapter that can be used
> to manipulate other device adapters, call µManager API  and perhaps
> return images.
>
> Just thinking...
>
>
> Cheers
>
> Karl Bellvé
> Biomedical Imaging Group
> Molecular Medicine
> University of Massachusetts Medical School
>
>
>
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>
>
>
> _______________________________________________
> micro-manager-general mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/micro-manager-general
>

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
micro-manager-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/micro-manager-general

smime.p7s (7K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: tcp/ip network layer for µManager

Karl Bellve-3


On Mon, Apr 3, 2017 at 1:19 PM Christian Carsten Sachs <[hidden email]> wrote:
Hello,

<snip>

The platform is planned to be published later this year, if interest in
the aforementioned RPC mechanisms exists, it might be possible to
release them ahead.

Best regards,
Christian Sachs

Christian,

Very interesting.  I would love to take a look at it. Quick question, what will the copyright license be for the project? 

Cheers 

Karl Bellvé
Biomedical Imaging Group
Molecular Medicine
University of Massachusetts Medical School

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
micro-manager-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/micro-manager-general
Reply | Threaded
Open this post in threaded view
|

Re: tcp/ip network layer for µManager

Christian Carsten Sachs
Hello Karl,

sorry for my belated response, I was busy preparing for a conference.

The project (part of my PhD-project) will likely be released under a
lenient open source license, like BSD or Apache license.
(This has so far not been a problem with other software developed here,
but ultimately I have to check back with my supervisors).

Unfortunately I cannot give a precise time schedule when I can push the
source online, as I'm nearing the final phase of my thesis and have a
multitude of todos to get done in the next months.

Best regards,
Christian

On 25.04.2017 17:56, Karl Bellve wrote:

>
>
> On Mon, Apr 3, 2017 at 1:19 PM Christian Carsten Sachs
> <[hidden email] <mailto:[hidden email]>> wrote:
>
>     Hello,
>
>     <snip>
>
>     The platform is planned to be published later this year, if interest in
>     the aforementioned RPC mechanisms exists, it might be possible to
>     release them ahead.
>
>     Best regards,
>     Christian Sachs
>
>
> Christian,
>
> Very interesting.  I would love to take a look at it. Quick question,
> what will the copyright license be for the project?
>
> Cheers
>
> Karl Bellvé
> Biomedical Imaging Group
> Molecular Medicine
> University of Massachusetts Medical School
>
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>
>
>
> _______________________________________________
> micro-manager-general mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/micro-manager-general
>

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
micro-manager-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/micro-manager-general

smime.p7s (7K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: (no subject)

Dominika Lyzwa
In reply to this post by Christian Carsten Sachs
 
Please unsubscribe this email address [hidden email]!
Thank you.
 

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
micro-manager-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/micro-manager-general