Optimizing Driver for Ahlborn ALMEMO 2890-9

Using SweepMe!
1

Hello everyone,

I used the Logger-Template to write a short driver for the ALMEMO 2890-9, a data logger from Ahlborn which can readout different types of sensors.
There are currently some performance issues.

Firstly, the device should be queried for its connected measurement channels when the constructor is called, as otherwise these will not appear in the driver’s variable list and therefore cannot be selected in the SweepMe environment. I have therefore implemented my own find_ports function and am not using the port manager.
After that, the specially implemented identify_channels function is called, which queries the device after outputting all measuring points.
However, this causes the SweepMe performance itself to suffer when you want to load the driver.

On the other hand, parsing the response string also takes a lot of time after the readout process. I use the functions request_result, read_result, and finally process_data for this.

Here is my own design of the functions. Maybe someone can help me optimize them so that SweepMe performance does not suffer.

main.py (17.8 KB)

1 Like
2

Hi,

thanks for using the forum!

I checked your driver and as you already describe, there are two problems:

  1. How to retrieve information from the device to configure it before the measurement
  2. speeding up the measurement

Regarding the 1.)
I have to say that SweepMe! is missing a Driver API function that allows to communicate with the device in order to refresh the GUI parameters. This is something that we have on our development list, but still looking for a robust way to implement it to handle the timeout errors when the wrong device is selected.

If you are working with a single device that has a fixed amount of sensors, my recommendation would be to create a hard-coded list of variables and units. If you have multiple devices with each having a fixed amount of sensors, you could create some kind of in-driver lookup dictionary and select your device in the GUI. If you have devices where the sensors constantly change, a workaround would be to create a GUI where the user can select the number of connected sensors and then manually select the sensors to be read out.

Here, use of dynamic GUI parameters as explained in the Wiki
https://wiki.sweep-me.net/wiki/Driver_Programming#Dynamic_GUI_parameters
might be helpful to populate the UI with as many Sensor selection fields as needed.

I will discuss this topic also once again with my colleagues. Optimal solution would be if a user could access the port during set_GUIparameter or update_gui_parameters.

Regarding 2.)
In general, it should be possible to use SweepMe!'s port manager during the measurement in semantic phase functions and still create own ports to something special before the measurement. This would not make your driver faster, but your could make use of the many convenience functions for the port handling like reading until the separator or the encoding.

Although the code looks a bit lengthy, I don’t think it is the reason for being slow. The use of the functions request_result, read_result, and process_data is correct.

You already retrieve the data with a single call using “S1” command so there is no chance to further parallelize the date readout. However, often there is the problem that terminator character is not correctly used. Then, the code always waits until the timeout. To check this, you can increase the timeout to larger values and see whether the measurement gets even slower.

I further wonder about the command “readall” that you use for the pyserial Port object. I was able to verify that the command exists, but did not find any official documentation about it. So, I would check whether the command is correctly used and how it handles the terminator of the incoming message.

Maybe you can write how long a single readout takes or how many values you can readout right now per second. More than 100 values will be difficult with a serial communication. If possible, one could also increase the baudrate of the COM port if the ALMEMO device allows to configure this.

In process_data you are redefining self.plottype and self.savetype which however has no influence anymore at this position. Variables, units, plottypes and savetypes must be defined latest during get_GUIparmeter or apply_gui_parameter. So you can safely remove them there.

Printing a lot of information at each process step, e.g. as you do during ‘call’ function might also slow down SweepMe!, so you could remove this as well or close the Debug widget during the measurement.

Hope these steps help you to proceed with a faster measurement. Let us know what you can figure out.

Thanks and best
Axel

3

Hello Axel,

First of all thank you very much for your quick and detailed response!
I was able to update the driver a little by using the port manager and preallocating the maximum number of measurement channels.
However, we still need to think about the interaction with the GUI, as the variables in the SweepMe backend are named differently due to the allocation in the constructor.
I have written an auxiliary function read_msg_until(), which enables faster and better reading of the logger.
The correct assignment of variables, measurement channels, and sensor typing continues to take place in the identify_channels() function and is initialized at the beginning of each measurement.
As for querying the device via “S1,” I have tried alternatives such as “MXX p,” which can control and query the “MXX” channel individually. However, this alternative is somewhat slower, as changing channels takes a little time.

If you have any further ideas, please let me know so that we can work together to program a nice driver that everyone can use.

Thank you again for your support and all the best for the future!

Mirko

main.py (17.2 KB)

1 Like
4

Hi Mirko,

I think it is ok to stick with the “S1” command if it sends the data of all active channels.
If it remains slow with using your new function “read_msg_until”, I would do some timing tests now. How man readings to you get right now per second and how many would you expect?

Can you also check whether the terminator character for reading messages is correct?

Use of port manager

  • You are using the key “delay” that is set to 0.5s which could also explain the extra time. Some older instruments are not able to process or buffer incoming messages when they are busy with processing a previous measurement. Then, it can help to add a “delay” to tell the port manager to always wait the given time before sending the next message. Can you comment this line and see whether it still works?

  • You function “read_msg_until” is actually what the port manager is doing when you use self.port.read() and no argument. It reads until the terminator and returns the message without the terminator. You only need to make sure that they keys “EOLread” and “EOLwrite” are correctly set in self.port_properties

General points

  • the drivers either support set_GUIparameter/get_GUIparameter or the newer functions update_gui_parameters/apply_gui_parameters. I recommend to use the newer functions and remove the older one avoid confusion. Also check the correct writing of update_gui_parameters please

  • update_gui_parameters allows to return a dictionary based on the current user selection. This way it is possible to add or remove keys depending on the user selection. For example, you could add multiple keys depending on the selected number of channels. My suggestion would be to hard-code it for now. Let the user define which channels and variables to retrieve. Then, check in the semantic phase “initialize” of the driver whether the variables exist. Otherwise, inform the user about the possible/supported channels.

  • You are using “self.value” to define the measurement results and returm then in “call”. However, “self.value” has a special meaning in SweepMe! drivers as this is the new setvalue /weep value that is handed over to the driver in the “apply” phase. For a Logger drive, this does not really hurt as there is nothing to set but a good practive would be to rename it to e.g. self.results = …

  • The command “S1” to get the results could be put to “measure” or to “request_result” depending on its meaning. If “S1” triggers the measurement, it should be put into “measure” as this is the semantic phase in which instruments should start acquiring the data. Some instruments immediately return the data, other need another command to request the result which then should be put into “request_result”. At the end if often does not matter where you put the commands into the semantic phase functions for a single driver as long as they are in the correct order. However, it matters if you combine multiple different instrument where you want them to do the same thing at the same moment, e.g. all should start the acquisition of new data in “measure”

I hope this helps to get further step forward. Maybe also some professional support is needed. In this case please contact me via direct message.

Thanks and best
Axel

5

Short explanation why it is not straight-forward to add functions to the driver API to retrieve variables before the measurement:

Basically, it is easy. One could give access to the port during update_gui_parameters and the driver developer could communicate with the instrument before defining the variables and units.

The update_gui_parameters-function is called multiple times in SweepMe! to make always sure the state of the driver is known. This can lead to some delays when multiple driver use this or when an instrument takes longer to respond. Of course, this could be done asynchronous. It gets more problematic when the port is not yet known or has changed, Then, the communication runs into a timeout or variables are returned from a wrong device.

If no variables are returned the UI of the module will not be correctly populated and the measurement cannot be started. So far our philosopy was that it should be possible to start a measurement directly after loading the setting. With an additional communication before the run, other inconvenient behavior is introduced that we need to consider and handle as well before adding such a feature.

Hope this explains a bit why we are sometimes in favor of a more hard-coded/preset solution where the driver could still check during initialize whether all configurations are ok.

6

Hello Axel,

thank you so much for your support and your time again! I have now revised the driver again. By using the functions updating_gui_parameters and apply_gui_parameters in combination with the function identify_channels, I have succeeded in creating and initializing all the necessary variables for the SweepMe environment.
The first data query is performed via the serial port selected by the user, which is why I can continue to use the port manager.

Thank you again for everything!

Best regards,

Mirko

main.py (13.5 KB)

1 Like
7

Oh and of course the termination character is correct but I have not yet conducted any further speed tests. I think the measure frequency is about 2 to 5 Hz now.

1 Like
8

Hello Mirko,

2 to 5 Hz does not sound that much indeed. As you already removed the key “delay” from self.port_properties, there is not much chance to improve the serial communication itself if you are sure the terminator is correct.

Some instruments allow to adjust averages or the integration time. Can you check whether it is possible to set/change such parameters to increase the readout speed?

Additionally, you could test any other communication command to see whether another one is processed faster by the instrument, e.g. by sending and reading it 100 times in “initialize” and measure the time using time.time()

Please also check that there is no Delay or Hold module in the Sequencer that slows down your measurement.

Best, Axel