Here’s the error print from my custom driver version. Seems like it breaks a lot more in the newer sweepme versions; I haven’t been heavily editing it.
Python Error:
Traceback (most recent call last):
File “MeasClasses.py”, line 2394, in wrapper
File “C:\Users\Public\Documents\SweepMe!\CustomDevices\Logger-Thorlabs_PM100_DJW\main.py”, line 147, in initialize
self.sensor_status = self.get_sensor_status()
File “C:\Users\Public\Documents\SweepMe!\CustomDevices\Logger-Thorlabs_PM100_DJW\main.py”, line 252, in get_sensor_status
sensor_info = self.get_sensor_info()
File “C:\Users\Public\Documents\SweepMe!\CustomDevices\Logger-Thorlabs_PM100_DJW\main.py”, line 305, in get_sensor_info
answer = self.port.read()
File “pysweepme\Ports.py”, line 628, in read
File “pysweepme\Ports.py”, line 880, in read_internal
File “pyvisa\resources\messagebased.py”, line 486, in read
File “pyvisa\resources\messagebased.py”, line 442, in _read_raw
File “pyvisa\ctwrapper\functions.py”, line 2337, in read
File “pyvisa\ctwrapper\highlevel.py”, line 226, in _return_handler
File “pyvisa\highlevel.py”, line 251, in handle_return_value
pyvisa.errors.VisaIOError: VI_ERROR_TMO (-1073807339): Timeout expired before operation completed.
I see similar effects with one of the scope drivers I am currently writing. It will work fine for about 3 to 4 times but then, port read will time out. I was assuming that there is still some sort of configuration issue that leads to the driver not retrieving all data from the scope and that this data will somehow add-up and lock-up the buffer, leading to the observed time-out. Plus it doesn’t happen with a second scope driver I am writing although basic principle of function is the same.
So the driver you wrote worked flawless but startet to show the time-out issues randomly with an increasing occurance while the sweep-me programm versioning proceeded?
For a couple of instruments we also observed VI_ERROR_TMO when developing drivers. What helped in our cases was adding a sleep between sending/reading.
time.sleep(0.5) # add this before sending the command
self.port.write("SENS:...") # command to send to the instrument
time.sleep(0.5) # add this before read()
result = self.port.read()
Not 100% sure why these sleep() help, but sometimes it seems sending multiple commands to the instrument too quickly, and the instrument just ignores the commands. Then it won’t send a response and read() times out. Sleep() before the read() shall make sure the instrument had the time to process the command and send the result, so at the time of read() there is already data in the buffer.
You could try adding generous sleeps in your drivers, and if that works reduce the sleep so it is enough to have an effect, but not too big to delay the measurement too much.
As to why this happens more often with newer SweepMe! versions, I have no clue. PyVISA, the library which actually handles the lowlevel communication with the VISA Runtime hasn’t changed since SweepMe! 1.5.6.3.
It seems like port timing is tricky in general. I feel like half of writing drivers is figuring out the time.sleep() durations. Christian, your assessment is correct. I had the driver in a reliable state after significant usage and testing. It had a device reset command that I commented-out to fight these kind of failure-to-start errors. Higher versions of sweepme seem to have different port timings that provoke these errors. Seeing Felix’s response, maybe there’s some other optimization that sped up what used to be slow-enough to work.
What’s frustrating is that this issue is not reproducible in my IDE (Spyder). Here’s the dunder code (forgive the indentation problems from using the forum text editor)
#Test the driver here
if __name__ == "__main__":
import pysweepme
#Create driver instance using pysweepme, from custom device folder
driver_path = 'C:\\Users\\Public\\Documents\\SweepMe!\\CustomDevices'
pm100_driver_name = "Logger-Thorlabs_PM100_DJW"
pm100_port_string_s121c = "USB0::0x1313::0x8072::1922348::INSTR"
# pm100_port_string_s405c = "USB0::0x1313::0x8072::1918190::INSTR"
pm100 = pysweepme.get_driver(pm100_driver_name, driver_path, pm100_port_string_s121c)
#Test driver in realistic sweepme sequence
pm100.initialize()
pm100.configure()
pm100.measure()
measurement = pm100.call() #Store this for viewing in spyder
pm100.deinitialize()
I can run this repeatedly with no port struggling/choking of any kind. But one run in sweepme (with just a loop and the logger) and it throws the error I started this thread with.
I’m sorry for the frustrating driver development process. But the non-reproducability in Spyder/pysweepme is actually a valuable insight, I think.
If you were willing to run a few tests, we could narrow down the difference between Spyder/pysweepme and SweepMe! bit by bit. As the error already occurs in initialize(), the concrete sequencer arrangement should not matter.
Here are my ideas for testing:
Can you also run the pysweepme script from your previous post from the command line? (python yourscript.py). Just to exclude Spyder could be doing something special, though I don’t expect any difference.
Are you using a plain python environment or an (ana)conda environment? In the latter case, could you try again with a standard python (virtual) environment?
Could you compare the versions of pysweepme, pyVISA and other installed libraries? SweepMe! library versions can be found in the <InstallationDirectory>\libs\requirements.txt file.
Running the script from SweepMe!:
Save your script in the directory C:\Users\Public\Documents\SweepMe!\Tools\PythonScripts
Remove the if __name__ == "__main__": condition
At the end, insert a pm100.port.close() - I assume your driver uses the port manager from pysweepme.
(pysweepme does not explicitly close ports, but usually this is done automatically when the script execution ends. In SweepMe! it needs to be added because it is running in the SweepMe! context and thus the execution is not ending until SweepMe! is closed)
You might need to restart SweepMe! for the script to show up. Then you can find it in the Tools → Python script window and run the script from there. Whether it works from here or not will show us if the issue is the environment or the way SweepMe! runs drivers.
(only needed to check if the one above works without the timeout problem): Create a driver like the one below and run a “measurement” with this driver (without loop). Difference to the step above is that it is not executed in the main thread but in the measurement thread.
from EmptyDeviceClass import EmptyDevice
class Device(EmptyDevice):
def __init__(self):
EmptyDevice.__init__(self)
self.shortname = "Test"
self.variables = ["dummy"]
self.units = ["dummy"]
self.plottype = [False]
self.savetype = [False]
def call(self):
print("start")
import pysweepme
#Create driver instance using pysweepme, from custom device folder
driver_path = 'C:\\Users\\Public\\Documents\\SweepMe!\\CustomDevices'
pm100_driver_name = "Logger-Thorlabs_PM100_DJW"
pm100_port_string_s121c = "USB0::0x1313::0x8072::1922348::INSTR"
# pm100_port_string_s405c = "USB0::0x1313::0x8072::1918190::INSTR"
pm100 = pysweepme.get_driver(pm100_driver_name, driver_path, pm100_port_string_s121c)
#Test driver in realistic sweepme sequence
pm100.initialize()
pm100.configure()
pm100.measure()
pm100.process_data()
measurement = pm100.call() #Store this for viewing in spyder
pm100.deinitialize()
print(measurement)
pm100.port.close()
print("end")
I hope that these steps will help to narrow down the key difference why it is working in your Spyder/pysweepme case but not in SweepMe!, so that we can continue from there on.
@Christian Could you perhaps also test if your scope driver (the one which also suffers these exceptions) works flawlessly when running from pysweepme?
@Felix: I’ll see what I can do. Hoever, I would like to note that this morning, I modified the driver whose waveform transfer always failed on its first aquisition but worked fine otherwise. I inserted the suggested wait loops (here: 0.2s) and that worked out. I think the reason was that when the scope booted up with default parameters, the first run changes so many settings in a short time that even so the trigger was successfull, setup was still not completed and therefore the waveform not ready to be transfered, yilding invalid preamble data.
I will add these wait times to the other, more fragile driver and see if this turns out to be the solution as well.
Thanks for the results. As the timeout also happens using the SweepMe! script tool, we can at least exclude the sequencer as the cause of the timeouts.
A few clarifications or details for the other tests:
From the Anaconda Navigator, there should be an option to open a terminal for your Anaconda environment. In this terminal, running python yourscript.py should run with the Anaconda-python interpreter. (Has nothing to do with the windows python)
Doing this test is a bit more work, as it requires to install a regular python (but possible even without admin). Here are the necessary steps: Download python 3.9 Windows Installer from Python Release Python 3.9.13 | Python.org and when installing choose Customize Installation. Only check pip, and uncheck all other options (in particular do NOT add python to the Path or environment variables - this ensures if will not interfere with your main anaconda installation). Without admin privileges it will install to something like C:\Users\YourName\AppData\Local\Programs\Python\Python39. Afterwards, open a regular command prompt and run C:\Users\YourName\AppData\Local\Programs\Python\Python39\python.exe -m pip install pysweepme==1.5.6.10 PyVISA==1.13.0 (you need to specifiy the full path). Finally you can run C:\Users\YourName\AppData\Local\Programs\Python\Python39\python.exe yourscript.py (might need to adjust the path to yourscript.py)
Which versions of pysweepme and PyVISA are installed in your Anaconda environment? If that’s different from the versions in the SweepMe! requirements.txt it might make a difference. And you are using SweepMe! 64bit? Which bitness is your Anaconda installation? Though I hope it’s not the bitness which causes the timeout.
Most interesting is probably point 3, if there is a difference in the PyVISA versions. If the other tests indicate that the significant difference between working driver and timeouts is Anaconda vs regular python, I would be out of ideas anyway, unfortunately.
I’m pleased to report that the problem is fixed, and sorry to report the reason why it was a problem in the first place.
Turns out I tested the following code in one of my driver versions in the past, but not in the one I was testing lately and reporting about:
self.port_properties = {
“timeout”: 10, # in seconds
“clear”: False, # there are problems if the port of the PM100 is cleared
}
This was your suggestion from Discord and it was good; I failed to implement this fix in all my versions. I will be consolidating my PM100 code soon, combining versions designed to be called by a CF and versions designed to be run independently in SweepMe, so consider this problem solved.