Driver communication issues due to non-threadsafe SDKs/DLLs

Using SweepMe!
1

When an SDK Dislikes Threads: Solving a Very Unusual SweepMe! Driver Issue

When writing device drivers, developers usually interact with instruments through well-established interfaces and protocols. Common examples include GPIB, VISA, USBTMC, Serial (RS-232), Modbus, SCPI, OPC UA, or TCP/IP-based communication. In other cases, manufacturers provide their own SDKs, DLLs, or Python packages that expose functions for controlling the hardware.

Recently, I came across an instrument that fell into the latter category. The manufacturer provided an SDK, and at first glance everything seemed to work perfectly. Communication was successful, measurements completed correctly, and the driver behaved exactly as expected.

At least for the first measurement.

The Problem

The strange behavior appeared when running a second measurement in SweepMe!.

The first measurement always completed successfully. However, every subsequent measurement failed when trying to reconnect to the device. The SDK reported that the communication port could not be opened.

What made the issue particularly confusing was that the exact same SDK calls worked flawlessly in a standalone pysweepme script. Even within SweepMe!, the first measurement run after a restart was successful every single time.

Naturally, the first suspicion was that some connection handle was not being properly released. After carefully reviewing the driver and verifying that all ports were being closed correctly, that explanation could be ruled out.

So what was different between the working and failing scenarios?

Tracking the Issue Down to Threads

After some experimentation, I reduced the problem to a minimal Python example and eventually discovered a surprising limitation of the SDK:

The SDK could not handle its functions being called from different threads.

If all SDK calls originated from the same thread, everything worked. As soon as a second thread attempted to use the SDK, communication started to fail.

This turned out to be highly relevant because SweepMe! is a GUI application. To keep the user interface responsive, measurements are executed in worker threads. A measurement run starts a worker thread, the thread performs the measurement, and once the measurement is finished, the worker thread terminates. On the next measurement run, a new worker thread is created.

From SweepMe!'s perspective, this is perfectly normal and exactly what should happen.

From the SDK’s perspective, however, it meant that the first measurement and the second measurement were executed from different threads — something it apparently could not tolerate.

The Idea: A Persistent SDK Thread

Once the root cause was identified, the solution became surprisingly straightforward.

Instead of allowing the SDK functions to be called directly from SweepMe!'s worker threads, I introduced a dedicated persistent thread inside the driver.

All SDK interactions are routed through this thread.

This means that SweepMe! can continue creating new worker threads for every measurement run, while the SDK only ever sees function calls coming from the same persistent thread.

The overall architecture looks like this:

Persistent Thread Architecture
Persistent Thread Architecture1826×1540 32.4 KB

Graph definition 
%%{init: {'theme': 'default', 'themeVariables': {
    'fontFamily': 'ZT Nature Medium',
    'clusterBkg': '#E0E9FF',
    'mainBkg': '#b3f4b9 '
}}}%%
flowchart LR

    subgraph GUI["SweepMe! GUI Thread"]
        direction TB
        s1["\n\n\n\n\n\n"]
        APP["SweepMe! Application"]
        s2["\n\n\n\n\n\n"]

        s1 --> APP
        APP --> s2

        style s1 fill:none,stroke:none
        style s2 fill:none,stroke:none
        linkStyle 0 stroke:none
        linkStyle 1 stroke:none
    end

    subgraph WT1["Measurement #1 Worker Thread"]
        D1["Driver Instance #1"]
        O1["Other drivers..."]
    end

    subgraph WT2["Measurement #2 Worker Thread"]
        D2["Driver Instance #2"]
        O2["Other drivers..."]
    end

    subgraph WT3["Measurement #3 Worker Thread"]
        D3["Driver Instance #3"]
        O3["Other drivers..."]
    end

    subgraph PS["SweepMe! Parameter Store"]
        subgraph SDKTHREAD["Persistent SDK Thread"]
            SDK["SDK Instance /\nSDK function calls"]
        end
    end

    GUI --> WT1
    GUI --> WT2
    GUI --> WT3

    D1 --> SDK
    D2 --> SDK
    D3 --> SDK

As far as the SDK is concerned, nothing ever changes.

A Word of Caution

Before anyone starts introducing persistent threads into every driver: this is a very specific solution to a very uncommon problem.

Writing thread-safe software requires a solid understanding of how threads, synchronization, queues, and shared resources interact. For the overwhelming majority of SweepMe! drivers, none of this is necessary.

In fact, this is the first time I have encountered an SDK that was sensitive to the thread from which its functions were called.

So please do not take this article as a recommendation that all drivers should be implemented this way. Instead, consider it a useful pattern to keep in mind for those rare situations where an SDK exhibits unexpected thread affinity.

And of course, if you are developing custom drivers and run into difficulties, we offer support agreements and development assistance. Sometimes a short consultation can save many hours of debugging and experimentation.

Keeping the Thread Alive Between Measurements

A normal driver instance only exists during the lifetime of a measurement. Therefore, simply storing the thread inside the driver would not be sufficient.

Fortunately, SweepMe! provides the Parameter Store.

The Parameter Store allows drivers to keep objects alive beyond the lifetime of a single driver instance and even across multiple measurement runs.

That makes it a perfect place to store a persistent SDK thread.

Implementing the Persistent Thread

For the implementation itself, I decided to use Python’s built-in threading module:

Although SweepMe! internally uses Qt threads, relying only on Python’s standard library keeps the solution self-contained and avoids introducing additional dependencies or licensing considerations for driver developers.

To communicate safely between the caller and the persistent thread, I used Python queues:

The resulting helper class looks like this:

class PersistentDllThread:
    def __init__(self):
        self.tasks = queue.Queue()
        self.thread = threading.Thread(target=self._run, daemon=True)
        self.thread.start()

    def _run(self):
        while True:
            func, args, kwargs, done, result = self.tasks.get()
            try:
                result["value"] = func(*args, **kwargs)
            except Exception as e:
                result["error"] = e
            finally:
                done.set()

    def call(self, func, *args, **kwargs):
        done = threading.Event()
        result = {}

        self.tasks.put((func, args, kwargs, done, result))
        done.wait()

        if "error" in result:
            raise result["error"]

        return result.get("value")​‌

How does it work?

The idea is simple:

  1. When the helper is created, it immediately starts a dedicated background thread.
  2. The thread continuously waits for work items from a queue.
  3. Whenever another part of the driver wants to execute an SDK function, it does not call the function directly.
  4. Instead, the function and its arguments are placed into the queue.
  5. The persistent thread retrieves the task and executes it.
  6. The caller waits until execution is complete and then receives either the result or, in case of an error, the exception.

This effectively transforms the persistent thread into a small command processor. Regardless of which SweepMe! worker thread initiates the request, the SDK call itself is always executed inside the same dedicated thread.

Storing the Thread in the Parameter Store

Next, I create the thread once and store it in the SweepMe! Parameter Store. If the thread already exists from a previous measurement run, it is simply restored.

class Device(EmptyDevice):
    _dll_thread = None
    store_thread_key = "MyInstrument_DLL_Thread"

    def __init__(self):
        super().__init__()

        # more init code

        # Create or Restore the persistent DLL thread - use the same thread for every measurement
        if (dll_thread := self.restore_parameter(self.store_thread_key)) is None:
            dll_thread = PersistentDllThread()
            self.store_parameter(self.store_thread_key, dll_thread)
        Device._dll_thread = dll_thread

This ensures that all driver instances share the same persistent thread.

Making the Pattern Convenient

Having to manually route every SDK call through the thread would become repetitive very quickly.

To simplify usage, I created a decorator:

def dll_thread(func):
    @wraps(func)
    def wrapper(*args, **kwargs):
        if Device._dll_thread is None:
            msg = "DLL Thread is not initialized."
            raise RuntimeError(msg)
        return Device._dll_thread.call(func, *args, **kwargs)
    return wrapper​‌

The decorator ensures that the decorated function is automatically executed in the persistent thread.

Using the Decorator

With the decorator in place, SDK communication becomes very clean:

class Device(EmptyDevice):

    # more functions ...

    def connect(self) -> None:
        # Place communication commands inside a nested function with the @dll_thread decorator:

        @dll_thread
        def _connect():
            result = SDK.openPorts(self.port_string)
            print(result)

        _connect()​‌

The nested _connect() function is submitted to the persistent thread, executed there, and the caller waits until completion.

Most importantly:

Every SDK function call is now guaranteed to originate from the same thread, regardless of how many measurement runs are started in SweepMe!.

In our case, this completely resolved the issue.

Looking Ahead

While this is the first instrument I have encountered that required such a workaround, the pattern proved very effective.

Because of that, I could imagine integrating a helper such as PersistentDllThread and its associated decorator directly into pysweepme in the future. This would make the solution readily available without every developer having to implement it from scratch.

For that reason, I would be very interested to hear from other driver developers:

  • Have you encountered SDKs that are sensitive to the thread they are called from?
  • Have you experienced similar reconnect or initialization issues after repeated measurement runs?
  • Or is this truly an exceptional case?

Feel free to share your experiences. If there are more instruments affected by this behavior, it may be worth providing built-in support for persistent SDK threads in pysweepme.

1 Like