The code looks good and should work in my oppinion.
One further guess: SweepMe! allows to use GPIB adapters from Prologix where the user can register a COM port via menu “File” → “Options” → “Ports”. If somebody uses such an adapter at your setup and has registered a COM port there, it could generate a conflict with opening the COM port in your case.
A way to further debug the driver or the problem is to use the object
self.port.port
https://wiki.sweep-me.net/wiki/Port_manager#Communication
Starting from the semantic phase function “connect” the driver has access to the object self.port which is a SweepMe! port object wrapping different communication interfaces.
With self.port.port you can access the underlying port object, e.g. from pyserial or pyvisa. It means that you have a look whether the port is open and do some modifications for testing purposes.
Also have a look at this issue with a similar COM port problem: