ACP hangs on initial exposure

[Dave Sliski]

Hi Bob,

On some evenings, we are seeing that one of the telescopes tries to start its first exposure and hangs. What I specifically mean by that is that Maxim has connected to the camera, reads out the sensor temp (in the display window), ACP in the console output will say the following



Using star at mag 3.7. Slew to star
Star slew to autofocus
wait for slew to complete
slew complete
Doing auto-center...
(taking 120 sec. exposure. HAlpha filter, binning = 1)
(Using 8 MHz readout mode)
(starting exposure)

and then nothing happens.

I've attached the screenshot of what is happening. I've also included the log. I tried to abort and run and it got stuck in the same place.

Where might I look to find the bug?

David

[Dave Sliski]

After I reset the autoguider error, I then relaunched. I thought we were running and went to sleep. However, I found out in the morning the mount got stuck. See log files attached. Any idea why this would happen?

[Dave Sliski]

Here is additional background from Arne who has been monitoring the situation

"The two problems that AP2 has are the "telescope can home" issue with the ASCOM driver, and a guider communication issue between MaximDL and ACP. The "can home" issue is solved by disconnecting the telescope from everything and redoing telescope setup in ACP, which gets you to the ASCOM chooser menu where you can click the appropriate box. Then reconnecting all of the software. This happens during Startup, as we home the mount, and often results in an "operator intervention" preventing any operation. The guider issue shows up when starting the first science field, where a 10-second guider image is taken. You either have ACP waiting for MDL to say the guider image is complete, or if that succeeds, you sometimes get a hang on the guider settle phase. The only way I've found to solve that is to disconnect MDL from everything and restart it."

From Arne 4/18/2024

David

[William Bristow]

Try...

In MaxIm, > Camera Control > Setup > Camera 1 > Options > Advanced > deselect both options "Camera threading on (recommended)" and "Filter wheel threading on (recommended).
Repeat the same steps for Camera 2.
See screen grab attached below.

Both these threading options (for both cameras) are normally defaulted to "Enabled" in MaxIm but some older camera drivers do not support separate threading and MaxIm may suffer a race condition lockup, which freezes the camera control.

There are no significant performance disadvantages by disabling these options except that if one of the cameras or filter wheels suffers a hardware problem and freezes then so will MaxIm because MaxIm and the imaging equipment are all sharing a single process thread and in this case there will be no MaxIm error logs written and you would need to use task manager to close MaxIm because it will no longer respond to mouse clicks.

As far as I can see the error logs only show that the ACP acquire script was aborting after beginning an exposure that failed or the ACP acquire script aborting after beginning a slew, both of those may be due to external hardware errors such as MaxIm reporting a camera error or when a Paramount suffers a stall when slewing.

Having moved on from a Paramount MX last year to a different mount I did have lots of issues with the Paramount's internal USB hub and cabling to the USB ports on the Versa Plate and replaced or repaired the internal USB cabling on several occasions. In the end I gave up using those internal ports and ran external USB cables. The most common issue was that the internal USB cables would gradually work loose and disconnect themselves from the mount control board or the cables would break in the DEC shaft due to being constantly bent and flexed.
If you are using the USB ports on the Paramount's Versa Plate for connection to the camera(s) be aware that the internal mounts USB cabling is subject to wear and breakages that will manifest as camera errors at particular slew angles when the cables are particularly stressed.

The Paramount may also suffer stalls when homing from a cold startup if the RA and DEC gears and worms require a clean and re-grease, or if the mounted equipment is close to the carrying capacity of the mount.

I no longer have access to a Paramount so I can't check this myself but I think I remember there is a user setting in the Paramount TCS that allows you to set the homing speed, pretty sure I reduced the homing speed on my MX by 50% to prevent stalling with a cold startup and a heavy payload.

If the mount is stalling intermittently only during initial homing and the mount has had the annual clean and re-grease of the worms and gears then reduce the homing speed, if the mount still stalls during initial homing with a slower homing speed configured then maybe back-off the worm spring plungers a half-turn.

If the mount is stalling intermittently during a regular slew then as above, possibly the worm spring plungers need backing off a little.

Advanced threading.jpg

[Bob Denny]

William — Thanks very much for helping David with this. I am at the Northeast Astro-Imaging Conference. It has been very busy here even during the talks. I think you’re on the right track. Again, thanks.

[Dave Sliski]

Thanks William,

Two of our three systems had camera threading turned on, non had the filter wheel threading turned on. I'll see if that change helps the first problem.

They payload on the MX+ mount is an astrophysics 130 mm refractor with a small guide telescope and a FLI PL 16803 camera so we should be way under the weight capacity.

I'll check the settings to see if changing those settings might help out with the errors we are seeing with the system on the MX+ mount (we don't see this error on the two scopes that are on the L350 mounts.

David

[William Bristow]

Hi David.

My experience after ten years of ownership of the original Paramount MX was that they could be a bit temperamental, in particular, if a mount stall occurs during homing or slewing (due to an overcurrent situation) the drive motors are effectively short-circuited to initiate an emergency stop and depending on various local variables, such as USB cable length, ground isolation etc., then the sudden power surge would cause the USB connection to be disrupted at the mount electronics end.

If that occurs and you then tried to carry out a reconnect to the mount from the controller application, without first power-cycling the mount, then the ASCOM driver would randomly have some of it's settings returned to default. I think because the mount does not respond to the connection attempt and remains invisible to the ASCOM driver which for some reason resets the ASCOM driver parameters.

Unfortunately, when working remotely you are not always aware that the mount has stalled, there is no reliable indication in TheSky, you need to be physically present to hear the alarm bleeps from mount controller and see the mount motor LED status colours and blink frequency to confirm that a motor stall has occurred and on which axis.

One thing I found improved the situation was to install a USB2 High Speed galvanic isolator in the USB2 cable between the controller computer and the Paramount.

The isolator doesn't stop the mount stalls, which can have a multitude of causes, but the isolator did help to prevent the control board USB interface in the mount from locking up after a mount emergency stop condition.

Whenever you have a Paramount error while homing or slewing and you can't be physically present to diagnose the mount alarm bleeps and LED's status then always power cycle the mount before trying to re-establish a driver connection to the mount from ACP, MaxIm or FocusMax and that *may* help to reduce the frequency of Software Bisque's ASCOM driver settings being reset.

William.

[Dave Sliski]

Hi William,

Thanks for the additional context. We stopped or never used the USB pass through so I don't think that could be one of our issues. But the mount hanging due to some odd mount stall due to some connectivity issue around the USB cord made me think you are onto something here, at least with our system. In later units they supplied a right angle stress relief dongle to lessen the issue you mentioned.

The grounding this is also interesting as that is something we have been working on at the observatory. We have seen grounding issues creep into a different telescope we were working on. Great ideas - really appreciate you sharing all of your knowledge.

Best,

David

[Eric Dose]

I had a MX+ for several years, operated remotely. Here were some things I found, usually the hard way.

* MX+ motors are substantial, and a motor stalling for mechanical reasons is unusual, short of something like egregiously bad balancing, wind gust torque, etc.
* The MX+ control board is very sensitive to electromagnetic pulses (through the air). Just the arcing from a household thermostat will freeze the MX+ and throw a TheSky error. Make sure there is no arcing source nearby. Grounding is generally good but just won't solve this particular problem.
* The MX+ USB system is a bit sensitive to EM noise. I used only high-end cables, added a ferrite to each end of each USB cable, then ran everything USB (except the camera) through an industrial-quality USB hub (which includes opto-isolators, USB-power regulation, RF filters, etc), and then my USB problems went away, for years. I'm convinced that all astronomy USB should use industrial USB hubs (except for the camera).
* Yes, strain relief is important.
* The MX+ pass-through port is tiny, so one must check all cables for strain through the mount's entire range of motion.Tedious, but less tedious than midnight problems in a remote scope.

Good luck
Eric

[Bob Denny]

One thing that might help is to list the changes made recently. These scopes apparently ran hands off for years. Or maybe not!