Dynamic library conflict between HP drivers and other tools; cannot even uninstall/reinstall![macOS]

[GwynethLlewelyn]

Hello there,

I have a HP OfficeJetPro 7740 connected to our home/office network (Ethernet, both via IPv4 and IPv6), and I'm using an old MacBook Pro (Retina, 15-inch, Mid 2014) with macOS Big Sur (11.6.3) to connect to it using HP Smart (latest version).

Regarding printing and accessing/configuring the printer, everything works fine. I can use all possible options when printing (e.g. single page, double page, colour, B&W, whatever — everything works), scanning, and faxing. I have no trouble getting HP Smart to recognise and configure the printer and use all the provided functionality. Similarly, the printing subsystem on the Mac is also working fine.

Furthermore, I also have no issues connecting to the embedded web server on the printer; all browsers I've used work fine, both using HTTP and/or HTTPS, and configuring the printer from the web page also works flawlessly.

That said, why should I be asking for support? 😅

Well, the real issue is hidden, several layers deep in the operating system. And it wasn't obvious to find what was going on.

For the past months, I have had a lot of difficulties connecting my iPhone via USB to the Mac. Even charging it that way had some issues; but the worst that happened was that the connection would get lost. macOS even tried to automatically add some extra software in order to 'fix' the issue, but this would not work — the connection would get lost in the middle of an attempted update, thus aborting it. Disconnecting and reconnecting the iPhone would just trigger the same chain of events.

Because this happened just intermittently, I automatically assumed that one of the following (or all) were true:

1. The iPhone and/or the MacBook Pro are getting old, and, with age, material fatigue sets in, and for some reason they lost the ability to provide a reliable USB connection with each other (note that connecting via Wi-Fi and/or Bluetooth works fine).
2. The particular USB cable I have is faulty (I tested with a lot of different cables, even non-Apple ones).
3. The USB port(s) (there are two on this MacBook Pro model) are getting old, and faulty; or, alternatively, the chip driving the USB subsystem is faulty (or overheating, or having some other electrical issue due to age).
4. Something on my USB configuration was wrong and, for some reason, it affected the connection with the iPhone.

Indeed, after experimenting with other USB devices, there was a similar experience: connections would be unreliable. Pen drives, for instance, would sometimes fail to be recognised, or abort a file transfer mid-way (again, I assumed that either the Mac or the drive were getting old and thus faulty). These problems do happen randomly, which is usually a sign of age and there is nothing more you can do. Still, the USB ports seem to have no problem charging devices, so I use them mostly for that purpose. It's only when using them for sending data that problems happen.

Of the list above, I could at least eliminate point 3 A: the USB chip also drives the Apple Internal Keyboard / Trackpad, and I have absolutely no problems with either. Then again, I'm assuming that the connection to those input devices is soldered directly on the motherboard; so, even if the USB chip is operational, the external ports might be somehow faulty.

You may be wondering where this all leads, since it doesn't seem to be related to HP at all!

Bear with me a moment — I'm well aware that this was a hard-to-track issue, and I'm explaining how I've diagnosed it, in the hope that others, in the future, will be able to figure out what is happening.

This weekend I brought out my old KOBO Touch (an eBook reader) which I hadn't updated in a while. While the software/firmgrade updates happen over Wi-Fi, adding my own eBooks (beyond those that I might have acquired via the KOBO bookstore) requires an USB connection to a computer. The KOBO Touch merely shows up as a normal USB drive, and you just copy eBooks in and out of it; alternatively, you can use specialised software to do that (I use Calibre), but it's not a requirement.

To my surprise, the eBook reader was also having trouble connecting (but charging was fine). Sometimes it was able to sustain a connection for a short while — I could see it as a mounted drive, and all the familiar folders and directories were in it. But as soon as I tried to upload/download large files, the connection would drop, and I'd get an error from the MacBook saying that I should unmount the Volume file before disconnecting an USB device.

Again — this apparently has absolutely nothing to do with the printer!

Unfortunately, it does.

On Darwin operating systems (which include macOS, iOS, iPadOS, etc. as well as the Darwin open-source operating system), applications are made out of 'construction blocks', known as dynamic libraries. Almost all contemporary operating systems work that way: the theory is that if two applications are using the same library (say, to draw things on the screen), the operating system only needs to load it once, and every application can share the same code in memory. This has lots of advantages, from preventing too much memory from being used; to make applications load faster (common dynamic libraries will have been already loaded from disk); and it makes it easier to update the low-level layers of the operating systems: just upgrade the dynamic libraries and all applications depending on them will be automatically updated, too — there is no need to recompile them. That's why, in general, an application originally developed for one version of macOS will work on subsequent releases without the application developer requiring to ship a new version of their app.

Putting it like this, it all sounds trivial, obvious, and pretty simple.

Unfortunately, it is anything but that. Operating systems will need to keep track of the various dynamic libraries scattered among the disk, to make sure that when an application launches, all the correct libraries are available, and load them accordingly. Because libraries call other libraries, there are complex dependency trees (more precisely, graphs, but I'm not going into such level of detail), and all of these have to be maintained. For instance, if an application draws something on the screen, it has to load a huge amount of libraries in order to do that, all the way down to the actual hardware driver, passing through all system libraries that draw windows on the screen and manages them. If any of these is broken — or is the 'wrong' library — everything may break down, and, ultimately, the application will not run. This is, very roughly, what happens when you run 32 bit applications on contemporary versions of macOS: the required dynamic libraries simply don't exist any more, thus the application fails, and there is nothing you can do about it. Similarly, pre-Catalina apps had a simpler digital certification sytem (i.e. the digital signature embedded by the developer on the application, so that macOS 'knows' it came from a legitimate developer), and this is why so many old (and largely abandoned...) applications will start giving all sorts of warnings when starting up — the old certification system has been superseded, and if it is still using the 'old' libraries, macOS will refuse to trust that application completely (in some cases, requiring the user to give manual permission to run; in others, deciding that the application is not trusted). There is a whole lot more to it, but this should give you a rough idea why it's not always easy to track errors in macOS.

So, what happens when, after having installed HP Smart (making sure that everything works properly with the printer), I connect my iPhone via USB?

Here is a sample output from my system logs:

Spoiler

Jan 24 18:53:12 gandalf MobileDeviceUpdater[41039]: Entered:_AMMuxedDeviceDisconnected, mux-device:377
Jan 24 18:53:12 gandalf AMPDevicesAgent[829]: Entered:_AMMuxedDeviceDisconnected, mux-device:377
Jan 24 18:53:12 gandalf MobileDeviceUpdater[41039]: Entered:__thr_AMMuxedDeviceDisconnected,mux-device:377
Jan 24 18:53:12 gandalf AMPDeviceDiscoveryAgent[460]: Entered:_AMMuxedDeviceDisconnected, mux-device:377
Jan 24 18:53:12 gandalf AMPDevicesAgent[829]: Entered:__thr_AMMuxedDeviceDisconnected, mux-device:377
Jan 24 18:53:12 gandalf AMPDeviceDiscoveryAgent[460]: Entered:__thr_AMMuxedDeviceDisconnected, mux-device:377
Jan 24 18:53:12 gandalf MobileDeviceUpdater[41039]: tid:ca5b - Mux ID not found in mapping dictionary
Jan 24 18:53:12 gandalf MobileDeviceUpdater[41039]: tid:ca5b - Can't handle disconnect with invalid ecid
Jan 24 18:53:12 gandalf AMPDevicesAgent[829]: tid:1d76b - Mux ID not found in mapping dictionary
Jan 24 18:53:12 gandalf AMPDeviceDiscoveryAgent[460]: tid:c4f7 - Mux ID not found in mappingdictionary
Jan 24 18:53:12 gandalf AMPDevicesAgent[829]: tid:1d76b - Can't handle disconnect with invalid ecid
Jan 24 18:53:12 gandalf AMPDeviceDiscoveryAgent[460]: tid:c4f7 - Can't handle disconnect with invalid ecid
Jan 24 18:53:13 gandalf com.apple.xpc.launchd[1] (com.hp.PSDrMonitorHelper[41201]): LaunchServices returned a bundle URL that does not match with the LoginItem's known association.
Jan 24 18:53:13 gandalf com.apple.xpc.launchd[1] (com.hp.PSDrMonitorHelper[41201]): Service exited with abnormal code: 78
Jan 24 18:53:13 gandalf com.apple.xpc.launchd[1] (com.hp.PSDrMonitorHelper): Service only ran for 0 seconds. Pushing respawn out by 10 seconds.
Jan 24 18:53:23 gandalf com.apple.xpc.launchd[1] (com.hp.PSDrMonitorHelper[41202]): LaunchServices returned a bundle URL that does not match with the LoginItem's known association.
Jan 24 18:53:23 gandalf com.apple.xpc.launchd[1] (com.hp.PSDrMonitorHelper[41202]): Service exited with abnormal code: 78

... etc., etc., these lines are repeated several times, over and over again...

It's a bit long, but you can see a few things happening: the system launcher (launchd) is attempting to launch whatever it needs to detect the iPhone and communicate with it.

The culprit seems to be the line that says: LaunchServices returned a bundle URL that does not match with the LoginItem's known association.

On the Web, you can find a few hints on what that means. For example, on this thread on Apple's Developer Support, the issue appears mysteriously, according to one Apple tech:

'Service Management login items are particularly sensitive to the state of the machine.'

On an unrelated discussion forum for Enpass (a password manager), this issue also seems to appear with that application (note: in fact, it appears for a lot of applications, HP's are by far not the only ones). There is a workaround listed, too, but more on that later. Similar errors, where launchd is respawning 'broken' apps over and over again, are, however, popular with HP applications, as this thread on SuperUser shows.

And I've tracked it down to (at least) one dynamic library used by HP's software, which, unfortunately, is also shared with the USB subsystem

So, what is the solution?

Well, theoretically, this is one of those cases where possibly several installations of all sorts of applications may, at some point, dropped a shared library in the 'wrong' place. At some point in time, some parts were upgraded, others weren't, so multiple copies for the same library may exist, even if they have different names (and version numbers). When the system needs to call a function on a dynamic library that it requires, it fires a request to the kernel to 'make available' the appropriate library — essentially telling it 'I need whatever library is required for calling a certain function X, please load it for me'. But in the case that the kernel is aware of the existence of two (or more!) different libraries that provide the same function — which one should be picked? What library has 'precedence'? The kernel doesn't know. So it attempts to load one of the libraries (possibly picked at random) and throws the warning message above — letting the user know that a choice was made on their behalf, just to keep the system running. Unfortunately, if the system happens to make the wrong decision, it will mean that the application will not run but throw an error instead.

The solution, therefore, is to make the system learn how to 'get it right', i.e. which dynamic library to pick, and which one to discard. I'm not aware of any mechanism that allows that, except for the obvious one: start installing things from scratch. Remove the application throwing the error and re-install it. That's actually the procedure recommended by HP technicians .

So, I did just that — but, unfortunately, for some unknown reason (nothing appeared in the logs!), the HP Uninstaller application couldn't finish its cleaning-up operations. It just started very briefly and stopped at the very beginning. Knowing how slow HP software runs on my MacBook Pro, I let it run for around seven hours or so (!), and the progress bar didn't advance a single pixel. Because there is no way of knowing what it was actually doing (possibly waiting for some special event from the operating system...), I proceeded to remove everything manually.

Now this is no mean task — HP software is scattered all around the disk, under the many Library folders. It's not just a question of throwing the HP applications into the Trash — if it were, you wouldn't need to have an HP Uninstaller, right?

There are a few handy utilities to help you out with the procedure — most of them paid utilities — which will sort of 'magically' remove everything related to a specific application. I'm not overly fond of such applications — it's hard to figure out what they're actually doing 'under the hood', and, who knows, they might be blindingly removing things that they aren't supposed to do — but I admit that some of them are rather useful. I just used App Cleaner & Installer (no link, since I don't really endorse it; there are more solutions available anyway); the trial version doesn't last for long, but even on the 'free' mode, it lists everything it finds related to an installed application — you just need to delete these files manually, most of the time manually from the command line (the paid version will obviously do that automatically), since many of those files may require superuser permissions to be changed or deleted.
I didn't stop with that application, but also actively searched for HP-related data — I found quite a lot of it, in fact. It took me some time, but I believe I have managed to catch everything. And then I could do a clean install of HP Smart, from scratch, without any 'legacy' configurations lying around.

This certainly improved my overall experience quite a lot!

No com.hp.PSDrMonitorHelper errors any more!
... alas, unfortunately for me, the conflicts with USB-attached devices persisted... so, although I learned a lot from this, ultimately I didn't fix my problem. Apparently, now the conflicts are with the Chromium-based browsers I use (Edge and Brave)... and so my quest goes on!