When Python Troubleshooting Goes Wrong (AKA How to Reinstall Python v3.7)

It was Thursday, mid-afternoon. The hours had rushed past the way they always do when you’re busy but everything is running smoothly. I had almost finished installing a custom Python application for testing and review when I ran into a strange issue.

When I started the app, it would grind for a few seconds, then display the message:

"ModuleNotFoundError: No module named '_lzma'".

I poked around and discovered this module is referenced from a standard Python module named “lzma” which implements file compression/decompression using an algorithm of the same name (lzma = Lempel–Ziv–Markov):

I had multiple versions of Python installed on my Ubuntu workstation and as a test I attempted to manually import the module. This worked in Python versions 2.7 and 3.6 but did not work in 3.7 (the interpreter reported the same error as the custom application). I’m relatively new to Python and the online consensus seemed to be this was a very unusual error and to be safe I should nuke Python from orbit and reinstall. So I issued an APT removal command for Python v3.7 then turned around to take care of something else. *cue ominous music*

When I finally got a chance to return to my workstation, I noticed it had quietly eaten itself. Several of the installed software packages were gone, including the graphical environment, and all I could do was reboot.

While I rebooted I did a quick post-mortem. I suspected (and still believe) my mistake was that I had forgotten I hadn’t actually used a package manager to install the latest Python but had installed it manually from source and I had targeted python3 instead of python3.7 in the removal command. Issuing the command with that target zapped several packages that had Python 3 dependencies. The APT tool does report what it’s going to do before it executes but since I had assumed I knew what the tool was doing and I was distracted by several spinning plates, I hadn’t fully read it. My oft-repeated phrase of “Measure twice, cut once,” echoed in my head.

Rebooting left me at a console login. I was able to log back in but not load any user interface. I attempted to restore broken packages but again had no luck (it’s hard to repair what’s no longer there). I finally decided to boot the Ubuntu 18.04 live CD and attempt a restore from the source media.

I didn’t see a “system refresh” or “reinstall and save user data” option on the CD; however, I was able to simulate a system refresh by selecting the “Other” install option, selecting the same partition mount points and (important!) checking the boxes to not format the partitions. After several tense minutes (and several cryptic notifications in the install log) I was able to log back into the Unity interface (XFCE was long gone) and relief washed over me in an awesome wave–all of my user data was exactly where I had left it.

The system was stable but was missing several APT packages. I reinstalled the critical packages and then set about reinstalling my Python v3.7 environment. To save time I decided to not install Python manually but install it from the deadsnakes PPA. The procedure I used is listed below for posterity (and by that I mean in case I do something this stupid again).

Installing Python v3.7 on Ubuntu via APT

Update the packages list and install any updates:

sudo apt install software-properties-common

Add the deadsnakes PPA to the sources list and confirm:

sudo add-apt-repository ppa:deadsnakes/ppa

At this point, install Python v3.7:

sudo apt install python3.7

And check for the version post-install:

python3.7 --version

I typically use pip to install Python packages and venv to set up virtualized Python environments for testing. To install both of these for Python v3.7, I also issued the following commands:

At this point my Python v3.7 development environment was restored. Manually test importing module “lzma” worked:

And creating a Python virtual environment worked:

I circled back around to the custom Python application and executed it. No module error. I put my headphones back on, loaded up the Stranger Things soundtrack and got back to work.

I’m very careful and meticulous but sometimes even experienced techs need to occasionally catch a puck in the teeth (in a non-Production environment, of course) to remember what the pain of inattention feels like.

© mylevel4cache.com, 2019. Unauthorized use and/or duplication of this material without express and written permission from this site’s author and/or owner is strictly prohibited. Excerpts and links may be used, provided that full and clear credit is given to mylevel4cache.com with appropriate and specific direction to the original content.