Python development environment 2020 edition, part 2

3 minute read

I got a few questions from my last article about my Python development environment. That is great! I’ll try to answer the questions below.

What about Docker?

What about it? I use Docker in production, I deploy code in Kubernetes, but I think that Docker for my development process is not worth the hassle. First, I have to create a new dependency on my process, create and manage images, change default configurations so Docker does not explode my / partition, and so on. The system isolation benefit that Docker can provide doesn’t help much on a local environment, specially for Python, that a virtual environment cannot provide. The storage management also complicates matters. Virtualenvs have been more than enough so far, but if the need arises, I definitely would use Docker, but I don’t think that is a panacea for everything.

Conda

Anaconda, usually called just Conda, is a bundle of the Python interpreter and libraries in a ready to install package. I like Conda, the idea is good, and I used in the past, specially when developing in a Windows environment. I even recommended Continuum, now called only Anaconda, as a Python environment to be deployed on the cluster on a company that I worked. But two things made me move away from Conda. First, they have a different approach for virtualenvs than the standard Python one, as they encapsulates more than just Python packages: they include C and FORTRAN libraries, compiled binaries from third-parties, and so on. As, again, this can be handy on a Windows environment, on Linux and production environments, the way they control the executable PATH is heavy-handed and very prone to breaking things inside the environment and outside Conda, on system level, a big no-no. The external libraries included can lead to a lot of complexities between Python library dependencies and system libraries. They also created a package repository outside Pipy, the official Python package repository, something that I don’t appreciate that much.

Ansible

I mentioned Ansible several times in the previous article. I use it extensively for configuration management, deployment, automation, you name it. Beyond being written itself in Python, making my workflow easier, it makes repetitive tasks reproducible and easier to maintain, even for local environments. It supports a ton of different OSes and is agentless, meaning that it doesn’t need an agent running on the target system to operate, relying on SSH connections. The playbooks are written in YAML in a declarative format and be very powerful when used with templates in Jinja. I strongly recommend to create a playbook with Ansible to automate any task you have to do more than once.

Jupyter plugins

That’s something that I used in the past but don’t use anymore. The basic configuration of JupyterLab satisfies 90% of my needs, and either I don’t have the time or don’t care enough to install plugins to account for that 10%. The usability is also a bit complicated, as there are Node parts on some plugins, external dependencies that are not Python, so it’s not as easy as doing a pip install. The only thing that I usually do on a longer lived system is to install a TeX distribution and pandoc, so I can export notebooks in PDF format.

Microsoft Windows

I’ve been using Windows professionally for more than one year now, and I have to say, most of the stuff that I’m required to do either it’s better on Windows (working with presentations with other Windows users) or I do on a remote system. I had to develop Python code directly on a Windows machine in the past and it was a pain. I tried do that a few weeks ago and the pain is still there, even more so with libraries that have external dependencies, like fbprophet. If I had to work locally with Python for longer, I would definitely change operating systems. On my personal computer I use Manjaro, a distribution based on Arch Linux, after years of Debian stable, so no problems on this front.

Conclusion part 2

So again, that’s it folks, nothing too fancy. Keep the questions coming!