When you’re choosing a base image for your Docker image, Alpine Linux is often recommended.Using Alpine, you’re told, will make your images smaller and speed up your builds.And if you’re using Go that’s reasonable advice.

But if you’re using Python, Alpine Linux will quite often:

Feb 11, 2021 Build a TensorFlow pip package from source and install it on Ubuntu Linux and macOS. While the instructions might work for other systems, it is only tested and supported for Ubuntu and macOS. The Conda packaging tool implements environments, that enable different applications to have different libraries installed. So when you’re building a Docker image for a Conda-based application, you’ll need to activate a Conda environment. Unfortunately, activating Conda environments is a bit complex, and interacts badly with the way Dockerfiles works. So how do you activate a Conda. Nov 16, 2020 Learn how to use a custom Docker base image when deploying trained models with Azure Machine Learning. Azure Machine Learning will use a default base Docker image if none is specified. You can find the specific Docker image used with azureml.core.runconfig.DEFAULTCPUIMAGE. The Conda package manager comes with all the dependencies you need, so you do not need to install everything separately. Both Conda and Docker are intended to solve the same problem, but one of the big differences/benefits of Conda is that you can use Conda without having root access. Conda should be easy to install if you follow these steps. Dec 09, 2020 That's my highest recommended solution but it may not be what you really want to do for many reasons. Especially if you are not familiar with docker and container usage, or really just want a good local install! In this post I will show you how to install NVIDIA's build of TensorFlow 1.15 into an Anaconda Python conda environment.

  1. Make your builds much slower.
  2. Make your images bigger.
  3. Waste your time.
  4. On occassion, introduce obscure runtime bugs.

Let’s see why Alpine is recommended, and why you probably shouldn’t use it for your Python application.

Why people recommend Alpine

Let’s say we need to install gcc as part of our image build, and we want to see how Alpine Linux compares to Ubuntu 18.04 in terms of build time and image size.

First, I’ll pull both images, and check their size:

As you can see, the base image for Alpine is much smaller.

Next, we’ll try installing gcc in both of them.First, with Ubuntu:

Note: Outside the very specific topic under discussion, the Dockerfiles in this article are not examples of best practices, since the added complexity would obscure the main point of the article.

To ensure you’re writing secure, correct, fast Dockerfiles, consider my Python on Docker Production Handbook, which includes a packaging process and >70 best practices.

We can then build and time that:

Now let’s make the equivalent Alpine Dockerfile:

And again, build the image and check its size:

As promised, Alpine images build faster and are smaller: 15 seconds instead of 30 seconds, and the image is 105MB instead of 150MB.That’s pretty good!

But when we switch to packaging a Python application, things start going wrong.

Let’s build a Python image

We want to package a Python application that uses pandas and matplotlib.So one option is to use the Debian-based official Python image (which I pulled in advance), with the following Dockerfile:

And when we build it:

The resulting image is 363MB.

Can we do better with Alpine? Let’s try:

And now we build it:

What’s going on?

Standard PyPI wheels don’t work on Alpine

If you look at the Debian-based build above, you’ll see it’s downloading matplotlib-3.1.2-cp38-cp38-manylinux1_x86_64.whl.This is a pre-compiled binary wheel.Alpine, in contrast, downloads the source code (matplotlib-3.1.2.tar.gz), because standard Linux wheels don’t work on Alpine Linux.

Why?Most Linux distributions use the GNU version (glibc) of the standard C library that is required by pretty much every C program, including Python.But Alpine Linux uses musl, those binary wheels are compiled against glibc, and therefore Alpine disabled Linux wheel support.

Most Python packages these days include binary wheels on PyPI, significantly speeding install time.But if you’re using Alpine Linux you need to compile all the C code in every Python package that you use.

Which also means you need to figure out every single system library dependency yourself.In this case, to figure out the dependencies I did some research, and ended up with the following updated Dockerfile:

And then we build it, and it takes…

… 25 minutes, 57 seconds! And the resulting image is 851MB.

Here’s a comparison between the two base images:

Base imageTime to buildImage sizeResearch required
python:3.8-slim30 seconds363MBNo
python:3.8-alpine1557 seconds851MBYes

Alpine builds are vastly slower, the image is bigger, and I had to do a bunch of research.

Can’t you work around these issues?

Build time

For faster build times, Alpine Edge, which will eventually become the next stable release, does have matplotlib and pandas.And installing system packages is quite fast.As of January 2020, however, the current stable release does not include these popular packages.

Docker install conda

Even when they are available, however, system packages almost always lag what’s on PyPI, and it’s unlikely that Alpine will ever package everything that’s on PyPI.In practice most Python teams I know don’t use system packages for Python dependencies, they rely on PyPI or Conda Forge.

Image size

Some readers pointed out that you can remove the originally installed packages, or add an option not to cache package downloads, or use a multi-stage build.One reader attempt resulted in a 470MB image.

So yes, you can get an image that’s in the ballpark of the slim-based image, but the whole motivation for Alpine Linux is smaller images and faster builds.With enough work you may be able to get a smaller image, but you’re still suffering from a 1500-second build time when they you get a 30-second build time using the python:3.8-slim image.

But wait, there’s more!

Alpine Linux can cause unexpected runtime bugs

While in theory the musl C library used by Alpine is mostly compatible with the glibc used by other Linux distributions, in practice the differences can cause problems.And when problems do occur, they are going to be strange and unexpected.

Some examples:

  1. Alpine has a smaller default stack size for threads, which can lead to Python crashes.
  2. One Alpine user discovered that their Python application was much slower because of the way musl allocates memory vs. glibc.
  3. I once couldn’t do DNS lookups in Alpine images running on minikube (Kubernetes in a VM) when using the WeWork coworking space’s WiFi.The cause was a combination of a bad DNS setup by WeWork, the way Kubernetes and minikube do DNS, and musl’s handling of this edge case vs. what glibc does.musl wasn’t wrong (it matched the RFC), but I had to waste time figuring out the problem and then switching to a glibc-based image.
  4. Another user discovered issues with time formatting and parsing.

Most or perhaps all of these problems have already been fixed, but no doubt there are more problems to discover.Random breakage of this sort is just one more thing to worry about.

Don’t use Alpine Linux for Python images

Conda

Unless you want massively slower build times, larger images, more work, and the potential for obscure bugs, you’ll want to avoid Alpine Linux as a base image.For some recommendations on what you should use, see my article on choosing a good base image.

Review the system requirements listed below before installing Anaconda Individual Edition. If you don’t want the hundreds of packages included with Anaconda, you can install Miniconda, amini version of Anaconda that includes just conda, its dependencies, and Python.

System requirements

  • License: Free use and redistribution under the terms of the ./eula.
  • Operating system: Windows 8 or newer, 64-bit macOS 10.13+, or Linux, including Ubuntu, RedHat, CentOS 6+, and others.
  • If your operating system is older than what is currently supported, you can find older versions of the Anaconda installers in our archive that might work for you. See Using Anaconda on older operating systems for version recommendations.
  • System architecture: Windows- 64-bit x86, 32-bit x86; MacOS- 64-bit x86; Linux- 64-bit x86, 64-bit Power8/Power9.
  • Minimum 5 GB disk space to download and install.
Docker

On Windows, macOS, and Linux, it is best to install Anaconda for the local user,which does not require administrator permissions and is the most robust type ofinstallation. However, if you need to, you can install Anaconda system wide,which does require administrator permissions.

Silent mode install

Docker Install Conda-build

You can use silent mode toautomatically accept default settings and have no screen prompts appear duringinstallation.

Using Anaconda on older operating systems

Conda

We recommend upgrading your operating system. Most OS that are no longersupported in the latest Anaconda are no longer getting security updates.Upgrading your OS allows you to get the latest packages, performanceimprovements, bug fixes, etc.

To use Anaconda on older operating systems, download from our archive.You will not be able to use conda to update or install packages beyondthe Anaconda version noted in the table below, unless you limit it toversions available at the time that particular version of Anacondawas released.You can see what was available by checking the package table archives.

Outdated operating system support
Operating systemHow to install Anaconda
macOS 10.10-10.12; Windows 7Use the command line or graphical installers for Anaconda versions 2019.10 and earlier. Download from our archive.
macOS 10.9

Use the command line or graphical installers for Anaconda versions5.1 and earlier.

Docker install postgresql client. Apr 05, 2016 But the thing is, in my case, I am using Compose so I can immediately instance new containers for new clients (docker-compose -p clientname up) and Compose will create a custom network clientnamedefault, it will create the containers with names clientname appserver1 and clientname server-postgresql1 and more importantly, it will create.

Note

Qt and other packages released after Anaconda Distribution 5.1 (February 15th, 2018)may not work on macOS 10.9, so it may be necessary to not update certain packages beyond this point.

macOS 10.7 and 10.8Use the command line installers for Anaconda versions 4.2 and earlier.
macOS 10.5 and 10.6

Use the command line installers for Anaconda versions 1.8 and earlier.

Windows XPUse Anaconda versions 2.2 and earlier.
Centos5 (or equivalent)Use Anaconda versions 4.3 and earlier.

Installing Anaconda on a non-networked machine (air gap)

  1. Obtain a local copy of the appropriate Anaconda installer for the non-networked machine. You can copy the Anaconda installer to the target machine using many different methods including a portable hard drive, USB drive, or CD.
  2. After copying the installer to the non-networked machine, follow the detailed installation instructions for your operating system.

Note

You can install offline copies of both docs.anaconda.com and enterprise-docs.anaconda.com by installing the conda package anaconda-docs: condainstallanaconda-docs

You can install offline copies of documentation for many of Anaconda’s open-source packages by installing the conda package anaconda-oss-docs: condainstallanaconda-oss-docs

Other ways to get Anaconda or Miniconda

You can find the official Anaconda or Miniconda AMIs on the AWS Marketplace.

Docker Install Conda Packages

You can find the official Anaconda and Miniconda Docker images on Docker Hub.

Docker Install Conda Linux

If you have a CDH cluster, you can install the Anaconda parcel using Cloudera Manager. The Anaconda parcel provides a static installation of Anaconda, based on Python 2.7, that can be used with Python and PySpark jobs on the cluster.

Troubleshooting

If you experience errors during the installation process,review our Troubleshooting topics.

Coments are closed
Scroll to top