Packaging is a very exciting process for the coder. Packaging allows you to share your work with the world and step into a structured, real developer position rather than dealing with bits and pieces of code here and there. And I don’t mean position in a corporate sense here, after all you know your position better than everyone else.

So, this tutorial is about packaging and it can help you navigate the pretty standardized steps of Python packaging and all the intricacies that come with it.

Estimated Time

10 mins

Skill Level

Upper-Intermediate

Exercises

na

Posts from the Series

Course Provider

Provided by HolyPython.com

This page is a practical summary version of all the steps involved in Python library packaging. You can also find links for longer versions of each step with more detailed explanations, comments and remedies for potential pitfalls at the bottom of this page along with a Python packaging checklist.

If you’re reading this tutorial you might already have a cool invention or an improved version of some other program. In that case congratulations! You’re one step closer to creating wealth for yourself and also for the world.

If you don’t have something tangible yet don’t worry, it’s something that comes with sustained practice. I don’t mean to be corny but, just continue coding and exploring and if you stay at it it’s almost guaranteed to happen. When the inspiration is there though, it’s advised not to sleep on it, so when the moment comes, don’t put it off, don’t delay, act and realize!

So, here is a summary of the major Python packaging cornerstones:

  1. Getting the Code Ready for Packaging
  2. Hosting at Github
  3. Open Source Software Licenses
  4. Necessary Files (setup.py, readme.md, license, __init__.py, requirements.txt, setup.cfg)
  5. Local Installation, tests and __init__.py File
  6. Uploading to PyPI with Twine
Long version links at the bottom.
A routine startup portrait, members discussing in front of computers

At some point in your coding journey, eventually you realize the power of publishing. Open source is really cool and it enables sharing technical knowledge and intellectual products at an unprecedented level. But only when you experience first hand:

  • An open source program development on Github
  • Publishing your program on a library like Pypi. Maybe at different stages like:
    • Pre-alpha,
    • Alpha,
    • Beta,
    • Release Candidate,
    • Stable Release)

you really start to feel the incredible power of open source synergies.

It is recommended to every programmer, yet, the knowledge gap seems to prevent many people from getting to that stage.

It’s not that difficult, but there is an open source terminology involved which can be confusing for a lot of people.

This is a huge value loss for everyone, for the programmer, for his environment, eventually for the whole world. Because you never know how far that programmer is going to be able to go if those gaps are bridged and dots are connected.

So, hopefully this Python tutorial empowers coders at the beginning stages of their journey and helps them understand how to package Python code, how to create a library, how to create a module. You might not necessarily be a beginner programmer, but you might not have discovered these topics yet or had the opportunity or occasion for it.

Nevertheless, understanding library structures and entering open source world really expands a coders horizons and we recommend stepping into this knowledge space as soon as possible since the benefits are so high.

You might have just learned how to print “Hello World” or you might actually be producing advanced level programs already and just not have stumbled upon this knowledge yet. In the end, it’s all good. Everyone’s journey is unique and it’s important to embrace and strive to progressively move forward.

 

Let’s check out the general frame of the packaging process and if you’d like to go deeper in any of the steps we have links for that too.

Python Packaging Preparation

After you have a functioning code you will want to do a certain type of cleaning up on it because, code in a library is slightly different than the code in a local .py file that’s used for personal tasks and development. Here are some common practices:

  • Clean up the unnecessary comments
  • Clean up the part or all of print functions not to cloud user’s terminal too much
  • If something can be expressed as a user-defined function, then consider building some functions.
  • Tidy up via Python classes. Try to place everything inside a class or multiple classes. User-defined functions can be run from inside the class when needed. For example:
    • class A:
      • def alpha():
  • A.alpha()

Before we advance, and after certain improvements on your code you also need a few tools for packaging so you might want to get them ready. These are:

  1. A Github account
  2. A PyPI account
  3. Command Prompt that works with Python (or PowerShell), we suggest Anaconda Command Prompt as a practical and complete solution.
  4. twine library installed in Python for uploading to PyPI

Creating a Github Repository

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Ut elit tellus, luctus nec ullamcorper mattis, pulvinar dapibus leo.

python setup.py sdist

Choosing a license

If you’re sharing your code you need a license so that important aspects of usage, development, price, modification, reselling, responsibility, sublicensing etc. are more clear.

Most common open source licenses are:

Apache-2.0

GNU GPL

GNU LGPL

MIT License

BSD Licenses

More on open source licenses here.

Necessary Files

You need a couple of files that you might not be familiar with in order to package your Python library and turn it into an installable repository. These files are:

setup.py

requirements.txt (optional)

readme.md (optional)

setup.cfg (optional)

__init__.py (optional)

So, before being misleading, although it seems only setup.py is obligatory for Python installation and this is technically true, all of the files above are pretty obligatory in practical sense. You don’t really want to miss any of them.

python setup.py sdist

Creating a Release on Github

Github is a great platform for coding collaboration, version management and open source sharing. It’s free and you simply sign up and create a new repository.

You might want to skip the license selection for your project at the very beginning if you’re not informed on the topic yet.

Local Installation & Testing

First off, you can just navigate to the root folder which hosts your package and import the package after launching Python there. This is as simple as navigating with cd command and then launching Python and trying to import the library as intended.

Eventually though you will want to make a real installation to see if everything is going through smoohtly. 

You can use the code below for a local installation, first create a source distribution, it will create a Python egg and source distribution in the folder where your package exists.

Installing a library locally is very simple and straightforward. Just navigate to the package’s root folder where setup.py can be found and execute the command below:

pip install .

This one will properly install the package inside the site-packages folder under your Python installation folder.

It can be more convenient if you don’t want to have to navigate to the specific directory to import your library and it’s still a local installation and has nothing to do with uploading.

After installation is successful, you can just import your library like any other library:

import [library_I_developed]

At this point, you can make the final checks, see if library is installing and functioning as it’s supposed to, you can also check if __init__.py files are functioning properly and creating the correct importing structure you intend for your users.

Uploading to PyPI

If everything is looking absolutely fine it’s time to step into the wild. Wild in this case is Python Package Index or PyPI.

Egg and source distribution terminology can be confusing. Source distribution is just your library packaged in a .zip or .tar.gz compressed archive formats. Installation occurs through these source distributions.

Egg is pretty much the same thing and it’s a zip file, it’s just a bit specialized and distinct so it’s extension is .egg instead of .zip. You can still open it as a zip file and check out the content if you’re curious.

So, let’s create the source distribution:

python setup.py sdist

This will create a source distribution which is basically a tar.gz or zip file as mentioned above.

Now you have a source distribution ready to be shipped to PyPI however, it’s beneficial to add one more step. It takes a couple of seconds but it can save you lots of headache.

Using twine check all the distributions inside the package. If there is an error with the setup.py file or readme.md file or any of the other ones, it will tell you what the error is so you can fix it and ship again.

import twine

twine check dist/*

Once everything clears 100%, it’s time to upload. Just execute the command below:
Don’t forget to import twine if you haven’t before.

twine upload dist/*

You’ll be asked your PyPI username and then password, just type those in order and press enter and your package will be uploaded.

If update to PyPI is successful, that means that now your library can be installed on any computer with Python and internet connection all around the world. All that’s needed is the execution of this simple pip command:

pip install [Library_Name]

Just get rid of the brackets around your library name when you’re actually installing it.

Updating the package in PyPI

You will likely have new versions every now and then just create a new source distribution and repeat the check and upload steps and your package will be updated automatically.

python setup.py sdist
import twine

twine check dist/*
twine upload dist/*

After that you might want to update your library in your computer from PyPI. Just run this code for that:

pip install [Library_Name] --upgrade 

If you still have question, each step included here has its own long version below and maybe the answers you’re looking for are there. It can still be a good idea to check them out even if you don’t have any questions since there are some cool knowledge included regarding each step of Python packaging.

Recommended Posts