My Python Working Notes

Most Python programming books cover the basics of python programming but don’t always deal very much with practical implementation and usage.

As someone who is relatively new to python I come across lot’s of things that are probably obvious for an experienced programmer but not so obvious when you are just starting out.

These are my Notes that I’ve made while creating basic Python programs that may be of use. They cover:

• Installing Python windows and Linux
• Multiple Python versions
• PIP-Python Package Manager
• Python file and directory structure
• Python Modules
• Running programs

Installing Python

The python crash course resources has instructions for installing Python on windows, Linux and MAC OS.

Multiple Python Versions and Installs

PIP is a package manger used to install Python modules and packages.

PIP normally comes installed with Python.

If it isn’t already installed you can install it by following these instructions.

You can have multiple python versions installed on the same machine, but it can cause problems if you aren’t very experienced.

One of the common problems I encountered was when I installed a package with PIP it would be installed to the wrong version/location.

You can use the command

pip –version

To see where pip will install new packages

To see where pip has installed a package type:

pip show package_name

PIP on Linux

On linux packages can be installed globally and locally.

Global packages are available to all users whereas local packages are restricted to the user that installed them.

Doting a simple pip install will usually try to install the package in a system folder usually usr/local/lib/python3.6/ which requires root access.

You can run as root using sudo to gain permissions to make the install work.

However another option and a better one is to install the package locally for that user and to do that you can use the –user option to tell pip that it is a local install.

The following screen shot taken from the PIP documentation shows some scenarios when you try to install a package using the –user option that is already available in a system folder.

The default action for pip is not to do the install as it doesn’t need to.

PIP versions and install Location

You will find different versions of pip installed on your system usually pip,pip3,pip3.6.

Although you will see instructions saying do pip install packagename following those instructions may not work because pip will install the package under the wrong version of python.

So doing

pip install paho-mqtt

May install the paho-mqtt module under python2.7.python3.4 or python3.6 depending on the versions you have on the system and your location when running the command.

On windows changing into the python folder works. That is to install the module for python3.4 then go to the folder that contains the python3.4 files to run the command.

Alternatively use pip3.4 if available. Pip3.4 is simply a shortcut or link to the correct pip.

Again using the command pip –version will show you where pip will install the files

Common pip commands

• pip install package-name
• pip uninstall package-name
• pip install -U pip          #upgrade pip
• python -m pip install -U pip       #upgrade pip on windows.

On Linux if you run as sudo and pip3 then the packages are installed in the /usr/local/lib/python3.5/dist-packages folder.

In this case python3.5 is the latest version.

If you don’t run as sudo then they installed in the home folder under .local/lib/python3.5/dist-packages.

PIP summary

When using pip be aware that it is important to check where pip is installing your packages.

Useful resources

• Python crash course-PIP
• Common pip commands
• PIP documentation

Video- Installing The Mqtt Python Client and Other Modules Using PIP

Linux Folders and Python

• usr/bin has binaries and links from generic e.g 2 to 2.7 etc
• usr/lib -main python files and also the idle
• /usr/local/lib/ – distribution packages that you have installed. E.g the paho mqtt client would be installed there.
• .local/lib -locally installed packages (–user option)

Python Packages

A python package is a collection of python modules. It is effectively a folder containing python files.

A python package directory has to have a __init__.py file.

The presence of this file indicates to the Python interpreter that the directory/folder contains python modules.

Although the __init__.py file can contain configuration information is is also often completely blank.

See this article be Pythonic:__init__.py for more details.

Modules

A python module is a file containing python code. a module can define classes,functions and variables.

A module can be imported into another Python program or run directly.

Namespaces and Modules

A module also defines a namespace. Two modules can contain functions and variable with the same name as they are effectively in two separate namespaces.

Example two modules, m1 and m2 each contain a function add().

To access the add function from modules m1 you need to use the following syntax:

m1.add()

To access the add function from modules m2 you need to use the following syntax:

m2.add()

You might find this article and this article Useful

Importing Modules

When using the import module format then the module is imported with its namespace and so to access a function in that module you will need to prefix it with the module name.

If you use the from module import function format then you can access the function without prefixing it with the module name. Ref Dive into Python

The __main__ module

When you run a script by typing:

python myscript.py

The Python interpreter runs it as module called __main__ which gets its own namespace.

If you imported the script then it would have the module name myscript.

You will also encounter the following code:

if __name__ == "__main__":
    # execute only if run as a script
    main()

This is found in python modules that can be imported or executed directly. These articles give a reasonable explanation

• __main__ — Top-level script environment
• Learn Python Main function

Where are my Packages being installed?

On Windows the packages are placed in the Lib/site-packages directory which is under the root of the python install. That is if you install python to python34 then new packages installed with pip will be placed in the python34\Lib\site-packages folder.

On linux the packages seem to be installed either in one of these locations:

• usr/local/lib/python3.5/dist-packages
• usr/lib/python3/dist-packages
• usr/lib/python3.5/dist-packages

You can find out where Python expects the packages to be found using the following simple script

import site
print (site.getsitepackages())

You can also do it from the python command prompt as follows:

Running Scripts

From the command line on Windows or linux use:

python myprog,py

If you have several versions on Python available e.g. python 2 and  3, then you can use

python3 myprog.py

to start your program using the Python3 Interpreter.

To find out which version of python is the default version use:

python --version

If Your script needs a particular version to work then use:

import sys
assert sys_version_info >=(2,5)

The SheBang Line

At the top of scripts written for Linux you often see this line.

#!/usr/bin/env python

This lets you run your program without starting the interpreter so you use simply:

myprog.py

However the above may not work if you have multiple python versions installed as it will use the default version.

In this case you will need to be explicit and use

/usr/bin/python3.5

To run using python3.5

On Windows you can also add a Shebang line but you need to specify the file location:

#! c:\python34\python.exe

Will run your program with the python3.4 interpreter.

A useful addition to you scripts is the following lines which will print out the version of python the script is using:

import sys
print(sys.executable)

Here is a good overview of the shebang line and the env program (note refers to node but same for python).

Exiting Python Scripts

Do you use quit,exit,sys,exit etc? Good explanation here.

Seems best to use:

raise SystemExit

Questions?

The above covers the main areas where I have had difficulty. You’re experience will be different you can help me improve the usefulness of this page by letting me know any areas that have caused problems by using the contact page or the comments form below.

Resources:

• Python Classes
• Lamda functions
• Logging with Python

Save

Save

Save