Most Python programming books cover the basics of python programming but don’t always deal very much with practical implementation and usage.
As someone who is relatively new to python I come across lot’s of things that are probably obvious for an experienced programmer but not so obvious when you are just starting out.
These are my Notes that I’ve made while creating basic Python programs that may be of use. They cover:
- Installing Python windows and Linux
- Multiple Python versions
- PIP-Python Package Manager
- Python file and directory structure
- Python Modules
- Running programs
Installing Python
The python crash course resources has instructions for installing Python on windows, Linux and MAC OS.
Multiple Python Versions and Installs
PIP is a package manger used to install Python modules and packages.
PIP normally comes installed with Python.
If it isn’t already installed you can install it by following these instructions.
You can have multiple python versions installed on the same machine, but it can cause problems if you aren’t very experienced.
One of the common problems I encountered was when I installed a package with PIP it would be installed to the wrong version/location.
You can use the command
pip –version
To see where pip will install new packages
To see where pip has installed a package type:
pip show package_name
PIP on Linux
On linux packages can be installed globally and locally.
Global packages are available to all users whereas local packages are restricted to the user that installed them.
Doting a simple pip install will usually try to install the package in a system folder usually usr/local/lib/python3.6/ which requires root access.
You can run as root using sudo to gain permissions to make the install work.
However another option and a better one is to install the package locally for that user and to do that you can use the –user option to tell pip that it is a local install.
The following screen shot taken from the PIP documentation shows some scenarios when you try to install a package using the –user option that is already available in a system folder.
The default action for pip is not to do the install as it doesn’t need to.
PIP versions and install Location
You will find different versions of pip installed on your system usually pip,pip3,pip3.6.
Although you will see instructions saying do pip install packagename following those instructions may not work because pip will install the package under the wrong version of python.
So doing
pip install paho-mqtt
May install the paho-mqtt module under python2.7.python3.4 or python3.6 depending on the versions you have on the system and your location when running the command.
On windows changing into the python folder works. That is to install the module for python3.4 then go to the folder that contains the python3.4 files to run the command.
Alternatively use pip3.4 if available. Pip3.4 is simply a shortcut or link to the correct pip.
Again using the command pip –version will show you where pip will install the files
Common pip commands
- pip install package-name
- pip uninstall package-name
- pip install -U pip #upgrade pip
- python -m pip install -U pip #upgrade pip on windows.
On Linux if you run as sudo and pip3 then the packages are installed in the /usr/local/lib/python3.5/dist-packages folder.
In this case python3.5 is the latest version.
If you don’t run as sudo then they installed in the home folder under .local/lib/python3.5/dist-packages.
PIP summary
When using pip be aware that it is important to check where pip is installing your packages.
Useful resources
Video- Installing The Mqtt Python Client and Other Modules Using PIP
Linux Folders and Python
- usr/bin has binaries and links from generic e.g 2 to 2.7 etc
- usr/lib -main python files and also the idle
- /usr/local/lib/ – distribution packages that you have installed. E.g the paho mqtt client would be installed there.
- .local/lib -locally installed packages (–user option)
Python Packages
A python package is a collection of python modules. It is effectively a folder containing python files.
A python package directory has to have a __init__.py file.
The presence of this file indicates to the Python interpreter that the directory/folder contains python modules.
Although the __init__.py file can contain configuration information is is also often completely blank.
See this article be Pythonic:__init__.py for more details.
Modules
A python module is a file containing python code. a module can define classes,functions and variables.
A module can be imported into another Python program or run directly.
Namespaces and Modules
A module also defines a namespace. Two modules can contain functions and variable with the same name as they are effectively in two separate namespaces.
Example two modules, m1 and m2 each contain a function add().
To access the add function from modules m1 you need to use the following syntax:
m1.add()
To access the add function from modules m2 you need to use the following syntax:
m2.add()
You might find this article and this article Useful
Importing Modules
When using the import module format then the module is imported with its namespace and so to access a function in that module you will need to prefix it with the module name.
If you use the from module import function format then you can access the function without prefixing it with the module name. Ref Dive into Python
The __main__ module
When you run a script by typing:
python myscript.py
The Python interpreter runs it as module called __main__ which gets its own namespace.
If you imported the script then it would have the module name myscript.
You will also encounter the following code:
if __name__ == "__main__": # execute only if run as a script main()
This is found in python modules that can be imported or executed directly. These articles give a reasonable explanation
Where are my Packages being installed?
On Windows the packages are placed in the Lib/site-packages directory which is under the root of the python install. That is if you install python to python34 then new packages installed with pip will be placed in the python34\Lib\site-packages folder.
On linux the packages seem to be installed either in one of these locations:
- usr/local/lib/python3.5/dist-packages
- usr/lib/python3/dist-packages
- usr/lib/python3.5/dist-packages
You can find out where Python expects the packages to be found using the following simple script
import site print (site.getsitepackages())
You can also do it from the python command prompt as follows:
Running Scripts
From the command line on Windows or linux use:
python myprog,py
If you have several versions on Python available e.g. python 2 and 3, then you can use
python3 myprog.py
to start your program using the Python3 Interpreter.
To find out which version of python is the default version use:
python --version
If Your script needs a particular version to work then use:
import sys assert sys_version_info >=(2,5)
The SheBang Line
At the top of scripts written for Linux you often see this line.
#!/usr/bin/env python
This lets you run your program without starting the interpreter so you use simply:
myprog.py
However the above may not work if you have multiple python versions installed as it will use the default version.
In this case you will need to be explicit and use
/usr/bin/python3.5
To run using python3.5
On Windows you can also add a Shebang line but you need to specify the file location:
#! c:\python34\python.exe
Will run your program with the python3.4 interpreter.
A useful addition to you scripts is the following lines which will print out the version of python the script is using:
import sys print(sys.executable)
Here is a good overview of the shebang line and the env program (note refers to node but same for python).
Exiting Python Scripts
Do you use quit,exit,sys,exit etc? Good explanation here.
Seems best to use:
raise SystemExit
If you found this guide useful then perhaps you would like to Buy Me a Coffee
Questions?
The above covers the main areas where I have had difficulty. You’re experience will be different you can help me improve the usefulness of this page by letting me know any areas that have caused problems by using the contact page or the comments form below.
Resources:
reading this I am even more confused…
I don’t know how many python versions are installed on my PC. I installed one directly, another one from the Windows store.
When I try to check where my paho-mqtt is installed, I get:
D:\Git_Repositories\Stromzaehler\MQTT\sql-logger\sql-logger>pip show paho-mqtt
Name: paho-mqtt
Version: 1.4.0
Summary: MQTT version 3.1.1 client class
Home-page: http://eclipse.org/paho
Author: Roger Light
Author-email: roger@atchoo.org
License: Eclipse Public License v1.0 / Eclipse Distribution License v1.0
Location: c:\users\ggoer\appdata\local\packages\pythonsoftwarefoundation.python.3.7_qbz5n2kfra8p0\localcache\local-packages\python37\site-packages
Requires:
Required-by:
But I can’t use it in
import paho.mqtt.client as mqtt
it looks like when I call a python script in the command line, a different python version not containing paho-mqtt is used. But I have no idea, how to check and how to use only one version of python.
I understand your frustration I’ve experienced the same. AT the top of your scripts add the following code
import sys
print(“Using “,sys.executable)
this will tell you the python version the script is running from
When you run the script then use the full pathname to the python version.
For example I have versions 3.4 and 3.6 installed in c:\python34 and c:\python36
to run the script using python3.4 I use
c:\python34\python.exe scriptname
PIP is located in c:\python34\Scripts and there are sever versions fortunately one is called pip3.4.exe
So running pip3.4 will run that version. You can check using pip3.4 -V
If I run pip it runs the 3.6 version as it was the last one installed.
There is also a pip3.6 version in c:\python36\Scripts so I could use pip3.6 to explicitly run that version. I only assume that the same is true with 3.7
Hope that helps a little
rgds
steve
hi
first thanks for your blog. it helps me to get into mqtt!
regarding to python in windows – i installed python 2.7.15 and by default it was installed at c:\python27
and i also installed python 3.7. when installed it i check for all user and it was installed on C:\Program Files (x86)\Python37-32.
then i write a script to manage the PATH of the python.exe file.
the script name is p2p3.bat and i place it in c:\windows\system32
————-
@echo off
rem if “%1″==”” (echo hello) else (echo bye)
if “%1″==”” goto help
if not %1==p2 if not %1==p3 goto help
if %1==p2 set path=c:\python27;c:\python27\scripts;%path%
if %1==p3 set path=C:\Program Files (x86)\Python37-32;C:\Program Files (x86)\Python37-32\scripts;%path%
title %1 server
start title %1 client
goto end
:help
echo off
echo This command set python environment
echo[
echo It is assume that python2 is at c:\python27
echo and python3 is at c:\program file (x86)\python3-32
echo[
echo to work with p2 write:
echo p2p3 p2
echo to work with p3 write:
echo p2p3 p3
goto end
:end
———
so in the cmd i write
p2p3 p2
and can work with python 2.7.15
and if i write p2p3 p3
i can work with python 3.7
Nir