Introduction

Overview

Teaching: 15 min
Exercises: 5 min

Questions

What is python?

Why might I want to use python?

Objectives

Understand what python is.

Understand why scientists use python.

This is a brief introduction to python programming, focused on the Raspberry Pi and assuming you’ve done an Arduino tutorial. It does not otherwise assume you know much, if anything, about programming. This guide will walk you through installing python and using it to write small programs that allow to you interface between the Raspberry Pi and other devices. A subsequent guide will focus on using Python to work with the General Purpose Input/Output (GPIO) – the little pins that stick up along the edge of the Raspberry Pi board.

Scientists use Python for many programming tasks, chief among them these days are bioinformatics (working with sequence data), data analysis (analyzing large data sets), and figure construction. Our focus here is different: mainly receiving information from one source and doing something with it: transforming it, saving it, etc.

Python is an interpreted programming language that has has become very popular in scientific contexts. There is a huge library of python resources that make it useful for many purposes. Just like with model organisms, what makes it useful is less any particular essential characteristic of python itself and more the body of pre-existing work one can draw from to support new work going forward.

Python was created by Guido von Rossum beginning in the late 1980s. Since then, it’s gone through three generations. The current version (3) is probably the one that new projects should use going forward, although many old projects and code still use version 2. And for a particular project, if a library of existing code would be useful, it might still be appropriate to use version 2.

In C and C-based languages (like bash and the Arduino programming language), white space is generally meaningless. Loops, if-statements, and functions are delimited using curly braces {}. In Python, however, the indentation (either tabs or spaces) determines the structure.

Python does not require you to declare variables. In Arduino programming language, you need to specify what kind of data a variable contains. Python, on the other hand, will decide for you based on what the initial data looks like. Knowing this can help you debug problems if you find that operations in python yield surprising results.

Note that python is extremely well documented and there are many tutorials. The lessons here are designed to present a small subset of python that will be the most useful for the kinds of projects that interface with hardware via the shell.

Key Points

Python is an interpreted scripting language used widely in science.

Python has a large library of useful resources for scientific computing.

Installation and Setup

Overview

Teaching: 30 min
Exercises: 10 min

Questions

Is python installed?

Where is python installed?

Which python should I use?

How should I manage python versions and libraries?

Objectives

Create a directory in your home directory for our work. (Or, better yet, create a repository at github and clone it into your home directory. Github has an excellent python .gitignore file that will prevent you from accidentally including lot of stuff with your commits. Install it and look at it.)

$ mkdir pythonpractice
$ cd pythonpractice

You can check to see to whether python is installed using which:

$ which python
/usr/bin/python
$ which python2
/usr/bin/python2
$ which python3
/usr/bin/python3

Raspbian Buster Lite comes with both python 2 and python 3 pre-installed. Currently, “python” is a symlink to python 2, but this is likely to change.

As was indicated in the introduction, python 3 is the current, stable version of python and new projects should almost certainly be written to use it, if possible. However, there is a large base of existing code and libraries written for python 2 and, if these resources are valuable, it might be worth using python 2 instead.

If python is not installed, you can install it using apt, e.g.

sudo apt install python3

But this should not be necessary. Note that these commands need to be run as root to modify the installed system.

As you work with python, you may discover that for one project, you need to use incompatible versions of python or python libraries. Python addresses this problem by allowing you to create a virtual environment (venv) that maintains a version of python and its libraries separate from the overall system, to provide more stability and to prevent changes made to support one project from stepping on resources required for another.

You may need to install venv.

sudo apt install python3-venv

This will allow you to create a virtual python environment in this directory and then activate it.

$ python3 -m venv .venv
$ source .venv/bin/activate

Once you’ve done this, your command line prompt will be prefaced with a reference to your virtual environment.

Try This

Use “which” to see which python3 will be used for execution now.

If you want to get out your virtual environment, you can run “deactivate.” Or log out.

Note that these commands are not prefaced with sudo, since this is creating a virtual environment that is owned by user. This also allows the user to manage this environment without affecting the rest of the system.

Similarly, to install libraries, the python package installer pip3 (or just pip for python2), should be run as the ordinary user and not via sudo. And it installs the library in our virtual environment, so it won’t affect or interact with other projects we’re trying to use.

A library we’re likely to need is the serial library, so let’s install it now.

$ pip3 install pyserial

We’ll talk more about using libraries in the last episode of this lesson.

Consult python documentation for more information about virtual environments and pip.

Key Points

There are (probably multiple versions of Python already installed.

Virtual environments allow you to set up a custom environment for each project.

Hello World

Overview

Teaching: 10 min
Exercises: 10 min

Questions

How do I create and run a script in python?

Objectives

Setup and run a script in python

The first program anyone writes in a new language is typically “hello world.” This episode should be mostly a review of things you’ve already learned in the lessons about the bash shell and the Arduino.

Using the bash shell and a text editor, you can create a text file that will contain the python script. The convention for python scripts is that they will have the extension “.py” so let’s create a file called “hello.py”.

#! /usr/bin/env python3
print("Hello world!")

The first line of a script includes #! (called a “shebang”) which is treated as a comment by python, but tells the bash shell to execute the referenced program and pass the contents of the file into the program.

In many scripts, you’ll see a reference to the program directly. You’ll recall we used “which” to see which version(s) of python the shell was going to use. This, instead, asks the “env” program to use the version of python our environment calls for. This should do the right thing and use the virtual environment we created.

You will need to make your file executable before you can run it.

$ chmod a+x hello.py

Try This

Use “ls” to inspect the permissions of your file to ensure the execute bit is set correctly.

Since you’re working on a file that is not on your PATH, the shell can only find your file if you point to it directly. The easiest way is putting a dot-slash in front of the file name.

$ ./hello.py
Hello world!

Try This

Can you think of another way to point at your file? Try executing it using a different path.

Key Points

Create a text file

Use a shebang to reference the executable

Flow Control

Overview

Teaching: 30 min
Exercises: 10 min

Questions

How can you affect the flow of execution in python?

Objectives

while, try, for, if/then/else can direct the flow of execution

In arduino programming all of the operation was contained in two functions: one called “setup” that was called once, and one called “loop” that was called over and over again.

In python, commands don’t need to be enclosed in functions: they simply get executed in order and the program will exit when it reaches the end. You can replicate the functionality of the arduino “loop” function by using “while”.

The while command takes a test as an argument and as long as that test evaluates as “true”, the progam will execute the steps inside the while loop as rapidly as possible. The simplest test that will evaluate as “true” is to just say “True.”

Before executing the next program, be ready to type a Ctrl-C to stop execution.

#! /usr/bin/env python3
while True:
  print("Hello world!")

In this example, the program will output forever until interrupted. A Ctrl-C will do it. But notice that this stops execution ungracefully, wherever the program was, it just stops with no cleanup.

If you use “try”, you can catch errors or interrupts and clean up before exiting. This becomes more important if you’re opening files, writing to disk, or using serial connections all of which you probably should shut down gracefully rather than leaving to chance.

#! /usr/bin/env python3
try:
  while True:
    print("Hello world!")
except:
  print("Done!")

In this example, you should be able to type Ctrl-C and see the statement “Done!” to show

This particular structure works like “loop” in Arduino – that is, it just loops forever. But you can also look only while certain conditions are true. But if you only want to look for a certain number of times, you can use “for.”

#! /usr/bin/env python3
try:
  for x in range(0,4):
    print("Hello world!")
except:
  print("Done!")

Question

What is the value of “x” during each iteration?

You can use “if” statements to test for conditions which can include “else” clauses for what to do in exceptions.

#! /usr/bin/env python3
try:
  for x in range(0,4):
    if (x <3):
      print("Hello world!")
    else:
      print("Goodbye world!")
except:
  print("Done!")

Try this

You can test the modulo as a condition determine whether a value divides evenly by a divisor, e.g. “x % 2 == 0” will only be true for even numbers.

This only scratches the surface for how you might direct flow control.

Key Points

Loops can be unconditional or conditional

Variable in Python

Overview

Teaching: 30 min
Exercises: 10 min

Questions

How to store and manipulate data symbolically?

Objectives

Assign values to variable

Convert data from one type to another

In our last Flow Control examples, we snuck a variable (x) into the code. Any word, that isn’t a reserved word (i.e. the name of some python command) can be defined as a variable.

By convention, python recommends using lower case words for variable names with underscores if necessary to aid reability.

Let’s add another variable “place”. You can assign a value to a variable using an equals sign.

#! /usr/bin/env python3

times = 4
try:
  for x in range(0,times):
    if (x < (times -1)):
      print("Hello world!")
    else:
      print("Goodbye world!")
except:
  print("Done!")

Now let’s add another variable “place”, this time we need to put the value in quotes.

#! /usr/bin/env python3

times = 4
place = "world"
try:
  for x in range(0,times):
    if (x < (times - 1)):
      print("Hello " + place + "!")
    else:
      print("Goodbye " + place + "!")
except:
  print("Done!")

Python will determine when a value is placed in a variable what type the variable is. We can inspect the type of a variable using the “type” command.

#! /usr/bin/env python3

times = 4
place = "world"
try:
  for x in range(0,times):
    if (x < (times - 1)):
      print("Hello " + place + "!")
    else:
      print("Goodbye " + place + "!")
except:
  print("Done!")

print(type(times))
print(type(place))

In our examples, “times” is an integer (non-decimal numbers) and “place” is a string (a sequence of alphanumeric characters).

The other most common data type you’re likely to encounter is “float” which can be used for most simple decimals.

#! /usr/bin/env python3

times = 4
place = "world"
try:
  for x in range(0,times):
    progress = x / (times - 1)
    if (x < (times - 1)):
      print("Hello " + place + "!")
    else:
      print("Goodbye " + place + "!")
except:
  print("Done!")

print(type(times))
print(type(place))
print(type(progress))

Question

What is the value of “progress” at the end? What is the type? Can you explain why?

A primary consideration is trying to compare variables of different types. Consider the following script:

#! /usr/bin/env python3

i = int(1)
f = float(1)
s = str("1")

if (i == f):
    print("int equals float")

if (i == s):
    print("int equals string")

if (f == s):
    print("float equals string")

if (f == float(s)):
    print("float equals float(string)")

Question

Which tests return TRUE? What happens if you use different values for i, f, and s?

Variables themselves can evaluate to TRUE under certain circumstances.

#! /usr/bin/env python3

i = int(1)
f = float(1)
s = str("1")

if (i):
  print("int is TRUE")

if (f):
  print("float is TRUE")

if (s):
  print("string is TRUE")

if (u):
  print("undefined is TRUE")

Question

Which variables evaluate to TRUE? Under what circumstances?

There many other data types with more esoteric uses – there is, for example, a BOOL data type that can only be TRUE or FALSE. And types for complex numbers. For most simple projects, however, these types are not needed.

Key Points

Python is strongly, dynamically typed

Formatting Output

Overview

Teaching: 30 min
Exercises: 10 min

Questions

How can python output data?

How can the format of the output be controlled?

Objectives

Use the two forms of formatting for strings

There are several ways to format output in python. We’ll skip the oldest, used in python 2, though you may still see it in older code.

One generally formats strings (lines of alphanumeric characters). You might want to do this to emit messages – especially for debugging – or for output.

You can just add strings together.

Create a file called “format.py” with the following code and execute it.

#! /usr/bin/env python3
a = "hello"
b = "world"
line = a + b
print(line)

Question

Is something missing? How can you fix the output?

Sometimes the problem is not a lack of whitespace, but too much – or line endings that make it difficult to make comparisons or format output correctly.

#! /usr/bin/env python3
a = "hello\n" # \n is a newline character
b = "world"
a = a.strip() # this removes white space from both ends
line = a + b
print(line)

There are also lstrip() and rstrip() functions. And you can provide a list of the characters you want removed if you want to be precise.

If you want to do more than just concatenate or assemble strings, you can use the format function. This uses placeholders in a string as a template and then inserts variables in order.

#! /usr/bin/env python3
a = "hello\n" # \n is a newline character
b = "world"
a = a.strip() # this removes white space from both ends
line = '{} to the whole {}.'.format(a,b)
print(line)

In fact, you most commonly see it simply embedded in the print function.

#! /usr/bin/env python3
a = "hello\n" # \n is a newline character
b = "world"
a = a.strip() # this removes white space from both ends
print('{} to the whole {}.'.format(a,b))

The newest style of formatting is called “f-strings”. It’s like format, but you can just declare the variables inside the curly braces.

#! /usr/bin/env python3
a = "hello\n" # \n is a newline character
b = "world"
a = a.strip() # this removes white space from both ends
print(f'{a} to the whole {b}.')

There are a vast array of other functions you can combine with format or f-strings.

Finally, we’ve been printing our output to the screen. But you might want to write the output to a file.

#! /usr/bin/env python3
a = "hello\n" # \n is a newline character
b = "world"
a = a.strip() # this removes white space from both ends
print(f'{a} to the whole {b}.')
file = open("output.txt","a")
file.write(f'{a} to the whole {b}.')
file.close()

If you inspect the contents of output.txt you’ll see the same line was saved there.

Question

What happens if you run format.py again? How does the contents of output.txt change? Can you get the output to end up on separate lines?

In the file open function, we specified “a” (for append”) as the mode to open the file in, which allows you to add to the existing file. If you use “w” you will overwrite the contents of the file each time you run the program.

Key Points

Combining strings and variables can help structure output.

There’s more than one way to do it.

Useful Libraries

Overview

Teaching: 0 min
Exercises: 0 min

Questions

What else can python do?

Objectives

Use libraries to extend python’s functionality.

Python is designed so that the core functionality is limited. This makes the language smaller and able to launch and run more quickly with a lower memory footprint. Much of the higher level functionality is available through a giant set of “libraries” that offer additional functions that can be imported at runtime.

The Python Standard Libraries are part of python itself. These libraries will be available for use (i.e. “import”) without additional installation. There are also public repositories, like the Python Package Index that include a universe of additional libaries to draw from. These libraries probably need to installed and managed (via pip) to be available.

To use the functions in a library, it needs to be included by means of an “import” statement.

time

The time library has an array of functions for accessing and using system information related to measuring time. Unlike an Arduino, a Raspbery Pi has a real-time clock that will use the network (if available) to maintain an extremely accurate current time value. But there are also functions that allow access to other kinds of time values or to measure the elapsed time between two points. The most basic use might be to access the local time and then emit a string formatted as a time stamp every five seconds.

#! /usr/bin/env python3
import time
while True:
  timestamp = time.strftime("%Y-%m-%d %H:%M:%S", time.localtime())
  print(timestamp)
  time.sleep(5)

sys

The sys library provides functions that support creating a unix utility. Many of its utilities are about the script being able to determine what kind of platform its being run on or the configuration of the system. These are mostly beyond the scope of what we’re likely to do. But there are a few functions that can help you take advantage of the unix shell.

The first is to accept arguments when you start the program. Create a script called “systest.py”

#! /usr/bin/env python3
import sys
if(len(sys.argv) == 2):
    print(f'The argument was {sys.argv[1]}.')
    print(f'The name of the script was {sys.argv[0]}.')
else:
    print("Usage: systest.py [argument]")

The sys library populates the sys.argv variable with all of arguments that were on the command line that invoked the script. Note that the reference to the script includes any path elements that were referenced to execute the script. Sometimes people write a single program with multiple functions that can behave differently depending on how the program was invoked.

Note also that this example contains some basic error checking. If you’re expecting an argument, you might not want to continue if the argument is absent. Typically, programs will emit a “Usage” statement that explains what the program is expecting.

There is another library, argparse, that provides more robust functions for command line arguments, including flags and switches, and hooks for easily adding help messages for each flag and argument. But for simple input, the sys library is often enough.

The sys library offers many more functions, including the ability to write not just to STDOUT, but also to STDERR.

A Brief Digression:

There are three important file descriptors in unix programming. A unix program run on the command line will typically print output to STDOUT that will show up in the terminal window. All of the print statements we’ve looked at print to STDOUT. Error messages should be directed to STDERR, which will typically also show up in the terminal window, although it can be directed elsewhere. And if you direct information into a unix program, the program can receive it on STDIN.

#! /usr/bin/env python3
import sys
if(len(sys.argv) == 2):
    print(f'The argument was {sys.argv[1]}.')
    print(f'The name of the script was {sys.argv[0]}.')
else:
    sys.stderr.write(f"Usage: systest.py [argument]\n")

If you run this program without an argument, you won’t notice any difference. But if you redirect STDOUT somewhere else – say, to a file

$ ./systest.py > output

In this case, the Usage statement will not be redirected into the file, but will show up in the terminal window. Otherwise, the user would get no notification that the script wasn’t working as expected.

pyserial

The pyserial library allows python to send and receive data via serial devices. This library is not one of the Python Standard Libraries and you may recall that we installed it in the first episode of this unit.

We’re going to pull together everything we’ve learned in this lesson to build a script that can receive data from an arduino program. In the rpi-arduino serial episode we created a sketch that transmitted pairs of numbers recorded from a potentiometer via the serial connection. If you run this program while an arduino is connected and emitting values, you should see those values printed, prefaced by the timestamp we learned to create above.

Note that this script will keep running until killed with Ctrl-C, but note that we catch the interrupt and gracefully close the serial connection before stopping.

#! /usr/bin/env python3

import serial,time # yes, even though the library is "pyserial"
ser = serial.Serial('/dev/ttyACM0', baudrate = 9600) # open the connection
try:
  while 1:
    if(ser.in_waiting >0): # check for a line to be read
      line = ser.readline() # read the line from the serial connection
      line = line.decode('utf-8') # convert to UTF-8 encoding
      line = line.rstrip()
      timestamp = time.strftime("%Y-%m-%d %H:%M:%S", time.localtime())
      #print(line)
      print(f'"{timestamp}",{line}') #enclose timestamp (a string) in quotes
except:
  ser.close() # close the serial connection
  print("done.")

Try this

In the Arduino unit, we wrote sketches that also received data via the serial interface. Can you extend this python script to have it send data to the Arduino to modify the behavior of the sketch?

Key Points

Libraries are collections of useful code.

You can make your own libaries too.

Python on Raspberry Pi

Introduction

Overview

Key Points

Installation and Setup

Overview

Try This

Key Points

Hello World

Overview

Try This

Try This

Key Points

Flow Control

Overview

Question

Try this

Key Points

Variable in Python

Overview

Question

Question

Question

Key Points

Formatting Output

Overview

Question

Question

Key Points

Useful Libraries

Overview

time

sys

A Brief Digression:

pyserial

Try this

Key Points