Week 12 - Recap - and - Revision
Week 12 - Recap - and - Revision
ITECH1400 – Foundations of
Programming
• This is your last chance to ask questions about anything you might be unsure
about or not understand!
2
What is Problem Solving?
3
Routine and Non-Routing Problems
• Routine (simple)
problems are the
kinds of problems
have simple solutions and
that occur regularly.
• Non-Routine
(complicated &
complex)
problems are more
difficult and often
require devising
strategies to solve.
4
Characteristics of Wicked Problems
• Difficult to define.
• No clear solution.
• Socially complex.
Source: Australian Public Service Commission. (2012). Tackling wicked problems : A public policy perspective. Retrieved
from: http://www.apsc.gov.au/publications-and-media/archive/publications-archive/tackling-wicked-problems
5
The Importance of Problem Solving
― Martin Yate, Knock 'Em Dead 2016: The Ultimate Job Search Guide
6
The Six-Step Problem Solving Process
• There are many different problem-solving methods, and the six-step method is
just one of them.
• The problem for most people is that they do not use one process to solve
problems and issues or to make decisions.
• Another problem is that people are not consistent in how they solve problems!
They do not find something that works and then do it the same way over and
over to be successful!
7
Step 1 - Identifying the Problem
• When identifying a problem the definition of the problem needs to be clear,
concise and unambiguous.
• You might consider using the Five W's to help narrow this down:
• Who,
• What,
• When,
• Where, and
• Why.
• Note: 'Why' considers the impact of the problem - that is, why we should solve
it, rather than why the problem is occurring.
8
Step 2 - Determining the Root Cause(s) of a Problem
• Sometimes a Fishbone Diagram is used to help determine the root cause of a
problem. In such a diagram you start by agreeing on a definition of the problem,
and then break up to problem into separate chains of possible cause and effect.
9
Step 2 - Determining the Root Cause(s) of a Problem (Cont'd)
• For example, the below diagram charts possible causes and effects of why a
basketball player might miss a free throw:
10
Further Steps
• Some problem solving models have additional steps such as:
• 7 - Prevent Recurrence
• Now that you've investigated the problem and understand it, how can
you stop this problem from ever occurring again?
• For example, the 8-step model is used by car manufacturers such as Ford and
Toyota when working on the engineering problems of creating vehicles.
11
Problem Solving Strategies and Techniques
• Some of the most commonly used problem solving strategies include:
• Trial and error,
• Using some form of algorithm to reach a solution, and
• Using some form of heuristic to reach a solution.
Source: 12
https://www.khanacademy.org/science/health-and-medicine/executive-systems-of-the-brain/cognition-2014-03-27T18:40:04.738Z/
Software Development Methodologies
• Largely regardless of programming domain or language, we tend to use one of
a handful of techniques to successfully design software that functions as
expected.
• Scrum.
• We'll take a brief look at the main characteristic of each methodology over the
next few slides, and then we can have a chat about which one we think is the
best method and why that might be.
13
The Waterfall / Waterflow Model
• The waterfall model is the most traditional model of the three where the lifetime
of the developed system goes through a fixed sequence of steps.
Fun fact: The maintenance of a system usually accounts for the vast majority of it's lifetime costs! Not convinced? See: 14
https://en.wikipedia.org/wiki/Long_tail
Agile Development
• Agile Development mostly refers to using a more cyclic iterative model of
software development.
Note: In this model you don't end up with a single finished product at the end, you get multiple prototypes as you go 15
along until you either accept a final prototype or do a final 'clean' re-write of the final prototype's functionality.
Agile Development (Cont'd)
• Another way of looking at the same process is like this, where you build up the
final application from a series of prototypes:
• In the Scrum method, the shareholders (i.e. people you're building the software
system for) are closely involved at the beginning, and during each sprint.
16
Abstraction
• When we talk about abstraction - what we're really talking about is
simplification.
• When we simplify something, our brains are better able to deal with it - because
there's less to think about!
• This makes life easier for us, and makes it less likely that we'll make a mistake
or error.
17
Abstraction (Cont'd)
• If we wanted to write a program to store the record the progress of a student
in a course - here are some of the student attributes that we could store:
• But which of these attributes are the most important ones to store student
scores?
18
Abstraction (Cont'd)
• Taking that big list of attributes of a Student, we can abstract away the ones
we don't need to handle student scores, leaving us with something like this:
• So we've crunched down our list of 30+ attributes to just three - which are the
only ones required to uniquely identify a student, and we've added a few
attributes to track the student's scores.
Question: Could we crunch our three attributes down any further? If so, should we? 19
Modelling
• Modelling is the process where we look at a problem or situation and analyse it
so that we can unambiguously describe it.
• To model a system there are a number of commonly used tools and formats
which we can use, and which you should be familiar with.
20
Activity Diagrams / Flowcharts
• Activity Diagrams show the decisions / branching of a program or system
based on the state of the system.
• For example, a Process Order activity diagram could look something like this:
21
Class Diagrams
• We typically draw class diagrams in a box listing the class name, the
attributes / properties of the class, and the methods (i.e. operations we can
perform) using objects of this class.
Attributes / Properties
Methods / Operations
22
Source: Riley, D., Hunt, Kenny A, & ProQuest. (2014). Computational thinking for the modern problem solver (Chapman &
Hall/CRC textbooks in computing). Boca Raton, FL: CRC Press. p.118.
22
Use Case Diagrams
• Although use cases imply the sequence of events, they do so in a loose
fashion (unlike flowcharts which have specific start and end points to the flow).
System boundary
Function / Action
Source: Riley, D., Hunt, Kenny A, & ProQuest. (2014). Computational thinking for the modern problem solver (Chapman & 23
Hall/CRC textbooks in computing). Boca Raton, FL: CRC Press. p.122.
Algorithms
• Algorithms are sequences of steps which are carried out in a specific order to
produce a desired outcome.
• Although you might not have any 'book knowledge' of algorithms, you actually
use them every single day.
• All that makes these processes algorithms is writing down the series of steps
you perform, along with any branch conditions (can't find toothpaste? phone is
flat, TV is switched off, etc.).
• Once you have the steps and their sequence worked out the desired outcome
will be the same every time!
24
Algorithms (Cont'd)
• Computers use algorithms for absolutely everything. They control what
happens when you switch your computer on, log in, launch an application etc.
• Really, you can think of all computer programs as algorithms - they're just
sequences of steps along with branching points that determine the state of the
machine at any given time. So in this way, you can think of algorithms as the
text-only version of flowcharts!
Equivalent
25
Pseudocode
• When creating or writing down algorithms, we commonly use pseudocode instead
of a specific programming language.
26
Sorting Algorithms - Bubble Sort
• Here's our starting list of un-sorted values:
14 33 27 35 10
• We compare the left-most two values (14 and 33) and see that they're already
sorted, so we leave them alone:
14 33 27 35 10
• We move to the right and compare the next two values (33 and 27) and see
that they are not sorted, so we swap them:
14 27 33 35 10
• We move to the right again and compare the next two values (33 and 35) and
see that they're sorted, so we leave them alone:
14 27 33 35 10
27
Sorting Algorithms - Bubble Sort (Cont'd)
• We move on to the final pair of values (35 and 10) and see that they aren't
sorted so we swap them:
14 27 33 10 35
• Now that we're at the end of the list we go back to the beginning and see that
14 and 27 are still sorted:
14 27 33 10 35
• We move across again and see that 33 and 10 not sorted though, so we need
to swap them:
14 27 10 33 35
28
Sorting Algorithms - Bubble Sort (Cont'd)
• We look at the final 33 and 35, and as they're sorted we leave them alone:
14 27 10 33 35
14 10 27 33 35
14 10 27 33 35
29
Sorting Algorithms - Bubble Sort (Cont'd)
• Back at the beginning, 14 and 10 are in the wrong order so we swap them:
10 14 27 33 35
10 14 27 33 35
• Finally, we go all the way through the list again from the beginning and this time
we don't make ANY swaps at all - this indicates that our list is now sorted!
10 14 27 33 35
• Then we compare it to the next value, and the next - and so on - until we either
find a value that is lower than our current lowest value, or we hit the end of the
list:
14 33 27 35 10
14 33 27 35 10 Nope!
14 33 27 35 10
31
Sorting Algorithms - Selection Sort (Cont'd)
• When we get to the 10 it is lower, so that becomes our lowest value:
14 33 27 35 10
• As we've now hit the end of the list, we swap our current lowest value with the
first element, so now we have one sorted element:
10 33 27 35 14
• Then we start from the next index along and make that our lowest:
10 33 27 35 14
• 27 is not smaller than 35, but 14 is smaller than 27 - so 14 is our new lowest:
10 33 27 35 14
10 33 27 35 14
32
Sorting Algorithms - Selection Sort (Cont'd)
• As we've now hit the end of the list we swap our lowest value (14) with the
value in index 1 (i.e. the 2nd location - so 33 and 14 swap places):
10 33 27 35 14
10 14 27 35 33
• Next up we're starting at index 2 (3rd location) and marking that as our lowest:
10 14 27 35 33
• Neither 35 nor 33 are lower than 27, so when we get to the end of our list 27 is
still our lowest value:
10 14 27 35 33
10 14 27 35 33
33
Sorting Algorithms - Selection Sort (Cont'd)
• Technically we swap our lowest value (27) with index 2 (3rd location) here - but
it's already in index 2 so doing so doesn't change anything:
10 14 27 35 33
10 14 27 35 33
• Aaaand we've hit the end of the list again so we swap the first unsorted index
(i.e. 35) with our current lowest (33), then as there's only a single 'unsorted'
value at the right-hand side, it'll be our largest and we've get a sorted list!
10 14 27 33 35
• WE CARE! Because the other thing that we often need to do is (you guessed
it):
Searching!
And we can search data faster if it has been sorted!
35
Executing Programs
• There are three different ways in which we can convert a program in a high-level
language like Python into a version that can be executed on a CPU:
• Compilation
• All program code is converted into machine code before the program
executes. Once a program has been compiled into machine code it is
ready to be executed - this typically means that the code can execute
very quickly.
• Pure Interpretation
• Each line of program code in converted into machine code for execution
(one line at a time) as the program executes. This typically means that
the code executes less quickly than compilation.
• Hybrid Implementation
• In hybrid implementation, the program code is translated into an
intermediate programming language which is designed to be able to be
interpreted quickly before the program executes. Once this has been
done, it is this intermediate code that is actually interpreted into machine
code. This typically results in code that executes relatively quickly.
Note: So the execution speed hierarchy from fastest to slowest is: Compilation, then Hybrid Interpretation, then Pure 36
Interpretation.
Executing Programs (Cont'd)
• To paraphrase the previous slide, we could say:
• Compilation
• Your source code is converted to an executable file BEFORE you run it.
• Pure Interpretation
• Your source code is converted into executable instructions AS you run it.
• Hybrid Implementation
• Your source code is converted to an intermediate format BEFORE you
run it (compilation)…
• Think of Java - your Test.java gets compiled into Test.class, and then
that Test.class gets interpreted into machine code in order to actually
run!
37
Pure Interpretation
• Pure interpretation is largely the opposite approach to compilation.
Source code
Interpreter to machine
Input data to go Machine code
code
with program
Computer
Results
38
Standard Mathematical Operators
• Now we know about some of the most common types of data we might want to
work with, it's time to move on to some of the operators than help us perform
calculations.
39
Some Other Operators
• While we don't need to use these right away, there are a few other operators that
sometimes come in handy - for example:
• Modulus
• The symbol for modulus is %
• For example, we might say x = 10 % 3, and x would be 1 because it's the
remainder when we divide 10 by 3 (i.e. 10 divided by 3 is 3 remainder 1).
• Power
• To raise a value to a power we use **
• For example, we might say 3 ** 2 to mean 32 which is 9.
• Equal to
• To compare two primitive values we use two equals signs ==.
• For example, is 4 == 4 (True), is 4 == 5 (False).
• Not Equal To
• Just like it sounds but using the symbols !=, so for example 4 != 4 is False
(because 4 is equal to 4), 4 != 5 is True because 4 is not equal to 5 etc.
40
Operator Precedence
• The order in which operators are applied to data is really important, and there's
a simple rule that governs which "value-pairs" in a calculation get calculated
before others!
2 + 10 * 5 = ?!?
• Clue: BODMAS!
41
Operator Precedence (Cont'd)
• In the previous question, we could either say the answer is 60 by calculating:
(2 + 10) * 5 = 60
• If you said 52 then you'd be correct! Because according to BODMAS, the order
in which we calculate the "value-pairs" is:
• Brackets
• Orders (i.e. powers and roots like 32 or )
• Division
• Multiplication
• Addition
• Subtraction
42
The input Function
• When we want to get console input from the user, we use the input method.
• When using the input function, the type of the data returned is always a string.
43
The input Function (Cont'd)
• The reason we're going to have a problem is because we're getting some data
and assigning it to a variable called number…
• …and then we're trying to add together that number variable and an integer -
but the input function converts whatever data is entered into a string!
44
The input Function (Cont'd)
• So how do we get around this problem?
• We have to convert the entered data into the type we want it to be!
• For example:
• The int function attempts to convert whatever is in the brackets after it into an
integer.
• It will fail if we give it data which cannot be converted into an int, for example if
we wrote the following then the conversion would fail:
• They both work the exam same way and provide the same result.
• Also, this "conversion" has a special name - it's called a cast. So you might
"cast an int to a float" or "cast a string to an int" etc.
• Make sure you know what casting is (i.e. changing the data type of a variable) -
it's a very important thing to know. Trust me on this.
46
String / Print Substitutio
• So we could write some code like this:
• We can also achieve the same output as the above by making the final line:
• Each set of curly-braces, { }, in the above line will have the relevant variable
substituted in its place, in the order they're declared:
47
String / Print Substitution (Cont'd)
• We can do a better job by substituting like this:
0 1 2 0 1 2
• This has the benefit that when have a numbered (or "indexed") substitution, we
could do something like this if we wanted and it would all work out alright:
Note: The order that we specify the variables to substitute is always the same, the first is 0, the second is 1 and so on! 48
Branching
• Branching is a term used to describe how we can do one thing out of a range
of possible things based on one or more conditions.
• You already know a lot about branching - you do it everyday in your lives!
• IF it's raining AND you have an umbrella AND you're outside, you might
PUT THE UMBRELLA UP.
Simples! 49
Conditional Operators
• When we branch in a programming language, we use conditional operators
to help us specify the conditions under which we want to do one thing, or a
different thing (i.e. the behaviour of our program changes depending on what's
going on!)
• To do this we can use any of the conditional operators from the table below*:
* = there are one or two others like "===" (strict equality), but we're not too fussed about them right now. 50
"if" Statements
• Whenever we want to perform one operation or another, different operation
depending on circumstances, we can use an if statement.
if (some-condition-is-true):
# do-something
• What's really important here is that we put the colon ( : ) after the closing
braces of our condition, and that we tab-indent the code that we want to run if
the condition is true!
• For example:
if (3 < 4):
print("three is less than four!")
print("three is still less than four!")
Note: We can perform this 'if-test' without using brackets if we want, i.e. if 3 < 4: instead of if (3 < 4): 51
Using "if" with "else"
• Commonly we might want to do one thing under certain conditions, and
something else otherwise - and the way that we do this is to use if statements
in combination with else statements, like this:
currentDay = "Tuesday"
weekendStartDay = "Saturday"
if (currentDay == weekendStartDay):
print("Yay!")
print("What fun stuff shall we do?") Which one of
else: these two
print("Boooo!") blocks will run?
print("Work work work... Bah!")
52
The elif Statement
• Fortunately for us, we don't have to indent things out to absurdity - we can use
the elif statement, instead!
• For example, the previous code can now be written like this:
cost = 0
num = int( input("Please enter the number 1, 2, 3 or 4:
") )
if (num == 1):
cost = 1000
elif (num == 2):
No ridiculous amount of
cost = 2000 indentation required!
elif (num == 3):
cost = 3000
elif (num == 4):
cost = 4000
53
Short-hand Notation to Manipulate Variables (Cont'd)
• We can perform mathematical operations and assignment the long way…
myNumber = 5
myNumber = myNumber + 10 # myNumber is now 15
myNumber = myNumber * 2 # myNumber is now 30
myNumber = myNumber - 6 # myNumber is now 24
myNumber = myNumber / 2 # myNumber is now 12
• …or we can use a short-hand equivalent (which just saves us some typing):
myNumber = 5
myNumber += 10 # myNumber
is now 15
myNumber *= 2 # myNumber is now
30
myNumber -= 6 # myNumber is now
24
myNumber /= 2 # myNumber is now
12
54
Using OR in Python
• All languages have some manner of OR operator, and in Python we use the
word 'or' for this operation.
If it is raining OR it is cold:
Wear a jacket.
55
Using OR in Python (Cont'd)
• Let's say that we have two boolean variables (True/False values) called cond1
and cond2 (i.e. conditions 1 and 2).
• We can see from the above table that for an OR operation to be true, either
condition A must be true OR condition B must be true.
• And if both of them are true, then great! But we only needed one to be true to
get a result of true!
56
Using OR in Python (Cont'd)
• Now let's write some code to make sure we can see how this works in Python.
Take another look at the pseudo-code below:
• If we were thirsty, but not too hot, we might write this in Python as follows:
thirsty = True
tooHot = False
• Which of the two print statements will get executed in the code above?
57
Using OR in Python (Cont'd)
• When we're comparing boolean values in an if-statement we can use a little
bit of shorthand, if we'd like to, in order to save ourselves some typing.
• We can do the same to check if something is False by using the NOT operator,
which is an exclamation mark ( ! ).
58
Using OR in Python (Cont'd)
• Now let's get some feedback from the user, and act on it. But first, just to make
things clear, let's write some pseudo-code which describes what we want to do:
• We could then write the following Python code to perform the above:
59
Using OR in Python (Cont'd)
• So far, we've been performing one of two actions depending on whether at least
one condition has been true. Instead of directly jumping into some code, we can
also do things like set other variables. For example:
wearJacket = False
if (wearJacket == True):
print("Brrr! Definitely jacket weather!");
else:
print("It's quite nice out today - no jacket req'd!")
60
Converting Cases
• We can make comparing input easier on ourselves by converting the input into
either uppercase or lowercase. Let's try it out:
name = "Bruce Wayne"
name = name.upper()
print(name)
61
Using AND in Python
• Now we know how the OR operator works, let's take a look at the AND operator,
which is designated by an ampersand symbol: &
• We generally use the AND operator, when we want something to happen if more
than one condition must be met.
62
Using AND in Python (Cont'd)
• To take this a bit further, let's say that we have two boolean variables called
cond1 and cond2 (i.e. conditions 1 and 2).
• If we create a truth-table for the AND operator using these two boolean
variables, we get this:
• We can see from the above table that for a AND operation to be true, both
condition A AND condition B must be true.
63
Using AND in Python (Cont'd)
• Let's give this a go - take a look at the following pseudo-code:
mobileHasCharge = True
mobileHasSignal = False
if (mobileHasCharge == True) and (mobileHasSignal == True):
print("Call me!")
else:
print("The number you have dialled cannot be
reached.")
• Which of the two print statements will get executed in the code above?
64
Using a Mixture of Data Types with Conditionals
• So far we've been using the same types of data for our AND or OR statements,
but we don't have to - we can mix up the data types if we like:
• In the above code, if any single one of the three conditions that we're testing for
evaluates to False, then the program will print "It's just another day..."
65
Creating and Working With Ranges of Numbers
• Python allows us to create a set of integers between zero and an end value by
using the range statement.
• We can then print out that range of numbers using the print statement, for
example:
End value
print( range(5) )
• When we run the above command, it will display all the whole numbers from
zero up until it's at "one step less than" the end value we specified, going up
by 1 per time.
[0, 1, 2, 3, 4]
66
Creating and Working With Ranges of Numbers (Cont’d)
• We can also use a two parameter version of the range statement where we
specify a starting value and an ending value.
• Again, when we run the above command, it will display all the whole numbers
from the first number we specified, up until it's at "one step less than" the end
value we specified.
[4, 5, 6, 7, 8, 9]
67
Creating and Working With Ranges of Numbers (Cont'd)
• When we just saw the range function, we provided it:
• A range-start value of 4, and
• A range-end value of 10.
• Because we only provided two parameters (i.e. 4 and 10) to the range statement,
the numbers in our range increased by 1 each time as the default behaviour.
• But what if we wanted the numbers in our range to increase by 2's? Or 5's? Or
7's? In this case, we can specify a third parameter, which controls how much to
increase the value of the numbers in our set by!
• This might sound complicated - but it's really very simple! Try running this:
print ( list( range(0, 10, 2) ) )
• If we run the above command, we'll generate a set of numbers starting at 0 and
going up by 2 each time until the value is one step less than our end value of
10:
[0, 2, 4, 6, 8]
68
Creating and Working With Ranges of Numbers (Cont'd)
• The technical term for this "one step less than" behaviour is that the range
statement returns us a set of numbers in what's called a:
half-inclusive range
• In maths, you might see this specified as:
[Start-value, End-value)
• The square bracket [ means that the number is INCLUDED in the range.
• The parenthesis ) means that the number is NOT INCLUDED in the range.
69
Variables and Lists
• As we've seen, a variable is a named piece of data, and it usually only contains
a single piece of information.
• A list, on the other hand, is still a named piece of data - but this time it usually
contains more than a single piece of information, for example:
70
Variables and Lists (Cont'd)
• Let's just take a quick look at our Beatles example again:
• If you've programmed before, you might recognise this looks like an array -
however, while arrays are typically fixed in size Python lists are dynamic in
size, and they can also store different types of data (unlike typical arrays).
• Every item in the list has a numbered location called its index that starts at 0
and goes up to one-less-than-the-number-of-items.
• beatles[0] is "John",
• beatles[1] is "Paul",
• beatles[2] is "George", and
• beatles[3] is "Ringo".
• In Python, because it's a dynamically typed language this is not the case and
we're free to mix and match data types.
72
Adding and Removing Elements to/from Lists
• Lists are dynamic structures, which means that not only can we modify the data
in a list, we can add things to them and remove things from them as we please.
For example:
shoppingList.remove('Milk')
print(shoppingList) # Prints ['Bread', 'Sugar']
del shoppingList[2]
print(shoppingList) # Prints ['Bread', 'Bananas']
Note: When we call remove and give a specific item, only the first element that matches is removed, not all of the 73
matching elements!
Multidimensional Lists
• We can embed lists inside other lists to create multidimensional lists if we want
to, for example:
first = [1, 2, 3]
second = [4, 5, 6]
third = [7, 8, 9]
myList = [first, second, third]
• You can't remove elements from a tuple. Tuples have no remove or pop
methods.
• We can find elements in a tuple, since this doesn’t change the tuple, and we
can also use the in operator to check if an element exists in the tuple (because
again, the tuple is not modified).
• So why would we want tuples at all? The reason is that it allows us to create a
kind of read-only data structure, that once set cannot be modified (not on
purpose, not by accident, not maliciously by a third-party).
76
Loop Types
• When we use loops in modern programming languages, there are only three
different types of loops that we can use:
• We can use a for loop, or
• We can use a while loop, or
• We can use a do-while loop equivalent (there is no exact do-while loop in
Python - but we can create something that works the same way).
• Those are our only three options (we don't use GOTO statements anymore -
it's not considered good practice in modern programming)
• The type of loop that we decide to use (for, while or do-while) depends on two
important factors that we have to ask ourselves:
1. Do we know exactly how many times the loop should execute (yes/no),
and
Note: We'll come back to these two questions shortly - first, let's take a look at each type of loop in action! 77
"For" Loops in Python
• The syntax for writing a for loop in Python is:
• Let's try it out by using a loop to print out the values from 1 to 10:
78
"For" Loops in Python (Cont'd)
• The set of data that we "loop through" (the correct term for this is: iterate)
doesn't have to be just numbers though - we can iterate through any kind of set!
• Let's write a short loop that prints out all the names of the Beatles band
members:
79
"While" Loops in Python
• A while loop continues to run while a certain condition is true, for example:
counter = 1
while (counter <= 5):
print("value of counter is: {}".format(counter) )
counter += 1
Quiz: What would happen if we forgot to put the counter += 1 line inside our while loop? 80
"While" Loops in Python (Cont'd)
• Another example of a while loop might be to keep looping until the user
provides some input which causes us to terminate the loop, for example:
arrived = False
while (arrived == False):
reply = input("Are we there yet? (y/n)")
if (reply == "y"):
arrived = True
print("Finally! Phew!")
• This loop will only end when we finally provide "y" as our reply, which sets our
arrived variable to True!
• As the loop is created to run only "while (arrived == false)" – when this is no
longer the case, the loop no longer runs and we print out our "Finally! Phew!"
message.
Note: The specific value used to stop a while loop from executing is called a sentinel. 81
"While" Loops in Python (Cont'd)
• What will happen when we run the code below?
print("Finally! Phew!")
• This time, the loop will never execute because the initial condition of arrived
being False is never met!
• So in this case, we'd skip the loop entirely, and go straight to the "Finally!"
message.
82
"Do-While" Loops in Python
• Sometimes, we want a loop to always execute at least once – and in most
languages, we'd use what's called a do-while loop for this – but in Python,
there is no such thing, so we have to make our own!
counter = 1
while True:
if (counter <= 5):
print( "Value of counter is: {0}".format(counter) )
counter += 1
else:
break
• In the above loop, when counter is greater than 5 the else condition runs, which
calls the break statement - and the break statement breaks us out of the loop
so we don't get stuck in an infinite loop!
83
"Do-While" Loops in Python
• The same do-while loop:
counter = 1
while True:
if (counter <= 5):
print( "Value of counter is: {0}".format(counter) )
counter += 1
else:
break
When to use "do-while" loops:
84
Multiplying Strings (Cont'd)
• By multiplying strings we can also do things like this:
85
Substrings
• When working with strings, we can extract part of a string (which is called a
substring), like this:
• When we use this [start:end] notation, what we're really saying is:
• Start and include the first character we specify as the start of our range,
• But exclude the last character that we specified as the end of our range.
• Just like the range statement: range(0, 3) gives use the set of data [0,1,2],
when extracting strings using [start:end], we include the start of the range, but
exclude the last element of the range.
86
Substrings (Cont'd)
• To print a string, we know we can use Python code like this.
• But what if we only wanted to print out the first 9 characters? In this case, we
can print out the word Knowledge (which is 9 characters) by specifying a range
that goes from character 0 to character 9 like this:
[start-location-inclusive : end-location-exclusive]
87
Substrings (Cont'd)
• So in our example we said:
• But if we omit (which just means do not include) a value either side of the colon
in [start : end] then Python takes that as:
• [:9] means "Go from the start up to but not including the 10th character",
and
• [9:] means "Go from the 10th character to the end of the string".
88
Finding Substrings
• If we want to find a substring within a string in Python, we can use the find
function, which works like this:
• If the substring exists in the string it will return the character number of the
first occurrence of the substring, or
• If the substring does not exist in the string, it will return the value -1.
89
Determining If A String Is Purely Alphabetical or Numerical
• Sometimes we may want to check if something is a number or a letter - and
again, Python comes to the rescue with the isalpha() and isdigit() functions.
s = "123456"
print( s.isdigit() ) # True - only digits in string
s = "12Three456"
print( s.isdigit() ) # False - not purely digits
s = "ElvisPresley"
print( s.isalpha() ) # True - all chars alphabetical
s = "Elvis Presley"
print( s.isalpha() ) # False - space ' ' is not a
letter!
90
Working with Files in Python
• Like any good language, Python allows us to work with files, but also like any
good language, we need to know what we want to do with the files before we
open them. The options we have available are:
• The difference between options 2 and 3 above is that if a file exists and has
some data in it already:
• If we open it for writing, any existing file contents will be discarded, and
• If we open it for appending, any new data we write can be added to the
end of the existing data in the file.
91
Opening a File in Python
• Before we can read from a file, we have to open the file, like this:
or
or
or
92
Reading Lines from a File
• We can open a file for reading, read a line from the file and print it like this:
• Once we've read the first line of a file, each subsequent call to readline will
read the next line of the file, so to get the second and third lines of the file we
could write:
second = file.readline()
print(second)
third = file.readline()
print(third)
file.close()
The close function is
explained in detail on the
next slide!
93
Reading Lines from a File (Cont'd)
• When we've finished working with a file, we should always close the file for two
important reasons:
1. When we close a file, we free up any resources like RAM and buffers that
Python allocated itself in order to work with the file, and
2. If we've opened a file and modified it, the changes only get written to the
file when the file is closed! So if we create a new file or modify an existing
file and forget to close it - the changes won't "stick"!
• Another important thing happens when we close a file, and that's that the file
pointer (which is just a piece of memory which stores where we are in the file
gets reset back to zero).
94
Counting Elements in a List
• When we have a list of data in Python, we can find out how many elements are
in the list by using the len function (short for length), for example:
• ...sometimes it's good to know how many elements we're working with, so after
running the code at the top of the slide, another way to print each element in
turn could be by using code like the below:
Note: If we'd already calculated the size variable as size = len(beatles) then we could use range(0, size) 95
What is a function?
• A function is just a named "block of code", that we can execute by calling it by
name!
• Functions are defined in Python by using the def keyword, like this:
def sayHello():
print("Hello!")
• In the above example, our function is called sayHello(), it doesn't take any
parameters (which we'll discuss very shortly), and the entire function is just a
single line of code which prints Hello! to the screen.
• To execute the above function (i.e. make it run), we can simply call the function
by name, like this:
sayHello()
96
Functions and Parameters (Cont'd)
• We can use parameters / arguments like this:
def sayHello(name):
print("Hello, {}!".format(name) )
sayHello("Bob")
• What's happening is: The data that we provide when we call the function (which
is called an actual parameter or argument) gets passed to the function.
• When this happens, that data gets given its own name for use inside the
function (called a formal parameter).
• So inside the sayHello function, the variable called name (formal parameter)
stores whatever information we passed to the function when we called it (actual
parameter or argument) - which in this case is just the string data "Bob". 97
Using Multiple Parameters
• We've seen that we can create a functions that take no parameters, and a
single parameter as input to work with - but we can also write functions that
take multiple parameters just as easily:
printSum(5, 3)
98
Using Multiple Parameters (Cont'd)
• When using our printSum function, we could call it like this:
printSum(5, 3)
printSum(3, 5)
• Both of the above calls will produce the exact same output, because it doesn't
matter if we add 5 plus 3, or if we add 3 plus 5 - either way the answer is
always 8!
• No problems so far...
99
Using Multiple Parameters (Cont'd)
# This won't work because our printSum function takes
# two parameters and we aren't giving it any parameters!
printSum()
Quiz: What about if we called printSum("5", "3") - would that work? 100
The Order of Parameters
• Now let's look at a function where the order of the parameters is more important.
If we had a function that took two parameters and used them like this:
• Then if we called the function like below we wouldn't have any problems:
displayFavourites(6, "Pizza")
Fave number: 6 Food: Pizza
displayFavourites("Pizza", 6)
101
Returning a Value from a Function
# Function to add five to a number and return the result
def addFive(number):
number += 5
return number
luckyNumber = 6
print( "Initially, luckyNumber is {0}".format(luckyNumber) )
102
Multiple Assignment
• Unlike most programming languages, Python allows us to specify multiple
values to return from a function - and we can do this by simply listing the values
we'd like to return separated by commas (i.e. 1,3,5 ).
• We've seen that we can specify the value for a variable, like this:
someNumber = 123
• But we can also create and assign multiple variables in one statement, like
this:
103
Multiple Assignment (Cont'd)
• By allowing us to both assign and return multiple values at once by separating
them by commas, we can now do things like swap the value of two variables,
like this:
104
Multiple Assignment (Cont'd)
• Let's do another example of returning multiple values from a function:
print(added) # 8
print(subtracted) # 2
• Let's do one final example, this time to find the highest and lowest values in a
list…
105
A Simple Cup Class
Either side of the word init
class Cup(): are two underscores!
def display(self):
print( "Owner: {0}".format(self.owner) )
print( "Contents: {0}".format(self.contents) )
• So what we've said here is that a Cup object has two properties that we care
about - one called owner and the other called contents (i.e. what's in the cup!).
• The __init__ method is called the constructor and will run automatically
when we create an object of type Cup.
• When we create a Cup or display a cup we don't provide the self parameter -
that gets automatically provided.
106
A Simple Cup Class (Cont'd)
• Now that we have a Cup class - this is a data type that we can use! Or to put it
another way - we can now create cups! Let's give it a go!
• At this point we have two Cup objects, called cup1 and cup2. But we also
wrote a function (called a method when it's part of a class) called displayCup()
- so let's use it!
cup1.displayCup()
Owner: Al
Contents: tea
cup2.displayCup()
Owner: Brenda
Contents: coffee
• Don't get me wrong - you should definitely know about if you're going to be a
programmer…
• …but it's a little bit more in-depth than was necessary for this course so we'll
just say that it's something that's good to know, but not essential to know for the
exam.
108
Categories of Testing
• The two most major categories of testing are:
• Black Box testing, and
• White Box testing.
• Black Box testing is when you are testing a program which you do not have
access to the source code for.
• In this way, the software is a 'black box' – you can see what it does, but you
probably don't know exactly how it's doing it.
109
Categories of Testing (Cont'd)
• White Box testing on the other hand, is where you have access to the source
code and can see exactly what's happening internally.
• This is the kind of testing you do when you're writing the software yourself, or if
the source code is made available to you.
• In white box testing you have the ability to fix the problem yourself (if you have
the programming knowledge to do so), while with black box testing the most
you can typically do is report an error (i.e. raise a bug report) if something isn't
working the way it should.
110
Unit Testing
• Unit Testing is a type of white box testing (i.e. source code is available) where
you test each individual section of the code (i.e. function or class) to ensure it's
all working as desired.
• To do this, you build up a series of small tests that exercise some aspect of
your code, and then run the series of tests to see if each unit passes or fails the
test.
• In this way, when you add more code to your software you can just add a
couple of additional unit tests to make sure it's all working…
• …while all the previous tests still exist to make sure that your new code didn't
break something in your existing code!
This section of the lecture is based on chapter 11 of Python Crash Course, No Starch Press 2016. 111
Unit Tests and Test Cases
• The module unittest from the Python standard library provides tools for
testing your code.
• A unit test verifies that one specific aspect of a function’s behaviour is correct.
• A test case is a collection of unit tests that together prove that a function
behaves as it’s supposed to within the full range of situations you expect it to
handle.
• A good test case considers all the possible kinds of input a function could
receive and includes tests to represent each of these situations.
• A test case with full coverage includes a range of unit tests covering all the
possible ways you can use a function.
• Achieving full coverage on a large project can be daunting! But often it's good
enough to just write tests for your code’s critical behaviours and then aim for full
coverage only if the project starts to see widespread use. Phew!
112
Responding to a Failing Test
• What do you do when a test fails?
• Assuming you’re checking the right conditions, a passing test means the
function is behaving correctly and a failing test means there’s an error in the
new code you wrote.
• So when a test fails, don’t change the test! Instead, fix the code that caused
the test to fail by examining the changes you just made to the function, and
figure out how those changes broke the desired behaviour!
• The best option here is to make the middle name optional. Once we do, our test
for names like 'bob dylan' should pass again, and we should be able to accept
middle names as well. Let's give it a shot…
113
Assert Methods
• Let's try out each of the assert methods - starting with the ones that check for
equality or inequality:
• In these examples we've just put a few simple literal values, but commonly your
assertions will compare against the return value from a function (e.g.
assertTrue(square_value(3), 9) would return true if our square_value function was
working correctly).
114
Assert Methods (Cont'd)
• Next let's take a look at examples of the assertions dealing with lists…
• The assertNotIn functions work just like you'd expect them to:
115
Assert Methods (Cont'd)
• The final assertion we'll look at is whether a function raises an exception when
given specific data to work with. For example:
def square_value(some_number):
if ( str(some_number).isdigit() == false):
raise Exception('Value must be of a numerical
type!')
else:
return some_number * some_number
• So our function will square and return a value if we give it a number to work with,
otherwise it will raise an exception. We can test for this via assertion like this:
• Exceptions are raised with a raise statement, which consists of the following:
• The raise keyword,
• A call to the Exception() function, and
• A string with a helpful error message passed to the Exception() function.
• For example, we could enter the following into the interactive shell:
117
Raising Exceptions (Cont'd)
• If there are no try and except statements covering the raise statement
that raised the exception, the program simply crashes and displays the
exception’s error message.
• Often it’s the code that calls the function, not the function itself, that
knows how to handle an exception. So you will commonly see a raise statement
inside a function and the try and except statements in the code calling the
function.
• For example, let's write some code prints a box shape out of characters and that
raises exceptions if the arguments passed to the function don't meet some criteria:
118
Raising Exceptions (Cont'd)
print(symbol * width) Still in
for i in range(height - 2): boxPrint
print(symbol + (' ' * (width - 2)) + symbol)
function
print(symbol * width)
• When running the code above will call the boxPrint function three times,
resulting in the following output:
**********
* *
**********
An exception happened: Width must be greater than 2.
An exception happened: Symbol must be a single character.
119
Logging Levels
• Logging levels provide a way to categorise your log messages by importance.
There are five logging levels, described below, which go from least to most
important – and messages can be logged at each level using a different logging
function.
120
Logging Levels (Cont'd)
• The benefit of logging levels is that you can change what priority of
logging message you want to see when you run your program.
• After developing your program some more, you may only be interested in
errors. In that case, you can set basicConfig()’s level argument to
logging.ERROR so that it will only show ERROR and CRITICAL messages
and skip any messages logged at DEBUG, INFO, or WARNING levels.
• Finally, so that you don't have to go all the way through your code disabling
each logged message, you can simply call logging.disable() and specify a
logging level to supress all log messages at that level or lower.
or
• You can find and use a library that has the ability to perform the
desired functionality.
• Another way of looking at this would be to say that you either have to build your
own tools to perform the job, or you can use the tools that other people have
built to perform similar jobs.
• Pretty much every conceivable tool that people need has already been
developed and can be used, so there's often little point in 're-inventing the
wheel' - the wheel exists! So let's use it!
122
What is Performance?
• The performance of software is basically how quickly a piece of software
executes either in general, or to perform a given task.
• No large delays between the work starting and the work completing.
• …sometimes the amount of work that has to be done is so large (for example,
rendering a complex frame of animation in a CAD / rendering package) that we
just have to accept that it will take a long time!
Note: Performance with regard to responding to user interaction is called responsiveness - but this is really a type of performance. 123
What is Efficiency?
• Efficiency is the act of using as little resources as possible to achieve a
desired outcome.
• We might be able to make our code use less memory (good!)…but it will
then take longer to process (bad!)
• Or, we might be able to use less storage access (which can be slow) so
that's good…but then the program will use more memory.
Quiz: Would you prefer an application to require twice the RAM, but then it runs at twice the speed? Or maybe an 124
application to take twice as long to load, but then require only half the storage space?
What is Optimisation?
• Optimisation is modifying code to streamline it - so that the same amount of
work can be performed in less time than before, or that the same (or very similar)
results can be obtained by performing less work!
• Analysing our code to find where we can either avoid or reuse the results of
previous calculations, and
125
Systematic Performance Tuning
• The general workflow to performance tune an application is as follows:
3. Identify the part of the system that is critical for improving the performance.
This is called the bottleneck.
126
Performance Analysis
• Performance analysis, also called profiling, is the investigation of a program's
behaviour using information gathered as the program executes.
• Hotspots - which are the pieces of code which run most often, and
• Bottlenecks - which are the pieces of code that run slowly, and thus slow
down the entire program.
• You can think of bottlenecks as small traffic jams on a freeway - only a
small part of the freeway might be jammed up, but it still slows down
everything that must pass through it!
127
Algorithm performance and Big-O Notation
• The more work we give a program to do, the longer it takes to do that work.
• But… what is the relationship between the increased work and increased time
taken to perform that work?
• Some of the more common categories of performance for a piece of code (from
fastest to slowest) are:
128
Algorithm performance and Big-O Notation (Cont'd)
• If we think of Big-O notation as a measure of the number of operations we need
to perform for a given number of ‘things’ (i.e. data elements or such), then we
end up with a table like this:
• As such, if we can use algorithms which have growth rates that are n2 or lower,
then the chances are they’ll operate quickly on our data sets - but if we hit O(n!)
we might be in trouble =/
130
Threading
• Back when most computers had only a single CPU, single threaded code was
fine – it would happily work away as fast as it could, and that’s the most we
could expect from it.
• …so if your code is single threaded, on a quad-core CPU, then you’re using at
most 25% of the possible computational capacity of the device (or even just
12.5% if it has two hardware threads per core!).
• Using multiple threads to perform work requires that we divide the work up to
be shared by multiple CPUs, which means more effort for us (as developers) –
but significantly more performance out of our code.
• There are various threading libraries which can be used to make life easier for
us to share out work between our CPU cores – which one you use (if any – you
can always will depend on the platform / language you’re working on.
131
Mandelbrot - Python Multi-Threaded Version
• After all our hard-work in refactoring the code to execute in parallel these are
our results when calculating using between 1 and 10 threads…
132
Mandelbrot - Python Multi-Process Pooled Version
• Pools are a way to allow Python to spin-up as many worker processes as it
needs to get a job done.
• The default is to create a worker for each CPU core or hardware thread. As the
machine I'm working on has 2 hardware-threads per core, and 4 cores Python
will spin up eight workers to perform our calculations.
import time
import matplotlib.pyplot as plt
import multiprocessing as mp
from functools import partial
133
Mandelbrot - Python Multi-Process Pooled Version (Cont'd)
• At the end of the day, each worker will calculate an eighth of the entire fractal,
delivering us our glorious Mandelbrot plot as before, but…
+ Work!
Note: The eight top-most 8 Python processes are doing work (one worker per hardware thread), and the last Python worker is 134
just the command prompt waiting for the workers in the pool to finish!
Mandelbrot - Python Multi-Process Pooled Version (Cont'd)
• Funnily enough - both yes and no - depending on the circumstances!
135
Mandelbrot - GPU Version
• At this stage we've pretty much hit the limit of how fast Python can generate the
Mandelbrot set. There might be some small code optimisation tweaks, but
nothing too major - we're simply using all the processing capacity the CPU has
available.
• However, the CPU typically the only processing device on a machine, we also
have a GPU (i.e. graphics processing unit)… and these tend to have not just 4
or 8 cores, but thousands of cores!
3,584
CUDA
cores!
Note: CUDA is Nvidia's version of OpenCL - it's a general purpose computation framework that can tap into all the graphics 136
processing cores to crunch numbers rather than draw graphics. Further reading: https://developer.nvidia.com/cuda-zone
GPU
• Because using the GPU via CUDA or OpenMP or such in Python is a little
beyond the scope of this course - I've put together some C++ code that
calculates the Mandelbrot fractal using the GPU and GLSL (the openGL Shading
Language).
• Our previous best was around 2 seconds per frame - but running this
processing on the GPU runs it at around 100 frames per second! (i.e. approx.
10 milliseconds or 0.01 seconds per frame) Result! =D
• Video: https://youtu.be/f3fOoB9b9mE
Further reading: OpenMP - http://www.openmp.org/ , CUDA - https://developer.nvidia.com/about-cuda 137
Profiling
• A profiler is a tool that can be run alongside your code as it executes - and it
times how long each function call takes.
• So if you want your program to run faster, once you know which parts of your
code are the slowest ones you know where you should concentrate your
optimisation effort!
• Let's take the example of a video game as an application which has very tight
performance requirements.
• If the game is required to run at 60 frames per second (fps), then it only has
one-sixtieth (1/60) of a second to process and render each frame…
• In that time the game may have to process audio, game logic, artificial
intelligence, networking AND draw the graphics!
138
Profiling (Cont'd)
• Here's a section of the output from profiling the code:
• ncalls is the number of calls to the function, tottime is the total time spent in a given function
(NOT including calls to sub-functions), percall is tottime divided by ncalls.
• cumtime is the cumulative time spent in this and all subfunctions, percall is cumtime /
ncalls and filename:lineno(function) is what function was called and where it was called
from.
Note: Cumulative time is probably the most important metric to us out of all of these. 139
Profiling (Cont'd)
ncalls tottime percall cumtime percall filename:lineno(function)
1000 0.090 0.000 0.597 0.001 test.py:13(do_audio)
1000 0.259 0.000 1.070 0.001 test.py:17(do_game_logic)
1 0.002 0.002 5.406 5.406 test.py:21(run_game)
1000 0.039 0.000 0.343 0.000 test.py:9 (do_ai)
• So looking at the above, we can see that our do_ai, do_audio and
do_game_logic functions each ran 1000 times…
• …and do_ai ran for 0.343 seconds, do_audio ran for 0.597 seconds and
do_game_log ran for 1.07 seconds.
• As such, if we wanted things to run faster, then we should probably look into
optimising the do_game_logic function first, as that's the one taking up most of
our processing time!
140
Performance in General (Cont'd)
• Once you have your data structures and algorithms - there are two basic rules
for writing efficient code:
• Look at that! I just did not dig a tunnel to China! And I could not dig five
hundred more just as fast!
142
Performance Tips - Use Constants Where Possible
• Python doesn't have constants (i.e. values that cannot be changed) - but most
other languages do.
• The reason being is that if a compiler or interpreter knows a value will never
be changed, it may be able to apply some of its own optimisation logic to
the code which it might otherwise not be able to perform.
Note: The image isn't especially related to what we're talking about - but it's a great quote and I assure you it's the truth. So if
143
you're one of those people who hate change, expect to spend a lot of your time hatin'. Or even better, learn to embrace change.
Performance tips - Trade speed for increased resource use
• Let’s say we’re working on an app which used particle systems, such as a
fireworks simulation (like rockets that zoom up into the air and explode).
• For 1,000 particles – we’re now up to 6,000 new random values per explosion.
• Let’s say we have 100 fireworks, and one explodes on average once every half
second – we’re now looking at 12,000 new random values per second!
• Generating random values is typically quite quick… but it’s not all that quick.
And it’s certainly not free!
144
Performance Tips - Use pools
• We kind-of used pools in the last example on random numbers, but to do this
properly we’d create some kind of wrapper-class which operates on the type of
resource we want to re-use.
• A better technique which avoids all this setup and tear down is that we:
1. Perform the expensive set-up just once, then
2. Re-use the system/connection we’ve set up as required, and finally
3. Tear down the system/connection at the end when we’re finished.
• …which results in our code having the resources it requires available far
quicker than having to do the setup each time.
145
School of Science, Engineering and Information Technology
Please Note this very carefully, as final test arrangements have changed for
this semester:
The final test will open at 1:00 pm AEDT and close at 3:10 pm AEDT.
You must start the final test at 1:00 pm AEDT.
The final test will NOT be open for 24 hours
School of Science, Engineering and Information Technology
Any questions?
Note: There is no lab session this week - start your revision instead!