Python Numpy 教程
Justin Johnson, 杜客(译), freeopen(编) 2017-12-11 [速查手册] #handbookfreeopen: 原译文没有目录导航,阅读起来不方便。补全少量漏翻的地方。虽然内容很基础,但需要时快速查阅一下还是很不错的。
Introduction
Python是一门伟大的通用编程语言,在一些常用库(numpy, scipy, matplotlib)的帮助下,它又会变成一个强大的科学计算环境。
我们期望你们中大多数人对于Python语言和Numpy库比较熟悉,而对于没有Python经验的同学,这篇教程可以帮助你们快速了解Python编程环境和如何使用Python作为科学计算工具。
一部分同学对于Matlab有一定经验。对于这部分同学,我们推荐阅读 numpy for Matlab users 页面。
本教程内容涵盖:
- Basic Python: Basic data types (Containers, Lists, Dictionaries, Sets, Tuples), Functions, Classes
- Numpy: Arrays, Array indexing, Datatypes, Array math, Broadcasting
- SciPy: Image operations, MATLAB files, Distance between points,
- Matplotlib: Plotting, Subplots, Images
Basics of Python
Python是一种高级的,动态类型的多范型编程语言。很多时候,大家会说Python看起来简直和伪代码一样,这是因为你能够通过很少行数的代码表达出很有力的思想。举个例子,下面是用Python实现的经典的quicksort算法例子:
return
=
=
=
=
return + +
print
# Prints "[1, 1, 2, 3, 6, 8, 10]"
Python versions
Python有两个支持的版本,分别是2.7和3.4。这有点让人迷惑,3.0向语言中引入了很多不向后兼容的变化,2.7下的代码有时候在3.4下是行不通的。在这个课程中,我们使用的是2.7版本。
如何查看版本呢?使用python --version
命令。
Basic data types
和大多数编程语言一样,Python拥有一系列的基本数据类型,比如整型、浮点型、布尔型和字符串等。这些类型的使用方式和在其他语言中的使用方式是类似的。
Numbers: 整型和浮点型的使用与其他语言类似。
= 3
# Prints "<class 'int'>"
# Prints "3"
# Addition; prints "4"
# Subtraction; prints "2"
# Multiplication; prints "6"
# Exponentiation; prints "9"
+= 1
# Prints "4"
*= 2
# Prints "8"
= 2.5
# Prints "<class 'float'>"
# Prints "2.5 3.5 5.0 6.25"
需要注意的是,Python中没有 x++
和 x--
的操作符。
Python也有内置的长整型和复杂数字类型,具体细节可以查看文档。
Booleans: Python实现了所有的布尔逻辑,但用的是英语,而不是我们习惯的操作符(比如&&
和||
等)。
= True
= False
# Prints "<class 'bool'>"
# Logical AND; prints "False"
# Logical OR; prints "True"
# Logical NOT; prints "False"
# Logical XOR; prints "True"
Strings: Python对字符串的支持非常棒。
= # String literals can use single quotes
= # or double quotes; it does not matter.
# Prints "hello"
# String length; prints "5"
= + + # String concatenation
# prints "hello world"
= % # sprintf style string formatting
# prints "hello world 12"
字符串对象有一系列有用的方法,比如:
=
# Capitalize a string; prints "Hello"
# Convert a string to uppercase; prints "HELLO"
# Right-justify a string, padding with spaces; prints " hello"
# Center a string, padding with spaces; prints " hello "
# Replace all instances of one substring with another;
# prints "he(ell)(ell)o"
# Strip leading and trailing whitespace; prints "world"
如果想详细查看字符串方法,请看文档。
Containers
Python有以下几种容器类型:列表(lists)、字典(dictionaries)、集合(sets)和元组(tuples)。
Lists
列表就是Python中的数组,但是列表长度可变,且能包含不同类型元素。
= # Create a list
# Prints "[3, 1, 2] 2"
# Negative indices count from the end of the list; prints "2"
= # Lists can contain elements of different types
# Prints "[3, 1, 'foo']"
# Add a new element to the end of the list
# Prints "[3, 1, 'foo', 'bar']"
= # Remove and return the last element of the list
# Prints "bar [3, 1, 'foo']"
列表的细节,同样可以查阅文档.。
Slicing: 为了一次性地获取列表中的元素,Python提供了一种简洁的语法,这就是切片。
= # range is a built-in function that creates a list of integers
# Prints "[0, 1, 2, 3, 4]"
# Get a slice from index 2 to 4 (exclusive); prints "[2, 3]"
# Get a slice from index 2 to the end; prints "[2, 3, 4]"
# Get a slice from the start to index 2 (exclusive); prints "[0, 1]"
# Get a slice of the whole list; prints "[0, 1, 2, 3, 4]"
# Slice indices can be negative; prints "[0, 1, 2, 3]"
= # Assign a new sublist to a slice
# Prints "[0, 1, 8, 9, 4]"
Loops: 我们可以这样遍历列表中的每一个元素:
=
# Prints "cat", "dog", "monkey", each on its own line.
如果想要在循环体内访问每个元素的指针,可以使用内置的enumerate
函数
=
# Prints "#1: cat", "#2: dog", "#3: monkey", each on its own line
List comprehensions: 在编程的时候,我们常常想要将一种数据类型转换为另一种。下面是一个简单例子,将列表中的每个元素变成它的平方。
=
=
# Prints [0, 1, 4, 9, 16]
使用列表推导,你就可以让代码简化很多:
=
=
# Prints [0, 1, 4, 9, 16]
列表推导还可以包含条件:
=
=
# Prints "[0, 4, 16]"
Dictionaries
字典用来储存(键, 值)对,这和Java中的Map
差不多。你可以这样使用它:
= # Create a new dictionary with some data
# Get an entry from a dictionary; prints "cute"
# Check if a dictionary has a given key; prints "True"
= # Set an entry in a dictionary
# Prints "wet"
# print(d['monkey']) # KeyError: 'monkey' not a key of d
# Get an element with a default; prints "N/A"
# Get an element with a default; prints "wet"
del # Remove an element from a dictionary
# "fish" is no longer a key; prints "N/A"
想要知道字典的其他特性,请查阅文档。
Loops:在字典中,用键来迭代更加容易。
=
=
# Prints "A person has 2 legs", "A cat has 4 legs", "A spider has 8 legs"
如果你想要访问键和对应的值,那就使用iteritems
方法:
=
# Prints "A person has 2 legs", "A cat has 4 legs", "A spider has 8 legs"
Dictionary comprehensions:和列表推导类似,但是允许你方便地构建字典。
=
=
# Prints "{0: 0, 2: 4, 4: 16}"
Sets
集合是独立不同个体的无序集合。示例如下:
=
# Check if an element is in a set; prints "True"
# prints "False"
# Add an element to a set
# Prints "True"
# Number of elements in a set; prints "3"
# Adding an element that is already in the set does nothing
# Prints "3"
# Remove an element from a set
# Prints "2"
和前面一样,要知道更详细的,查看文档.
Loops:在集合中循环的语法和在列表中一样,但是集合是无序的,所以你在访问集合的元素的时候,不能做关于顺序的假设。
=
# Prints "#1: fish", "#2: dog", "#3: cat"
Set comprehensions:和字典推导一样,可以很方便地构建集合:
=
# Prints "{0, 1, 2, 3, 4, 5}"
Tuples
元组是一个值的有序列表(不可改变)。从很多方面来说,元组和列表都很相似。和列表最重要的不同在于,元组可以在字典中用作键,还可以作为集合的元素,而列表不行。例子如下:
= # Create a dictionary with tuple keys
= # Create a tuple
# Prints "<class 'tuple'>"
# Prints "5"
# Prints "1"
文档有更多元组的信息。
Functions
Python函数使用def
来定义函数:
return
return
return
# Prints "negative", "zero", "positive"
我们常常使用可选参数来定义函数:
# Prints "Hello, Bob"
# Prints "HELLO, FRED!"
Classes
Python对于类的定义是简单直接的:
# Constructor
= # Create an instance variable
# Instance method
= # Construct an instance of the Greeter class
# Call an instance method; prints "Hello, Fred"
# Call an instance method; prints "HELLO, FRED!"
更多类的信息请查阅文档。
Numpy
Numpy是Python中用于科学计算的核心库。它提供了高性能的多维数组对象,以及相关工具。如何熟悉MATLAB, 你可以从这份教程开始学习Numpy.
Arrays
一个numpy数组是一个由不同数值组成的网格。网格中的数据都是同一种数据类型,可以通过非负整型数的元组来访问。维度的数量被称为数组的阶,数组的大小是一个由整型数构成的元组,可以描述数组不同维度上的大小。
我们可以从列表创建数组,然后利用方括号访问其中的元素:
= # Create a rank 1 array
# Prints "<class 'numpy.ndarray'>"
# Prints "(3,)"
# Prints "1 2 3"
= 5 # Change an element of the array
# Prints "[5, 2, 3]"
= # Create a rank 2 array
# Prints "(2, 3)"
# Prints "1 2 4"
Numpy还提供了很多其他创建数组的方法:
= # Create an array of all zeros
# Prints "[[ 0. 0.]
# [ 0. 0.]]"
= # Create an array of all ones
# Prints "[[ 1. 1.]]"
= # Create a constant array
# Prints "[[ 7. 7.]
# [ 7. 7.]]"
= # Create a 2x2 identity matrix
# Prints "[[ 1. 0.]
# [ 0. 1.]]"
= # Create an array filled with random values
# Might print "[[ 0.91940167 0.08143941]
# [ 0.68744134 0.87236687]]"
其他数组相关方法,请查看文档。
Array indexing
Numpy提供了多种访问数组的方法。
Slicing:和Python列表类似,numpy数组可以使用切片语法。因为数组可以是多维的,所以你必须为每个维度指定好切片。
# Create the following rank 2 array with shape (3, 4)
# [[ 1 2 3 4]
# [ 5 6 7 8]
# [ 9 10 11 12]]
=
# Use slicing to pull out the subarray consisting of the first 2 rows
# and columns 1 and 2; b is the following array of shape (2, 2):
# [[2 3]
# [6 7]]
=
# A slice of an array is a view into the same data, so modifying it
# will modify the original array.
# Prints "2"
= 77 # b[0, 0] is the same piece of data as a[0, 1]
# Prints "77"
你可以同时使用整型和切片语法来访问数组。但是,这样做会产生一个比原数组低阶的新数组。需要注意的是,这里和MATLAB中的情况是不同的:
# Create the following rank 2 array with shape (3, 4)
# [[ 1 2 3 4]
# [ 5 6 7 8]
# [ 9 10 11 12]]
=
# Two ways of accessing the data in the middle row of the array.
# Mixing integer indexing with slices yields an array of lower rank,
# while using only slices yields an array of the same rank as the
# original array:
= # Rank 1 view of the second row of a
= # Rank 2 view of the second row of a
# Prints "[5 6 7 8] (4,)"
# Prints "[[5 6 7 8]] (1, 4)"
# We can make the same distinction when accessing columns of an array:
=
=
# Prints "[ 2 6 10] (3,)"
# Prints "[[ 2]
# [ 6]
# [10]] (3, 1)"
Integer array indexing:当我们使用切片语法访问数组时,得到的总是原数组的一个子集。整型数组访问允许我们利用其它数组的数据构建一个新的数组:
=
# An example of integer array indexing.
# The returned array will have shape (3,) and
# Prints "[1 4 5]"
# The above example of integer array indexing is equivalent to this:
# Prints "[1 4 5]"
# When using integer array indexing, you can reuse the same
# element from the source array:
# Prints "[2 2]"
# Equivalent to the previous integer array indexing example
# Prints "[2 2]"
整型数组访问语法还有个有用的技巧,可以用来选择或者更改矩阵中每行中的一个元素:
# Create a new array from which we will select elements
=
# prints "array([[ 1, 2, 3],
# [ 4, 5, 6],
# [ 7, 8, 9],
# [10, 11, 12]])"
# Create an array of indices
=
# Select one element from each row of a using the indices in b
# Prints "[ 1 6 7 11]"
# Mutate one element from each row of a using the indices in b
+= 10
# prints "array([[11, 2, 3],
# [ 4, 5, 16],
# [17, 8, 9],
# [10, 21, 12]])
Boolean array indexing:布尔型数组访问可以让你选择数组中任意元素。通常,这种访问方式用于选取数组中满足某些条件的元素,举例如下:
=
= # Find the elements of a that are bigger than 2;
# this returns a numpy array of Booleans of the same
# shape as a, where each slot of bool_idx tells
# whether that element of a is > 2.
# Prints "[[False False]
# [ True True]
# [ True True]]"
# We use boolean array indexing to construct a rank 1 array
# consisting of the elements of a corresponding to the True values
# of bool_idx
# Prints "[3 4 5 6]"
# We can do all of the above in a single concise statement:
# Prints "[3 4 5 6]"
为了教程的简短,有很多数组访问的细节我们没有详细说明,可以查看文档。
Datatypes
每个Numpy数组都是数据类型相同的元素组成的网格。Numpy提供了很多的数据类型用于创建数组。当你创建数组的时候,Numpy会尝试猜测数组的数据类型,你也可以通过参数直接指定数据类型,例子如下:
= # Let numpy choose the datatype
# Prints "int64"
= # Let numpy choose the datatype
# Prints "float64"
= # Force a particular datatype
# Prints "int64"
更多细节查看文档。
Array math
基本数学计算函数会对数组中元素逐个进行计算,既可以利用操作符重载,也可以使用函数方式:
=
=
# Elementwise sum; both produce the array
# [[ 6.0 8.0]
# [10.0 12.0]]
# Elementwise difference; both produce the array
# [[-4.0 -4.0]
# [-4.0 -4.0]]
# Elementwise product; both produce the array
# [[ 5.0 12.0]
# [21.0 32.0]]
# Elementwise division; both produce the array
# [[ 0.2 0.33333333]
# [ 0.42857143 0.5 ]]
# Elementwise square root; produces the array
# [[ 1. 1.41421356]
# [ 1.73205081 2. ]]
和MATLAB不同,*
是元素逐个相乘,而不是矩阵乘法。在Numpy中使用dot
来进行矩阵乘法:
=
=
=
=
# Inner product of vectors; both produce 219
# Matrix / vector product; both produce the rank 1 array [29 67]
# Matrix / matrix product; both produce the rank 2 array
# [[19 22]
# [43 50]]
Numpy提供了很多计算数组的函数,其中最常用的一个是’sum’:
=
# Compute sum of all elements; prints "10"
# Compute sum of each column; prints "[4 6]"
# Compute sum of each row; prints "[3 7]"
想要了解更多函数,可以查看文档。
除了计算,我们还常常改变数组或者操作其中的元素。其中将矩阵转置是常用的一个,在Numpy中,使用T
来转置矩阵:
=
# Prints "[[1 2]
# [3 4]]"
# Prints "[[1 3]
# [2 4]]"
# Note that taking the transpose of a rank 1 array does nothing:
=
# Prints "[1 2 3]"
# Prints "[1 2 3]"
Numpy还提供了更多操作数组的方法,请查看文档.
Broadcasting
广播是一种强有力的机制,它让Numpy可以让不同大小的矩阵在一起进行数学计算。我们常常会有一个小的矩阵和一个大的矩阵,然后我们会需要用小的矩阵对大的矩阵做一些计算。
举个例子,如果我们想要把一个向量加到矩阵的每一行,我们可以这样做:
# We will add the vector v to each row of the matrix x,
# storing the result in the matrix y
=
=
= # Create an empty matrix with the same shape as x
# Add the vector v to each row of the matrix x with an explicit loop
= +
# Now y is the following
# [[ 2 2 4]
# [ 5 5 7]
# [ 8 8 10]
# [11 11 13]]
这样是行得通的,但是当’x’矩阵非常大,利用循环来计算就会变得很慢很慢。我们可以换一种思路:
# We will add the vector v to each row of the matrix x,
# storing the result in the matrix y
=
=
= # Stack 4 copies of v on top of each other
# Prints "[[1 0 1]
# [1 0 1]
# [1 0 1]
# [1 0 1]]"
= + # Add x and vv elementwise
# Prints "[[ 2 2 4
# [ 5 5 7]
# [ 8 8 10]
# [11 11 13]]"
Numpy广播机制可以让我们不用创建vv
,就能直接运算,看看下面例子:
# We will add the vector v to each row of the matrix x,
# storing the result in the matrix y
=
=
= + # Add v to each row of x using broadcasting
# Prints "[[ 2 2 4]
# [ 5 5 7]
# [ 8 8 10]
# [11 11 13]]"
y = x + v
这一行,即使x
的维度为(4, 3)
, v
的维度为(3,)
,但由于广播机制,v
在计算时的实际维度为(4, 3)
,其中每行是v
的副本,然后再每个元素相加求得结果。
对两个数组使用广播机制要遵守下列规则:
- 如果数组的秩不同,使用1来将秩较小的数组进行扩展,直到两个数组的尺寸的长度都一样。
- 如果两个数组在某个维度上的长度是一样的,或者其中一个数组在该维度上长度为1,那么我们就说这两个数组在该维度上是相容的。
- 如果两个数组在所有维度上都是相容的,他们就能使用广播。
- 如果两个输入数组的尺寸不同,那么注意其中较大的那个尺寸。因为广播之后,两个数组的尺寸将和那个较大的尺寸一样。
- 在任何一个维度上,如果一个数组的长度为1,另一个数组长度大于1,那么在该维度上,就好像是对第一个数组进行了复制。
支持广播机制的函数是全局函数。哪些是全局函数可以在文档中查找。
下面是一些广播机制的使用:
# Compute outer product of vectors
= # v has shape (3,)
= # w has shape (2,)
# To compute an outer product, we first reshape v to be a column
# vector of shape (3, 1); we can then broadcast it against w to yield
# an output of shape (3, 2), which is the outer product of v and w:
# [[ 4 5]
# [ 8 10]
# [12 15]]
# Add a vector to each row of a matrix
=
# x has shape (2, 3) and v has shape (3,) so they broadcast to (2, 3),
# giving the following matrix:
# [[2 4 6]
# [5 7 9]]
# Add a vector to each column of a matrix
# x has shape (2, 3) and w has shape (2,).
# If we transpose x then it has shape (3, 2) and can be broadcast
# against w to yield a result of shape (3, 2); transposing this result
# yields the final result of shape (2, 3) which is the matrix x with
# the vector w added to each column. Gives the following matrix:
# [[ 5 6 7]
# [ 9 10 11]]
# Another solution is to reshape w to be a column vector of shape (2, 1);
# we can then broadcast it directly against x to produce the same
# output.
# Multiply a matrix by a constant:
# x has shape (2, 3). Numpy treats scalars as arrays of shape ();
# these can be broadcast together to shape (2, 3), producing the
# following array:
# [[ 2 4 6]
# [ 8 10 12]]
广播机制能够让你的代码更简洁更迅速,能够用的时候请尽量使用!
Numpy 文档
这篇教程涉及了你需要了解的numpy中的一些重要内容,但是numpy远不止如此。可以查阅numpy文献来学习更多。
SciPy
Numpy提供了高性能的多维数组,以及计算和操作数组的基本工具。SciPy基于Numpy,提供了大量的计算和操作数组的函数,这些函数对于不同类型的科学和工程计算非常有用。
熟悉SciPy的最好方法就是阅读文档。我们会强调对于本课程有用的部分。
图像操作
SciPy提供了一些操作图像的基本函数。比如,它提供了将图像从硬盘读入到数组的函数,也提供了将数组中数据写入的硬盘成为图像的函数。下面是一个简单的例子:
# Read an JPEG image into a numpy array
=
# Prints "uint8 (400, 248, 3)"
# We can tint the image by scaling each of the color channels
# by a different scalar constant. The image has shape (400, 248, 3);
# we multiply it by the array [1, 0.95, 0.9] of shape (3,);
# numpy broadcasting means that this leaves the red channel unchanged,
# and multiplies the green and blue channels by 0.95 and 0.9
# respectively.
= *
# Resize the tinted image to be 300 by 300 pixels.
=
# Write the tinted image back to disk
MATLAB文件
函数scipy.io.loadmat
和scipy.io.savemat
能够让你读和写MATLAB文件。具体请查看文档。
点之间的距离
SciPy定义了一些有用的函数,可以计算集合中点之间的距离。
函数scipy.spatial.distance.pdist
能够计算集合中所有两点之间的距离:
# Create the following array where each row is a point in 2D space:
# [[0 1]
# [1 0]
# [2 0]]
=
# Compute the Euclidean distance between all rows of x.
# d[i, j] is the Euclidean distance between x[i, :] and x[j, :],
# and d is the following array:
# [[ 0. 1.41421356 2.23606798]
# [ 1.41421356 0. 1. ]
# [ 2.23606798 1. 0. ]]
=
具体细节请阅读文档。
函数scipy.spatial.distance.cdist
可以计算不同集合中点的距离,具体请查看文档。
Matplotlib
Matplotlib是一个作图库。这里简要介绍’matplotlib.pyplot’模块,功能和MATLAB的作图功能类似。
Plotting
matplotlib库中最重要的函数是Plot
。该函数允许你做出2D图形,如下:
# Compute the x and y coordinates for points on a sine curve
=
=
# Plot the points using matplotlib
# You must call plt.show() to make graphics appear.
只需要少量工作,就可以一次画不同的线,加上标签,坐标轴标志等。
# Compute the x and y coordinates for points on sine and cosine curves
=
=
=
# Plot the points using matplotlib
Subplots
可以使用subplot
函数来在一幅图中画不同的东西:
# Compute the x and y coordinates for points on sine and cosine curves
=
=
=
# Set up a subplot grid that has height 2 and width 1,
# and set the first such subplot as active.
# Make the first plot
# Set the second subplot as active, and make the second plot.
# Show the figure.
关于subplot
的更多细节,可以阅读文档。
Images
你可以使用imshow
函数来显示图像,如下所示:
=
= *
# Show the original image
# Show the tinted image
# A slight gotcha with imshow is that it might give strange results
# if presented with data that is not uint8. To work around this, we
# explicitly cast the image to uint8 before displaying it.