.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_gallery/scipy_numpy.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note Click :ref:`here ` to download the full example code .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_gallery_scipy_numpy.py: Numpy: arrays and matrices ========================== NumPy is an extension to the Python programming language, adding support for large, multi-dimensional (numerical) arrays and matrices, along with a large library of high-level mathematical functions to operate on these arrays. **Sources**: - Kevin Markham: https://github.com/justmarkham Computation time: import numpy as np l = [v for v in range(10 ** 8)] s = 0 %time for v in l: s += v arr = np.arange(10 ** 8) %time arr.sum() .. GENERATED FROM PYTHON SOURCE LINES 24-29 Create arrays ------------- Create ndarrays from lists. note: every element must be the same type (will be converted if possible) .. GENERATED FROM PYTHON SOURCE LINES 29-39 .. code-block:: default import numpy as np data1 = [1, 2, 3, 4, 5] # list arr1 = np.array(data1) # 1d array data2 = [range(1, 5), range(5, 9)] # list of lists arr2 = np.array(data2) # 2d array arr2.tolist() # convert array back to list .. rst-class:: sphx-glr-script-out Out: .. code-block:: none [[1, 2, 3, 4], [5, 6, 7, 8]] .. GENERATED FROM PYTHON SOURCE LINES 40-41 create special arrays .. GENERATED FROM PYTHON SOURCE LINES 41-48 .. code-block:: default np.zeros(10) np.zeros((3, 6)) np.ones(10) np.linspace(0, 1, 5) # 0 to 1 (inclusive) with 5 points np.logspace(0, 3, 4) # 10^0 to 10^3 (inclusive) with 4 points .. rst-class:: sphx-glr-script-out Out: .. code-block:: none array([ 1., 10., 100., 1000.]) .. GENERATED FROM PYTHON SOURCE LINES 49-50 arange is like range, except it returns an array (not a list) .. GENERATED FROM PYTHON SOURCE LINES 50-54 .. code-block:: default int_array = np.arange(5) float_array = int_array.astype(float) .. GENERATED FROM PYTHON SOURCE LINES 55-57 Examining arrays ---------------- .. GENERATED FROM PYTHON SOURCE LINES 57-64 .. code-block:: default arr1.dtype # float64 arr2.ndim # 2 arr2.shape # (2, 4) - axis 0 is rows, axis 1 is columns arr2.size # 8 - total number of elements len(arr2) # 2 - size of first dimension (aka axis) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none 2 .. GENERATED FROM PYTHON SOURCE LINES 65-67 Reshaping --------- .. GENERATED FROM PYTHON SOURCE LINES 67-72 .. code-block:: default arr = np.arange(10, dtype=float).reshape((2, 5)) print(arr.shape) print(arr.reshape(5, 2)) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none (2, 5) [[0. 1.] [2. 3.] [4. 5.] [6. 7.] [8. 9.]] .. GENERATED FROM PYTHON SOURCE LINES 73-74 Add an axis .. GENERATED FROM PYTHON SOURCE LINES 74-81 .. code-block:: default a = np.array([0, 1]) a_col = a[:, np.newaxis] print(a_col) #or a_col = a[:, None] .. rst-class:: sphx-glr-script-out Out: .. code-block:: none [[0] [1]] .. GENERATED FROM PYTHON SOURCE LINES 82-83 Transpose .. GENERATED FROM PYTHON SOURCE LINES 83-86 .. code-block:: default print(a_col.T) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none [[0 1]] .. GENERATED FROM PYTHON SOURCE LINES 87-88 Flatten: always returns a flat copy of the orriginal array .. GENERATED FROM PYTHON SOURCE LINES 88-94 .. code-block:: default arr_flt = arr.flatten() arr_flt[0] = 33 print(arr_flt) print(arr) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none [33. 1. 2. 3. 4. 5. 6. 7. 8. 9.] [[0. 1. 2. 3. 4.] [5. 6. 7. 8. 9.]] .. GENERATED FROM PYTHON SOURCE LINES 95-96 Ravel: returns a view of the original array whenever possible. .. GENERATED FROM PYTHON SOURCE LINES 96-103 .. code-block:: default arr_flt = arr.ravel() arr_flt[0] = 33 print(arr_flt) print(arr) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none [33. 1. 2. 3. 4. 5. 6. 7. 8. 9.] [[33. 1. 2. 3. 4.] [ 5. 6. 7. 8. 9.]] .. GENERATED FROM PYTHON SOURCE LINES 104-123 Summary on axis, reshaping/flattening and selection --------------------------------------------------- Numpy internals: By default Numpy use C convention, ie, Row-major language: The matrix is stored by rows. In C, the last index changes most rapidly as one moves through the array as stored in memory. For 2D arrays, sequential move in the memory will: - iterate over rows (axis 0) - iterate over columns (axis 1) For 3D arrays, sequential move in the memory will: - iterate over plans (axis 0) - iterate over rows (axis 1) - iterate over columns (axis 2) .. figure:: ../images/numpy_array3d.png .. GENERATED FROM PYTHON SOURCE LINES 123-127 .. code-block:: default x = np.arange(2 * 3 * 4) print(x) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none [ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23] .. GENERATED FROM PYTHON SOURCE LINES 128-129 Reshape into 3D (axis 0, axis 1, axis 2) .. GENERATED FROM PYTHON SOURCE LINES 129-133 .. code-block:: default x = x.reshape(2, 3, 4) print(x) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none [[[ 0 1 2 3] [ 4 5 6 7] [ 8 9 10 11]] [[12 13 14 15] [16 17 18 19] [20 21 22 23]]] .. GENERATED FROM PYTHON SOURCE LINES 134-135 Selection get first plan .. GENERATED FROM PYTHON SOURCE LINES 135-138 .. code-block:: default print(x[0, :, :]) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none [[ 0 1 2 3] [ 4 5 6 7] [ 8 9 10 11]] .. GENERATED FROM PYTHON SOURCE LINES 139-140 Selection get first rows .. GENERATED FROM PYTHON SOURCE LINES 140-143 .. code-block:: default print(x[:, 0, :]) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none [[ 0 1 2 3] [12 13 14 15]] .. GENERATED FROM PYTHON SOURCE LINES 144-145 Selection get first columns .. GENERATED FROM PYTHON SOURCE LINES 145-148 .. code-block:: default print(x[:, :, 0]) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none [[ 0 4 8] [12 16 20]] .. GENERATED FROM PYTHON SOURCE LINES 149-155 Simple example with 2 array Exercise: - Get second line - Get third column .. GENERATED FROM PYTHON SOURCE LINES 155-162 .. code-block:: default arr = np.arange(10, dtype=float).reshape((2, 5)) print(arr) arr[1, :] arr[:, 2] .. rst-class:: sphx-glr-script-out Out: .. code-block:: none [[0. 1. 2. 3. 4.] [5. 6. 7. 8. 9.]] array([2., 7.]) .. GENERATED FROM PYTHON SOURCE LINES 163-164 Ravel .. GENERATED FROM PYTHON SOURCE LINES 164-168 .. code-block:: default print(x.ravel()) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none [ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23] .. GENERATED FROM PYTHON SOURCE LINES 169-172 Stack arrays ------------ .. GENERATED FROM PYTHON SOURCE LINES 172-176 .. code-block:: default a = np.array([0, 1]) b = np.array([2, 3]) .. GENERATED FROM PYTHON SOURCE LINES 177-178 Horizontal stacking .. GENERATED FROM PYTHON SOURCE LINES 178-181 .. code-block:: default np.hstack([a, b]) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none array([0, 1, 2, 3]) .. GENERATED FROM PYTHON SOURCE LINES 182-183 Vertical stacking .. GENERATED FROM PYTHON SOURCE LINES 183-186 .. code-block:: default np.vstack([a, b]) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none array([[0, 1], [2, 3]]) .. GENERATED FROM PYTHON SOURCE LINES 187-188 Default Vertical .. GENERATED FROM PYTHON SOURCE LINES 188-191 .. code-block:: default np.stack([a, b]) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none array([[0, 1], [2, 3]]) .. GENERATED FROM PYTHON SOURCE LINES 192-196 Selection --------- Single item .. GENERATED FROM PYTHON SOURCE LINES 196-203 .. code-block:: default arr = np.arange(10, dtype=float).reshape((2, 5)) arr[0] # 0th element (slices like a list) arr[0, 3] # row 0, column 3: returns 4 arr[0][3] # alternative syntax .. rst-class:: sphx-glr-script-out Out: .. code-block:: none 3.0 .. GENERATED FROM PYTHON SOURCE LINES 204-209 Slicing ~~~~~~~ Syntax: ``start:stop:step`` with ``start`` *(default 0)* ``stop`` *(default last)* ``step`` *(default 1)* .. GENERATED FROM PYTHON SOURCE LINES 209-218 .. code-block:: default arr[0, :] # row 0: returns 1d array ([1, 2, 3, 4]) arr[:, 0] # column 0: returns 1d array ([1, 5]) arr[:, :2] # columns strictly before index 2 (2 first columns) arr[:, 2:] # columns after index 2 included arr2 = arr[:, 1:4] # columns between index 1 (included) and 4 (excluded) print(arr2) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none [[1. 2. 3.] [6. 7. 8.]] .. GENERATED FROM PYTHON SOURCE LINES 219-221 Slicing returns a view (not a copy) Modification .. GENERATED FROM PYTHON SOURCE LINES 221-226 .. code-block:: default arr2[0, 0] = 33 print(arr2) print(arr) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none [[33. 2. 3.] [ 6. 7. 8.]] [[ 0. 33. 2. 3. 4.] [ 5. 6. 7. 8. 9.]] .. GENERATED FROM PYTHON SOURCE LINES 227-228 Row 0: reverse order .. GENERATED FROM PYTHON SOURCE LINES 228-232 .. code-block:: default print(arr[0, ::-1]) # The rule of thumb here can be: in the context of lvalue indexing (i.e. the indices are placed in the left hand side value of an assignment), no view or copy of the array is created (because there is no need to). However, with regular values, the above rules for creating views does apply. .. rst-class:: sphx-glr-script-out Out: .. code-block:: none [ 4. 3. 2. 33. 0.] .. GENERATED FROM PYTHON SOURCE LINES 233-239 Fancy indexing: Integer or boolean array indexing ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Fancy indexing returns a copy not a view. Integer array indexing .. GENERATED FROM PYTHON SOURCE LINES 239-247 .. code-block:: default arr2 = arr[:, [1, 2, 3]] # return a copy print(arr2) arr2[0, 0] = 44 print(arr2) print(arr) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none [[33. 2. 3.] [ 6. 7. 8.]] [[44. 2. 3.] [ 6. 7. 8.]] [[ 0. 33. 2. 3. 4.] [ 5. 6. 7. 8. 9.]] .. GENERATED FROM PYTHON SOURCE LINES 248-249 Boolean arrays indexing .. GENERATED FROM PYTHON SOURCE LINES 249-258 .. code-block:: default arr2 = arr[arr > 5] # return a copy print(arr2) arr2[0] = 44 print(arr2) print(arr) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none [33. 6. 7. 8. 9.] [44. 6. 7. 8. 9.] [[ 0. 33. 2. 3. 4.] [ 5. 6. 7. 8. 9.]] .. GENERATED FROM PYTHON SOURCE LINES 259-261 However, In the context of lvalue indexing (left hand side value of an assignment) Fancy authorizes the modification of the original array .. GENERATED FROM PYTHON SOURCE LINES 261-265 .. code-block:: default arr[arr > 5] = 0 print(arr) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none [[0. 0. 2. 3. 4.] [5. 0. 0. 0. 0.]] .. GENERATED FROM PYTHON SOURCE LINES 266-267 Boolean arrays indexing continues .. GENERATED FROM PYTHON SOURCE LINES 267-275 .. code-block:: default names = np.array(['Bob', 'Joe', 'Will', 'Bob']) names == 'Bob' # returns a boolean array names[names != 'Bob'] # logical selection (names == 'Bob') | (names == 'Will') # keywords "and/or" don't work with boolean arrays names[names != 'Bob'] = 'Joe' # assign based on a logical selection np.unique(names) # set function .. rst-class:: sphx-glr-script-out Out: .. code-block:: none array(['Bob', 'Joe'], dtype=' 0).sum() # counts number of positive values (rnd > 0).any() # checks if any value is True (rnd > 0).all() # checks if all values are True # random numbers np.random.seed(12234) # Set the seed np.random.rand(2, 3) # 2 x 3 matrix in [0, 1] np.random.randn(10) # random normals (mean 0, sd 1) np.random.randint(0, 2, 10) # 10 randomly picked 0 or 1 .. rst-class:: sphx-glr-script-out Out: .. code-block:: none array([0, 0, 0, 1, 1, 0, 1, 1, 1, 1]) .. GENERATED FROM PYTHON SOURCE LINES 313-337 Broadcasting ------------ Sources: https://docs.scipy.org/doc/numpy-1.13.0/user/basics.broadcasting.html Implicit conversion to allow operations on arrays of different sizes. - The smaller array is stretched or “broadcasted” across the larger array so that they have compatible shapes. - Fast vectorized operation in C instead of Python. - No needless copies. Rules ~~~~~ Starting with the trailing axis and working backward, Numpy compares arrays dimensions. - If two dimensions are equal then continues - If one of the operand has dimension 1 stretches it to match the largest one - When one of the shapes runs out of dimensions (because it has less dimensions than the other shape), Numpy will use 1 in the comparison process until the other shape's dimensions run out as well. .. figure:: ../images/numpy_broadcasting.png Source: http://www.scipy-lectures.org .. GENERATED FROM PYTHON SOURCE LINES 337-348 .. code-block:: default a = np.array([[ 0, 0, 0], [10, 10, 10], [20, 20, 20], [30, 30, 30]]) b = np.array([0, 1, 2]) print(a + b) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none [[ 0 1 2] [10 11 12] [20 21 22] [30 31 32]] .. GENERATED FROM PYTHON SOURCE LINES 349-350 Center data column-wise .. GENERATED FROM PYTHON SOURCE LINES 350-353 .. code-block:: default a - a.mean(axis=0) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none array([[-15., -15., -15.], [ -5., -5., -5.], [ 5., 5., 5.], [ 15., 15., 15.]]) .. GENERATED FROM PYTHON SOURCE LINES 354-355 Scale (center, normalise) data column-wise .. GENERATED FROM PYTHON SOURCE LINES 355-360 .. code-block:: default (a - a.mean(axis=0)) / a.std(axis=0) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none array([[-1.34164079, -1.34164079, -1.34164079], [-0.4472136 , -0.4472136 , -0.4472136 ], [ 0.4472136 , 0.4472136 , 0.4472136 ], [ 1.34164079, 1.34164079, 1.34164079]]) .. GENERATED FROM PYTHON SOURCE LINES 361-384 Examples Shapes of operands A, B and result: :: A (2d array): 5 x 4 B (1d array): 1 Result (2d array): 5 x 4 A (2d array): 5 x 4 B (1d array): 4 Result (2d array): 5 x 4 A (3d array): 15 x 3 x 5 B (3d array): 15 x 1 x 5 Result (3d array): 15 x 3 x 5 A (3d array): 15 x 3 x 5 B (2d array): 3 x 5 Result (3d array): 15 x 3 x 5 A (3d array): 15 x 3 x 5 B (2d array): 3 x 1 Result (3d array): 15 x 3 x 5 .. GENERATED FROM PYTHON SOURCE LINES 388-391 Exercises --------- Given the array: .. GENERATED FROM PYTHON SOURCE LINES 391-394 .. code-block:: default X = np.random.randn(4, 2) # random normals in 4x2 array .. GENERATED FROM PYTHON SOURCE LINES 395-398 - For each column find the row index of the minimum value. - Write a function ``standardize(X)`` that return an array whose columns are centered and scaled (by std-dev). .. rst-class:: sphx-glr-timing **Total running time of the script:** ( 0 minutes 0.012 seconds) .. _sphx_glr_download_auto_gallery_scipy_numpy.py: .. only :: html .. container:: sphx-glr-footer :class: sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: scipy_numpy.py ` .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: scipy_numpy.ipynb ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_