numpy basics, creating arrays, indexing and slicing

NumPy (Numerical Python) is a popular Python library for numerical and scientific computing. It provides support for large, multi-dimensional arrays and matrices, along with a collection of mathematical functions to operate on these arrays. NumPy is a fundamental library for data manipulation and analysis in the Python ecosystem and is widely used in various scientific and engineering applications.

Here are some of the key features and capabilities of NumPy:

Multidimensional Arrays: NumPy provides the ndarray object, which is a highly efficient and flexible array data structure. These arrays can have any number of dimensions and are the building blocks for many scientific and mathematical computations.

Element-Wise Operations: NumPy allows you to perform element-wise operations on arrays, making it easy to apply mathematical operations to entire arrays without explicit loops.

Mathematical Functions: NumPy includes a wide range of mathematical functions for operations like basic arithmetic, linear algebra, statistics, and more. These functions are highly optimized for performance.

Broadcasting: NumPy has a powerful broadcasting feature that allows you to perform operations on arrays with different shapes. This can simplify many array operations and make your code more concise.

Indexing and Slicing: You can access and manipulate elements within NumPy arrays using advanced indexing and slicing techniques.

Integration with Python: NumPy seamlessly integrates with Python and can be combined with other libraries like SciPy, Matplotlib, and pandas to create comprehensive data analysis and visualization pipelines.

Efficiency: NumPy is implemented in C and can take advantage of hardware acceleration, making it much faster than using built-in Python lists for numerical operations.

Open Source: NumPy is open-source and widely adopted, making it a community-driven project with active development and a large user base.

Interoperability: NumPy can interact with data in other formats, including text files, databases, and other data storage mechanisms, making it a versatile tool for data processing.

Data Handling: NumPy arrays can store a wide range of data types, including integers, floats, and custom data types, providing flexibility for various data types and domains.

Data Filtering and Masking: NumPy provides tools for filtering and masking data based on conditions, making it easier to manipulate and analyze data based on specific criteria.

Overall, NumPy is a fundamental library for data manipulation and scientific computing in Python, and it forms the foundation for many other libraries and tools in the Python ecosystem. Its efficient handling of numerical data and mathematical operations makes it a valuable resource for researchers, data scientists, engineers, and anyone working with numerical data in Python.

The ancestor of NumPy, Numeric, was originally created by Jim Hugunin with contributions from several other developers. In 2005, Travis Oliphant created NumPy by incorporating features of the competing Numarray into Numeric, with extensive modifications. NumPy is open-source software and has many contributors.Here are some of the things it provides:

• ndarray, a fast and space-efficient multidimensional array providing vectorized arithmetic operations and sophisticated broadcasting capabilities

• Standard mathematical functions for fast operations on entire arrays of data without having to write loops

• Tools for reading / writing array data to disk and working with memory-mapped files

• Linear algebra, random number generation, and Fourier transform capabilities

• Tools for integrating code written in C, C++, and Fortran

The NumPy ndarray: A Multidimensional Array Object

One of the key features of NumPy is its N-dimensional array object, or ndarray, which is a fast, flexible container for large data sets in Python. Arrays enable you to perform mathematical operations on whole blocks of data using similar syntax to the equivalent operations between scalar elements:

A numpy array is a grid of values, all of the same type, and is indexed by a tuple of non negative integers. In NumPy dimensions are called axes.The number of dimensions is the rank of the array; the shape of an array is a tuple of integers giving the size of the array along each dimension.

We can initialize numpy arrays from nested Python lists, and access elements using square brackets.Python array indexing start from 0.Matrix operations can be done with numpy arrays.

Creating numpy arrays

#Creating a 1D array

import numpy as np

arr_1d = np.array([1, 2, 3, 4, 5])

# Creating a 2D array

arr_2d = np.array([[1, 2, 3], [4, 5, 6]])

import numpy as np

A = np.array([[1, 2, 3], [3, 4, 5]]) #Array of integers

print(A)

[[1 2 3]

[3 4 5]]

A = np.array([[1.1, 2, 3], [3, 4, 5]]) # Array of floats

print(A)

[[1.1 2. 3. ]

[3. 4. 5. ]]

A = np.array([[1, 2, 3], [3, 4, 5]], dtype = complex) # Array of complex numbers

print(A)

[[1.+0.j 2.+0.j 3.+0.j] [3.+0.j 4.+0.j 5.+0.j]]

Arrays of zeros

import numpy as np

za = np.zeros( (2, 3) )

print(za)

Output:

[[0. 0. 0.]

[0. 0. 0.]]

Arrays of ones

oa= np.ones( (2,3), dtype=np.int32 )

print(oa)

Output:

[[1. 1. 1.]
 [1. 1. 1.]]

Identity matrix

import numpy as np

print(np.eye(4,4))

[[1. 0. 0. 0.]

[0. 1. 0. 0.]

[0. 0. 1. 0.]

[0. 0. 0. 1.]]

Constant array

import numpy as np
print(np.full((4,4),5))

[[5 5 5 5]

[5 5 5 5]

[5 5 5 5]]

An uninitialized array

import numpy as np

print(np.empty((4,2))

[[2.37663529e-312, 2.14321575e-312],

[2.37663529e-312, 2.56761491e-312],

[1.18831764e-312, 1.10343781e-312],

[2.02566915e-322, 0.00000000e+000]]

Nine numbers from 0-2

import numpy as np

print(np.linspace(0,2,9))

[0. 0.25 0.5 0.75 1. 1.25 1.5 1.75 2. ]

Using arange() and reshape()

import numpy as np

A = np.arange(4)

print('A =', A)

A = [0 1 2 3]

B = np.arange(12).reshape(2, 6)

print('B =', B)

B = [[ 0 1 2 3 4 5]

[ 6 7 8 9 10 11]]

print(np.arange(2, 10, dtype=float))

[ 2., 3., 4., 5., 6., 7., 8., 9.]

print(np.arange(2, 3, 0.1) )
[ 2. , 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 2.8, 2.9]

Crearting arrays with random numbers

A=np.random.random((2,3)) #Return random floats in the half-open interval [0.0, 1.0).

print(A)

[[0.5516055 0.27255692 0.74313995]

[0.39238878 0.63832042 0.11740813]]

# Create a 2x3 array of random numbers between 0 and 1

A= np.random.rand(2, 3)

print(A)

[[0.1240735 0.62342374 0.45980557]

[0.18353189 0.36371283 0.38268776]]

# Create a 3x3 array of random integers between 0 and 9

A = np.random.randint(0, 10, (3, 3))

[[4 6 5]

[3 8 8]

[5 2 0]]

import numpy as np

# Create a 1D array with 5 random values from a standard normal distribution

A = np.random.randn(5)

print(A)

[-0.7843627 0.8237941 -0.42392379 -1.4803646 0.79355273]

# Create a 2D array with random values

A = np.random.randn(3, 4)

print(A)

[[-0.34814217 -0.70096532 0.18079399 0.9191105 ]

[-2.32021114 -1.12144335 -0.60344652 2.07601588]

[-0.51966003 0.32837054 0.54501516 -0.11058631]]

import numpy as np

# Create a 1D array with 5 random uniform values between 1 and 5

A = np.random.uniform(1, 5, 5)

print(A)

[4.43387645 4.55246073 3.96244595 4.35692682 3.26164963]

# Create a 2D array with random uniform values

A = np.random.uniform(0, 1, (3, 4))

print(A)

[[0.85469177 0.74871351 0.67757149 0.52523758]

[0.61111166 0.3522932 0.34703503 0.16185291]

[0.32944132 0.89028168 0.09788003 0.53375285]]

import numpy as np

# Create a 1D array with random samples from a given array

choices = np.array([10, 20, 30, 40, 50])

# Randomly select 3 values from 'choices'

random_samples = np.random.choice(choices, 3)

print(random_samples)

[20 40 20]

Convert a list into numpy array ( university question)

# importing library

import numpy

# initializing list

lst = [1, 7, 0, 6, 2, 5, 6]

# converting list to array

arr = numpy.array(lst)

# displaying list

print ("List: ", lst)

# displaying array

print ("Array: ", arr)

#output

List: [1, 7, 0, 6, 2, 5, 6]
Array: [1 7 0 6 2 5 6]

Attributes of the array

import numpy as np

arr = np.array([[1, 2, 3], [4, 5, 6]])

# Shape of the array

shape = arr.shape # (2, 3)

# Data type of the array

dtype = arr.dtype # int64

# Number of dimensions

ndim = arr.ndim # 2

# Size (total number of elements)

size = arr.size # 6

The data type or dtype is a special object containing the information the ndarray needs to interpret a chunk of memory as a particular type of data:

print(type(A)) # prints the class type

print(A.dtype) # print the data type

complex128

You can explicitly convert or cast an array from one dtype to another using ndarray’s astype method:

A=np.arange(10)

print(A)

[0 1 2 3 4 5 6 7 8 9]

print(A.astype(np.float))

[0. 1. 2. 3. 4. 5. 6. 7. 8. 9.]

Internal memory layout of an ndarray

An instance of class ndarray consists of a contiguous one-dimensional segment of computer memory (owned by the array, or by some other object), combined with an indexing scheme that maps N integers into the location of an item in the block. The ranges in which the indices can vary is specified by the shape of the array. How many bytes each item takes and how the bytes are interpreted is defined by the data-type object associated with the array.

Data in new ndarrays is in the row-major (C) order, unless otherwise specified, but, for example, basic array slicing often produces views in a different scheme.

Adding and removing elements

import numpy as np

A=np.array([10,20,30])

print(A)

[10 20 30]

A=np.append(A,[40,50])

print(A)

[10 20 30 40 50]

A=np.insert(A,0,100)

print(A)

[100 10 20 30 40 50]

A=np.array([[1,2],[3,4]])

print(A)

[[1 2]

[3 4]]

adding a row

A=np.append(A,[[5,6]],axis=0)

print(A)

[[1 2]

[3 4]

[5 6]]

adding a column

A=np.append(A,[[5],[6]],axis=1)

print(A)

[[1 2 5]

[3 4 6]]

deleting an element

A=np.array([10,20,30,40,50,60,70,80])

print(A)

[10 20 30 40 50 60 70 80]

A=np.delete(A,1)

print(A)

[10 30 40 50 60 70 80]

deleting a row

A=np.array([[10,20,30],[40,50,60],[70,80,90]])

print(A)

[[10 20 30]

[40 50 60]

[70 80 90]]

A=np.delete(A,1,axis=0)

print(A)

[[10 20 30]

[70 80 90]]

Indexing and slicing 1-D array

One-dimensional arrays can be indexed, sliced and iterated over, much like lists and other Python sequences.

import numpy as np

A=np.arange(10)

print(A)

[0 1 2 3 4 5 6 7 8 9]

print(A[0])

print(A[-1])

print(A[0:3])

[0 1 2]

A[0:3]=100

A[3]=200

print(A)

[100 100 100 200 4 5 6 7 8 9]

As you can see, if you assign a scalar value to a slice, as in A[0:3] = 100, the value is propagated (or broadcasted henceforth) to the entire selection. An important first distinction from lists is that array slices are views on the original array. This means that the data is not copied, and any modifications to the view will be reflected in the source array:

slice=A[5:9]

print(slice)

[5 6 7 8]

slice[:]=200

print(A)

[100 100 100 3 4 200 200 200 200 9]

B=np.arange(10)

print(B)

[0 1 2 3 4 5 6 7 8 9 ]

print(B[0:8:2])

[0 2 4 6]

print(B[8:0:-2])

[8 6 4 2]

print(B[:4])

[0 1 2 3]

print(B[5:])

[5 6 7 8 9]

print(B[::-1])

[9 8 7 6 5 4 3 2 1 0]

Indexing and slicing on 2-D array

A=np.array([[1,2,3],[4,5,6],[7,8,9]])

print(A)

[[1,2,3],

[4,5,6],

[7,8,9]]

print(A[2]) # row 2

[7 8 9]

print(A[2,2])

print(A[2][2])

print(A[1:,1:]) #row 1 and 2 column 1 and 2

[[5 6]

[8 9]]

print(A[:2,1:]) # row 0 and 1 column 1 and 2

[[2 3]

[5 6]]

print(A[:,:1])# all rows column 0

[[1]
 [4]
 [7]]

A[:,:1]=10

print(A)

[[10  2  3]
 [10  5  6]
 [10  8  9]]

Consider the following two-dimensional array named arr2d [[1, 2, 3], [4, 5, 6], [7, 8, 9]] Write the output of following Python Numpy expressions: ( University question)

1. arr2d[:2]

array([[1, 2, 3],

[4, 5, 6]])

2. arr2d[:2, 0:]

array([[1, 2, 3],

[4, 5, 6]])

3. arr2d[1, :2]

array([4, 5])

4. arr2d[:2, 1:] = 0

array([[1, 0, 0],

[4, 0, 0],

[7, 8, 9]])

Write output of the following ( University question)

import numpy as np
arr1 = np.arange(6).reshape((3, 2))
arr2 = np.arange(6).reshape((3,2))
arr3 = arr1 + arr2[0].reshape((1, 2))
print(arr3)

output:

[[0 2]

[2 4]

[4 6]]

arr1=[[0 1]
[2 3]
[4 5]]

arr2=[[0 1]
[2 3]
[4 5]]
arr2[0].reshape(1,2)=
[[0 1]]
arr1+arr2[0].reshape(1,2)
[[0 2]
[2 4]
[4 6]]

index -1 in reshape function

In NumPy, when you use -1 as a dimension in the reshape method, it is a special value that is often used to automatically infer the size of that dimension based on the size of the original array and the other specified dimensions. This can be quite handy when you want to reshape an array while keeping the total number of elements constant.

Here's how it works:

Suppose you have an array with a known shape, and you want to reshape it, but you don't want to specify the size of one dimension explicitly. You can use -1 for that dimension, and NumPy will calculate it for you based on the other specified dimensions, ensuring that the total number of elements remains the same.

For example, if you have a 2D array arr with shape (6, 4) and you want to reshape it into a 2D array with a shape of (3, -1), NumPy will automatically calculate the size of the second dimension based on the total number of elements. In this case, the second dimension will be 8 because 3 * 8 = 24, which matches the total number of elements in the original array.

Here's how you can use -1 in the reshape method:

import numpy as np

arr = np.arange(24).reshape(6, 4)

reshaped_arr = arr.reshape(3, -1) # Automatically calculates the size of the second dimension

print(arr)

print(reshaped_arr)

[[ 0 1 2 3]

[ 4 5 6 7]

[ 8 9 10 11]

[12 13 14 15]

[16 17 18 19]

[20 21 22 23]]

[[ 0 1 2 3 4 5 6 7]

[ 8 9 10 11 12 13 14 15]

[16 17 18 19 20 21 22 23]]

Boolean array Indexing

Boolean array indexing lets you pick out arbitrary elements of an array. Frequently this type of indexing is used to select the elements of an array that satisfy some condition.

import numpy as np

A=np.random.randn(3,2)

print(A)

[[-0.71292301 0.52865595]

[-0.54578822 -0.48479499]

[-0.01538739 0.00882706]]

print(A[A<0])

[-0.71292301 -0.54578822 -0.48479499 -0.01538739]

Consider student names and their marks are stored in another array marks.Students names are repeated and we can get all those rows of marks array.

import numpy as np
names=np.array(['biju','bini','binu','bini'])
marks=np.array([[30,40,40],[45,46,47],[38,40,45],[47,48,30]])
index=names=='bini'
print(index)

[False True False True]
print(marks[index])

[[45 46 47]

[47 48 30]]

Selecting data from an array by boolean indexing always creates a copy of the data,even if the returned array is unchanged.

Fancy Indexing

We can select particular row or column based on index array which stores the index

A=np.empty([4,2])

print(A)

[[2.37663529e-312 2.14321575e-312]

[2.37663529e-312 2.56761491e-312]

[1.18831764e-312 1.10343781e-312]

[2.02566915e-322 0.00000000e+000]]

index=[2,3,1]

print(A[index])

[[1.18831764e-312 1.10343781e-312]

[2.02566915e-322 0.00000000e+000]

[2.37663529e-312 2.56761491e-312]]

index=[-1,-3]

print(A[index])

[[2.02566915e-322 0.00000000e+000]

[2.37663529e-312 2.56761491e-312]]

Passing multiple index arrays does something slightly different; it selects a 1D array of elements corresponding to each tuple of indices:

print(A[[1,2],[1,1]]) # prints elements at (1,1) and (2,1)

[2.56761491e-312 1.10343781e-312]

Another way is to use the np.ix_ function, which converts two 1D integer arrays to an indexer that selects the square region: (1,0) (1,1) (2,0) (2,1).

print(A[np.ix_([1,2],[0,1])])

[[2.37663529e-312 2.56761491e-312]

[1.18831764e-312 1.10343781e-312]]

Keep in mind that fancy indexing, unlike slicing, always copies the data into a new array.

Universal Functions

A universal function, or ufunc, is a function that performs element wise operations on data in ndarrays. You can think of them as fast vectorized wrappers for simple functions that take one or more scalar values and produce one or more scalar results.

Unary function takes single array and apply the operation to all values.

A=np.arange(10)

print(np.sqrt(A))

[0. 1. 1.41421356 1.73205081 2. 2.23606798 2.44948974 2.64575131 2.82842712 3. ]

print(np.exp(A))
[1.00000000e+00 2.71828183e+00 7.38905610e+00 2.00855369e+01 5.45981500e+01 1.48413159e+02 4.03428793e+02 1.09663316e+03 2.98095799e+03 8.10308393e+03]

print(np.square(A))

[ 0 1 4 9 16 25 36 49 64 81]

The binary functions take two arrays as arguments and return a single array.

x=np.array([3,4,5,6])

y=np.array([1,4,7,2])

print(np.minimum(x,y))

[1 4 5 2]

print(np.maximum(x,y))

[3 4 7 6]

print(np.mod(x,y)) # find x mod y

[0 0 5 0]

Data Processing

A typical use of where in data analysis is to produce a new array of values based on another array.Suppose you had a matrix of randomly generated data and you wanted to replace all positive values with 2 and all negative values with -2.This is very easy to do with np.where

A=np.random.randn(4,4)

B=np.where(A>0,2,-2)

print(A)

print(B)

[[-1.01775201 -0.73483517 0.10462159 -0.23697366]

[ 0.23281261 0.13014115 1.82079278 -0.72670015]

[-2.67186248 -0.8649474 -0.25756318 0.49680316]

[-0.65459274 0.17070326 3.29936106 0.14854436]]

[[-2 -2 2 -2]

[ 2 2 2 -2]

[-2 -2 -2 2]

[-2 2 2 2]]

statistical methods

A=np.array([[1,2,3],[4,5,6]])
print(A)
[[1 2 3]

[4 5 6]]

print(np.min(A))

1
print(np.min(A,axis=0))

[1 2 3]

print(np.min(A,axis=1))

[1 4]

print(np.max(A))

6
print(np.max(A,axis=0))

[4 5 6]

print(np.max(A,axis=1))
[3 6]
print(np.sum(A))

print(np.sum(A,axis=0))

[5 7 9]

print(np.sum(A,axis=1))

[ 6 15]

print(np.mean(A))

3.5

print(np.mean(A,axis=0))

[2.5 3.5 4.5]

print(np.mean(A,axis=1))

[2. 5.]

print(np.var(A))

2.9166666666666665

print(np.var(A,axis=0))

[2.25 2.25 2.25]

print(np.var(A,axis=1))

[0.66666667 0.66666667]

print(np.std(A))

1.707825127659933

print(np.std(A,axis=0))

[1.5 1.5 1.5]

print(np.std(A,axis=1))

[0.81649658 0.81649658]

print(np.cumsum(A))

[ 1 3 6 10 15 21]

print(np.cumsum(A,axis=0))

[[1 2 3]

[5 7 9]]

print(np.cumsum(A,axis=1))

[[ 1 3 6]

[ 4 9 15]]

print(np.cumprod(A))

[ 1 2 6 24 120 720]

print(np.cumprod(A,axis=0))

[[ 1 2 3]

[ 4 10 18]]

print(np.cumprod(A,axis=1))

[[ 1 2 6]

[ 4 20 120]]

Counting number of +ve elements

A=np.array([1,2,3,-1,-4])

print((A>0).sum())

Searching and Sorting

Search

np.where: This function returns the indices of elements in an array that satisfy a specified condition. You can use it to search for elements based on a condition.

import numpy as np

arr = np.array([1, 2, 3, 4, 5])

# Find indices of elements greater than 3

indices = np.where(arr > 3)

print(indices)

# Outputs:

(array([3, 4]),)

np.argmax and np.argmin: These functions return the index of the maximum and minimum values in an array, respectively.

import numpy as np

arr = np.array([10, 20, 5, 30, 15])

# Find the index of the maximum value

max_index = np.argmax(arr) # 3 (index of 30)

print(max_index)

# Find the index of the minimum value

min_index = np.argmin(arr) # 2 (index of 5)

print(min_index)

#output

Sorting

np.sort: This function sorts the elements of an array in ascending order along the specified axis. By default, it sorts in-place.

import numpy as np

arr = np.array([5, 2, 9, 1, 5])

# Sort the array in ascending order

sorted_arr = np.sort(arr)

print(sorted_arr)

To sort the columns of the 2D array independently, you can use the np.sort function with axis=0. Here's how you can sort the columns in ascending order:

import numpy as np

arr = np.array([[5, 2], [9, 1]])
print(arr)
# Sort the array in ascending order row wise
sorted_arr = np.sort(arr,0) # column wise sort
print(sorted_arr)

[[5 1]

[9 2]]

sorted_arr = np.sort(arr,1) # row wise sort
print(sorted_arr)

[[2 5]

[1 9]]

np.argsort: This function returns the indices that would sort an array. It provides the indices that, when used to index the original array, would result in a sorted array.

import numpy as np

arr = np.array([5, 2, 9, 1, 5])

# Get the indices for sorting

indices = np.argsort(arr)

print(indices)

[3 1 0 4 2]

np.partition: This function rearranges the elements in an array such that the k-th element is in its final sorted position. Elements before the k-th position are smaller than the elements after the k-th position.

import numpy as np

arr = np.array([3, 1, 4, 1, 5, 9, 2, 6])

# Rearrange so that the 3rd smallest element (k=3) is in its final position

partitioned_arr = np.partition(arr, 3)

print(partitioned_arr)

[1 1 2 3 5 9 4 6]

Set operations

uniqe elements in sorted order