NumPy Multiply and Matrix Multiplication

---

SETUP

import numpy as np

a = np.array([1, 2, 3, 4])
b = np.array([10, 20, 30, 40])

A = np.array([[1, 2],
              [3, 4]])

B = np.array([[5, 6],
              [7, 8]])

A_rect = np.array([[1, 2, 3],
                   [4, 5, 6]])          # shape (2, 3)

B_rect = np.array([[7,  8],
                   [9,  10],
                   [11, 12]])           # shape (3, 2)

---

PART 1 — ELEMENT-WISE MULTIPLICATION

---

What it is

Element-wise: each element multiplied by the corresponding element.
Arrays must be the same shape (or broadcastable).
Output shape = input shape.

a = [1, 2, 3, 4]
b = [10,20,30,40]
    ↓  ↓  ↓  ↓
  [10, 40, 90, 160]

---

1. np.multiply() and * operator

# These are identical — * calls np.multiply under the hood
a * b                         # → [10, 40, 90, 160]
np.multiply(a, b)             # → [10, 40, 90, 160]

# 2D element-wise
A * B                         # → [[ 5, 12],
                              #    [21, 32]]
np.multiply(A, B)             # → same

---

2. Element-wise with a scalar — broadcasting

a * 3                         # → [ 3,  6,  9, 12]
np.multiply(a, 3)             # → [ 3,  6,  9, 12]

A * 10                        # → [[10, 20],
                              #    [30, 40]]

# Scale rows differently — broadcast a 1D array
A * np.array([1, 2])
# row 0: [1*1, 2*2] = [1, 4]
# row 1: [3*1, 4*2] = [3, 8]
# → [[1, 4],
#    [3, 8]]

---

3. Element-wise with broadcasting — different shapes

# (3,1) × (1,4) → (3,4)  all combinations
col = np.array([[1], [2], [3]])         # shape (3, 1)
row = np.array([[10, 20, 30, 40]])      # shape (1, 4)

col * row
# → [[10, 20, 30, 40],
#    [20, 40, 60, 80],
#    [30, 60, 90,120]]

# Scale each row of A_rect by a different factor
scales = np.array([1, 2])              # shape (2,) → broadcast over (2,3)
A_rect * scales[:, np.newaxis]
# row 0 × 1, row 1 × 2
# → [[1,  2,  3],
#    [8, 10, 12]]

---

4. In-place multiplication — *=

arr = np.array([1., 2., 3., 4.])
arr *= 5
print(arr)                    # → [ 5., 10., 15., 20.]
# modifies array directly — no new array allocated

---

5. np.multiply.reduce() — product along axis

np.multiply.reduce(a)         # → 24  (1×2×3×4)  same as np.prod(a)
np.multiply.reduce(A, axis=0) # → [3, 8]   product down each column
np.multiply.reduce(A, axis=1) # → [2, 12]  product across each row

---

PART 2 — MATRIX MULTIPLICATION

---

What it is

Matrix multiplication: rows of left × columns of right, summed.
Shape rule: (m, k) @ (k, n) → (m, n)
             ↑           ↑
         inner dims must match

A (2×2) @ B (2×2) → C (2×2)

C[0,0] = A[0,:] · B[:,0] = 1×5 + 2×7 = 19
C[0,1] = A[0,:] · B[:,1] = 1×6 + 2×8 = 22
C[1,0] = A[1,:] · B[:,0] = 3×5 + 4×7 = 43
C[1,1] = A[1,:] · B[:,1] = 3×6 + 4×8 = 50

---

6. @ operator — the standard way

A @ B
# → [[19, 22],
#    [43, 50]]

# Rectangular: (2,3) @ (3,2) → (2,2)
A_rect @ B_rect
# → [[ 58,  64],
#    [139, 154]]

# 1D dot product: (4,) @ (4,) → scalar
a @ b                         # → 1×10 + 2×20 + 3×30 + 4×40 = 300

---

7. np.matmul() — explicit matrix multiply

np.matmul(A, B)               # → [[19, 22], [43, 50]]  same as A @ B
np.matmul(A_rect, B_rect)     # → [[ 58, 64], [139, 154]]

# matmul does NOT accept scalars — use np.dot or * for that
np.matmul(A, 2)               # → ValueError
A * 2                         # → correct for scalar multiplication

---

8. np.dot() — works for 1D, 2D, and scalars

# 1D: dot product → scalar
np.dot(a, b)                  # → 300  (same as a @ b)

# 2D: matrix multiplication
np.dot(A, B)                  # → [[19, 22], [43, 50]]  same as A @ B

# Scalar: element-wise scale
np.dot(A, 3)                  # → [[ 3,  6], [ 9, 12]]

# Mixed 1D × 2D
np.dot(np.array([1, 2]), A)   # → [7, 10]   (1×2) @ (2×2) = (1×2)

---

9. Shape Rules — what works and what doesn't

# ✅ Square × square — inner dims match (2,2)@(2,2)
A @ B                         # → (2,2)

# ✅ Rectangular — inner dims match (2,3)@(3,2)
A_rect @ B_rect               # → (2,2)

# ✅ Matrix × vector — (2,2)@(2,) → (2,)
A @ np.array([1, 2])          # → [5, 11]

# ✅ Vector × matrix — (2,)@(2,2) → (2,)
np.array([1, 2]) @ A          # → [7, 10]

# ❌ Inner dims don't match — (2,3)@(2,3) → Error
A_rect @ A_rect               # → ValueError

# ❌ Swapped order changes result (matrix mult is NOT commutative)
A @ B                         # → [[19, 22], [43, 50]]
B @ A                         # → [[23, 34], [31, 46]]  ← different!

---

10. Batch Matrix Multiplication — 3D arrays

# Stack of matrices — shape (batch, m, k) @ (batch, k, n) → (batch, m, n)
batch_A = np.random.default_rng(42).random((5, 2, 3))  # 5 matrices of (2,3)
batch_B = np.random.default_rng(0).random((5, 3, 4))   # 5 matrices of (3,4)

result  = batch_A @ batch_B    # → shape (5, 2, 4)
# matmul applies independently to each of the 5 matrix pairs

---

PART 3 — RELATED OPERATIONS

---

11. np.outer() — outer product of two 1D vectors

# outer product: every element of a combined with every element of b
x = np.array([1, 2, 3])
y = np.array([4, 5, 6])

np.outer(x, y)
# → [[ 4,  5,  6],
#    [ 8, 10, 12],
#    [12, 15, 18]]
# shape: (len(x), len(y))

# Same result using broadcasting
x[:, np.newaxis] * y          # → same (3,3) array

---

12. np.inner() — inner product

# 1D: same as dot product
np.inner(a, b)                # → 300

# 2D: inner product of last axes
np.inner(A, B)
# → A[i,:] · B[j,:] for all i,j
# → [[17, 23],
#    [39, 53]]
# Note: NOT the same as A @ B

---

13. np.tensordot() — generalized dot product over specified axes

# axis=-1 default — like dot product for last axis
np.tensordot(A, B, axes=1)    # → [[19, 22], [43, 50]]  same as A @ B

# Sum over specific axes
np.tensordot(A, B, axes=([1], [0]))  # → same as A @ B

# axes=0 — outer product
np.tensordot(a[:3], a[:3], axes=0)
# → [[1, 2, 3],
#    [2, 4, 6],
#    [3, 6, 9]]

---

14. Element-wise square — np.square() vs matrix square

# Element-wise square — each element squared
np.square(A)                  # → [[ 1,  4],
                              #    [ 9, 16]]
A * A                         # → same
A ** 2                        # → same

# Matrix square — matrix multiplied by itself
A @ A                         # → [[ 7, 10],
                              #    [15, 22]]
np.linalg.matrix_power(A, 2)  # → same

---

15. Hadamard product vs matrix product — side by side

A = np.array([[1, 2],
              [3, 4]])

B = np.array([[5, 6],
              [7, 8]])

# Hadamard (element-wise)
A * B
# → [[ 5, 12],
#    [21, 32]]
# Each position multiplied independently

# Matrix multiplication
A @ B
# → [[19, 22],
#    [43, 50]]
# Rows × columns, summed

---

PART 4 — LINEAR ALGEBRA EXTRAS

---

16. Transpose — needed to align shapes

A.T                           # → [[1, 3],
                              #    [2, 4]]

# Common pattern: A @ A.T or A.T @ A
A @ A.T                       # → [[ 5, 11],
                              #    [11, 25]]   symmetric matrix

A.T @ A                       # → [[10, 14],
                              #    [14, 20]]   also symmetric

---

17. np.linalg — matrix operations

# Determinant
np.linalg.det(A)              # → -2.0  (1×4 - 2×3)

# Inverse — A @ inv(A) = Identity
np.linalg.inv(A)              # → [[-2. ,  1. ],
                              #    [ 1.5, -0.5]]

# Verify: A @ inv(A) ≈ Identity
A @ np.linalg.inv(A)          # → [[1., 0.], [0., 1.]]

# Solve linear system Ax = b
b_vec = np.array([5, 11])
x     = np.linalg.solve(A, b_vec)    # → [1., 2.]
A @ x                                # → [5., 11.]  ✓

# Eigenvalues and eigenvectors
vals, vecs = np.linalg.eig(A)
# vals → [-0.372, 5.372]
# vecs → eigenvectors as columns

# Matrix rank
np.linalg.matrix_rank(A)      # → 2

# Frobenius norm (default)
np.linalg.norm(A)             # → 5.477  sqrt(1+4+9+16)

---

18. np.linalg.matrix_power() — raise matrix to a power

np.linalg.matrix_power(A, 2)  # → A @ A
np.linalg.matrix_power(A, 3)  # → A @ A @ A
np.linalg.matrix_power(A, -1) # → inverse of A  (same as linalg.inv)
np.linalg.matrix_power(A, 0)  # → Identity matrix

---

PART 5 — COMMON MISTAKES

---

19. Mistake reference

# ❌ Using * for matrix multiplication — gives element-wise
A * B
# → [[ 5, 12], [21, 32]]   WRONG if you wanted matrix multiply

# ✅ Use @ for matrix multiplication
A @ B
# → [[19, 22], [43, 50]]   CORRECT

# ❌ Forgetting shape rules
A_rect @ A_rect               # ValueError — (2,3) inner ≠ (2,3)

# ✅ Transpose to fix shape
A_rect @ A_rect.T             # (2,3) @ (3,2) → (2,2)  ✓
A_rect.T @ A_rect             # (3,2) @ (2,3) → (3,3)  ✓

# ❌ Assuming commutativity
A @ B == B @ A                # False — matrix mult order matters

# ❌ np.matmul with scalar
np.matmul(A, 3)               # ValueError

# ✅ Use * for scalar
A * 3                         # → [[ 3,  6], [ 9, 12]]

---

🧠 Quick Reference

Operation	Syntax	Shape rule
Element-wise multiply	a * b or np.multiply(a,b)	same shape or broadcastable
Scalar multiply	a * 3	any shape
Dot product (1D)	a @ b or np.dot(a,b)	(n,) @ (n,) → scalar
Matrix multiply	A @ B or np.matmul(A,B)	(m,k) @ (k,n) → (m,n)
Matrix × vector	A @ v	(m,n) @ (n,) → (m,)
Outer product	np.outer(x, y)	(m,) × (n,) → (m,n)
Element-wise square	A ** 2 or np.square(A)	same shape
Matrix square	A @ A	(n,n) @ (n,n) → (n,n)
Transpose	A.T	(m,n) → (n,m)
Inverse	np.linalg.inv(A)	square matrix only
Solve Ax=b	np.linalg.solve(A, b)	A square, b vector
Matrix power	np.linalg.matrix_power(A,n)	square matrix only

---

🧠 Mental Model

Two completely different operations — easy to confuse:

Element-wise  *    →  same position × same position
                       shape stays the same
                       [1,2] * [3,4] = [3, 8]

Matrix mult   @    →  rows × columns, then sum
                       inner dims must match, outer dims survive
                       (2,3) @ (3,4) → (2,4)

When to use which:
  *   →  scaling, masking, component-wise operations
  @   →  linear transformations, dot products, neural networks

Shape rule for @:
  (m, k) @ (k, n) → (m, n)
        ↑↑
   these must match

Order matters for @:
  A @ B  ≠  B @ A   (unlike multiplication of scalars)

np.dot vs @ vs np.matmul:
  @          →  preferred, clean syntax, same as matmul
  np.matmul  →  same as @ but no scalar support
  np.dot     →  works for scalars too, legacy API