3.1. Basis as an Approximation of Functions

In mathematics and data science, the concept of a basis plays a fundamental role in approximating complex functions. A basis is a set of functions (or vectors) that can be combined to represent other functions within a given space. This idea allows us to break down a complicated function into simpler building blocks, making it easier to analyze and compute.  Basis: From \(n\)-Dimensional Vectors to Functions.   In \(n\)-dimensional space, a vector can be represented as a combination of basis vectors. For example, a vector \(\mathbf{v}\) in \(n\)-dimensional space can be written as: \[ \mathbf{v} = c_1 \mathbf{e}_1 + c_2 \mathbf{e}_2 + \cdots + c_n \mathbf{e}_n, \] where \(\mathbf{e}_1, \mathbf{e}_2, \ldots, \mathbf{e}_n\) are the basis vectors, and \(c_1, c_2, \ldots, c_n\) are the coordinates of the vector with respect to this basis. The basis vectors provide a "coordinate system" for the space, allowing us to describe any vector in terms of these building blocks.  For ...

2.3. Gradients and Their Role in Learning: The Gradient Descent Method

이미지
  Partial Derivatives For a scalar function \(f(x, y, z)\) of multiple variables, the derivative with respect to a single variable is called a partial derivative. It is denoted as: \[ \frac{\partial f}{\partial x}, \quad \frac{\partial f}{\partial y}, \quad \frac{\partial f}{\partial z}. \] The partial derivative with respect to \(x\) is defined as: \[ \frac{\partial f}{\partial x} = \lim_{\Delta x \to 0} \frac{f(x + \Delta x, y, z) - f(x, y, z)}{\Delta x}, \] where \(y\) and \(z\) are held constant. Partial derivatives measure the rate of change of \(f(x, y, z)\) with respect to one variable, while treating the other variables as constants. They are fundamental in multivariable calculus and have applications in fields such as physics, engineering, and medical imaging.  Example. The relationship between blood pressure, cardiac output, and vascular resistance is modeled using Ohm's Law for the circulatory system: \[ P = Q \cdot R, \] where \(P\) is the blood pressure,  \...

2.2. The Derivative: A Ratio in the Limiting Sense

이미지
The concept of a derivative is deeply connected to the idea of a ratio. Specifically, for a scalar function \(f(x)\) of a single variable \(x\), the derivative is the limit of the ratio of the change in the function's value (\(f(x)\), the dependent variable) to the change in its independent variable (\(x\), the independent variable). More precisely, the derivative is defined as: \[ f'(x) = \frac{d}{dx} f(x) = \lim_{\Delta x \to 0} \frac{f(x + \Delta x) - f(x)}{\Delta x}. \] The derivative represents the instantaneous rate of change of \(f(x)\) with respect to \(x\), making it a ratio in the limiting sense. In practical terms, it quantifies the sensitivity of \(f(x)\) to small changes in \(x\), describing how the function's value changes locally.  In many applications, ratios and derivatives are used to quantify rates of change: In physics, the ratio of distance traveled to time elapsed gives velocity. Taking the derivative of distance with respect to time yields instanta...

2.5. Vector Fields and Line integral

  Vector Fields A  vector field  assigns a vector to every point in space. Examples include hemodynamics, where vector fields are used to understand blood flow patterns and detect abnormalities such as stenosis or aneurysms, and respiratory dynamics, where vector fields model airflow in the lungs to study ventilation efficiency.  Let \(\mathbf{r} = (x, y, z)\). A vector field is defined as: $$F(\mathbf{r}) = F_x(\mathbf{r}) \hat{i} + F_y(\mathbf{r}) \hat{j} + F_z(\mathbf{r}) \hat{k},$$ where:  \( F_x(\mathbf{r}) \): The \(x\)-component of the vector field.  \( F_y(\mathbf{r}) \): The \(y\)-component of the vector field.  \( F_z(\mathbf{r}) \): The \(z\)-component of the vector field.  \( \hat{i}, \hat{j}, \hat{k} \): Unit vectors in the \(x\)-, \(y\)-, and \(z\)-directions, respectively.  Example 1: Blood Velocity Field   In a cylindrical blood vessel, the velocity of blood flow can be described as a vector field. For laminar flow, ...

2.4. Taylor Expansion and Approximation

이미지
Taylor expansion is a fundamental tool for approximating functions using their derivatives.  1D Taylor Expansion. Let \( f(x) \) be a smooth and differentiable scalar function of one variable \( x \) near a point \( x_0 \). The Taylor expansion of \(f(x)\) around \( x_0 \), up to the second order, is given by: \[ f(x) = f(x_0) + f'(x_0)(x - x_0) + \frac{1}{2}f''(x_0)(x - x_0)^2 + \frac{1}{3!}f'''(x_0)(x - x_0)^3+O(|x - x_0|^4). \] The term \( O(|x - x_0|^4) \) (pronounced "Big-O of \(|x - x_0|^4\)") describes the error in the approximation. Specifically: \[ \frac{O(|x - x_0|^4)}{|x - x_0|^4} \text{ is bounded. }  \] This notation simplifies the representation of higher-order terms, which become egligible compared to the lower-order terms when \( x \) is close to \( x_0 \).   In machine learning, Taylor expansion is used to approximate non-linear activation functions, such as the sigmoid function: \[ \sigma(x) = \frac{1}{1 + e^{-x}}. \] For small \( x ...

2.6. Divergence Theorem and Stokes' Theorem

Divergence Theorem The Divergence Theorem relates the flux of a vector field through a closed surface to the divergence of the field within the enclosed volume. Mathematically: \[\iint_{\partial V} \mathbf{F} \cdot \mathbf{n} \, dS = \iiint_V (\nabla \cdot \mathbf{F}) \, dV,\] where:  \( \mathbf{F} \): Vector field (e.g., flow velocity or electric field).  \( \partial V \): Closed surface enclosing the volume \(V\).  The divergence of a vector field \(\mathbf{F} \), denoted as \(\nabla \cdot \mathbf{F}\), is a scalar quantity that measures the rate at which the vector field spreads out or converges at a point. It is defined as: \[ \nabla \cdot \mathbf{F} = \frac{\partial F_x}{\partial x} + \frac{\partial F_y}{\partial y} + \frac{\partial F_z}{\partial z}, \] Proof of Divergence Theorem  Step 1: Verification for a Cuboid. Consider a cuboidal volume: \[ V = \{ (x, y, z) : 0 < x < a, 0 < y < b, 0 < z < c \}. \] The integral over \( V \) is: \[ \i...

2.1 Vectors

이미지
Under Construction Vectors are mathematical entities characterized by both magnitude and direction. They are essential for describing physical quantities such as blood flow velocity, forces acting on joints, and features in image analysis for deep learning. A vector in three-dimensional space is written as: \[ \mathbf{v} = v_x \hat{i} + v_y \hat{j} + v_z \hat{k}, \] where \(\hat{i} = (1,0,0)\), \(\hat{j} = (0,1,0)\), and \(\hat{k} = (0,0,1)\) are the unit vectors along the \(x\)-, \(y\)-, and \(z\)-axes, respectively. The components \(v_x\), \(v_y\), and \(v_z\) represent the magnitude of the vector in the \(x\)-, \(y\)-, and \(z\)-directions.  Vectors can describe the speed and direction of blood flow in vessels, facilitating the analysis of hemodynamics. They are also used to model forces acting on prosthetic joints or tissues, aiding in the study of biomechanics and prosthetic design. A vector in \(n\)-dimensional space is written as: \[ \mathbf{x} = (x_1, x_2, x_3, \l...