6  Chapter 5: Numerical Methods - II

6.1 Module 5: Numerical Methods - II

6.1.1 Topic Summary

This module advances numerical techniques for equations and fitting. Gauss elimination solves linear systems by forward elimination and back-substitution, while Jacobi and Gauss-Seidel use iterative methods for convergence, with Gauss-Seidel updating values immediately for faster results. Least squares minimizes squared residuals to fit lines \(y = ax + b\), using normal equations \(\sum y = a\sum x + nb\) and \(\sum xy = a\sum x^2 + b\sum x\), applied to data points for trend estimation. Ordinary Differential Equations (ODEs) are solved numerically: Euler’s method \(y_{n+1} = y_n + h f(x_n, y_n)\) for simple steps, Runge-Kutta 4th order for higher accuracy using weighted slopes, and predictor-corrector methods like Milne’s for improved stability by predicting and correcting values. Simpson’s 3/8 rule \(\frac{3h}{8}(y_0 + 3y_1 + 3y_2 + y_3)\) approximates integrals with cubic polynomials (Burden and Faires 2015).

Tips for Beginners

  • Gauss Elimination: Eliminate variables step-by-step to upper triangular, then back-substitute.

  • Iterative Methods: Jacobi uses old values, Gauss-Seidel uses updated. Converge when diagonally dominant.

  • Least Squares: Minimize \(\sum (y_i - (ax_i + b))^2\). Solve normal equations for \(a,b\).

  • ODEs: Euler simple but inaccurate; RK4 better with 4 slopes. \(h\) small for accuracy.

  • Integration: Simpson 3/8 for 4 points, more accurate than 1/3.

  • Common Mistakes: Wrong order in elimination, not checking convergence in iterations, forgetting squares in least squares.

  • Practice: Solve 2x2 systems first, then larger. Plot ODE solutions to visualize.

6.1.2 Solved Problems

  1. Euler’s Method: Use Euler’s method to solve \(y' = y - x\) with initial condition \(y(0) = 1\) for \(x = 0.1\) using step size \(h=0.1\).

    • Solution: Euler’s formula is \(y_{n+1} = y_n + h f(x_n, y_n)\). Here, \(f(x, y) = y - x\). At \(n=0\), \(x_0=0\), \(y_0=1\), \(f(0,1)=1-0=1\). Thus, \(y_1 = 1 + 0.1 \times 1 = 1.1\).

  2. Runge-Kutta 4th Order: Solve \(y' = x + y\) with \(y(0) = 1\) for one step using Runge-Kutta 4th order method with \(h=0.2\).

    • Solution: Compute the four slopes: \(k_1 = f(0,1) = 0+1=1\), \(k_2 = f(0.1, 1.1) = 0.1+1.1=1.2\), \(k_3 = f(0.1, 1.2) = 0.1+1.2=1.3\), \(k_4 = f(0.2, 1.3) = 0.2+1.3=1.5\). Then, \(y_1 = 1 + \frac{0.2}{6} (1 + 2\times1.2 + 2\times1.3 + 1.5) = 1 + \frac{0.2}{6} (1 + 2.4 + 2.6 + 1.5) = 1 + \frac{0.2}{6} \times 7.5 = 1 + 0.25 = 1.25\).

  3. Gauss Elimination: Solve the system of equations: \(2x + y = 3\), \(x + 2y = 3\).

    • Solution: Write in matrix form and perform elimination. Subtract half of first equation from second: \(2x + y = 3\), \(0x + 1.5y = 1.5\). Thus, \(y=1\), then \(x=1\).

  4. Jacobi Method: Solve \(10x + y = 11\), \(x + 10y = 11\) using Jacobi method with initial guess \((1,1)\), perform one iteration.

    • Solution: Jacobi formulas: \(x = \frac{11 - y}{10}\), \(y = \frac{11 - x}{10}\). Starting with \(x=1, y=1\), new \(x = \frac{11-1}{10}=1\), \(y=\frac{11-1}{10}=1\). Converged.

  5. Simpson’s 3/8 Rule: Approximate \(\int_0^1 x^3 \, dx\) using Simpson’s 3/8 rule with \(h=0.5\).

    • Solution: Points: \(x=0,0.5,1.0\), \(y=0, 0.125, 1.0\). Simpson’s 3/8 rule: \(\frac{3h}{8} [y_0 + 3y_1 + 3y_2 + y_3]\) but for 3 points, it’s \(\frac{h}{3} [y_0 + 4y_1 + y_2]\) wait, actually for Simpson 1/3 it’s that, but 3/8 is for 4 points. Wait, for 3 points, use 1/3 rule: \(\frac{h}{3} [y_0 + 4y_1 + y_2] = \frac{0.5}{3} [0 + 4*0.125 + 1] = \frac{0.5}{3} * 1.5 = 0.25\).

  6. Normal Equations: State the normal equations for fitting a straight line \(y = ax + b\) using the method of least squares.

    • Solution: Minimizing the sum of squares of the vertical deviations gives the two normal equations: \(\sum y = a\sum x^2 + b\sum x\) and \(\sum y = a\sum x + nb\).

  7. Curve Fitting Calculation: Fit a straight line to the data (1, 2), (2, 5), (3, 7), (4, 10) using least squares.

    • Solution: Set up a summation table to get: \(\sum x=10\), \(\sum y=24\), \(\sum x^2=30\), \(\sum xy=73\). Plugging into the normal equations gives \(30a + 10b = 73\) and \(10a + 4b = 24\). Solving these yields \(a=2.6\) and \(b=-0.5\), producing the line \(y = 2.6x - 0.5\).

  8. Euler’s Method: Apply Euler’s method to find \(y(0.1)\) given \(\frac{dy}{dx} = x - y\), initial condition \(y(0) = 1\) with step size \(h=0.1\).

    • Solution: Euler’s iterative formula is \(y_{n+1} = y_n + h f(x_n, y_n)\). Substituting initial variables: \(y_1 = 1 + 0.1(0 - 1) = 0.9\).

  9. Gauss-Seidel Setup: Solve the system \(10x + y + z = 12\), \(2x + 10y + z = 13\), \(x + y + 5z = 7\) using Gauss-Seidel method.

    • Solution: The equations are diagonally dominant. Rearrange to solve for the variables iteratively: \(x = (12 - y - z)/10\), \(y = (13 - 2x - z)/10\), and \(z = (7 - x - y)/5\). You iterate through these equations substituting the most recently updated values until they converge.

  10. Principle of Least Squares: What is the principle of least squares used for curve fitting?

  • Solution: It minimizes the sum of the squares of the residuals (vertical deviations) between the observed data points and the fitted line.
  1. Curve Fitting Example: Fit a straight line \(y = ax + b\) to the data: \((1, 2), (2, 4), (3, 5), (4, 7)\) using the method of least squares.
  • Solution: Normal equations resolve to \(30a + 10b = 53\) and \(10a + 4b = 18\). Yielding \(y = 1.6x + 0.5\).
  1. Euler’s Method Example: Apply Euler’s method to find \(y(0.1)\) for \(dy/dx = y - x\) with \(y(0)=2\) and \(h=0.1\).
  • Solution: \(y(0.1) = 2 + 0.1(2-0) = 2.2\).

6.1.3 Practice Problems

  • Use Euler’s method to solve \(y' = -2xy\) with \(y(0)=1\) for \(x=0.2\), \(h=0.1\).

  • Apply Runge-Kutta 4th order to \(y' = y - x^2\) with \(y(0)=0\), \(h=0.5\).

  • Solve the system: \(x + 2y - z = 4\), \(2x - y + 3z = 9\), \(-x + y + 2z = 5\) using Gauss elimination.

  • Solve the system: \(3x + 2y - z = 1\), \(2x - 2y + 4z = -2\), \(-x + 0.5y - z = 0\) using Gauss elimination.

  • Use Jacobi method for \(4x - y = 1\), \(-x + 4y = 1\), initial guess \((0,0)\), perform two iterations.

  • Evaluate \(\int_0^2 e^x \, dx\) using Simpson’s 3/8 rule with \(n=3\) (adjust for 4 points).

  • Use the Trapezoidal rule to evaluate \(\int_1^2 \frac{1}{x} dx\) taking h = 0.5.

  • Evaluate \(\int_0^1 \frac{1}{1+x} dx\) using Simpson’s 1/3rd rule with h=0.25.

  • Solve the following system using the Jacobi iteration method (4 iterations): 5x - y + z = 10; 2x + 4y = 12; x + y + 5z = -1.

  • Use Runge-Kutta method of fourth order to find \(y(0.2)\) given \(dy/dx = x^2 + y\), \(y(0)=1\). Take step-size, h = 0.1.

  • Solve \(dy/dx = 2x + y, y(0) = 1\) to find \(y(0.4)\) using Runge-Kutta 4th order for early steps and Milnes predictor-corrector for the final step.

  • Solve \(dy/dx = x + y, y(0)=1\) for \(x \in [0, 0.4]\) with \(h=0.2\). Use Runge-Kutta 4th order for \(y(0.2)\) and predictor-corrector method for \(y(0.4)\).

Exam Preparation Tips

  • NR: Choose x0 close to root, iterate until convergence.

  • Systems: Use matrices for Gauss, check diagonal dominance for iterations.

  • Least Squares (straight line): Set up tables for sum \(x, y, xy, x^2\).

  • ODEs: Euler for quick, RK4 for exams. Predictor-corrector combines methods.

  • Integration: Choose rule based on points: Trap (2), Simpson 1/3 (3), 3/8 (4).

  • Common pitfalls: Not updating values in Gauss-Seidel, wrong slopes in RK4, forgetting to square residuals in least squares.

  • Practice: Implement steps manually, use calculators for computations.