Newton’s Method is the most powerful root finding algorithm.
There are several methods to find Newton’s Method, the first is to use Taylor’s Theorem, reproduced here:
To solve a root-finding problem, we are ultimately trying to solve the simple equation \(f(x)=0\). So, given that the Taylor Polynomial is equal to \(f(x)\), we are simply trying to solve:
If we assume that x and \(x_0\) are close together, then we can say that the \(x-x_0\) term in each part of the Taylor Polynomial is close to zero. And even more importantly, that the error term is very close to zero.
The above is what we are left with taking a Taylor Polynomial with n = 1, and discarding our small error term. We can algebraically manipulate the above to get:
The above is Newton’s Method. Note that an important constraints is that \(f’(x_0) \ne 0\). This doesn’t make the method weaker, because if \(f’(x_0) = 0\), then you can see \(f(x_0)\) must equal zero, and therefore \(x_0\) is a root.
Geometric Interpretation of Newton’s Method
Start with the equation for the tangent line:
If you take the tangent line at the point \(x_0,f(x_0)\) shown in the image above, you will see that it appears somewhat close to the root. The closer your \(x_0\) gets to the true root x, the closer the tangent line will also be. Much like bisection method, this is an iterative approach where we move closer and closer to the true root with each iteration.
To get our new \(x_0\) for the next iteration, we can simply look at where the tangent line of the previous \(x_0\) intersects with the x-axis.
for img src: math.lamar.edu
x0 = 1 %The initial value f = @(x) x^2 - 2 %The function whose root we are trying to find fprime = @(x) 2*x %The derivative of f(x) tolerance = 10^(-7) %7 digit accuracy is desired epsilon = 10^(-14) %Don't want to divide by a number smaller than this maxIterations = 20 %Don't allow the iterations to continue indefinitely haveWeFoundSolution = false %Have not converged to a solution yet for i = 1 : maxIterations y = f(x0) yprime = fprime(x0) if(abs(yprime) < epsilon) %Don't want to divide by too small of a number % denominator is too small break; %Leave the loop end x1 = x0 - y/yprime %Do Newton's computation if(abs(x1 - x0) <= tolerance * abs(x1)) %If the result is within the desired tolerance haveWeFoundSolution = true break; %Done, so leave the loop end x0 = x1 %Update x0 to start the process again end if (haveWeFoundSolution) ... % x1 is a solution within tolerance and maximum number of iterations else ... % did not converge end
The code given in lecture is given on the lecture slides.
Pros and Cons
You only need one guess for Newton’s Method to work, it converges very quickly, and it can find roots of functions that only “touch” the x-axis, like \(f(x)=x^2\).
On the other hand, Newton’s Method does not always converge. You also need a GOOD guess. You also need \(f’(x) \ne 0\).
What if we don’t know f prime
You can simply approximate \(f’\) by
If you use this approximation to replace \(f’\) in Newton’s Method, it is known as the SECANT Method.
This is now your iterated term for SECANT Method.
Rate of Convergence (to be completed)
If you have an initial guess that is close to the solution, it converges very quickly.