Neural Euler's Method

\[mean = \frac{\displaystyle\sum_{i=1}^{n} x_{i}}{n}\]

In classical data modelling problems, we have N pairs of data (x_i, y_i) that are used in training of model. To make a prediction, we feed a new data point x_star into model to find y_star. We assume that there is an inherit relationship between pairs of (x,y). In other words, there is an unknown function F that maps X to Y. The machine learning approaches can be used to approximate the the function F.

How would we like to approximate F?

We can try to approximate the function F with curve fitting or neural networks. By using neural networks, we assume that there is a continous and differentiable relationship between x and y. Therefore, our approximation of F is differentiable. What about modelling derivative of F instead of itself? In terms of parametric efficiency, modeling the derivative is practical.

\(F(x)=\int f(x) d x\) where $$ F^{\prime}(x)=f$ $

\[\frac{d y}{d x} \approx f(x, y)\]

We are planning the model the derivative of F, approximating an ODE that governs the function F. Solving the approximated ODE is equivalent of function approximation of F. We can utilize numerical methods to solve it since some ODE’s cannot be solved analytically. In this post, we will use a simple numerical method which is Euler’s Method.

What is Euler’s Method?

Euler Method is simply re-writing the basic definition of gradient at a point with a fixed step size \(\delta\).

\[\frac{d y}{d x} \approx \frac{y(x+\delta)-y(x)}{\delta}\] \[y(x+\delta)=y(x)+\delta \frac{d y}{d x}\]
Written on