Curve fit / interpolate data

[AllCombinations]

Hi.

I don't know much about interpolation or curve fitting and need help, please.

Fit/approximate a function through the points (0,0), (7,11), (13,33).

 

The type of function is of the form y=tan(exp(ax^2+bx+c)).

 

Any help is appreciated. I hope I have given enough information.

 

Thanks.

 

 

[studiot]

You have three unknowns, a, b and c

 

You can obtain three simultaneous equations linking them, by substituting the values for x and y at the known points.

 

You then need to solve the three equations for the coefficients a, b and c.

 

The first one should yield c directly, but think carefully about it, and note that angles will be measured in radians.

Edited August 1, 2015 by studiot

[AllCombinations]

studiot,

 

I thought about trying it as a system of equations but the point through the origin is giving me trouble for c. I get:

 

0=tan(exp©)

 

which leads to

 

0=exp©

 

which doesn't solve for c because ln(0) is undefined. So if we ignore the zero solution we have, in trying elimination and substitution, a system of three unknowns with only two equations.

 

Thanks.

[studiot]

 

0=exp©

 

which doesn't solve for c because ln(0) is undefined. So if we ignore the zero solution we have, in trying elimination and substitution, a system of three unknowns with only two equations.

 

You are not trying hard enough.

 

[math]\tan \left\{ {{e^{a{x^2} + bx + c}}} \right\} = 0,\quad at\,x = 0[/math]

 

[math]\tan \left\{ {{e^{ + c}}} \right\} = 0[/math]

 

[math]\tan \left\{ {something} \right\} = 0,\quad something \ne 0[/math]

 

[math]{e^{ + c}} = something[/math]

 

Can you think of something, not equal to zero, that makes tan(something) zero?

 

Hint tan is a periodic function.

Edited August 1, 2015 by studiot

[AllCombinations]

Ah, quite right. Tangent is periodic. So we could use c=ln(Pi). And that allows for a and b to be solved for.

 

That was easier than I expected. I guess I was intimidated by the composition of transcendental functions.

 

That is a satisfactory method for three points. If, supposing we had ten points or a thousand, are there easier methods for this kind of interpolation (if I am using the right word) than substitution and elimination? Yet, even as I ask that, it strikes me as silly to imagine there being one method specifically for the form of the one I gave you.

 

I guess what I am trying to ask is, 1) is there a numerical method that specializes in transcendental curve-fitting, and 2) is there any such thing as a "universal" method of interpolation where one might specify the desired format and the data set and get out an appropriate function?

 

Thanks.

[studiot]

You can solve for a, b and c because you have exactly the same number of unknowns as equations, so any solution (just have the same number does not guarantee solutions) is exact.

 

If you have more equations than unknowns then, as always, you have redundant information.

 

There are various methods of dealing with this, mostly statistical, which is what I expect you are thinking of.

 

You can assume the form of the fitting equation and calculate a set of auxiliary equations minimising some functions of the deviations that yields a unique set of coefficients. the usual is to minimise the squares of the deviations producing what is known as 'least squares fitting'.

We use the squares because they are all positive and thus we can minimise their absolute value.

 

Other statistical methods are available if we want to weight the deviations to remove outliers etc.

 

 

Another approach entirely may be more appropriate if we are concerned with the slope of the fitting equation at the end points because we can gain some extra equations by differentiating it and fitting the derived curve, at least at the ends.

The most popular of this type is are called spline curves.

 

This whole subject is huge.

 

By the way, as a start do you know the difference between interpolation and extrapolation?

[John Cuthber]

I wonder where you get a function like

y=tan(exp(ax^2+bx+c)).

[studiot]

I wonder where you get a function like

y=tan(exp(ax^2+bx+c)).

 

Yes I thought it seemed a bit contrived, but I did briefly wonder if there was any relation to the Langevin function.

[AllCombinations]

I am not sure what the difference between extrapolation and interpolation is. I think interpolation is the fit of a function through a finite set of points and extrapolation might be looking at what a function does beyond the data set?

 

As for "seeming contrived," I am learning about polynomial interpolation and the instructor made a side note that there are an infinite number of functions and that "most if not all" could be interpolated, and that random function was the example of one such function. The points were my own made up numbers for the sake of asking my question. If I did not provide both data points and the structure of the function to be fit, I was sure to be accused of not providing enough information.

 

I do find the idea of interpolation or curve-fitting or whatever it is called fascinating. I like the idea of there being actual methods for finding/approximating an infinite number of points between two points that are known. It is amazing that math can be manipulated that way. I mean, I know that math is a vast subject, but this is all new to me.

 

Being pleasantly surprised by this concept, my immediate inquiry was... is there one single method for the interpolation of any number of points in any number of variables in any coordinate system and in any random format, such as the composition of polynomials, exponentials, and trigonometric functions? It's just a curiosity. Does there exist one "almighty" method, for lack of terminology?

 

You said the subject is vast. How many methods are there? There must be hundreds, I take it, considering how many kinds of functions there are.

[studiot]

Here is the STRRGI (Studio T Rapid Rough Guide to Interpolation and other itches)

 

Let us say we have a number of data points and wish model this by a curve of known algebraic form, so that we can use the curve to find data at points we do not have the true (or measured) value for.

 

With reference to the figures, I have used large crosses to denote the known data points where it is assumes the values are exact and labelled these points A, B, C, D and E.

Points where e want to estimate the value the data would take at these points are labelled F and G in all cases.

 

 

 

 

First comes interpolation.

Inter is Latin for between and interpolation is used to estimate the data value between two points.

Obviously many different curves can be employed, but we want one that is ‘well behaved’.

That is it does not go off to infinity between A and B or execute wiggles etc but in some way moves smoothly and continuously from A to B.

I have shown a straight line and a polynomial (probably a quadratic) that have this property.

Point F is located on the straight line and Point G on the quadratic, but both have the same x coordinate.

So you can see that we already have two different possible estimates for the data value at this x.

 

But we may have a set or table of data values that show a curve, not a straight line.

If we model with a polynomial, the curve that passes through (has the same values as the data at ) A, B,C, D and E is called the collocating polynomial. It is said to collocate (be the same as or fit exactly) the data at these points.

If we can generate such a polynomial we can have good confidence that points F and G on the curve will be pretty good estimates of the data at their locations.

 

Lastly we come to extrapolation.

Extra is Latin for outside and this is where we extend the supposed line beyond the interval bracketed by A and B.

In the diagram F is close to B and we can reasonably hope that the estimate that F represents is a good match to the actual data.

The further we go the more dicy this assumption becomes. It is a bit like leverage. The longer the lever greater the displacement. This is shown by Point G in the diagram.

Point G also shows something else.

Your constructed function (don’t worry about my contrived comment it only meant it was an exercise) was a tangent.

The tangent goes of to + infinity and then returns from – infinity as shown.

So if the data function really does this there is huge scope for error around this point.

 

 

 

Now to change the subject entirely.

Before the days of calculating machines, tables of mathematical functions were drawn up. Someone had to calculate all these values.

How did they do this? You may well ask.

Well not by exhaustive computations on series or whatever. That would have been horrific.

The method used was the method developed by Newton and which he was playing about with when he developed the Calculus.

 

It is called the method of finite differences, is was a way to obtain data values between entries in a table, from the existing assumed exact ones.

The method can be used to obtain any desired level of accuracy if enough exact data points are available. That is the tables can be interpolated to any number of significant figures.

Thus it is a method of interpolation and many observed empirical tables such as the international steam tables are still used in this way.

Original sine, tangent and logarithmic tables were prepared in this way.

 

As an introduction look at the final diagram which has 5 columns of figures.

 

The first column is a list of the exact data, 1, 8, 27 etc

The second column is the difference between successive real data entries so the first one is (8-1 = 7)

The third column is the difference between these ‘differences’ So 19 – 7 = 12

The fourth column is the difference between these differences, which you will note are all the same

The fifth column is again differences but they are all zero.

All subsequent columns would also be zero.

 

If you are observant you will note that the original data is cubic and the columns become constant on the fourth.

This is no accident. A quadratic becomes constant on the third etc.

 

So with enough data we can either use these values to reinstate a missing data point in the list, calculate the next one or insert a point partway between two existing ones.

Because numerical data tables were once so important a huge body of theory has been developed about the best ways to do this.

 

[overtone]

I wonder where you get a function like

y=tan(exp(ax^2+bx+c)).

It looked familiar to me from a time when I was dealing with the mathematics of ecology and epidemiology - similar stuff would turn up in the middle of models.

Edited August 3, 2015 by overtone

[AllCombinations]

studiot: That is really fascinating. Thank you for taking the time to do all of that. You explained it really well. In your opinion, what subject or subjects should I study in order to learn more about interpolation and its various associated methods? Numerical analysis? Any other information is welcome. Thanks again for giving your time to share all of that. And no worries about the "contrived" remark. I wondered but I understand now.

 

overtone: That is interesting. Interesting too that math has applications in those subjects, though I guess it makes sense when I consider topics like population, predator-prey models, and the spread of disease.

[studiot]

 

In your opinion, what subject or subjects should I study in order to learn more about interpolation and its various associated methods? Numerical analysis? Any other information is welcome.

 

and

 

Being pleasantly surprised by this concept, my immediate inquiry was... is there one single method for the interpolation of any number of points in any number of variables in any coordinate system and in any random format, such as the composition of polynomials, exponentials, and trigonometric functions? It's just a curiosity. Does there exist one "almighty" method, for lack of terminology?

 

I am very flattered to think that the whole of my rushed explanation was crystal clear since you haven't asked any technical questions, but I'm sure it could be improved.

So don't hesitate to ask about what was said.

It would be useful at this point to ask what your level of mathematical/technical background is.

The second quote sounds like you have seen the finite element grids used in modern analysis of just about everything.

Do you know in principle what a diffeential equation is? I am not asking if you know lots of solutions, just if you have the general idea.

Then I can give a better answer to the first quote above.

[AllCombinations]

I understood your explanation, though I did not understand how your triangular grid relates to interpolation.

I also understand the concept of a differential equation but I do not know what a finite element grid is, at least not by name.

I don't know if that answered your question or not. Apart from taking a few local classes I am almost entirely self-taught by buying books and reading them, and from online courseware by Khan Academy, MIT, etc. on youtube.

Teaching myself is something I can do. The hardest part is knowing what subjects to study and in what order. For instance, I just taught myself linear algebra and now I am just starting books on tensor analysis (the book by Bishop and Goldberg and another one by Grinfeld who also has a lecture series on Youtube) but I could just have easily tried tensors first only to realize I needed linear algebra as a prerequisite. I was lucky in this case.

I didn't always have a particular goal in mind because small schools are the only ones I have ever had available to me and they do not teach anything further along than multivariable calculus and first-order ordinary differential equations. Now that I have explored on my own I find the subject of interpolation to be particularlly fascinating... I just don't know what to study or what direction to even head in. I mentioned numerical analysis but I don't really know if that is the right direction?

I don't spend much time on forums but I thought it was a good time to first ask if there is one single method for fitting most functions before I further explored the appropriate subjects (whatever they turn out to be) or if the only way to learn how to fit a wide range of functions would be to study interpolation and curve-fitting in more depth.

If I strike you as someone who doesn't really seem to know what they are doing, you are correct. That is why I was seeking advice.

 

In summary, I understand calculus, (basic) differential equations, and linear algebra, I am now in the first few pages of a book on tensors, and I am interested in learning either about a general method for fitting a lot of different kinds of functions (algebraic and transcendental, anyway) or in seeking out a wide range of these methods, or both.

[studiot]

Here are a few short thoughts, as it's getting late here.

 

Taught yourself linear algebra.

That's something to be proud of and the subject is the basis for many (if not most) of useful mathematics.

 

Tensors take you back into the theoretical arena and a really a part of linear mathematics anyway.

 

Interpolation is definitely in the applied mathematics camp, sub category numerical methods.

Note I said numerical methods. There is a much more theoretical subject of numerical analysis.

 

No there is not one (or even a few) grand methods, there are many approaches.

We can continue to talk about them if you like, this is a very pleasant thread, much better than those where there is constant arguing for the sake of it.

But remember my views are skewed towards the applied side and I make no apology for this.

 

I would say that you would be better advised to look at Hamilton - Lagrange methods and what is called "The Calculus of Variations", rather than pursue tensor methods very deeply. The purpose of the calculus of variations is to determine the best curve through data points according to some pre established criterion.

 

Finite element analysis is when you draw up a (multidimensional) grid or mesh and fit it to whatever you are analysing.

You must have seen this in the movies or adverts where a computer 'scans' the hero or a car or something.

My avatar picture is a finete element mesh overlaid on a human face.

 

Again we can talk about all this and I will also explain the finite difference table further.

[AllCombinations]

Constant fighting and stupidity is why I try to avoid most forums. I agree, this is a nice one.

 

The calculus of variations? I will certainly look in to it. To "determine the best curve through data points according to some pre established criterion" sounds like exactly what I have in mind.

 

I like the beauty of pure mathematics but I also lean toward application. That the physical world speaks in mathematics is one of the things I like most about mathematics.

 

Have a good night.

[AllCombinations]

So I Googled Hamilton - Lagrange and somehow ended up reading information on the Euler-Lagrange method. I watched https://www.youtube.com/watch?v=08vJyA-XD3Q which shows that the optimal path between two points is a straight line.

 

My immediate question on this is how do we find the shortest path through, say, three points? I do not mean two linear equations that go from point a to point b and then from point b to point c, but rather how might one go about finding an optimal path? Does it become too complicated? Or is it a matter of there being many solutions?

 

Right now I am not thinking about gravitational fields or air resistance, but only in pure terms as in interpolation.

 

I welcome any input, insight, or suggeted reading.

[studiot]

Google the Brachistochrone problem.

[AllCombinations]

Very nice. So, it would seem, for any three points a circle can be found to fit through them, unless they are colinear in which case the circle would have an infinite radius.

 

So could interpolation be described as a form of optimization given a set of restraints? For example, an optimization through n points with the additional condition/restraint that the form must be a polynomial/rational/exponential/etc. Or perhaps I have it backwards and we should look at it as a form type, for instance a sine wave, that must fulfill the additional condition that it pass through a set of "special" points, special in that they are not what the sine wave would pass through on their own.

 

Is that right, or is interpolation nothing to do with optimization?

[studiot]

Don't forget you entitled this thread curve fitting and interpolation.

 

Pure mathematics only admits one situation and it is deterministic.

 

That is you know the true function and its values at certain data points and want intermediate ones between the points.

There is no choice of curve(s) to fit.

 

For example you have sine tables with the sine at every degree and want the sine of 23^o 30'.

 

Numerical mathematics adds two more scenarios.

 

Firstly you have a set of data points, but you dont know true function at all.

 

For example you have a set of tide tables with the tide height every 10 minutes and want to correct a set of soundings to mean sea level.

You have the soundings at random times in the interval covered by your tables so you need the tide heights at say 11 minutes, 14 minutes, 18 minutes etc.

 

 

Finally you may know the exact function, but it may be to difficult to handle so you choose a simpler one that is near enough.

It is this third method that is the basis of all the finite element computer programs for structural engineering, fluid mechanics, electric and magnetic field plotting and so on.

The calculus of variations and Lagrange-Hamiltonian mechanics are ways to choose such simpler functions.

[AllCombinations]

Thanks for all of your input. I appreciate the help. I think I have a clearer understanding of the difference between interpolation and optimization now.