Is linear algebra confined to solving systems of linear equations?

[hobz]

It seems like there is some hidden fundamentals to linear algebra that supersedes the systems of linear equations. For example, in DSP, complex exponentials are considered eigenvectors to LTI systems. This is very fascinating, and I would like to know if this is based on systems of linear equations, or there is some way of transferring stuff into linear algebra.

[the tree]

Aside from linear algebra having perfectly interesting results in of itself (apparently, although I never found them so fascinating), it has applications in a lot of other maths. Solving systems of linear differential equations is quite a useful one, but also using matrices to examine transformations in geometry and rather unsurprisingly as a useful tool in cryptography.

[Bignose]

You can always "linearize" any non-linear system of equations and then use the linear methods of solution. This is the basis on most of the computational solvers out there. Like, the system of fluid mechanics equations -- the mass, linear momentum, and energy equations are highly non-linear. Especially if you start to introduce things like turbulence and temperature dependence on the physical properties of the fluid (both of which change the effective or actual viscosity of the fluid).

 

Basically, to linearize a non-linear system, you "freeze" some set of the terms -- the non-linear parts -- and then solve the linear remaining part. Then, with the new solution, you update the non-linear parts with the new information, "freeze" them with the new information and re-solve. And repeat. Until some sort of convergence occurs.

 

The real tricks are in how big of a step you take -- i.e. if the solution from the frozen set of equations differs a lit from the previous solution, how much of that difference do you accept? If you accept a large difference, then you may miss a lot of what the non-linearity in the problem was supposed to be doing. If you take too small of the difference, you are essentially resolving the same problem as the previous one which is wasted time.

 

There are many incredibly non-linear problems that can be solved successfully this way. The entire field of computational solutions is based on it, and computational solutions is very, very rich. I am most familiar with the computational fluid dynamics side, but the solution of solid dynamics is equally rich, maybe even more so.

[Country Boy]

Your question is, in a sense, answered by the title of Halmo's classic book on linear algebra: "Finite Dimensional Vector Spaces". Linear algebra is basically the study of finite dimensional vector spaces. You can use "vectors" to represent the solutions to systemes of linear equations and so that becomes a small part of Linear Algebra. Solutions to linear differential equations can be represented as vector spaces as well. But that's only a very small part of linear algebra. You will find "eigenvalues" and "eigenvectors" (defined in linear algebra) in all parts of mathematics.

 

Essentially, "vector spaces" encapsulate our basic concept of "linear" behavior and so almost anything involving some concept of "linearity" might come under the heading of linear algebra. And, as Bignose said, you linearize, that is, approximate by a linear function, any "smooth" function locally so linear algebra techniques can be used to get approximate solutions to problems that, themselves, do not fall under "linear algebra".

 

Some linear problems, involving PARTIAL differential equations or spaces of functions, involve infinite dimensional vector spaces and so come under "Functional Analysis" rather than "Linear Algebra".

[hobz]

So basically, linear implies "well understood" and "easily solvable" (though not by my personal interpretation of the term "easy". Perhaps easier solvable is more accurate).

 

I realized that my initial question was not very accurately put forward. "Stuff" is rather closely related to "undefined".

Can I (and how can I learn how to) transform e.g. classical mechanics into LA? Say a system of springs, masses and dampers connected in parallel and series?

Another example, the movement of a pendulum that is, where the output is location at time t.

 

What is the physical significance of the eigenvectors? E.g. what do eigenvectors (and values) say about a system? My limited insight inhibits me from defining "system" very accurately, but let's say that a configuration of springs and masses can be transformed into LA, what then do eigen- tell me?

What can eigen- tell me in general (or by an example)?

[ajb]

If you are interested in "rephrasing" classical mechanics in terms of linear algebra, what you really need to to is study some differential geometry.

 

In particular, we have things called "vector bundles", which are (roughly) a collection of vector spaces parametrised by another space called a manifold. The important thing here is that at each point on the manifold we have a vector space, so at least locally we can use linear algebra.

 

Classical mechanics is described (in most cases) by a vector bundle called the cotangent bundle. I don't want to get too advanced at this point, but if you are interested you should look up symplectic geometry.

[Harish Kumar]

But with today's computing power, is it really necessary to convert non-linear equations to linear equations? May be you get results quickly, but you lose the accuracy you will get if you wait for a few hours or a few days!

[ajb]

But with today's computing power, is it really necessary to convert non-linear equations to linear equations? May be you get results quickly, but you lose the accuracy you will get if you wait for a few hours or a few days!

 

And a lot of people do solve all kinds of non-linear equations (differential and so on) using numerical methods. However, I think one looses a real understanding when doing so. Though I am not so naive to think that numerical methods have no place in mathematics. Sometimes things are too difficult (usually not even clear if a solution exists in a nice form!) or you need a quick direct answer. Numerical methods are great for exploring systems and getting a good feel for what is going on.

[Bignose]

There are plenty of non-linear solution methods proposed, but no general one. Whereas linearization and updating is general and guaranteed to be convergent when certain rules are followed (like step size and eigenvalue restrictions). Accuracy can also be easily obtained with a good analysis of the residuals and a grid independence study.

 

Actually, I think that one could argue that "with today's computing power" the drive to develop super-speedy solutions methods is lessened. If you use a slowly convergent method, an easy way to speed up solution is to get a bigger faster computer. I don't think that it is a great argument, but one that does hold at least some weight.

 

 

But with today's computing power, is it really necessary to convert non-linear equations to linear equations? May be you get results quickly, but you lose the accuracy you will get if you wait for a few hours or a few days!

[TonyMcC]

I think when the result of an equation is irrational (e.g. square root of 2) geometry is the only way to express it. The fact that you cannot accurately measure the line that you so easily have drawn is another thing!

[Dave]

And a lot of people do solve all kinds of non-linear equations (differential and so on) using numerical methods. However, I think one looses a real understanding when doing so. Though I am not so naive to think that numerical methods have no place in mathematics. Sometimes things are too difficult (usually not even clear if a solution exists in a nice form!) or you need a quick direct answer. Numerical methods are great for exploring systems and getting a good feel for what is going on.

 

I agree to a certain extent, but I'm not sure I agree with the idea that you're always losing out by using numerics. For instance, I do CFD and we're looking at pipe flow. Nobody is even close to explaining, theoretically, the kinds of localised structures that I simulate on a daily basis. Hopefully, eventually, it will come -- before then, we have to settle with numerics

 

Even in fully-understood nonlinear systems (if any exist), numerics provides a very handy way of checking your theory, and allows you to actually visualise systems.

[ajb]

I agree to a certain extent, but I'm not sure I agree with the idea that you're always losing out by using numerics. For instance, I do CFD and we're looking at pipe flow. Nobody is even close to explaining, theoretically, the kinds of localised structures that I simulate on a daily basis. Hopefully, eventually, it will come -- before then, we have to settle with numerics

 

Even in fully-understood nonlinear systems (if any exist), numerics provides a very handy way of checking your theory, and allows you to actually visualise systems.

 

I cannot disagree with this.

[khaled]

Linear Systems are interesting when it comes to its applications ...

 

however, is it true that not all formulas are transformable to linear system ..?

[the tree]

however, is it true that not all formulas are transformable to linear system ..?

mmmyeah, kind of, I guess. For pretty much any system you can linearise it at individual points and use linear techniques at those individualise points but such techniques aren't useful at all points and finding the points where they are useful isn't necessarily possible.