Random Vectors and their inner product.

[Prometheus]

Let's say i have two random vectors, X and Y, both of length n*. The elements of either vector are not independent, and X and Y themselves are not independent. All elements are non-negative.

I want to explore the behaviour of the inner product of X and Y as .

Not sure where to begin though: does anyone know of any good references/advice to get me started?

 

I'm sure i once saw a paper that showed if the RVs are independent then the inner product converges to zero, but can't for life of me dig it up: that'd be a good start.

 

Cheers.

 

*Edit: i meant of n dimensions.

Edited April 19, 2017 by Prometheus

[Bender]

If all elements have to be positive or zero, the inner product can only be zero if at every index one of the vectors is zero, which is not very random.

[Sensei]

Let's say i have two random vectors,

In how many dimensions?

 

X and Y, both of length n.

So,

if they're 3D, Xx,Xy,Xz and Yx, Yy, Yz,

you know from the start that:

 

sqrt(Xx^2+Xy^2+Xz^2)=sqrt(Yx^2+Yy^2+Yz^2)

sqrt(Xx^2+Xy^2+Xz^2)=n

sqrt(Yx^2+Yy^2+Yz^2)=n

 

therefor

 

Xx^2+Xy^2+Xz^2=Yx^2+Yy^2+Yz^2

 

In 2D:

 

sqrt(Xx^2+Xy^2)=sqrt(Yx^2+Yy^2)

sqrt(Xx^2+Xy^2)=n

sqrt(Yx^2+Yy^2)=n

 

Xx^2+Xy^2=Yx^2+Yy^2

 

so once we randomize the first Xx:

Xx = sqrt(Yx^2+Yz^2-Xy^2) // Xx = rand();

 

then we cannot randomize Xy, because it has to have length equal n

Xx = (float) rand() / MAX_INT;

sqrt(Xx^2+Xy^2)=n

Xx^2+Xy^2=n^2

Xy^2=n^2-Xx^2

Xy=sqrt(n^2-Xx^2) // either n and Xx are known already.. then how can we randomize Xy.. ?

 

 

Normalization of vector is division each component by it's length

Xx / n = Xx'

Xy / n = Xy'

Yx / n = Yx'

Yy / n = Yy'

 

Length will be 1.

Why going n to infinity, if they will be just vector in 2D in range 0.0...1.0 per component, multiplied by n.. ?

 

Not sure where to begin though: does anyone know of any good references/advice to get me started?

I would start from writing small C/C++ program that would randomize vectors in loop,

and draw them on screen, to be able to analyze them.

Edited April 19, 2017 by Sensei

[Bender]

 

Edit: I think if the magnitude of the vectors is kept constant, the inner products would converge to zero.

 

By "length" do you mean dimension or magnitude?

[Prometheus]

Very sorry to have wasted your time but i mis-stated the problem: n is the number of dimensions, not the length of the vectors. I'll edit that.

 

I did think about running a simulation, but thought i'd check if there were any analytic work on it first. Plus i work in matlab and that's slow.....

[Bender]

What about the magnitude? Is it restricted?

[Prometheus]

The magnitude is not restricted at all.

 

Btw, the case where it converges to zero was not with vectors with non-negative elements.

[Bender]

About Matlab: do you use a for-loop? Matlabs for-loop is notoriously slow. You have to combine the vectors and use matrix calculations instead. This is much faster.

 

I don't see how it could converge to zero. I think the series would look like a 1D random walk.

[Sensei]

The magnitude is not restricted at all.

 

So each component range is 0...+infinity, right?

 

Btw, the case where it converges to zero was not with vectors with non-negative elements.

 

 

I don't see how it could converge to zero. I think the series would look like a 1D random walk.

 

If vector component, in f.e. 2D is x,y, can be in range 0...+infinity

Then dot product will be

(+infinity) *(+infinity) + (+infinity)*(+infinity) + .... (repeat as many as dimensions)

 

I made such little C++ code:

#include <stdio.h>
#include <stdlib.h>
#include <time.h>

int main( int argc, int *argv[] )
{
    srand( clock() );
    float max_value = 1000000;
    int max_i = 1000000;
    for( int i = 0; i < max_i; i++ )
    {
        float xx = (float) rand() * max_value / RAND_MAX;
        float xy = (float) rand() * max_value / RAND_MAX;
        float yx = (float) rand() * max_value / RAND_MAX;
        float yy = (float) rand() * max_value / RAND_MAX;

        float dot = xx * yx + xy * yy;
        printf( "%f\n", dot );
    }
    return( 0 );
}

After using it from command-line,

It generated series of rows (,exe >data,csv), that can be loaded from OpenOffice, and sorted, and show graph (max_value=1000):

 

 

3-dimensions version:

#include <stdio.h>
#include <stdlib.h>
#include <time.h>

int main( int argc, int *argv[] )
{
    srand( clock() );
    float max_value = 1000000;
    int max_i = 1000000;
    for( int i = 0; i < max_i; i++ )
    {
        float xx = (float) rand() * max_value / RAND_MAX;
        float xy = (float) rand() * max_value / RAND_MAX;
        float yx = (float) rand() * max_value / RAND_MAX;
        float yy = (float) rand() * max_value / RAND_MAX;
#if 1
        float zx = (float) rand() * max_value / RAND_MAX;
        float zy = (float) rand() * max_value / RAND_MAX;
#endif
        float dot = xx * yx + xy * yy;
#if 1
        dot += zx * zy;
#endif
        printf( "%f\n", dot );
    }
    return( 0 );
}

Graph from 3D version (max_value =1,000,000)

Edited April 19, 2017 by Sensei

[studiot]

Hilbert Sequence Space (symbol l²) is an infinite dimensional vector space with a defined inner product.

 

Rⁿ and Cⁿ are not infinite dimensional since n is defined as a number and therefore cannot be infinite.

 

https://en.wikipedia.org/wiki/Hilbert_space

 

Edit

note you will need to scroll down to the section about Hilbert sequence space, the first part of the Wiki article is about finite dimensional Hilbert spaces.

Edited April 19, 2017 by studiot

[Prometheus]

 

So each component range is 0...+infinity, right?

 

Correct.

 

 

I made such little C++ code:

 

Sorry, you lost me with this code: what are you up to?

[Xerxes]

Rⁿ and Cⁿ are not infinite dimensional {vector spaces} since n is defined as a number and therefore cannot be infinite.

No, I don't think this is quite correct. If [math]n \in \mathbb{N}[/math] and [math]\mathbb{N}[/math] is infinite but countable i.e has cardinality [math]\aleph_0[/math], then the vector spaces [math]\mathbb{R}^n[/math] and [math]\mathbb{C}^n[/math] are necessarily infinite dimensional vector spaces.

[wtf]

No, I don't think this is quite correct. If [math]n \in \mathbb{N}[/math] and [math]\mathbb{N}[/math] is infinite but countable i.e has cardinality [math]\aleph_0[/math], then the vector spaces [math]\mathbb{R}^n[/math] and [math]\mathbb{C}^n[/math] are necessarily infinite dimensional vector spaces.

Jeez man that's not right. What is the dimension of [math]\mathbb R^2[/math]?

 

If you're thinking of [math]\mathbb R^{\mathbb N}[/math], the direct product of countably many copies of the reals; or else perhaps [math]\oplus_{\mathbb N} \mathbb R[/math], the direct sum of countably many copies of the reals, both of those are infinite-dimensional vector spaces.

 

The direct sum is the set of functions on the naturals that are zero at all but finitely many places. The direct product doesn't have that restriction. It's like the distinction between the collection of formal polynomials versus the collection of formal power series.

 

The direct sum has countably infinite dimension. The direct product can not possibly have countably infinite dimension. Why is that? Because linear combinations are defined as being finite. So the obvious basis [math]\{e_i\}[/math] doesn't work for the direct product, while it does for the direct sum.

Edited April 25, 2017 by wtf

[studiot]

No, I don't think this is quite correct. If [math]n \in \mathbb{N}[/math] and [math]\mathbb{N}[/math] is infinite but countable i.e has cardinality [math]\aleph_0[/math], then the vector spaces [math]\mathbb{R}^n[/math] and [math]\mathbb{C}^n[/math] are necessarily infinite dimensional vector spaces.

 

Being a simple soul, I like to look at it this way.

 

 

n is the dimension of the vector space.

 

n is number; no number is infinite.

 

hence n is finite, however large.

 

This is the old chestnut confusion of the difference between the cardinality of a set and the maximum value (or otherwise) of an element.

 

Nice artwork by the way, it won't display those symbols here for me.

[wtf]

Nice artwork by the way, it won't display those symbols here for me.

Math markup works fine for me on this site.

[math]\mathbb C[/math]

renders as [math]\mathbb C[/math]. Doesn't work for you?

[studiot]

I write formulae in MathType and copy/paste.

 

So yes the symbols are available in MathType, but do not paste into this forum.

Matrices have to be adjusted as well, but a lot of other stuff works just fine.

[wtf]

Nice talkin' to y'all over the years on this site. I'm probably going to get banned by humorless scolds who can't read through a tongue-in-cheek post to the deeper more serious point, which I then explained in detail.

 

All the best.

[Xerxes]

No, I don't think this is quite correct. If [math]n \in \mathbb{N}[/math] and [math]\mathbb{N}[/math] is infinite but countable i.e has cardinality [math]\aleph_0[/math], then the vector spaces [math]\mathbb{R}^n[/math] and [math]\mathbb{C}^n[/math] are necessarily infinite dimensional vector spaces.

Good God, did I really say this?

 

It appears that I did - I cannot imagine what I was thinking, as it is quite clearly nuts.

 

Apologies

Edited April 26, 2017 by Xerxes

[studiot]

No problems we are all friends here and friends help when someone has had too much tequila.

 

[Xerxes]

!!!!!

 

+1