Are there any rules governing the relationship between information and the size of its description?

[Frog]

For instance if x is one bit of information, how many similar bits (y) are necessary to describe x ? *It seem obvious that y > x, I just wanted to know had any rules been established.

 

This would be done without using compression, predictions, sets or subsets, and each bit of information must be individually described.

 

*Could a clone be considered as a valid description, in the physical or theoretical sense? Would this mean the smallest possible uncompressed description of a system can be no smaller than the system itself.

 

Btw I know very little about computer science, physics or maths so a link to a '________ for dummies' site would not be considered inappropriate.

Edited April 25, 2013 by Frog

[Frog]

Possibly a better way to ask the question would be, can a system ever describe itself?

 

Edit

No that's silly, just ignore it until I find out how to delete it.

Edited April 26, 2013 by Frog

[pwagen]

I can't seem to find what I was looking for in regards to this, but you might want to look up something like Huffman coding.

 

http://en.wikipedia.org/wiki/Huffman_coding

 

While the numbers I was looking for escape me, if my memory serves me, it actually has the potential to encode information to take up surprisingly few bits.

 

I'm not exactly sure what you mean by individually describing each bit. If so, since a bit is the smallest amount of information, that would simply mean x = y?

[Frog]

Thanks, your link helped a lot. I didn't really understand it but now I know the relationship I was looking for was 'redundancy' to message, and this will vary depending on the code used.

[ewmon]

I've noticed an inverse relationship between the relative size of an object and the length of its name (if not by outright letters, then by syllables or words). The reason for this being, at least, diminutive endings (see "spago" below) and designations needed to distinguish one of many small things (see leg bones below). Exceptions certainly exist.

 

Examples —

• The "world" versus "Proxima Centauri" aka "V645 Centauri" (the nearest star to our solar system)
• "spago" (Italian for small cord) versus "spaghetti" (diminutive of "spago") versus "spaghettini" diminutive of "spaghetti"
• Some leg bones – "femur" versus "tibia", "patella" and "fibula" versus the "intermediate cuneiform bone" (one of those little bones in the ankle)
• "Person" versus "rocket scientist" versus "Robert Hutchings Goddard" (a particular rocket scientist)

[Bignose]

For instance if x is one bit of information, how many similar bits (y) are necessary to describe x ? *It seem obvious that y > x, I just wanted to know had any rules been established.

I don't think that the above is even necessarily true.

 

My 'bit' of information (and I assume you don't mean bit as 0 or 1), may be sets of input & output.

 

e.g. inputs are {1, 2, 3, 4, 5, ....} and outputs are {1, 4, 9, 16, 25, ...}

 

The information here is possible infinite.

 

But I think it is valid to describe that information by output = input^2. Certainly smaller than all the information.

Edited April 27, 2013 by Bignose

[DevilSolution]

In communication terms its relative to the architecture or protocol being used, in codec terms you could do it with 1 bit pwm probably but again its relative to the specific file type and stream.

 

Its not necessarily the amount of information, more a case of the layers of abstraction involved or if single layer then how many commands are required

 

if you simply have 2 functions, play and stop, then only 1 bit is required .... etc etc, pwm allows for 1 bit to read in analogue values and convert to digital meaning you can get alot of description from 1 track in serial.

Edited May 10, 2013 by DevilSolution

[EdEarl]

Possibly a better way to ask the question would be, can a system ever describe itself?

 

Edit

No that's silly, just ignore it until I find out how to delete it.

 

Not silly, see: http://www.nyx.net/~gthompso/quine.htm for a few programs that can write or print themselves.