Are there any rules governing the relationship between information and the size of its description?

[Frog]

For instance if x is one bit of information, how many similar bits (y) are necessary to describe x ? *It seem obvious that y > x, I just wanted to know had any rules been established.

 

This would be done without using compression, predictions, sets or subsets, and each bit of information must be individually described.

 

*Could a clone be considered as a valid description, in the physical or theoretical sense? Would this mean the smallest possible uncompressed description of a system can be no smaller than the system itself?

 

Btw I know very little about computer science, physics or maths so a link to a '________ for dummies' site would not be considered inappropriate.

Edited April 26, 2013 by Frog

[Enthalpy]

Information may have an objective definition that allows to evaluate its amount, but only under strict assumptions, like "all combinations of bits are possible and have equal probability", which is not common life.

 

Compression is possible - and often works well - because data is not a set of random bits. But "not random bits" means that one finds a relation of some kind, and this is no more objective theory and methods, it's subjective and intelligence.

 

Imagine you have an executable programme. Zip makes general assumptions and compresses it by a factor of 3. You may see it's written for the i386 and then you can compress it easily by a factor of 10. Or you can recognize that it includes subroutines from the MFC or some other layer, remove them from the programme and replace them with calls, and gain more size. Or you can recognize that the whole programme is just msvcrt.dll, and its description takes 10 letters.

 

Or you can replace hundreds of previously elementary particles by the combinations of a handful of quarks - you may consider that as information compression. Before inventing the quarks, no method, no algorithm would have told that the set of then elementary particles contained less information than thought.

 

Forgive me that drift: I consider the size of its description as a good indicator of how complicated a technical project is, and how many chances it has to work properly.