What is information?

[Mr Skeptic]

The question is, what is information? And how would you measure it?

 

In my opinion, information is any data at all, and measured in bits (or a similar system). However, some data can be compressed. Because the same data can be reconstituted from a potentially smaller amount of data, it makes sense to count the smallest amount possible as the amount of information, with the compressibility factor a measure of the redundancy of the information.

 

A subtype of information would be communication-information. Messages depend on the context of the situation, so what would be information to one person may be gibberish to another. Also, this gives another way to measure the information value of a message. For a message, the amount of information in the message itself is rather meaningless, and what people are interested in is what is new to them. Telling someone the sky is blue, for example, might not be considered information since they learn nothing new. Sending someone the entire contents of a website and sending them a link to it (if they have internet access) would be essentially equivalent. Politicians like to blather without saying anything definitive, so lots of words with very little information. I'm not sure how one might go about measuring this however, since to properly measure one would require the entire knowledge of the receiver as well as of the message.

 

Also a closely related question, how can information be generated?

[ajb]

There is a technical definition of information formulated by Claude E. Shannon. Heuristically, it is "the amount of surprise in a message".

 

Shannon's information theory looks very much like statistical mechanics. He uses entropy as a measure of the information.

 

Take a look at the Wikipedia article here.

 

Shannon's 1948 paper can be found here (as a pdf).

 

A quick google search will bring up lecture notes etc...

[cypress]

The question is, what is information? And how would you measure it?

 

In my opinion, information is any data at all, and measured in bits (or a similar system). However, some data can be compressed. Because the same data can be reconstituted from a potentially smaller amount of data, it makes sense to count the smallest amount possible as the amount of information, with the compressibility factor a measure of the redundancy of the information.

 

As ajb indicates, Shannon provides a good measure of information content when one is wishing to compare the quality and efficiency of systems that transcribe, translate and transmit encoded streams. In this case one has an objective standard in measuring information based on the ability of the system to reproduce the original stream.

 

I don't see how data and information can be treated as synonyms as you are suggesting though.

 

There are many kinds of information though as well. The compression algorithm is a form of information in that it is an instruction set that when executed it regenerates a stream of data. There are two objective measures in this case. One is in terms of shannon information and the ability of the instruction set to reproduce the original stream, the other is based on the degree of compressibility.

 

A subtype of information would be communication-information. Messages depend on the context of the situation, so what would be information to one person may be gibberish to another. Also, this gives another way to measure the information value of a message. For a message, the amount of information in the message itself is rather meaningless, and what people are interested in is what is new to them. Telling someone the sky is blue, for example, might not be considered information since they learn nothing new. Sending someone the entire contents of a website and sending them a link to it (if they have internet access) would be essentially equivalent. Politicians like to blather without saying anything definitive, so lots of words with very little information. I'm not sure how one might go about measuring this however, since to properly measure one would require the entire knowledge of the receiver as well as of the message.

 

This is another kind of information in that it informs. It eliminates alternatives. The challenge in measuring information content by Shannon's formulations is to know the size and probability distribution of the entire set of alternatives. In some cases this is known but in others it may not be known.

 

This thread is an outgrowth of a topic from the religion section that dealt with a particular kind of information which is digitally encoded functional information. These words for example are digitally encoded and they inform in that they eliminate alternatives but they may not always generate functional systems when processed. Computer code is functional and encoded.

 

Also a closely related question, how can information be generated?

 

How information can be generated would depend on the kind of information to be generated. If we were to accept your definition that information is data (which seems quite clearly false since data is data and requires analysis before it has any objective meaning) there are countless ways to generate data. However Information that informs by eliminating options and is coherent in that when processed generates functional systems and is encoded and is transcribed onto a neutral carrier is unique in that the only known source of this kind of information is an intelligent mind.

[pioneer]

There is useful and useless information. For example, I tell someone about my trip to the beach. They then tell another, leaving out some of the original information while adding their own flair to the story, so it sounds more interesting in conversation. They tell another person, who then begins with this data. They might recreate this new information for another, or add their own modification to the information, etc. After it is all said and done there is useful and useless information in the final stream, even if the entire process is called information.

 

There is an alternative. Someone looks at a phenomena and relays the useful information. Another gets this useful information and contrives a device that makes use of this information. The information grows into additional useful information.

 

Just adding entropy to information will still create information, but the most part it creates useless information. But since every dog has his day, it can also periodically create some additional useful information. A better filter is based on order and not entropy, where useful information is transformed in the light of some form of organization filter. The contrived device in the above example, would use principles of say engineering to filter the information, and restructure it accordingly.

 

When we get a stream of both useful and useless information, we need an structuring filter to extract one from the other similar to a chemical extraction or distillation. Another filter may then extrapolate the smaller amount of useful information to help extend the useful information; fills in the blanks formerly occupied by the useless information.

Edited August 8, 2010 by pioneer

[Mr Skeptic]

As ajb indicates, Shannon provides a good measure of information content when one is wishing to compare the quality and efficiency of systems that transcribe, translate and transmit encoded streams. In this case one has an objective standard in measuring information based on the ability of the system to reproduce the original stream.

 

No... He (and Shannon) specifically said "messages". Changing this context changes the meaning.

 

I don't see how data and information can be treated as synonyms as you are suggesting though.

 

I do. The same data might or might not be meaningful, depending on context. Since no one can consider all the context, the prudent choice is to either treat the data as information or ignore it.

 

There are many kinds of information though as well. The compression algorithm is a form of information in that it is an instruction set that when executed it regenerates a stream of data. There are two objective measures in this case. One is in terms of shannon information and the ability of the instruction set to reproduce the original stream, the other is based on the degree of compressibility.

 

Compression is the ability of the instruction set to reproduce the original stream, whereas Shannon's message information is most definitely not. Shannon can accept lossy compression that cannot reproduce the original stream, so long as the same meaning is understood.

 

This is another kind of information in that it informs. It eliminates alternatives. The challenge in measuring information content by Shannon's formulations is to know the size and probability distribution of the entire set of alternatives. In some cases this is known but in others it may not be known.

 

This thread is an outgrowth of a topic from the religion section that dealt with a particular kind of information which is digitally encoded functional information. These words for example are digitally encoded and they inform in that they eliminate alternatives but they may not always generate functional systems when processed. Computer code is functional and encoded.

 

My words are digitally encoded (when written) and also functional.

 

How information can be generated would depend on the kind of information to be generated. If we were to accept your definition that information is data (which seems quite clearly false since data is data and requires analysis before it has any objective meaning) there are countless ways to generate data. However Information that informs by eliminating options and is coherent in that when processed generates functional systems and is encoded and is transcribed onto a neutral carrier is unique in that the only known source of this kind of information is an intelligent mind.

 

OK, I set as my example lottery numbers. These are pure information (according to Shannon and my own compressibility definitions), yet they were originally randomly generated.

[cypress]

No... He (and Shannon) specifically said "messages". Changing this context changes the meaning.

 

Messages works for me.

 

 

 

I do. The same data might or might not be meaningful, depending on context. Since no one can consider all the context, the prudent choice is to either treat the data as information or ignore it.

 

Please show me how treating something as if it was information makes it information. You can call a cat a dog but even if you do, it remains a cat.

 

 

 

Compression is the ability of the instruction set to reproduce the original stream, whereas Shannon's message information is most definitely not. Shannon can accept lossy compression that cannot reproduce the original stream, so long as the same meaning is understood.

 

Relevance? I think we are in general agreement here.

 

My words are digitally encoded (when written) and also functional.

 

I fail to understand your point. Please explain. Also please offer a process that when it operates on your words, produces a functional system because I don't see how your words are functional in the sense I mean.

 

OK, I set as my example lottery numbers. These are pure information (according to Shannon and my own compressibility definitions), yet they were originally randomly generated.

 

I agree they fit the shannon and your definition of information. I am not certain they can be generated by random processes alone though. You still have a problem that the basic symbols require a mind to decipher the message generated to obtain meaning, so even in this example you have a long ways to go.

 

I find it irrelevant to the earlier question if lottery numbers can be generating by some random processes. It does not seem to address the question I asked.

 

Can you offer a mechanism to derive digitally encoded coherent functional information as I previously described?

[Mr Skeptic]

Messages works for me.

 

For me too, so long as this definition isn't used out of the context of messages.

 

Please show me how treating something as if it was information makes it information. You can call a cat a dog but even if you do, it remains a cat.

 

How can you call something a dog if you can't ever tell the difference between it and a cat? If you call such a thing a cat then it might as well be a cat, and if you call it a dog it might as well be a dog.

 

Relevance? I think we are in general agreement here.

 

Well, that you got something backwards (shannon information and compressibility).

 

I fail to understand your point. Please explain. Also please offer a process that when it operates on your words, produces a functional system because I don't see how your words are functional in the sense I mean.

 

My words function in informing humans of things. They also function as part of several compute programs. Yet another function of my words is to change the pattern of light pixels on computer screens.

 

I agree they fit the shannon and your definition of information. I am not certain they can be generated by random processes alone though. You still have a problem that the basic symbols require a mind to decipher the message generated to obtain meaning, so even in this example you have a long ways to go.

 

I find it irrelevant to the earlier question if lottery numbers can be generating by some random processes. It does not seem to address the question I asked.

 

The point is that Shannon information can be created via random processes. The event of which the message informs is a random event.

 

Can you offer a mechanism to derive digitally encoded coherent functional information as I previously described?

 

What is the relevance of this? It looks an awful lot like word soup for playing "gotcha" with.

[Guest Girish]

Information technology is what we know as IT these days.

In engineering colleges (b.tech degrees), the syllabus will be little similar to computer science but difference will be that in CS both Hardware and Software knowledge will be given while in IT they will concentrate on software mainly.

 

for job purpose, both IT and CS are good in IT and ITES sector, while Electrical , Mechanical and Civil are evergreen trades which will give you jobs any time (even when IT sector are not doing good.)

[cypress]

How can you call something a dog if you can't ever tell the difference between it and a cat? If you call such a thing a cat then it might as well be a cat, and if you call it a dog it might as well be a dog.

 

But since in many cases we can distinguish the difference between encoded information and data, when we can we should.

 

 

My words function in informing humans of things. They also function as part of several compute programs. Yet another function of my words is to change the pattern of light pixels on computer screens.

 

Yes there are different kinds of information.

 

The point is that Shannon information can be created via random processes. The event of which the message informs is a random event.

 

Sure but it is an uninteresting. It is a direct result of the definition for a particular kind of information; namely Shannon information. To say that this shows that random processes generates information is a tautology. It is circular logic.

 

However since it is a definition for just one kind of information, we should not expect it to be true of all kinds of information. I doubt random processes can generate a compression algorithm, and I doubt random processes can generate all the text in an encyclopedia, and I doubt a random process can generate digitally encoded functional information.

 

What is the relevance of this? It looks an awful lot like word soup for playing "gotcha" with.

 

This was the question that spawned this thread. At some point we should advance to this question. I am anxious to get to the point.

[Mr Skeptic]

But since in many cases we can distinguish the difference between encoded information and data, when we can we should.

 

But you almost never can. For example, if you give me a bunch of gibberish it could still be, for example, a lottery number, a telephone number, a message encrypted via a one-time-pad. It is the context and not the contents that determine whether it has meaning.

 

Sure but it is an uninteresting. It is a direct result of the definition for a particular kind of information; namely Shannon information. To say that this shows that random processes generates information is a tautology. It is circular logic.

 

In case you're wondering, tautologies are always true. Calling something circular logic doesn't mean anything when you don't know the meaning of circular logic.

 

However since it is a definition for just one kind of information, we should not expect it to be true of all kinds of information. I doubt random processes can generate a compression algorithm, and I doubt random processes can generate all the text in an encyclopedia, and I doubt a random process can generate digitally encoded functional information.

 

Well, it turns out you are wrong. A counterexample is a genetic algorithm. What a genetic algorithm does is start with random data and from simple rules weed out the least useful of that data.

 

This was the question that spawned this thread. At some point we should advance to this question. I am anxious to get to the point.

 

OK then but it would be best to be clear from the start what exactly we are talking about. I'm sure we agree on what "digitally encoded" means. What about "coherent", "functional", and "information"? And since we are no longer talking about Shannon information, please avoid using theorems related to Shannon information for this definition of yours.

 

And as to my questioning the relevance, if we are going to use this to talk about life, it would be kind of useless to define the above if it does not describe the information contents of a cell. For example, "digitally encoded" does not match.

[Saryctos]

Also a closely related question, how can information be generated?

I would say that information is generated by a change in any state of being. Even time plays a role. Given an object which stays constant, there is information in this unchanging state due to changing time.

[cypress]

In case you're wondering, tautologies are always true. Calling something circular logic doesn't mean anything when you don't know the meaning of circular logic.

 

Yes, it is always true by definition as I said. Thanks.

 

Well, it turns out you are wrong. A counterexample is a genetic algorithm. What a genetic algorithm does is start with random data and from simple rules weed out the least useful of that data.

 

Genetic algorithms are not random. They are designed and thus far they all appear to import information from the designer to allow the intended processes to succeed. But please prove me wrong. Offer up a functional genetic algorithm that does not import information into the process to guarantee success.

 

OK then but it would be best to be clear from the start what exactly we are talking about. I'm sure we agree on what "digitally encoded" means. What about "coherent", "functional", and "information"? And since we are no longer talking about Shannon information, please avoid using theorems related to Shannon information for this definition of yours.

 

Why should we avoid using entropy and information theory to measure any kind of information? That makes no sense to me. References to use of this system of measurement go way beyond communication and messaging; way beyond shannon information. What justification do you offer for such an unreasonable position that breaks with years of generally undisputed tradition?

 

Coherent (a blueprint is an example): When processed the results are marked by an orderly, logical, and aesthetically consistent relation of parts.

 

Functional (Software is an example): When processed it results in a system that have function.

 

And as to my questioning the relevance, if we are going to use this to talk about life, it would be kind of useless to define the above if it does not describe the information contents of a cell. For example, "digitally encoded" does not match.

 

Incorrect. The base pairs form a base four digital code.

[Mr Skeptic]

Genetic algorithms are not random. They are designed and thus far they all appear to import information from the designer to allow the intended processes to succeed. But please prove me wrong. Offer up a functional genetic algorithm that does not import information into the process to guarantee success.

 

Why does it matter whether the genetic algorithm contains information? You're making a really big deal about something that doesn't matter. A genetic algorithm can produce many many functional designs. Add them all up, and the genetic algorithm can produce more information than it itself contains. Therefore, it must have generated the additional information.

 

Why should we avoid using entropy and information theory to measure any kind of information? That makes no sense to me. References to use of this system of measurement go way beyond communication and messaging; way beyond shannon information. What justification do you offer for such an unreasonable position that breaks with years of generally undisputed tradition?

 

Well, I shall give you an example. Suppose Bob defines information as a red-skinned fruit with white flesh of the Rosaceae family (ie, an apple). Do you suppose Bob can use the theories you mention to talk about his idea of "information"? No, whenever someone makes a new definition they need to show that whatever theorem they wish to use apply to that definition. This can be done either by showing the definition is in fact identical, or that the theorem still holds. It is hard work and usually people simply use the theorems with the original definitions.

 

Coherent (a blueprint is an example): When processed the results are marked by an orderly, logical, and aesthetically consistent relation of parts.

 

I see. This would be unlikely to result from a genetic algorithm -- while the designs genetic algorithms produce work well, they are often baffling to a human. A similar effect can be done by the famous coding while drunk, where the resulting program usually works but the person who wrote it usually has difficulty understanding it. Likewise, two different people might disagree on this. This strikes me as a subjective requirement.

 

Also, DNA is generally not considered orderly, logical, nor aesthetically consistent, so this definition would have limited applicability when talking about life.

 

Functional (Software is an example): When processed it results in a system that have function.

 

Nice and clear.

 

Incorrect. The base pairs form a base four digital code.

 

The base pairs are irrelevant without the proteins. Proteins have a digital sequence, but their folding is analogue, and what really matters is the analogue positioning of the groups at the active site and any binding sites. The information necessary for life is not digital.

[cypress]

Why does it matter whether the genetic algorithm contains information? You're making a really big deal about something that doesn't matter. A genetic algorithm can produce many many functional designs. Add them all up, and the genetic algorithm can produce more information than it itself contains. Therefore, it must have generated the additional information.

 

It matters because I said that random processes do not generate certain kinds of information. You responded that I was wrong and cited genetic algorithms as an exception to my claim. Now in the process you introduced a fallacy by changing the claim when you said that genetic algorithms start with random data and weed out the least useful. I should instead now say that you have not yet offered an example of a random process that generates the kind of information I described.

 

Well, I shall give you an example. Suppose Bob defines information as a red-skinned fruit with white flesh of the Rosaceae family (ie, an apple). Do you suppose Bob can use the theories you mention to talk about his idea of "information"? No, whenever someone makes a new definition they need to show that whatever theorem they wish to use apply to that definition. This can be done either by showing the definition is in fact identical, or that the theorem still holds. It is hard work and usually people simply use the theorems with the original definitions.

 

This point has already been addressed by noting that a cat by any other name is still a cat. An apple is still an apple even if Bob wants to believe otherwise. Publications widely make use of the examples I offered as kinds of information so I am not making up new definitions.

 

I see. This would be unlikely to result from a genetic algorithm -- while the designs genetic algorithms produce work well, they are often baffling to a human. A similar effect can be done by the famous coding while drunk, where the resulting program usually works but the person who wrote it usually has difficulty understanding it. Likewise, two different people might disagree on this. This strikes me as a subjective requirement.

 

Also, DNA is generally not considered orderly, logical, nor aesthetically consistent, so this definition would have limited applicability when talking about life.

 

Coherent refers to the output of the plan when processed, not necessarily the plan itself. Have you any example of a random process generating a large message that when processed by a natural system outputs a coherent system?

 

The base pairs are irrelevant without the proteins. Proteins have a digital sequence, but their folding is analogue, and what really matters is the analogue positioning of the groups at the active site and any binding sites. The information necessary for life is not digital.

 

The pattern of base pairs are transcribed and translated then processed to generate proteins and other components that including control networks that perform functions. DNA is the digitally encoded information storage for biological systems It is the source code for the body plans and functional controls in life.

 

Source code is nearly irrelevant without the compiler that transcribes the code into binary code. The firmware and hardware then processes the code and function is derived. Have you any example of random processes generating functional digital code?

[Mr Skeptic]

It matters because I said that random processes do not generate certain kinds of information. You responded that I was wrong and cited genetic algorithms as an exception to my claim. Now in the process you introduced a fallacy by changing the claim when you said that genetic algorithms start with random data and weed out the least useful. I should instead now say that you have not yet offered an example of a random process that generates the kind of information I described.

 

I'm not sure what your objection is. A random process obviously can generate any form of information you like. Choose any digital information you like, and I will tell you the probability that a pure random process can generate it. Hint: it won't be zero. Therefore it is mathematically certain that a random process can generate said information. The purpose of the genetic algorithm is simply to increase the odds of generating useful information, and to separate the useless information from the useful (the genetic algorithm requires some information to play this role, but it is not identical to the information that it is to generate).

 

This point has already been addressed by noting that a cat by any other name is still a cat. An apple is still an apple even if Bob wants to believe otherwise.

 

But what if no one can tell the difference? It is also my point that data by any other name is still data. All you've done is assert some sort of difference that makes data not always information, but not give an example of such nor told how to tell the difference.

 

Publications widely make use of the examples I offered as kinds of information so I am not making up new definitions.

 

Then that makes it easy to point out a proof that the theorems you want to use are applicable to your definition of information, since you say that someone has already done it for you. If you provide the links I'll search for "digitally encoded coherent functional information" myself, or you can point out where it mentions it if it uses other words.

 

Coherent refers to the output of the plan when processed, not necessarily the plan itself. Have you any example of a random process generating a large message that when processed by a natural system outputs a coherent system?

 

That is irrelevant. Unless you can show me who the "message" in DNA is addressed to, and how surprised they are at said "message", there really is no point in talking about messages in the context of life.

 

The pattern of base pairs are transcribed and translated then processed to generate proteins and other components that including control networks that perform functions. DNA is the digitally encoded information storage for biological systems It is the source code for the body plans and functional controls in life.

 

Source code is nearly irrelevant without the compiler that transcribes the code into binary code. The firmware and hardware then processes the code and function is derived.

 

Yes, but the protein information is analogue, and without functioning proteins there is no point in having the DNA. To be useful the information must be translated from digital DNA to analogue protein shapes. Source code, the compiler, and the resulting executable are indeed all digital, but that won't make protein shapes digital.

 

Have you any example of random processes generating functional digital code?

 

Did you miss the last several times I mentioned a genetic algorithm?

[cypress]

I'm not sure what your objection is. A random process obviously can generate any form of information you like. Choose any digital information you like, and I will tell you the probability that a pure random process can generate it. Hint: it won't be zero.

 

I did not ask for the mathematical probability of generating a particular string of digital values. I asked for an actual case of a natural random process generating for example the construction plans for a functional system. Do you have a process in operation today that is known to have accomplished such a task?

 

Therefore it is mathematically certain that a random process can generate said information.

 

Is a mathematical model reality or does it sometimes model reality? Are all mathematical models realistic and accurate?

 

The purpose of the genetic algorithm is simply to increase the odds of generating useful information, and to separate the useless information from the useful (the genetic algorithm requires some information to play this role, but it is not identical to the information that it is to generate).

 

You have thus far not provided what was requested. Your point is moot.

 

But what if no one can tell the difference? It is also my point that data by any other name is still data. All you've done is assert some sort of difference that makes data not always information, but not give an example of such nor told how to tell the difference.

 

What if? Sounds like speculation. I have made a distinction between different kinds of information. The fact that one can discern a construction blueprint from a table of data and lines of computer source code indicates I am correct.

 

Then that makes it easy to point out a proof that the theorems you want to use are applicable to your definition of information, since you say that someone has already done it for you. If you provide the links I'll search for "digitally encoded coherent functional information" myself, or you can point out where it mentions it if it uses other words.

 

wiki provides a summary of the branches and different kinds of information in this article.

 

See also the link below.

 

In particular these areas: Computer Science, Source coding, Linguistics, Crypography, Informatics, Electrical Engineering, and others.

 

Unless you can show me who the "message" in DNA is addressed to, and how surprised they are at said "message", there really is no point in talking about messages in the context of life.

 

Were that true we could also say there is not point in talking about the information content in digital source code. Source code is processed by the compiler and the results are executed to derive function. You will have a hard time convincing Bill Gates for example that source code is not information since it is not addressed to anyone and no one seems particularly surprised by the messages and yet he finds the digital code contained in biological systems uncannily computer-like only far more sophisticated than anything human designers have written to date.

 

Your point is false and you have not demonstrated that natural random processes do generate functional digital code of the size observed in computer systems and biological systems.

 

Yes, but the protein information is analogue, and without functioning proteins there is no point in having the DNA. To be useful the information must be translated from digital DNA to analogue protein shapes. Source code, the compiler, and the resulting executable are indeed all digital, but that won't make protein shapes digital.

 

The functional output of 90% of the computer systems I write code for is analog also. This is particularly true for control systems.

 

Did you miss the last several times I mentioned a genetic algorithm?

 

As near as I can tell, every genetic algorithm offered thus far succeeds because the designer designed them to succeed. They are neither random nor natural. Perhaps some day someone will offer one up that that succeeds on its own accord and does not import an information source that allows the system to succeed. Robert Marks and William Dembski have published several peer-reviewed papers on this point.

 

In short, your response does not seem to uncover anything to indicate the power of random processes with respect these unique kinds of information generated by humans and found in biological systems. It would be fascinating to hear of some natural process or system other than mind accomplishing such tasks, it appears that random systems can and do shuffle the information around but thus far, only a mind has been shown to be capable of generating this kind of information.

[Cap'n Refsmmat]

Perhaps some day someone will offer one up that that succeeds on its own accord and does not import an information source that allows the system to succeed. Robert Marks and William Dembski have published several peer-reviewed papers on this point.

 

Which papers are these? There are many on that page; I'm not sure which you refer to specifically.

[cypress]

Which papers are these? There are many on that page; I'm not sure which you refer to specifically.

 

All five of the main publications have bearing on the subject.

[ydoaPs]

I did not ask for the mathematical probability of generating a particular string of digital values. I asked for an actual case of a natural random process generating for example the construction plans for a functional system. Do you have a process in operation today that is known to have accomplished such a task?

Mutation and Natural Selection. See the Lenski experiment.

[cypress]

Mutation and Natural Selection. See the Lenski experiment.

 

What specific plan was generated in this experiment? How many bits of novel information was involved? How did this compare to blind search given the resources available to cause the two alterations involved? Did this example generate more information than one would expect by a blind search or less? Do you have sources to confirm any of this?

[AzurePhoenix]

Here's a selection of novel, observed adaptations generated by the completely random insertion, deletion or replacement of nucleotides.

[cypress]

Here's a selection of novel, observed adaptations generated by the completely random insertion, deletion or replacement of nucleotides.

 

I don't understand where you are coming from. Within the context of this discussion I don't see where these examples meet the conditions described. Perhaps I am wrong but can you point to the specific request made and explain how these fit?

 

I recall indicating that random processes move information around and it is documented in information theory (see the Marks and Dembski papers) that blind search is capable of generating modest quantities of information in proportion to the resources available and probability density of functional alternatives within the sample set. Do these examples fall outside the boundaries of what is predicted for blind search by information theory? If so that would be remarkable.

[Cap'n Refsmmat]

All five of the main publications have bearing on the subject.

 

Well, I see there's one called "LIFE'S CONSERVATION LAW: Why Darwinian Evolution Cannot Create Biological Information," which would address the topic quite well, except it's published in a non-peer-reviewed book published by one of the coauthors. Three others are conference proceedings, so I'll wait to judge them until the papers are properly peer-reviewed and published in a journal. There's a paper published in IEEE Transactions on Systems, Man and Cybernetics A, Systems & Humans, called "Conservation of Information in Search: Measuring the Cost of Success", and I'll address that. It's directly related to the topic at hand, so it's useful.

 

The paper is interesting. The paper uses the premise that a genetic algorithm cannot proceed any faster than a random search without "active information," such as the fitness function.

 

In random mutation, the active information comes from the following sources.

1) Choosing the most fit among mutated possibilities. The active information comes from knowledge of the fitness.

 

Makes sense. Descent with natural selection would be totally random if it weren't for the "natural selection" portion. Dembski also gives number of offspring as a factor; more mutated offspring = more information. Sure.

 

So...

 

We now offer examples of measuring the active information for these sources of mutation-based search procedures.

1) Choosing the Fittest of a Number of Mutated Offspring:

 

In evolutionary search, a large number of offspring is often generated, and the more fit offspring are selected for the next generation. When some offspring are correctly announced as more fit than others, external knowledge is being applied to the search, giving rise to active information. As with the child's game of finding a hidden object, we are being told, with respect to the solution, whether we are getting "colder" or "warmer" to the target.

 

Consider the special case where a single parent gives rise to [imath]\Phi[/imath] children. The single child with the best fitness is chosen and used as the parent for the next generation. If there is even a small chance of improvement by mutation, the number of children can always be chosen to place the chance of improvement arbitrarily close to one.

 

Very well. With a fitness function, you can almost certainly get improvement as long as enough children are made. Now, this case only selects the fittest child, and ignores that it's possible for several generations to occur with no significant increase in fitness. But we can see that fitness improvement is possible.

 

Dembski then presents several other examples, varying the parent/child scheme to make different scenarios. Each results in improved fitness. Each, however, depends on the information inherent in the fitness function.

 

Dembski then concludes as follows:

 

Endogenous information represents the inherent difficulty of a search problem in relation to a random-search baseline. If any search algorithm is to perform better than random search, active information must be resident. If the active information is inaccurate (negative), the search can perform worse than random. Computers, despite their speed in performing queries, are thus, in the absence of active information, inadequate for resolving even moderately sized search problems. Accordingly, attempts to characterize evolutionary algorithms as creators of novel information are inappropriate. To have integrity, search algorithms, particularly computer simulations of evolutionary search, should explicitly state as follows: 1) a numerical measure of the difficulty of the problem to be solved, i.e., the endogenous information, and 2) a numerical measure of the amount of problem-specific information resident in the search algorithm, i.e., the active information.

 

Thus, Dembski concludes that evolutionary algorithms do not generate information by themselves: they are dependent upon the information given by the fitness function. So how much does the fitness function contribute?

 

Well, it depends on the case. Take the case of a random string of bits which we hope to turn into a desired sequence by mutation and selection. With a random string of starting bits, the initial sequence can be close to or very different from the desired sequence; it will average to having half the bits being correct. Now, the information contained in the fitness function is fixed: the fitness function does not change depending on the initial random bit string. In this scenario, the information generated by the random mutation varies. In some cases, little mutation is required, as the initial string is close to what is desired. In other cases, much is required, as the initial string is far from what is desired. The contribution by the fitness function is the same.

 

So the contribution by mutation and selection is not limited by the fitness function, but merely dependent on the existence of such a function. With enough generations and children, optimization is nearly guaranteed.

 

Now, the algorithms Dembski evaluates differ from true natural selection in several ways. First, there isn't always one specific fitness function: in nature, there are many different factors that contribute to fitness, and not all are important in the same places at the same time. Second, nature has numerous trials continuing in parallel: Dembski considers a single chain of parents->children, but in nature there are numerous independent parents and children. Furthermore, improvements from one parent->child chain in the wild can be transferred to another through breeding. And finally, Dembski's calculations do not include the effect of gene duplication or transposition, merely random mutations.

 

On the other hand, fitness functions (and their active information) are provided by nature, simply by natural competition and events. They are subject to the condition in the above paragraph -- they aren't as well-defined and single-minded as an artificial one -- but they exist.

 

So, moral of the story: In a naive simulation, a fitness function is required for information to be generated at a rate better than random search. With a fitness function, optimization is likely after enough generations. Fortunately, reality provides fitness functions, or else "natural selection" wouldn't be called "selection."

 

So, cypress: do you have any evidence that natural selection functions differently, and cannot generate new information where a genetic algorithm can? Or that natural selection's fitness functions are somehow invalid, or incorrect, or unusable? Or that the maximum rate of evolutionary change, as calculated using information theory, fitness functions, and accommodations for millions of simultaneous reproducing and breeding organisms, cannot match the rate expected?

[cypress]

I take it fro your summary Cap'n that you find nothing to be critical of regarding the demonstrations by Marks and Dembski that genetic algorithms succeed only because the designers import "active information" into the algorithms and that without his active information the algorithms cannot succeed. Thank-you for the summary.

[Cap'n Refsmmat]

I take it fro your summary Cap'n that you find nothing to be critical of regarding the demonstrations by Marks and Dembski that genetic algorithms succeed only because the designers import "active information" into the algorithms and that without his active information the algorithms cannot succeed. Thank-you for the summary.

 

No. (a) They do not demonstrate that "designers" are required or that natural fitness functions (as provided by natural selection) are inadequate. (b ) They do not contest that mutation and selection can produce more information than is imported; that is, a fitness function can be simple, but the generated information can be comparatively complex.

 

So no, I don't see how the paper demonstrates that designers import active information that causes success. I see that the paper demonstrates that a fitness function is required for mutation with selection to succeed. Surprise! This has no bearing on whether evolution can succeed.

 

Also, when people clearly disagree with you, the better strategy is to attempt to understand their points, rather than to arrogantly reinterpret their posts to agree with you. Pissing others off makes them less likely to concede your points.