Information theory and coding problem

[wiggles2000]

Hello,

 

I've come upon a (probably very easy) problem while writing about information theory and transmission of information, but be that as it may, I'm stuck. Since it is a rather specific area of science, I'll give a brief introduction, so maybe somebody who isn't familiar with it can help me as well.

 

 

This is an excerpt that I am having trouble understanding.

 

"For simplicity, we will assume that our source emits symbols into the channel at a rate of one per second.

The transmission rate R is defined to be the average number of bits of information transmitted across the channel per second (the observant reader will note that R is nothing other than mutual information between source and receiver).

Suppose that we transmit information for t seconds, then we transmit, on average, tR bits of information.

If we are using a binary code, then the average total number of symbols transmitted in t seconds is floor(2^tR)."

 

 

My problem is this.

Let's take the simplest case, where t=1, and we have a noisless channel so R=1 as well.

So, according to the forumula, we get that the average total number of symbols transmitted in one second is 2, but the source emits symbols into the channel at a rate of one per second. The source sends one symbol per second, but two are transmitted. How can that be?

 

Or, if t=3, and R=1. We transmit information for three seconds, and a source emits one symbol per second, but somehow average number of symbols transmitted is 8? What am I missing?

 

 

Thank you in advance!

 

[pwagen]

While I haven't read the full chapter of what you're linking, it seems he's talking about symbols being sent, not bits. They're a bit different.

 

http://en.wikipedia.org/wiki/Symbol_(data)

 

In this case, it seems to me he's talking about pulses, in which case one bit would require two symbols. Kind of like

on(1/0) -> off

and in that case, the equations add up.

 

I'm not fully sure I'm correct, even if this would make sense to you. But I hope I am. For egotistical reasons.

[wiggles2000]

I don't quite understand what you mean when you talk about pulses?.

 

 

As I understood, relation between binary symbol and bit of information is 1-1.

 

R bits of information for t seconds is tR bits of information, which is by definition tR binary symbols. But that's obviously (judging by the excerpt from the book) not true.

 

[pwagen]

Sure. Look at the quote.

 

 

Suppose that we transmit information for t seconds, then we transmit, on average, tR bits of information.

If we are using a binary code, then the average total number of symbols transmitted in t seconds is floor(2^tR).

You see that one bit is not equal to one symbol.

 

Now, have a look at this quote, from here:

 

The term baud rate has sometimes incorrectly been used to mean bit rate, since these rates are the same in old modems as well as in the simplest digital communication links using only one bit per symbol, such that binary "0" is represented by one symbol, and binary "1" by another symbol. In more advanced modems and data transmission techniques, a symbol may have more than two states, so it may represent more than one binary digit (a binary digit always represents exactly two states). For this reason, the baud rate value will often be lower than the gross bit rate.

 

I'd suggest reading up on that Wikipedia article (until some electro wiz emerges), as I'm not fully certain how it works myself, and I'd be even worse at explaining it.