#separator:tab #html:true #tags:symbols-and-information-quiz #columns:Front Back

What is a symbol in the sense of information theory?

  1. A small picture, such as an icon on a screen.
  2. A written character such as a letter or a digit, that is, something you can write down.
  3. An agreed, distinguishable state.
  4. A single sensor reading, such as a particular brightness.

A symbol needs both: physics makes it distinguishable, and an agreement gives it a meaning.

Two teams send data with coloured light, and their LED shows four colours they have agreed on. What is the alphabet of their link?

  1. The 26 letters from A to Z that the link is meant to carry.
  2. The set of the four colours that both teams have agreed on as symbols.
  3. The order in which the colours of a message are sent.
  4. The mapping of the colours to bit sequences, such as green standing for 01.

The alphabet is the stock of agreed symbols, and a code then says what each of them stands for.

"

A link sends letters as coloured flashes of light. Who decides that a green flash means ""A""?

  1. The physics of light: each colour has a fixed wavelength, and from that the receiver recognises which character is meant.
  2. Sender and receiver together, by an agreement made before the transmission.
  3. A standard for light transmission that all devices follow so that they understand each other.
  4. The receiver alone, since it is the side that has to turn readings into characters.
"

Meaning is not in the signal but in an agreement that both sides made beforehand.

"

What is a bit?

  1. A tiny component inside a computer that either passes or blocks current.
  2. The amount of storage a computer needs for a single character such as ""A"".
  3. The amount of information in a decision between two equally likely possibilities.
  4. The smallest brightness a sensor can still measure.
"

One bit is the answer to one yes/no question whose two answers are equally likely.

An alphabet has four colours, all equally likely. How much information does a single received symbol carry, and why?

  1. 4 bits, because each of the four colours is a possibility of its own, and each possibility counts as one bit.
  2. 2 bits, because two yes/no decisions suffice to determine one of four colours.
  3. 1 bit, because the receiver makes one decision per symbol: it has arrived or it has not.
  4. It depends on the colour sent, because an easily recognised colour such as red carries more than a hard one such as yellow.

With N equally likely symbols, each carries log₂(N) bits, so four colours carry two.

A link switches from 2 to 8 colours and still sends the same number of symbols per second. How does the transfer time of the same file change?

  1. It drops to a quarter, because 8 divided by 2 is 4.
  2. It drops to a third: each symbol now carries 3 bits instead of 1.
  3. It stays the same, since the symbol rate has not changed.
  4. It drops to an eighth, one eighth per colour.

Time follows bits per symbol, log₂(2) = 1 against log₂(8) = 3, so a third.

More symbols per alphabet means more bits per symbol. Why not simply make the alphabet huge, say 256 colours?

  1. Because a lamp can mix only a limited number of distinct colours from its three channels.
  2. Because the receiver would need a lookup table for 256 colours whose search per symbol takes too long.
  3. Because the receiver must still tell the symbols apart reliably: the more there are, the closer their measurement regions crowd together.
  4. Because an alphabet needs only as many symbols as the text has distinct characters; further colours would go unused.

A larger alphabet carries more per symbol until noise makes neighbouring symbols indistinguishable.

A receiver recognises every colour without error; recognition accuracy is 100 percent. Yet the resulting text is nonsense. What is the most likely cause?

  1. The sensor delivers distorted readings, for instance because the gain is set too high.
  2. Sender and receiver use different mappings from colours to characters.
  3. Ambient light shifts the readings, so the receiver sees the colours but sorts them into the wrong regions.
  4. The alphabet has fewer colours than the text has distinct characters, so several characters end up on the same colour.

Perfect reception with different tables gives perfect nonsense.

An alphabet has 16 symbols. At most how many yes/no questions are needed to determine a received symbol with certainty?

  1. 4, because halving four times leads from 16 possibilities to one.
  2. 8, because each question halves, and half of 16 possibilities is 8.
  3. 15, because in the worst case you have to rule out 15 symbols one by one.
  4. 2, because every question has two answers, and two times two answers cover 16.

The number of halvings is log₂(16) = 4, and that is also the number of bits.

Which statements about bits are true?

  1. One bit distinguishes exactly two possibilities.
  2. A bit and a byte denote the same amount, just in different contexts.
  3. Any information can be represented as a sequence of bits.
  4. Bits exist only in computers, because only there are states stored electrically.
  5. A message contains more bits if its content is more important.

Bits count possibilities, independent of hardware and of importance.

An alphabet has four colours, so each symbol carries 2 bits. A 12-bit identifier is to be transmitted. How many symbols does that take at least?

  1. 12, one symbol per bit.
  2. 6, because 12 bits divided by 2 bits per symbol.
  3. 3, because 12 divided by 4 colours.
  4. 24, each bit twice to be safe.

Bits divided by bits per symbol gives the number of symbols.

Why is an alphabet of just two symbols in principle enough for any message, however complicated?

  1. It is not enough: text needs at least 26 symbols.
  2. Because two symbols lie furthest apart in the signal, so the receiver never confuses them.
  3. Because every symbol of a larger alphabet can be replaced by a fixed group of two-way symbols; it only costs length.
  4. Because computers work with two states internally anyway, so every message is already in that form before sending.

What a large alphabet achieves per symbol, two symbols achieve in a group.

"

In terms of symbols and information, what does this dictionary establish?

ALPHABET = {
    ""red"":    ""00"",
    ""green"":  ""01"",
    ""blue"":   ""10"",
    ""yellow"": ""11"",
}
  1. The four colours a lamp can produce, each with the code the manufacturer assigned to it.
  2. The agreement: which symbol carries which group of bits. The receiver needs exactly the same mapping, only in reverse.
  3. The order in which the colours are sent: first red, then green, then blue, then yellow.
  4. How many symbols are sent per second, because two bits per symbol determine the pace of the transmission.
"

The dictionary is the agreement in code, and it has to exist identically on both sides.

"

Sixteen faces, one of them is the secret. Question A is ""does it wear glasses?"", and eight of the sixteen do. Question B is ""is it face number 7?"". Which question is worth more on average?

  1. Question A: it yields a guaranteed 1 bit, question B only about a third of that on average.
  2. Question B: a yes solves the puzzle completely at once, which A can never do.
  3. Both the same: they are both yes/no questions, and a yes/no answer is one bit.
  4. That depends on luck and cannot be said in advance.
"

Only a question that splits the field in half is worth a whole bit; the expected value of a lopsided one is far lower.

"

In Morse code the most common letter, ""e"", is also the shortest: a single dot. Rare letters like ""q"" are long. Why does that unequal treatment pay off?

  1. Because short codes are less error-prone, so the most common letter should be the best protected.
  2. Because ""e"" is the most important letter and important letters should arrive first.
  3. Because it makes the average message shorter: what occurs often then costs little.
  4. Because an alphabet with codes of different lengths can hold more letters than one with equal lengths.
"

Frequent symbols get short codes, so the average message gets shorter.

A link can reliably manage only two symbols, bright and dark. It is still supposed to transmit an alphabet of 32 characters by grouping several flashes. How many flashes does one group need at least?

  1. 16, half the number of characters.
  2. 5, because two to the fifth is 32.
  3. 32, because every character needs one flash.
  4. 6, because one extra place is needed to separate the groups.

n flashes with two possibilities give 2ⁿ groups, and 2⁵ = 32.