Shannon 100

Collection Shannon 100

Organizer(s)
Date(s) 04/05/2024
00:00:00 / 00:00:00
5 15

Shannon’s Formula Wlog(1+SNR): A Historical Perspective

By Olivier Rioul

As is well known, the milestone event that founded the field of information theory is the publication of Shannon’s 1948 paper entitled "A Mathematical Theory of Communication". This article brings together so many fundamental advances and strokes of genius that Shannon has become the hero of thousands of researchers, praised almost as a deity. One can say without exaggeration that Shannon's theorems are the mathematical theorems which have made possible the digital world as we know it today. We first describe some of his most outstanding contributions, culminating with Shannon's emblematic capacity formula C = W. log(1+P/N) where W is the channel bandwidth and P/N is the channel signal-to-noise ratio (SNR). Incidentally, Hartley’s name is often associated with the same formula, owing to "Hartley’s rule": Counting the highest possible number of distinguishable values for a given amplitude A and precision D yields a similar expression log(1 + A/D). In the information theory community, the following "historical" statements are generally well accepted:(1) Hartley put forth his rule in 1928, twenty years before Shannon;(2) Shannon’s formula as a fundamental trade-off between transmission rate, bandwidth, and signal-to-noise ratio came unexpected in 1948;(3) Shannon’s formula is exact while Hartley’s rule is imprecise;(4) Hartley’s expression is not an appropriate formula for the capacity of a communication channel. We show that all these four statements are somewhat wrong:(1) Hartley’s rule does not seem to be Hartley’s. (2) At least seven other authors have independently derived formulas very similar to Shannon’s in the same year 1948 — the earliest published original contribution being a Note at the Académie des Sciences by a French engineer Jacques Laplume. (3) A careful calculation shows that Hartley’s rule does coincide with Shannon’s formula. (4) Hartley’s rule is in fact mathematically correct as the capacity of a communication channel, where the noise is not Gaussian but uniform, and the signal limitation is not on the power but on the amplitude. (This talk was presented in part at the MaxEnt 2014 conference in Amboise as a joint work with José Carlos Magossi (Unicamp, São Paulo State, Brasil)).

Information about the video

  • Date of publication 07/11/2016
  • Institution IHP
  • Format MP4

Domain(s)

Last related questions on MathOverflow

You have to connect your Carmin.tv account with mathoverflow to add question

Ask a question on MathOverflow




Register

  • Bookmark videos
  • Add videos to see later &
    keep your browsing history
  • Comment with the scientific
    community
  • Get notification updates
    for your favorite subjects
Give feedback