Estimation theory & the CRLB: bias, variance, and the best any estimator can do

What this course is — in one sentence

Probabilistic and statistical methods for sensing is the discipline of turning noisy measurements into numbers you would defend in review — an estimate, its uncertainty, and the argument that no honest analysis could have done much better. Everything in this course is detail underneath that single sentence.

This is the opening week of PRB504, assuming undergraduate probability plus comfort with Python and NumPy. The throughline of the whole course bears repeating: an estimate without a defensible uncertainty is not a result. This week builds the machinery behind the word "defensible" — the exact decomposition of estimation error, the information-theoretic floor beneath every unbiased estimator, and a worked sensing example where the floor can be written in closed form and read like a datasheet. By the end of the week you will be able to compute the best any estimator can do for a concrete radio-ranging problem, and to say when your own estimator is close enough to that best to stop tuning it.

How to use this document

This is the full script of a 100-minute session, organised into timed parts. Every derivation is written out at board pace — every symbol introduced before it is used, intermediate steps shown — because the material is being met for the first time and "it can be shown" is not teaching. Every number quoted from the workbook is an executed, reproducible output of workbook.ipynb (single seeded generator, default_rng(7), fully offline); those numbers are in-silico Monte-Carlo results unless explicitly labelled otherwise. One set of numbers in Part 9 is measured real-world data, cited from the lab's corpus, and is labelled as such everywhere it appears. Concept checks — "stop and convince yourself" — sit between parts; do them, they are cheaper than discovering the gap in an exam.

Part 1 · The problem nobody ordered (0:00–0:10)

Start with a picture. It is 1943, and a radar operator is watching a cathode-ray screen. A pulse went out; somewhere over the sea it bounced off an aircraft; a faint echo has come back and drawn a bump on the trace. The distance to the aircraft is the speed of light times half the round-trip delay — trivial physics. Except the bump is not a point. It is smeared by the pulse shape, corrupted by receiver noise, and jittering from sweep to sweep. Two operators reading the same screen give two different ranges. So a question arises that nobody had ordered from mathematics: out of all the ways to read this noisy echo, which one is best — and how good is best?

That question outlived the war. The engineers at the wartime radar laboratories could build filters that made the bump sharper, and after 1945 the theorists — most visibly in Woodward's 1953 monograph on probability and information theory applied to radar — asked the deeper question: is there a limit, set by the noise and the signal energy and nothing else, below which no range-reading scheme can push its error? There is. It is this week's subject. And it is not a radar fact; it is a fact about estimation itself, which is why the same bound will show up today in a WiFi ranging problem, in the GPS receiver in your pocket, and in a microscope that won a Nobel Prize.

Here is the same situation in this lab's clothes. A receiver measures the signal strength of a transmitter's packets and must answer: how far away is it? The physics gives a clean average law — received power falls off with the logarithm of distance — but every individual measurement is corrupted by shadowing, the slowly-varying attenuation from walls, bodies, and furniture. So the estimate of distance is a random variable, and three questions follow in strict order:

1. How wrong is my estimator on average (bias), and how much does it scatter (variance)?
2. What is the best any estimator could do on this data — is my scatter my fault, or the physics' fault?
3. Where should I spend my next measurement to learn the most?

These are, respectively, this week's three main acts: the bias–variance decomposition, the Cramér–Rao lower bound, and Fisher information as experiment design. The third act ends at the edge of this week's frequentist toolkit, at a real result from the lab where the classical design rule gave the wrong answer — and that cliffhanger is precisely where Week 2's Bayesian machinery picks up.

Why does the floor matter practically? Sensing pipelines are tuned endlessly: more filtering, more features, deeper models. The CRLB is the one tool that says when to stop — once the estimator's variance sits on the physics floor, further tuning is rearranging noise. And it cuts the other way too: when a published system claims an error variance below the CRLB for its stated noise model, either the noise model is wrong or the claim is. Both readings are useful, and you will use both before this course is over.

Part 2 · The words you need (0:10–0:20)

A handful of terms recur all semester. Each is defined here in plain language; every one comes back in context within the hour.

• Estimator — a rule that turns data into a guess: \hat\theta = g(x_1,\dots,x_N). The hat marks a guess; the bare symbol \theta is the true, unknown quantity.
• Bias — the systematic part of the error: \mathbb{E}[\hat\theta] - \theta. An estimator that is wrong on average, in the same direction, is biased.
• Variance — the scatter of the guess around its own average, from one repetition of the experiment to the next.
• MSE — mean squared error, \mathbb{E}[(\hat\theta-\theta)^2]: the single number that scores total error, and the quantity that decomposes exactly into bias² + variance.
• Consistency — the estimator converges to the truth as data grows: more samples, and the guess pins down \theta.
• Efficiency — the estimator's variance reaches the theoretical floor; no unbiased competitor can beat it.
• Score — the slope of the log-likelihood with respect to the parameter: how loudly one observation argues for moving the estimate.
• Fisher information — the variance of the score: how much, on average, one observation can tell you about the parameter. Written I(\theta).
• CRLB — the Cramér–Rao lower bound: for unbiased estimators, \mathrm{Var}(\hat\theta) \ge 1/I(\theta). The floor.

Two of these — score and Fisher information — look abstract and are in fact mechanical: both are one derivative away from the likelihood you already know how to write down. We will compute both, by hand, twice today.

One idea underneath all nine words deserves saying out loud, because it is the single most common stumbling block in a first estimation course. The estimator is a random variable. The data x_1,\dots,x_N are random draws; the estimator is a fixed function of them; therefore its output is random too. When we say "the variance of the sample mean", we do not mean anything about one dataset — we mean: imagine re-running the entire experiment many times, computing the estimate each time, and looking at the scatter of those estimates. Every expectation in this lecture is an average over that imaginary ensemble of re-runs. The workbook makes the ensemble real: 20,000 replicates, every time. If a statement in this course ever confuses you, ask "over what ensemble is this expectation taken?" — the confusion usually dissolves.

Stop and convince yourself (1 minute). The sample mean \bar x = \frac{1}{N}\sum_i x_i of N i.i.d. draws with variance \sigma^2 has variance \sigma^2/N. Derive it: variances of independent variables add, and constants come out squared. If this took more than a minute, revisit it tonight — the whole lecture leans on it.

Part 3 · Bias, variance, and the exact decomposition (0:20–0:35)

The identity, derived

Take any estimator \hat\theta of any parameter \theta. We want its mean squared error, and the trick — the one trick of this part — is to add and subtract the estimator's own mean, \mathbb{E}[\hat\theta], inside the square:

Expand the square. Three terms appear: the square of the random part, the square of the constant part, and twice their product. The middle term, the cross-term, is

because the expectation of any random variable minus its own mean is zero by construction. What survives is an identity, not an approximation:

In plain words: total squared error = (systematic offset)² + (scatter). You can trade one for the other, and the trade is often worth making.

Dimensional sanity, a habit this course will enforce: if \theta is metres, both bias² and variance are metres², and so is the MSE. Every equation this week passes this check; make it a reflex to run it.

The first surprise: the divide-by-N estimator

The classic first surprise of estimation theory is that unbiasedness is not a virtue, just a property. Take N = 10 draws from a Gaussian with true variance \sigma^2 = 4 and estimate that variance two ways: dividing the sum of squared deviations from the sample mean by N, or by N-1.

First, the bias of the divide-by-N version, derived rather than asserted. Write each deviation from the sample mean in terms of the deviation from the true mean \mu:

which you can verify by expanding both sides (the cross-terms collapse because \sum_i (x_i - \bar x) = 0). Take expectations: the first sum has expectation N\sigma^2, and \mathbb{E}[(\bar x - \mu)^2] = \mathrm{Var}(\bar x) = \sigma^2/N, so the second term has expectation \sigma^2. Hence

The intuition is worth keeping: the sample mean is the point that minimises the sum of squared deviations of your particular sample, so deviations measured from it systematically underestimate deviations from the truth — by exactly one sample's worth of variance. Dividing by N-1 repairs the bias exactly; dividing by N leaves a systematic shortfall of \sigma^2/N = 0.4 in our scenario.

So the unbiased one is better? Measure total error and watch. The workbook runs 20,000 replicates of this experiment and finds: the biased estimator has bias -0.406 (theory -0.400) and MSE 3.052; the unbiased one has MSE 3.565. The biased estimator wins on total error by a factor of 0.856 — its small systematic shortfall buys a larger reduction in scatter. And the decomposition is verified to the third digit: bias² + variance reproduces the directly-computed MSE (3.053 vs 3.052). Unbiasedness cost real accuracy here. It often does.

The purer version: a shrunk mean

The same trade appears in distilled form if we deliberately shrink the sample mean: estimate \mu by \hat\mu_c = c\,\bar x with a constant c < 1. Both error components are now in closed form. The bias is \mathbb{E}[c\bar x] - \mu = (c-1)\mu; the variance is c^2 \sigma^2/N; so by the identity above,

Minimise over c: differentiate, -2(1-c)\mu^2 + 2c\sigma^2/N = 0, and solve:

Always strictly less than 1: some shrinkage always helps in MSE. The workbook evaluates this at the lecture scenario (\mu = 3, \sigma^2 = 4, N = 10) and gets c^\star = 0.9574 for a 4.4\% MSE reduction over the plain mean. Small — but look hard at the formula. To compute the optimal shrinkage you need \mu^2: the true value you are trying to estimate. The optimal amount of bias to accept depends on knowledge you do not have. Hold that thought. It is the doorway through which priors enter next week, and it is the exact mechanism behind the real lab result that closes this lecture.

Reproduce in Workbook §1.

Interlude: Stein's paradox (1956) — the cliffhanger

Before leaving this part, a historical grenade, thrown here deliberately and defused only later in the course. In 1956 Charles Stein proved something the statistical establishment initially refused to believe: when you estimate three or more Gaussian means simultaneously, the obvious estimator — each sample mean for its own parameter — is inadmissible. There exists an estimator with lower total MSE for every possible configuration of the true means, no matter what they are. James and Stein exhibited one explicitly in 1961: shrink all the sample means toward a common point, by an amount computed from the data themselves — sidestepping exactly the "you need the unknown truth" objection we just hit, because with several parameters in play the ensemble of estimates carries information about how much shrinkage is safe.

The result scandalised people because the problems seem unrelated — the classic teaching example estimates baseball batting averages and finds that shrinking every player toward the league average beats tracking each player separately. It stops being a paradox and becomes obvious machinery the moment you allow hierarchical models: the parameters are treated as draws from a shared population, and shrinkage toward the population mean is just coherent inference. That machinery — partial pooling — is precisely what the lab's calibration plane runs on, and Part 8 shows you the real numbers. For today, keep the moral: estimating many related things separately wastes information that their relatedness contains.

Stop and convince yourself (2 minutes). In the MSE decomposition, which term does averaging more data shrink, and which term does it leave untouched? (Variance falls as 1/N; bias is immune to sample size.) Now explain in one sentence why a consistent estimator must have both terms going to zero.

Part 4 · The score, Fisher information, and the floor (0:35–0:55)

The score

Now the central objects of the week, built slowly. Everything starts from the likelihood: the probability density of the data viewed as a function of the parameter, p(x;\theta). Its logarithm, \ell(\theta) = \log p(x;\theta), is the log-likelihood, and logs are taken for one honest reason: independent observations multiply probabilities, and we would rather add than multiply.

Differentiate the log-likelihood with respect to the parameter — not the data, the parameter — and you get the score:

The second form (chain rule on the log) is worth keeping visible; we use it twice below. The score is a random variable — it depends on the random x — and it answers a concrete question: given this one observation, in which direction and how strongly does the likelihood want the parameter moved?

First fact, derived: the score has mean zero when evaluated at the true parameter. Densities integrate to one for every value of \theta: \int p(x;\theta)\,dx = 1. Differentiate both sides with respect to \theta, moving the derivative under the integral (this interchange is legitimate under regularity conditions we will make honest in a moment):

where the middle step used the second form of the score, \partial_\theta p = s\,p. In words: at the true parameter, the observations argue for moving the estimate up exactly as often, and as loudly, as down. The votes cancel on average — which is what "true parameter" ought to mean.

Fisher information

Since the score has mean zero, its variance equals its mean square, and that quantity gets a name:

In plain words: the score says how strongly one observation votes to move the parameter; Fisher information is the average loudness of that vote. Flat likelihood → quiet votes → little information.

An equivalent form, one more differentiation away (differentiate \mathbb{E}[s]=0 once more under the integral and rearrange), is the curvature form:

Information is the expected downward curvature of the log-likelihood at its peak. A sharply peaked likelihood pins the parameter down — high information; a flat one leaves it loose — low information. The two forms agree whenever both exist, and you use whichever differentiates more cleanly.

Two properties make the definition sing. First, information adds. For N independent observations the log-likelihoods add, so the scores add; the scores are independent and zero-mean, so their variances add:

Information accumulates linearly in data; precision, being an inverse variance, therefore also grows linearly, and standard error shrinks only as 1/\sqrt{N}. This square-root law is the most expensive fact in experimental science — each additional decimal digit of precision costs a hundredfold more data — and it reappears in every part of this lecture. Second, information is a property of the measurement model alone. No estimator has been mentioned yet. That is the deep structural point of this whole part: the floor is computed before any algorithm is designed.

A short history, because the people matter

The name is earned. Ronald Fisher built the framework — likelihood, sufficiency, efficiency, and the information that now carries his name — in two papers, 1922 and 1925, that essentially founded estimation theory as a subject; before them, "statistics" mostly meant descriptive summaries and curve-fitting by taste. Fisher's question was exactly ours: of all ways to reduce data to an estimate, which extracts everything the data contain?

The bound itself has a double birth, on opposite sides of a world war. In 1945, Calcutta: C. R. Rao, then twenty-five years old and teaching at the Indian Statistical Institute, was asked by a student — the standard account says during a lecture course — whether there was a limit to how accurate an unbiased estimator could be. He derived the answer and published it that year in the Bulletin of the Calcutta Mathematical Society. In 1946, Stockholm: Harald Cramér, who had spent the war years largely cut off from the international literature, published the same inequality independently in his Mathematical Methods of Statistics — the book that trained the first postwar generation of mathematical statisticians. Fréchet (1943) and Darmois (1945) had versions too, and some texts say Fréchet–Darmois–Cramér–Rao; the field settled on Cramér–Rao. The lesson under the names: the result was ready — a question asked in a classroom in one hemisphere and a wartime book project in the other landed on the same inequality within months. And the toolkit they built is the working language of signal processing to this day; Kay's Fundamentals of Statistical Signal Processing: Estimation Theory (1993), the field's bible, is essentially a book-length elaboration of this one part of this one lecture.

The bound, proved

Setup: \hat\theta is any unbiased estimator, so \mathbb{E}[\hat\theta] = \theta for every value of \theta — the estimator tracks the truth wherever the truth sits. That "for every value" is load-bearing: it means we may differentiate the statement with respect to \theta.

Step 1 — differentiate the unbiasedness constraint. Write the expectation as an integral and differentiate under it:

using \partial_\theta p = s\,p again (note \hat\theta(x) does not depend on \theta — it is a function of data only, so it passes through the derivative). Because the score has mean zero, \mathbb{E}[\hat\theta s] = \mathrm{Cov}(\hat\theta, s), so:

Pause on how strange that is. Any unbiased estimator — however constructed, however clever — has covariance exactly one with the score. Unbiasedness welds the estimator to the score.

Step 2 — Cauchy–Schwarz. For any two random variables, \mathrm{Cov}(A,B)^2 \le \mathrm{Var}(A)\,\mathrm{Var}(B) — correlation cannot exceed one in magnitude. Apply it with A = \hat\theta, B = s:

and rearrange:

That is the whole proof: one differentiated constraint and one Cauchy–Schwarz. For N observations, replace I by I_N = N I_1.

In plain words: no unbiased estimator can be more precise than the data is informative. The bound is a property of the measurement model, computed before any estimator is designed — a specification, like a noise figure.

The fine print, stated honestly rather than hidden: the proof interchanged differentiation and integration twice, which requires the model to be smooth in \theta and — crucially — requires the support of the data (the set of possible observations) not to depend on \theta. The standard counterexample is estimating the endpoint of a uniform distribution on [0,\theta]: the support moves with the parameter, the interchange is illegal, and estimators there beat the naive "bound" handily, converging at rate 1/N rather than 1/\sqrt N. The CRLB is a theorem with hypotheses, not a slogan. For the Gaussian-family models of this course the hypotheses hold, but you check, every time — the equality-condition of Cauchy–Schwarz also tells you when the bound is achievable at finite N: exactly when the estimator is a linear function of the score, which is rare and precious.

The canonical example, fully worked and then verified

For one observation x \sim \mathcal N(\mu, \sigma^2) with \sigma known, the log-likelihood is

Check the mean: \mathbb{E}[x - \mu] = 0 — the score is centred, as proved in general. Its variance:

so the bound for N observations is \sigma^2/N. And the sample mean has exactly this variance — computed in your Part 2 concept check. The sample mean is efficient at every N, not just asymptotically; indeed it is a linear function of the total score, which is the Cauchy–Schwarz equality condition showing up in the flesh.

The workbook checks this by Monte Carlo across N from 2 to 500: the ratio of empirical variance to the bound stays within Monte-Carlo error of 1 across the full sweep — more than two orders of magnitude in N — and at the reference size N = 50 the ratio is 1.008. Because a single Monte-Carlo number is itself an estimate, the workbook then applies the lab's reporting canon and re-runs the N = 50 experiment under 12 independent seeds: the efficiency ratio comes out at mean 1.000, with a 95% band of [0.985, 1.012] — the bound is achieved, and the residual wiggle is quantified rather than hand-waved.

In general the maximum-likelihood estimator — pick the \theta that maximises the likelihood of what you saw — does not sit on the bound at finite N. The general theorem is that the MLE is asymptotically efficient: as N grows, its distribution approaches Gaussian with variance 1/I_N(\theta), squeezing onto the floor. This is the deep reason maximum likelihood is the default estimator of this course: it is the one general-purpose recipe guaranteed to waste nothing, eventually. The Gaussian mean is the lucky case where "eventually" is "always".

Reproduce in Workbook §2.

Stop and convince yourself (2 minutes). Compute the score and Fisher information for one observation from a Poisson distribution with rate \lambda: \log p = x\log\lambda - \lambda - \log x!, so s = x/\lambda - 1 and I_1(\lambda) = \mathrm{Var}(x)/\lambda^2 = 1/\lambda. What is the CRLB for N observations, and which familiar estimator achieves it? (The sample mean again — \mathrm{Var}(\bar x) = \lambda/N = 1/I_N.)

Part 5 · Interlude — what a 95% confidence interval says (0:55–1:00)

An aside that prevents a career of misstatements, placed mid-lecture on purpose, while the machinery is warm.

With \sigma known, the interval \bar x \pm 1.96\,\sigma/\sqrt{N} is a 95% confidence interval for \mu — and the \sigma/\sqrt N in it is exactly the square root of the CRLB, which is why this interlude lives here. What the "95%" quantifies is the procedure: across repeated experiments, intervals constructed this way trap the true parameter 95% of the time. The workbook runs the experiment 20,000 times and finds empirical coverage 0.950 — the procedure delivers exactly its advertised long-run rate.

What the number does not say is that the one interval you computed has a 95% probability of containing \mu. After the data are seen, the interval either contains the fixed parameter or it does not; there is no ensemble left to average over. Assigning a probability to that event requires treating \mu itself as random — which is a prior, which is Week 2, where the statement becomes legitimate under the name credible interval. Keeping these two readings separate is the difference between a defensible error bar and a misquoted one, and this course will test the distinction, because reviewers will.

Part 6 · Case study — the microscope that beat the diffraction limit (1:00–1:07)

Now a story about photons, told because it is the purest real-world instance of everything derived so far — and because it collected a Nobel Prize.

In 1873 Ernst Abbe derived the diffraction limit: a lens cannot focus light to a spot smaller than roughly half its wavelength, which for visible light means around 200 nanometres. Two fluorescent molecules closer together than that blur into one blob, and for 130 years that was accepted as the resolution wall of optical microscopy — engraved, literally, on Abbe's memorial. Most of cell biology's machinery lives below that scale.

The wall fell to an estimation-theory argument. A single fluorescent molecule, imaged through a microscope, produces a diffraction-blurred spot — the point-spread function — a couple of hundred nanometres wide. But that spot is not the molecule; it is a likelihood surface. Each detected photon is a draw from a distribution centred on the true molecular position, and locating the molecule is a parameter-estimation problem: estimate the centre of a distribution of known width from N_{\text{photons}} draws. You already know the answer's shape from Part 4. The precision of the centre estimate is not the spot width — it is the spot width divided by \sqrt{N_{\text{photons}}}, the square-root law again. Collect thousands of photons from one molecule and a 200-nanometre blur localises its centre to the order of nanometres. The bound on that precision is a Fisher-information calculation — worked out for microscopy in the early 2000s (a widely used practical formula by Thompson, Larson and Webb in 2002; the explicit CRLB treatment by Ober, Ram and Ward in 2004) — and well-engineered instruments approach it.

One estimation trick does not make an image, because the argument needs isolated molecules — two overlapping blurs break the model. The 2006 breakthrough (Betzig and Hess's PALM; Rust, Bates and Zhuang's STORM) was to switch fluorophores on stochastically in sparse batches: image a few isolated molecules, localise each to CRLB-grade precision, switch them off, repeat thousands of times, and assemble the super-resolution image point by estimated point. The 2014 Nobel Prize in Chemistry — Betzig, Hell, and Moerner — rewarded this circumvention of Abbe's limit, and the citation's "super-resolved fluorescence microscopy" is, at its computational heart, the mathematics of Parts 3 and 4 run at nanometre scale.

Mark the structural lesson, because it transfers verbatim to radio. The diffraction limit constrains the width of one measurement's blur; the CRLB constrains the estimate, and it rewards photon count with a square root. Our RSSI problem in the next part has exactly this anatomy: shadowing sets the blur (\sigma, in dB), packets play photons (n_{\text{pkt}}), and the achievable ranging precision will be blur over \sqrt{\text{count}}, times a geometry factor. Same theorem, different wavelength.

Part 7 · The worked sensing example: RSSI ranging under log-normal shadowing (1:07–1:22)

The model

Now the machinery earns its keep on our own hardware. The log-distance path-loss model (the workhorse of LOC502 week 2) says received power at distance d, referenced to d_0 = 1 m, is

with every symbol on the table: P the received power in dBm; P_0 the mean power at the 1-metre reference; n the path-loss exponent (how fast power decays — around 2 in free space, higher indoors); X_\sigma the shadowing term, Gaussian in dB (hence "log-normal" in linear units) with spread \sigma, typically several dB indoors — the aggregate effect of walls, bodies and furniture. The model is empirical but venerable, and it is the standard first model for exactly this reason: it makes the estimation theory exactly solvable.

The bound, derived in three moves

Move 1 — average the packets. Average n_{\mathrm{pkt}} packets at fixed distance and the mean power \bar P is Gaussian with mean \mu(d) = P_0 - 10 n \log_{10} d and variance \sigma^2/n_{\mathrm{pkt}} — the variance-of-the-mean fact from Part 2, doing its usual job.

Move 2 — information about the mean. From Part 4, a Gaussian observation with known noise variance v carries information 1/v about its mean. So \bar P carries information I(\mu) = n_{\mathrm{pkt}}/\sigma^2 about \mu.

Move 3 — the chain rule to distance. We want information about d, not \mu, and only the mean depends on d. The reparameterisation rule for Fisher information — substitute \partial_d \log p = (\partial_\mu \log p)\,\mu'(d) into the definition, and the factor \mu'(d)^2 comes out of the expectation — gives I(d) = I(\mu)\,\mu'(d)^2: information in a Gaussian mean parameter is (slope of mean / noise)². Differentiate the mean, remembering \log_{10} d = \ln d/\ln 10:

and inverting gives the floor:

Dimensional sanity: \sigma and 10n are both in dB (per decade of distance), so their ratio is dimensionless; times d gives metres; the packet count is a pure number under a square root. Metres. Good.

In plain words: the achievable ranging error is the true distance, scaled by the shadowing-to-slope ratio, divided by the square root of the packet count. Every term is a knob you can read.

Three engineering consequences, straight off the formula

1. Averaging packets buys only \sqrt{n_{\mathrm{pkt}}}. Ten times the packets, 3.2 times the precision. The square-root law of Part 4 — and of the microscope — now with a price tag in airtime: halving the error costs 4× the packets. Invert the bound for the packet count, n_{\mathrm{pkt}} = \big(d\sigma\ln 10 / (10 n\,\mathrm{std}_{\text{target}})\big)^2, and the workbook prints the budget for the lecture scenario: 10.0 packets for the 0.583 m floor we already have, 21.2 for 0.4 m, 84.8 for 0.2 m, 339.3 for 0.1 m. At typical beacon rates the last row is no longer a real-time system.
2. The environment enters as \sigma/n. Heavier shadowing (larger \sigma) or flatter path loss (smaller n) degrade the floor linearly — and neither is under the designer's control. This ratio is the environment, seen through the estimator.
3. Relative error is constant in d. The bound scales with d itself: RSSI ranging is a fixed-percentage instrument, which is why it degrades from useful to hopeless as distance grows, and why the literature treats RSSI as fundamentally accuracy-limited for ranging while remaining useful for proximity and fingerprinting (liu2022_9677 ↗; roy2022_18d3 ↗).

The numbers, computed and then attacked

The workbook instantiates the lecture scenario — d = 5 m, \sigma = 4 dB, n = 2.5, n_{\mathrm{pkt}} = 10 — and the bound evaluates to a standard deviation of 0.583 m, which is 11.7\% of the true distance. Plug the numbers yourself: 5 \times 4 \times \ln 10 \,/\, (10 \times 2.5 \times \sqrt{10}) = 0.583. That is the floor. No estimator, no neural network, no filter operating on this data under this noise model does better unbiasedly.

Then the workbook attacks the floor with the actual maximum-likelihood estimator. Invert the mean law: \hat d = 10^{(P_0 - \bar P)/(10 n)}. Run it 20,000 times: the Monte-Carlo standard deviation is 0.589 m, a ratio of 1.011 to the bound. The ML estimator is not unbiased here — exponentiating a Gaussian makes \hat d log-normal, and the measured bias is +0.028 m against a closed-form prediction of +0.034 m — but at realistic packet counts the bias is a rounding error against the 0.58 m scatter, and the estimator is, for engineering purposes, on the floor. This is asymptotic efficiency arriving early, exactly as advertised in Part 4. (All of these are in-silico Monte-Carlo numbers under the stated model; the model itself is the standard empirical fit to real indoor radio, but no packet in this section was measured.)

Reproduce in Workbook §3.

The point of this part is not the specific numbers — it is that the floor was computable before any estimator was written, from the measurement model alone. When LOC502 builds real BLE ranging pipelines, this bound is the yardstick their scatter gets compared against.

Case study — GPS, or the CRLB made planetary

You carry this week's theorem in your pocket, running continuously. A GPS receiver estimates four unknowns — three position coordinates and its own clock offset — from pseudorange measurements to visible satellites, each corrupted by noise of roughly similar size (atmosphere, orbit error, receiver noise). Linearise about the current position and the model is exactly our Gaussian-mean setup in four dimensions: each satellite contributes information along its own line of sight, and the total Fisher information matrix is assembled from the unit vectors to the satellites — the constellation's geometry, and nothing else, up to the shared noise scale.

Navigation engineers have a name for the resulting error amplification: GDOP, geometric dilution of precision. It is not an analogy to this lecture; it is this lecture. GDOP is computed as the square root of the trace of the inverted geometry matrix — that is, the square root of the trace of the CRLB covariance for the position-and-clock estimate, expressed in units of the per-satellite ranging error. When your phone reports poor accuracy in a street canyon, the pseudoranges have not necessarily gotten noisier — the visible geometry has gone information-poor: satellites clustered in a narrow patch of sky give nearly parallel lines of sight, nearly redundant votes, a nearly singular information matrix, and the bound blows up. Every satellite-visibility planning tool in the industry is, in the terms of this course, Fisher-information-aware measurement design — which is precisely the topic of the next part.

Stop and convince yourself (1 minute). In the RSSI bound, why does a packet heard near the transmitter carry more information about d than one heard far away? (Because I(d) \propto 1/d^2: the mean-power curve is steeper per metre at short range, so the same dB of noise obscures fewer metres.)

Part 8 · Information as design — and where the classical rule breaks (1:22–1:32)

From bound to design criterion

Look back at I(d) \propto 1/d^2 and turn the observation around. Fisher information depends on where you measure — so, before measuring, you can choose the configuration that maximises the information you will collect. With several parameters, information is a matrix, and the classical recipe — D-optimality — says: maximise \det I(\theta), the determinant being the volume of the confidence ellipsoid the data will buy you. For the simplest case — fitting a straight line with equal noise everywhere — D-optimality gives famously blunt advice: put your measurement points as far apart as possible. Steep leverage on the slope, maximal spread, maximal determinant.

The toy that breaks it honestly

The workbook probes that advice with a two-point toy carrying one realistic wrinkle: the noise is tail-heavy, \mathrm{Var}(\varepsilon \mid x) = \sigma_0^2 (1 + (x/1.5)^4), so measurements far from the centre are noisier — as extreme-count conditions are in real crowd-calibration data. Fit y = \alpha + \beta x from one point at -x and one at +x, and compare a narrow design (\pm 0.5) with a wide one (\pm 2.0):

• Both parameters unknown (flat prior): the wide design wins decisively — Monte-Carlo slope variance 0.523 against 1.977, a wide/narrow ratio of 0.265. Spread buys slope information faster than tail noise costs, and D-optimality's advice stands.
• Slope effectively known (strong prior): only the offset \alpha remains to estimate, spread buys nothing, and the tail noise now dominates: offset variance 2.083 wide against 0.522 narrow — the ratio flips to 3.989. The "optimal" wide design is four times worse.

Same instrument, same noise, same budget of two measurements — and the best design inverted, purely because of what was already known before measuring. Design optimality is not a property of the experiment alone; it is a property of the experiment plus the prior.

Reproduce in Workbook §4.

The lab tie-in — and this is a real-data result, not a simulation

The lab's cross-environment crowd-counting corpus (n = 939 recordings, 7 environments, 2 WiFi platforms) is calibrated by placing a small number of labelled anchor points in a new environment — the direct field analogue of choosing the two x-positions above. Fitted as a hierarchical model (hierarchical-calibration-shrinkage), the corpus delivered exactly the toy's verdict at full scale: frequentist extreme-spread D-optimal anchor placement beat random placement under a weak, flat prior — and inverted, actively hurting, under the strong nested (platform-stratified) prior that the pooled corpus justifies, because wide placement over-leverages the noisy count tails. The correct criterion is Bayesian-optimal design: maximise the posterior information, prior included.

And now the threads of this lecture tie themselves. The same hierarchical analysis explains the lab's measured recalibration numbers as empirical-Bayes shrinkage — the shrunk-mean idea of Part 3, whose optimal factor needed the unknown truth, resolved exactly the way Stein and James resolved it in 1956–61: let the ensemble of related environments supply the knowledge no single environment has. Per-environment estimation is the "each batting average separately" strategy; the lab's calibration plane is the league-average shrinkage strategy, run on WiFi hardware; and it wins for the same theorem-level reason. Week 2 builds the machinery to state all of this properly.

Part 9 · How the lab reports numbers — and what you can now do (1:32–1:40)

The reporting canon you will be held to

This course will grade your numbers the way the lab grades its own, so the canon goes on the table in week one.

Every number lives inside a named baseline bracket. A headline metric is uninterpretable without the ladder it sits on. The lab's cross-environment counting bracket, from the same real-data corpus as above: in-env oracle 0.72 < 2-anchor recalibration 1.32 < zero-shot transfer 1.75 < predict-the-mean 2.07 persons MAE. The ladder is what turns "MAE 1.32" into a statement: two anchors recover about 42% of the gap between zero-shot transfer and the in-environment ceiling. These four numbers are cited, measured, real-data results — the workbook renders the ladder but does not re-derive it.

Reproduce in Workbook §5 (rendering only; the values are cited, not derived).

Multi-seed CIs, not single runs. Any Monte-Carlo or learned result is quoted as mean / lo / hi over independent seeds — as done for the efficiency ratio in Part 4 (1.000, [0.985, 1.012], 12 seeds). One seed is an anecdote.

Effect sizes with intervals, over verdicts. A rank correlation certifies direction, not size: Spearman's rho reaches its extremes for any monotone relation, however shallow, so it cannot carry a magnitude claim. Magnitude claims are carried by effect sizes — a standardised difference such as Cohen's d, or the raw effect in physical units — with their intervals attached. "Significant" is not a result; a bounded effect is.

What you can now do

Taking stock, concretely. After this week you can: decompose any estimator's error into bias² + variance and decide whether a bias is worth buying; write down a likelihood, differentiate it into a score, and square-and-average into a Fisher information; state and prove the CRLB and check its regularity conditions; compute the closed-form ranging floor for a log-normal-shadowing radio link and read its three design consequences; recognise the same bound at work in a radar, a GPS receiver, and a super-resolution microscope; state exactly what a 95% confidence interval does and does not claim; and explain, with a toy and a real n = 939 result, why optimal experiment design cannot be decided without the prior. That last item is a genuinely current research question — this lab's, among others'.

Looking ahead

Week 2 turns to Bayesian inference — priors that earn their keep. Three loose ends from this week become its agenda: the shrinkage factor that needed the unknown truth (Part 3) becomes a posterior mean under a stated prior; the forbidden reading of the confidence interval (Part 5) becomes the credible interval, for which the probability statement is legitimate; and the design inversion (Part 8) becomes Bayesian-optimal experiment design, the criterion the lab's real anchor-placement result demands. The estimation-theoretic floor built this week does not disappear — it becomes the likelihood half of everything that follows.

Exam-style questions

Model-answer sketches follow each question; write your answer before reading the sketch.

Q1 (decomposition). Define bias, variance, and MSE for an estimator \hat\theta of \theta, and prove that \mathrm{MSE} = \text{bias}^2 + \text{variance} exactly. Then explain, without heavy algebra, why the divide-by-N variance estimator can beat the unbiased divide-by-(N-1) estimator in MSE.

Sketch. Definitions as in Part 2. Proof: insert \pm\mathbb{E}[\hat\theta] inside the square, expand, cross-term vanishes because \mathbb{E}[\hat\theta - \mathbb{E}\hat\theta] = 0. Explanation: dividing by the larger N shrinks every output of the estimator toward zero, introducing a small negative bias but reducing scatter by a larger amount at small N; MSE trades these off, and the workbook's N=10 case gives MSE 3.05 vs 3.57 in the biased estimator's favour.

Q2 (the bound). State the Cramér–Rao lower bound, including the assumptions it needs, and prove it. Give one model where the assumptions fail and say what goes wrong.

Sketch. Statement: for unbiased \hat\theta under regularity (smoothness in \theta; support of x not depending on \theta; differentiation–integration interchange valid), \mathrm{Var}(\hat\theta) \ge 1/I(\theta) with I(\theta) = \mathbb{E}[(\partial_\theta \log p)^2]. Proof: differentiate \int p = 1 to get \mathbb{E}[s]=0; differentiate \mathbb{E}[\hat\theta]=\theta to get \mathrm{Cov}(\hat\theta,s)=1; Cauchy–Schwarz. Failure case: uniform on [0,\theta] — support moves with \theta, interchange illegal, and e.g. the sample maximum converges at rate 1/N, "beating" the naive bound.

Q3 (worked bound). For the RSSI model P = P_0 - 10 n \log_{10} d + X_\sigma with X_\sigma \sim \mathcal N(0,\sigma^2) and n_{\text{pkt}} averaged packets, derive the CRLB for \hat d. With d = 5 m, \sigma = 4 dB, n = 2.5, n_{\text{pkt}} = 10, evaluate the bound, and compute how many packets a 0.2 m standard-deviation floor would require.

Sketch. Three moves of Part 7: averaged power is Gaussian with variance \sigma^2/n_{\text{pkt}}; I(\mu) = n_{\text{pkt}}/\sigma^2; chain rule with \mu'(d) = -10n/(d\ln 10) gives \mathrm{std}(\hat d) \ge d\sigma\ln 10/(10 n \sqrt{n_{\text{pkt}}}). Evaluation: 0.583 m. For 0.2 m: n_{\text{pkt}} = (5 \cdot 4 \cdot \ln 10 / (25 \cdot 0.2))^2 = 84.8, i.e. 85 packets — the workbook prints this row.

Q4 (interpretation trap). A colleague computes \bar x \pm 1.96\,\sigma/\sqrt N = [3.1, 4.3] and writes "there is a 95% probability that \mu lies between 3.1 and 4.3." Is the sentence correct? If not, state precisely what the 95% refers to, and name the object for which the colleague's sentence would be correct.

Sketch. Incorrect as written. The 95% is a property of the procedure: over repeated experiments, 95% of such intervals cover the fixed, non-random \mu (workbook check: empirical coverage 0.950). The colleague's sentence describes a Bayesian credible interval, which requires a prior on \mu.

Q5 (design). Explain D-optimality for straight-line fitting and why it recommends extreme spread. Then describe, using either the workbook's two-point toy or the lab's anchor-placement result, a situation where extreme spread is the wrong design, and state the corrected criterion.

Sketch. \det I maximised by maximal leverage on the slope → extreme spread under homoskedastic noise and a flat prior. With tail-heavy noise and a strong prior on the slope, only the offset remains to estimate; spread buys nothing and costs noise — toy ratio flips from 0.265 (wide wins) to 3.989 (wide loses); the lab's n = 939 real-data corpus showed the same inversion under its nested prior. Corrected criterion: Bayesian-optimal design — maximise posterior information, prior included.

Q6 (transfer). GPS receivers report a "dilution of precision" that worsens when visible satellites cluster in one part of the sky, even though each pseudorange is no noisier. Explain this in the language of this week — what object is degrading, and why does the CRLB capture it?

Sketch. The Fisher information matrix is built from the satellite line-of-sight geometry; clustered satellites give nearly parallel, nearly redundant measurement directions, so the information matrix approaches singularity and its inverse — the CRLB covariance — blows up along the poorly-observed directions. GDOP is the square root of the trace of that inverted geometry matrix: pure measurement design, no change in per-measurement noise.

Further reading

Annotated; the first two are this week's citations into the vault, the rest are the canonical texts behind the lecture's stories.

• Liu et al. (2022) — A Survey on Fundamental Limits of Integrated Sensing and Communication. The CRB/Fisher-information toolkit applied across ranging modalities; the RSS localisation bound (shadowing variance over squared path-loss exponent) generalises this week's worked example to full positioning. liu2022_9677 ↗
• Roy et al. (2022) — A survey on ubiquitous WiFi-based indoor localization from implementation perspectives. The log-distance path-loss model in deployment practice, and where RSSI-based systems sit against CSI-based ones. roy2022_18d3 ↗
• Fallani (2026) — IoT solutions for e-Health applications. A compact statement of why multipath shadowing fundamentally limits RSSI ranging — the physical mechanism behind the σ in this week's bound. fallani2026_04be ↗
• Lab hypothesis note — hierarchical-calibration-shrinkage. The n = 939 hierarchical fit, the shrinkage reading of the 2-anchor recovery, and the D-optimal placement inversion under the nested prior. hierarchical-calibration-shrinkage
• Kay (1993) — Fundamentals of Statistical Signal Processing, Volume I: Estimation Theory. The field's bible: every object of Parts 3–7, with a hundred worked signal-processing examples. Chapter 3 is this lecture's Part 4 at textbook depth.
• Fisher (1922, 1925) — On the mathematical foundations of theoretical statistics and Theory of statistical estimation. Where likelihood, efficiency, sufficiency and information enter the language. Hard reading, worth an afternoon for the shock of how much arrived at once.
• Rao (1945) and Cramér (1946) — the bound's independent births: a Calcutta journal paper answering a classroom question, and a chapter of the Stockholm book that trained postwar mathematical statistics.
• Efron & Morris (1977) — Stein's paradox in statistics (Scientific American). The most readable account of the 1956 inadmissibility result and the James–Stein estimator, batting averages included — and the gentlest possible on-ramp to Week 2's hierarchical thinking.
• Woodward (1953) — Probability and Information Theory, with Applications to Radar. The postwar synthesis of this lecture's opening story: estimation bounds for delay (range) measurement, written by one of the people who needed them first.