6.441-17 Additive Gaussian noise channel

LECTURE 17Last time:Differential entropy • Entropy rate and Burg’s theorem • AEP • Lecture outlineWhite Gaussian noise • Bandlimited WGN • Additive White Gaussian Noise (AWGN) • channelCapacity of AWGN channel• Application: DS-CDMA systems • Spreading • Coding theorem • Reading: Sections 10.1-10.3. White Gaussian Noise (WGN)WGN is a good model for a variety of noiseprocesses:-ambient noise-dark current-noise from amplifiersA sample at any time N(t) is GaussianIt is defined by its power spectrum: R(t)=N0δ(t)Its power spectral density is N0 over thewhole spectrum (flat PSD)What is its bandwidth?What is its energy?Do we ever actually have WGN?Bandlimited WGNWGN is always bandlimited to some bandwiidt W , thus the noise is passed through a bandpass filter The power density spectrum is now nonzeer only over an interval W of frequencyThe energy of the noise is then WN0 The Nyquist rate is then W and samples are W 1 apart in time Discrete-time samples of the bandlimited noise, which by abuse of notation we still denote N, are IID )NNi ∼N (0,σ2 AWGN ChannelWe operate in sampled time, hence already in a bandlimited world We ignore issues of impossibility of bandlimmitin and time-limiting simultaneously (one can obtain bounds for the limiting in time, and we are inherently considering very long times, as per the coding theorem) Yi = Xi + Ni Nyquist rate of the output is Nyquist rate of the input The input and the noise are independentThe input has a constraint on its average energy, or average variance Capacity of AWGN ChannelWe have a memoryless channel Same arguments as for DMC imply the Xis should be IID, yielding IID Yis The variance of Yi is σ2+ σ2 XN Hence we omit the i subscript in the followwin I(X; Y )= h(Y ) − h(Y |X)= h(Y ) − EX[h(Y − x|X = x)]= h(Y ) − EX[h(N|X = x)]= h(Y ) − h(N)≤ 21 ln �2πe(σ2 N )� − 21 N � X + σ2 ln �2πeσ2 To achieve inequality with equality, select the Xis to be IID ∼N (0,σ2 ) (could have Xany mean other than 0, but would affect second moment without changing variance, so not useful in this case), hence the Xis are themselves samples of bandlimited W GN CapacityofAWGNChannel Another view of capacity I(X; Y )= h(X) − h(X|Y )= h(X) − EY [h(X − αy|Y = y)]In particular, we can select α so that αY is the LLSE estimate of X from Y For X Gaussian, then this estimate is also the MMSE estimate of X from Y The error of the estimate is �= X − αyX The error is independent of the value y (recaal that for any jointly Gaussian random variables, the first rv can be expressed as the weighted sum of the second plus an independent Gaussian rv) The error is Gaussian Capacity of AWGN ChannelNote that this maximizes EY [h(X − αy|Y = y)] The �Xis are still IID EY [h(X − αy|Y = y)] = EY [h( �YX= y)]|X)≤ h1( �≤ 2 ln �2πeσ2 �� X We can clearly see that adding a constant to the input would not help Note that, if the channel is acting like a jammer to the signal, then the jammer cannno control the h(X) term, only can control the h(XY ) term |All of the above inequalities are reached with equality if the channel is an AWGN channel Capacity of AWGN ChannelThus, the channel is acting as an optimal jammer to the signal Saddle point: -if the channel acts as an AWGN channel, then the sender should send bandlimited W GN under a specific variance constraint (energy constraint) -if the input is bandlimited WGN, then undde a variance constraint on the noise, the channel will act as an AWGN channel The capacity is thus �σ2 σ21 Y � 1 �X �C = 2ln σ2 = 2ln 1+ σ2 NN for small SNR, roughly proportional to SNRApplication: DS-CDMA systemsConsider a direct-sequence code division multiipl access system Set U of users sharing the bandwidth Every user acts as a jammer to every other user Xij = �k∈U\{j} Xik + Ni The capacity for user j is +σ2 ⎛⎜⎝1+ ⎞ ⎟⎠σ21Xj σ2Xk N�k∈U\{j} i lnCj =2SpreadingWhat happens when use more bandwidth (spreading)? Let us revisit the expression for capacity:1 �σ2 �C = ln 1+ X 2 σ2 N Problem: when we change the bandwidth, we also change the number of samples Think in terms of number of degrees of freedom For time T and bandwidth W The total energy of the input stays the same, but the energy of the noise is proporttiona to W Or, alternatively, per degree of freedom, wehave the same energy per degree of freeddo for the noise, but the energy for the1input decreases proportionally to W SpreadingFor a given T with energy constraint E overthat time in the input, maximum mutualinformation in terms of nats per second is:21 TW ln �1+ E� WN0 limit as W →∞ is 2TNE 0 spreading is always beneficial, but its effect is bounded Geometric interpretation in terms of concavvit Coding theoremWe no longer have a discrete input alphabeet but a continuous input alphabet We now have transition pdfs fY x)|X(y|For any block length n, let nfY n|Xn(y|xn)= �in =1 fY |X(yi|xi) fXn(xn)= �ni=1 fX(xi) fY n(yn)= �ni=1 fY (yi) Let R be an arbitrary rate R 0 and for any rate R