6.441-19 Gaussian channels with feedback

LECTURE 19Last time:Gaussian channels: parallel • colored noise • inter-symbol interference • general case: multiple inputs and out₭ puts Lecture outlineGaussian channels with feedback • Upper bound to benefit of capacity • Reading: Section 10.6. Gaussian channels with feedbackIn the case of a DMC that there is no beneffi to feedback The same arguments extend to the case where we have continuous inputs and outpuut What happens in the case when the noise is not white? We can garner information about future noise from past noise Yi = Xi + Ni but now the Xi is also a function of the past Y s, within an energy per codeword constraint Gaussian channels with feedbackA code is now a mapping xi(M,Y i−1) from the messages in M = {1, 2,..., 2nR} and from Yi−1 onto reals under the constraint ENn �n 1 �in =1 xi(m,Y i−1)� ≤P∀m ∈{1, 2,..., 2nR}How do we define capacity? Let’s try:�1 � = max I (Xn; Yn)Cn,F B 1trace�ΛXn�≤P n n moreover I (Xn; Yn) = h(Yn) − h(Yn |Xn) = h(Xn) − h(Xn |Yn) but then select (X1,X2, . . . , Xn) = (0,N1, . . . , Nn−1) the mutual information blows up! �� Gaussian channels with feedback Let’s try: �1 �I (M; Yn)= maxCn,F B 1trace�ΛXnn n≤PNote: in the case of no feedback, then M and Xn are equivalent I (M; Yn) = h(Yn) − hn(Yn |M) M,Y i−1)= h(Yn) −h(Yi|=1in�M,Y i−1,Xi)= h(Yn) −h(Yi|=1in�M,Y i−1, Xi, Ni−1)= h(Yn) −h(Yi|=1in�Xi, Ni−1)= h(Yn) −= h(Yn) −h(Yi|=1in�i=1Ni−1)h(Ni|= h(Yn) − h(Nn)Gaussian channels with feedbackHow do we maximize I (M; Yn), or equivalenntl h(Yn) − h(Nn) Since a Gaussian distribution maximizes entroopy 1h(Yn) ≤ 2 ln �(2πe)n|ΛXn+Nn| � we can always achieve this by taking theXs to be jointly Gaussian with the past Y sXi = �i−1 j=1 αi,jYj + Vi + ci where Vi is mutually independent from the Yjs, for 1 ≤ j ≤ i − 1 and any constant ci will leave the autocorrelation matrix unchannged Note that the past Xs are a constaant so in particular we can select ci = − �i−1 j=1 αi,jxj so = �i−1Xi j=1 αi,jNj + Vi Gaussian channels with feedbackDo we have coding theorems? Joint typicality between input and output hold as a means of decoding WLLN of large numbers holds Sparsity argument for having multiple identiica mappings holds Converse: Fano’s lemma still holds, with M being directly involved in the bound Question: how does this compare to the non-feedback capacity? Gaussian channels with feedback Non-feedback capacity is simply Gaussian colored noise channel: Cn = max1trace�ΛXn��1I (Xn; Yn)� n ≤P n In this case I (Xn; Yn) = h(Yn) − h(Yn |Xn) = h(Xn + Nn) − h(Nn) which is maximized by taking Xn to be Gaussian colored noise determined using water-filling so Cn = max1 � 1 ln �|ΛXn+ΛNn|�� ntrace�ΛXn�≤P 2n |ΛNn| From our previous discussion, Cn,F B = 21 n ln �|ΛXΛnN+nNn|� || we can find this if we determine the αi,js, but this may not be easy An upper boundFact 1:ΛXn+Nn +ΛXn−Nn =2 �ΛXn +ΛNn�Look at elements in the diagonal and theoff-diagonalsFact 2:If C = A − B is symmetric positive definite,when A and B are also symmetric positive definite, then AB||≥|| Consider V ∼N (0,C),W ∼N (0,B) indepennden random variables Let S = V + W , then S ∼N (0,A) h(S) ≥ h(S|V )= h(W |V )= h(W ) so |A|≥ |B| An upper boundFrom fact 1:2(ΛXn +ΛNn) − ΛXn+Nn =ΛXn−Nnhence 2(ΛXn +ΛNn) − ΛXn+Nn is positive definite From fact 2: |ΛXn+Nn|≤|2(ΛXn +ΛNn)| =2n|(ΛXn + ΛNn)| Hence Cn,F B = 1 max � 21 n ln �|ΛXΛnN+nNn|�� ntrace�ΛXn�≤P || � 1 � ΛXn +ΛNn�� max ln2n≤ n 1trace�ΛXn�≤P 2n ||ΛNn|| ln(2) = Cn + 2 Writing on dirty paperSuppose that the sender knows the degradattio d exactly, what should he do? What should the receiver do? May not always be able to subtract d at the sender. Example: we try to send S uniformly distribbute over [−1, 1] select X such that (X + d) mod 2 = S X = S − d mod 2 and the receiver takes mod 2 MIT OpenCourseWarehttp://ocw.mit.edu 6.441 Information Theory Spring 2010 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.