6.041 / 6.431 2. Conditioning and Bayes' Rule

LECTURE 2 • Readings: Sections 1.3-1.4 Lecture outline Review• • Conditional probability • Three important tools: – Multiplication rule – Total probability theorem – Bayes’ rule Review of probability models • Sample space ! – Mutually exclusive Collectively exhaustive – Right granularity • Event: Subset of the sample space • Allocation of probabilities to events 1. P(A) ! 0 2. P(!)=1 3. If A" B = Ø, then P(A# B)= P(A)+ P(B) 3’. If A1,A2,... are disjoint events, then: P(A1 # A2 # ···)= P(A1)+ P(A2)+ ··· • Problem solving: – Specify sample space – Define probability law – Identify event of interest – Calculate... Conditional probability A B • P(A|B) = probability of A, given that B occurred – B is our new universe • Definition: Assuming P(B) $= 0, P(A | B)= P(A" B) P(B) P(A | B) undefined if P(B)=0 Die roll example X = First roll 1 2 3 4 4 3 2 Y = Second roll 1 • Let B be the event: min(X,Y)=2 • Let M = max(X,Y) • P(M =1 | B)= • P(M =2 | B)= Multiplication rule P(A ! B ! C) = P(A)P(B | A)P(C | A ! B) Models based on conditional probabilities • Event A: Airplane is flying above Event B: Something registers on radar screen P(A)=0.05 P(Ac)=0.95 P(B | A)=0.99 P(Bc | A)=0.01 P(B | Ac)=0.10 P(Bc | Ac)=0.90 P(A" B)= P(B)= P(A | B)= Multiplication rule P(A" B" C)= P(A) · P(B | A) · P(C | A" B) P(A) P(Ac) P(B | A) P(Bc | A) A Ac BUA CA UBcU BcU A CcA UBcU P(C | A B)U CA UBU Total probability theorem • Divide and conquer • Partition of sample space into A1,A2,A3 • Have P(B | Ai), for every i A1 B A2 A3 • One way of computing P(B): P(B)= P(A1)P(B | A1) + P(A2)P(B | A2) + P(A3)P(B | A3) Bayes’ rule • “Prior” probabilities P(Ai) – initial “beliefs” • We know P(B | Ai) for each i • Wish to compute P(Ai | B) – revise “beliefs”, given that B occurred A1 B A2 A3 P(Ai | B) = P(Ai " B) P(B) = P(Ai)P(B | Ai) P(B) = P(Ai)P(B | Ai) !j P(Aj)P(B | Aj) MIT OpenCourseWarehttp://ocw.mit.edu 6.041 /6.431 Probabilistic Systems Analysis and Applied ProbabilityFall 2010 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.