A [somewhat] Quick Overview of Probability

1 .A [somewhat] Quick Overview of Probability Shannon Quinn CSCI 6900

2 .Probabilistic and Bayesian Analytics Andrew W. Moore School of Computer Science Carnegie Mellon University www.cs.cmu.edu/~awm awm@cs.cmu.edu 412-268-7599 Note to other teachers and users of these slides. Andrew would be delighted if you found this source material useful in giving your own lectures. Feel free to use these slides verbatim, or to modify them to fit your own needs. PowerPoint originals are available. If you make use of a significant portion of these slides in your own lecture, please include this message, or the following link to the source repository of Andrew’s tutorials: http://www.cs.cmu.edu/~awm/tutorials . Comments and corrections gratefully received. Copyright © Andrew W. Moore [Some material pilfered from http://www.cs.cmu.edu/~awm/tutorials ]

3 .Probability - what you need to really, really know Probabilities are cool

4 .Probability - what you need to really, really know Probabilities are cool Random variables and events

5 .Discrete Random Variables A is a Boolean-valued random variable if A denotes an event , there is uncertainty as to whether A occurs. Examples A = The US president in 2023 will be male A = You wake up tomorrow with a headache A = You have Ebola A = the 1,000,000,000,000 th digit of π is 7 Define P(A) as “the fraction of possible worlds in which A is true” We’re assuming all possible worlds are equally probable

6 .Discrete Random Variables A is a Boolean-valued random variable if A denotes an event, there is uncertainty as to whether A occurs. Define P(A) as “the fraction of experiments in which A is true” We’re assuming all possible outcomes are equiprobable Examples You roll two 6-sided die (the experiment) and get doubles (A=doubles, the outcome) I pick two students in the class (the experiment) and they have the same birthday (A=same birthday, the outcome) a possible outcome of an “experiment ” the experiment is not deterministic

7 .Visualizing A Event space of all possible worlds Its area is 1 Worlds in which A is False Worlds in which A is true P(A) = Area of reddish oval

8 .Probability - what you need to really, really know Probabilities are cool Random variables and events There is One True Way to talk about uncertainty: the Axioms of Probability

9 .The Axioms of Probability 0 <= P(A) <= 1 P(True) = 1 P(False) = 0 P(A or B) = P(A) + P(B) - P(A and B) Events, random variables, …., probabilities “Dice” “Experiments”

10 .The Axioms Of Probability (This is Andrew’s joke)

11 .Interpreting the axioms 0 <= P(A) <= 1 P(True) = 1 P(False) = 0 P(A or B) = P(A) + P(B) - P(A and B) The area of A can’t get any smaller than 0 And a zero area would mean no world could ever have A true

12 .Interpreting the axioms 0 <= P(A) <= 1 P(True) = 1 P(False) = 0 P(A or B) = P(A) + P(B) - P(A and B) The area of A can’t get any bigger than 1 And an area of 1 would mean all worlds will have A true

13 .Interpreting the axioms 0 <= P(A) <= 1 P(True) = 1 P(False) = 0 P( A or B ) = P( A ) + P( B ) - P( A and B ) A B

14 .Interpreting the axioms 0 <= P(A) <= 1 P(True) = 1 P(False) = 0 P( A or B ) = P( A ) + P( B ) - P( A and B ) A B P(A or B) B P(A and B) Simple addition and subtraction

15 .Theorems from the Axioms 0 <= P(A) <= 1, P(True) = 1, P(False) = 0 P( A or B ) = P( A ) + P( B ) - P( A and B )  P(not A) = P(~A) = 1-P(A) P(A or ~A) = 1 P(A and ~A) = 0 P(A or ~A) = P(A) + P(~A) - P(A and ~A) 1 = P(A) + P(~A) - 0

16 .Elementary Probability in Pictures P(~A) + P(A) = 1 A ~A

17 .Another important theorem 0 <= P(A) <= 1, P(True) = 1, P(False) = 0 P( A or B ) = P( A ) + P( B ) - P( A and B )  P(A) = P(A ^ B) + P(A ^ ~B) A = A and (B or ~B) = (A and B) or (A and ~B) P(A) = P(A and B) + P(A and ~B) – P((A and B) and (A and ~B)) P(A) = P(A and B) + P(A and ~B) – P(A and A and B and ~B)

18 .Elementary Probability in Pictures P(A) = P(A ^ B) + P(A ^ ~B) B ~B A ^ ~B A ^ B

19 .Probability - what you need to really, really know Probabilities are cool Random variables and events The Axioms of Probability Independence

20 .Independent Events Definition: two events A and B are independent if Pr(A and B)=Pr(A)*Pr(B). Intuition: outcome of A has no effect on the outcome of B (and vice versa). We need to assume the different rolls are independent to solve the problem. You frequently need to assume the independence of something to solve any learning problem.

21 .Some practical problems You’re the DM in a D&D game. Joe brings his own d20 and throws 4 critical hits in a row to start off DM=dungeon master D20 = 20-sided die “Critical hit” = 19 or 20 What are the odds of that happening with a fair die? Ci=critical hit on trial i, i=1,2,3,4 P(C1 and C2 … and C4) = P(C1)*…*P(C4) = (1/10)^4

22 .Multivalued Discrete Random Variables Suppose A can take on more than 2 values A is a random variable with arity k if it can take on exactly one value out of {v 1 ,v 2 , .. v k } Example: V={ aaliyah , aardvark, …., zymurge , zynga } Example: V={ aaliyah_aardvark , …, zynga_zymgurgy } Thus…

23 .Terms: Binomials and Multinomials Suppose A can take on more than 2 values A is a random variable with arity k if it can take on exactly one value out of {v 1 ,v 2 , .. v k } Example: V={ aaliyah , aardvark, …., zymurge , zynga } Example: V={ aaliyah_aardvark , …, zynga_zymgurgy } The distribution Pr(A) is a multinomial For k=2 the distribution is a binomial

24 .More about Multivalued Random Variables Using the axioms of probability and assuming that A obeys… It’s easy to prove that And thus we can prove

25 .Elementary Probability in Pictures A=1 A=2 A=3 A=4 A=5

26 .Elementary Probability in Pictures A=aardvark A= aaliyah A =… A =…. A= zynga …

27 .Probability - what you need to really, really know Probabilities are cool Random variables and events The Axioms of Probability Independence, binomials, multinomials Conditional probabilities

28 .A practical problem I have lots of standard d20 die, lots of loaded die, all identical. Loaded die will give a 19/20 (“critical hit”) half the time. In the game, someone hands me a random die, which is fair (A) or loaded (~A), with P(A) depending on how I mix the die. Then I roll, and either get a critical hit (B) or not (~B) . Can I mix the dice together so that P(B ) is anything I want - say, p(B)= 0.137 ? P(B) = P(B and A ) + P(B and ~A ) = 0.1* λ + 0.5* (1- λ ) = 0.137 λ = (0.5 - 0.137)/0.4 = 0.9075 “mixture model”

29 .Another picture for this problem A (fair die) ~A (loaded) A and B ~A and B It’s more convenient to say “if you’ve picked a fair die then …” i.e. Pr(critical hit|fair die)=0.1 “if you’ve picked the loaded die then….” Pr(critical hit|loaded die)=0.5 Conditional probability: Pr(B|A) = P(B^A)/P(A) P(B|A) P(B|~A)