2831.sagews

John Travis - Mississippi College

Bayes Theorem

For a given sample space S, there may be a natural way to partition the space into disjoint sets--so that all elements in S belong to exactly one of the sets.  For example, separating a given set of people into males and females or dividing up a pocket full of change into pennies, nickles, dimes, quarters, etc. 

For disjoint sets, a Venn diagram is traditionally written using a collection of disconnected blobs--one for each set in the partition--in a box.  However, when sets form a partition it is often simpler to write a Venn diagram as a pie chart with each set in the partition corresponding to one sector of that pie chart.

Notationally, we will describe our partition of subsets as

$S = B_{1}\cup B_{2}\cup B_{3} \cdots \cup B_{N}$.

From this partitioned sample space, one may desire a conditional probabability for some outcome A.  For example, if A is the event that a random student selected from a particular class fails the course and $S = Males \cup Females$, then $P(fails | Male)$ and $P(fails | Female)$ might be well known from historical values.  However, given that a student who failed has set up an appointment to meet with the teacher, determining the likelihood that the person will be Male (eg. $P(Male | fails)$) is often a harder thing to quantify directly.  Bayes Theorem is the answer to solving this type of problem.

Derivation of Bayes Theorem:  From the definition of conditional probability and using the notation developed above:

$P(A \cap B_{k}) = P(A) P(B_{k}| A)$ or

$P(A \cap B_{k})$ = $P(B_{k})$\LaTeX$ P(A | B_{k}) $ and by transitivity

$P(A) P(B_{k}| A)$ = $P(B_{k})$\LaTeX$ P(A | B_{k}) $

Since $\bigcup B_{k}$ comprises all of S, then one may compute P(A) by adding up the probabilities of its parts--indeed, $A = (A\cap B_{1}) \cup (A\cap B_{2}) \cup (A\cap B_{3}) \cup \cdots \cup (A \cap B_{N})$.  In the diagram below, this is illustrated using the probabilities inside the inner circle.

Using the second formula with this partition of A (remember the $B_k$ are all disjoint) yields:

 ${\bf P(A)} = P(A\cap B_{1}) + P(A\cap B_{2}) + \cdots + P(A \cap B_{N})$ =  

$ = P(B_{1})P(A|B_{1}) + P(B_{2})$$P(A|B_{2})$$+ \cdots +$$P(B_{N})$$P(A|B_{N})$

On the other hand, using the third formula and solving yields:

$P(B_{k}| A) $ = $P(B_{k})$\LaTeX$ P(A | B_{k})/ {\bf P(A)}  $ 

Replacing the P(A) on the bottom with the bold formulation gives Bayes Theorem.

Therefore, Bayes Theorem is very useful when it is possible to determine the conditional probabilities $P(A|B_{k}), k=1 \cdots N$ but perhaps not so easy to compute $P(B_{k}| A) $.

%auto
##
##  Illustration of Bayes Theorem for introductory Probability and Statistics
##  
##  John Travis
##  Mississippi College
##  06/17/11
##
##  The user inputs:
##  - A list of probabilities corresponding to each sector of a partition
##  - A list of prior conditional probabilities P( A | B_1), etc.
##  Output:
##  - Two Venn diagrams.  The first describes the input data.  The second describes the posterior probabilities.
##  - Numerical values for everything, if desired.
##
##  An updated version of this worksheet may be available at http://sagenb.mc.edu
##

#  This function is used to convert an input string into separate entries
def g(s): return str(s).replace(',',' ').split()

@interact
def _(Partition_Probabilities=input_box('0.35,0.25,0.40',label='$P(B_1),P(B_2),...$'),
        Conditional_Probabilities=input_box('0.02,0.01,0.03',label='$P(A|B_1),P(A|B_2),...$'),
        print_numbers=checkbox(True,label='Numerical Results on Graphs?'),
        auto_update=False):
            
    Partition_Probabilities = g(Partition_Probabilities)
    Conditional_Probabilities = g(Conditional_Probabilities)
    
    n = len(Partition_Probabilities)
    n0 = len(Conditional_Probabilities)
    
    if n<>n0:
        print html("<font size=+2 color=red>You must have the same number of partition probabilities and conditional probabilities.</font>")
        
    else:                               # input data streams now are the same size!
        colors = rainbow(n)
        accum = float(0)                # to test whether partition probs sum to one
        ends = [0]                      # where the graphed partition sectors change in pie chart 
        mid = []                        # middle of each pie chart sector used for placement of text
        p_Bk_given_A = []               # P( B_k | A )
        pA = 0                          # P(A)
        PP=[]                           # array to hold the numerical Partition Probabilities 
        CP=[]                           # array to hold the numerical Conditional Probabilities 

        for k in range(n):
            PP.append(float(Partition_Probabilities[k]))
            CP.append(float(Conditional_Probabilities[k]))
            p_Bk_given_A.append(PP[k]*CP[k] )
            pA += p_Bk_given_A[k]
            accum = accum + PP[k]
            ends.append(accum)
            mid.append((ends[k]+accum)/2)
#
#  Marching along from 0 to 1, saving angles for each partition sector boundary.
#  Later, we will multiple these by 2*pi to get actual sector boundary angles.
#
        if abs(accum-float(1))>0.0000001:     #  Due to roundoff issues, this should be close enough.                     
            print html("<font size=+1 color=red>Sum of probabilities should equal 1.</font>")
        
        else:                           # probability data is sensible
 
#        
#  Draw the Venn diagram by drawing sectors from the angles determined above
#  First, create a circle of radius 1 to illustrate the the sample space S
#  Then draw each sector with varying colors and print out their names on the edge
#
            G = circle((0,0), 1, rgbcolor='black',fill=False, alpha=0.4,aspect_ratio=True,axes=False,thickness=5)
            for k in range(n):
                G += disk((0,0), 1, (ends[k]*2*pi, ends[k+1]*2*pi), color=colors[mod(k,10)],alpha = 0.2)
                G += text('$B_'+str(k+1)+'$',(1.1*cos(mid[k]*2*pi), 1.1*sin(mid[k]*2*pi)), rgbcolor='black')
                
            G += circle((0,0), 0.6, facecolor='yellow', fill = True, alpha = 0.1, thickness=5,edgecolor='black') 
    
#  Print the probabilities corresponding to each particular region as a list and on the graphs
            if print_numbers:               

                html("$P(A) = %s$"%(str(pA),))
                for k in range(n):
                    html("$P(B_{%s}|A)$"%(str(k+1)),)
                    html("$ = %s$"%str(p_Bk_given_A[k]/pA),)
                                        
                    G += text(str(p_Bk_given_A[k]),(0.4*cos(mid[k]*2*pi), 0.4*sin(mid[k]*2*pi)), rgbcolor='black')
                    G += text(str(PP[k] - p_Bk_given_A[k]),(0.8*cos(mid[k]*2*pi), 0.8*sin(mid[k]*2*pi)), rgbcolor='black')
        
#  This is essentially a repeat of some of the above code but focused only on creating the smaller inner circle dealing
#  with the set A so that the sectors now correspond in area to the Bayes Theorem probabilities


            accum = float(0)                        
            ends = [0]                     # where the graphed partition sectors change in pie chart 
            mid = []                       # middle of each pie chart sector used for placement of text
            for k in range(n): 
                accum += float(p_Bk_given_A[k]/pA) 
                ends.append(accum)
                mid.append((ends[k]+accum)/2)
            H = circle((0,0), 1, rgbcolor='black',fill=False, alpha=0,aspect_ratio=True,axes=False,thickness=0)
            H += circle((0,0), 0.6, facecolor='yellow',fill=True, alpha=0.1,aspect_ratio=True,axes=False,thickness=5,edgecolor='black')
            
            for k in range(n):
                H += disk((0,0), 0.6, (ends[k]*2*pi, ends[k+1]*2*pi), color=colors[mod(k,10)],alpha = 0.2)
                H += text('$B_'+str(k+1)+'|A$',(0.7*cos(mid[k]*2*pi), 0.7*sin(mid[k]*2*pi)), rgbcolor='black')
                    
        #  Now, print out the bayesian probabilities using the smaller set A only
    
            if print_numbers:
                for k in range(n):
                    H += text(str( N(p_Bk_given_A[k]/pA,digits=4) ),(0.4*cos(mid[k]*2*pi), 0.4*sin(mid[k]*2*pi)), rgbcolor='black')
                    
            html.table([[G,H],['Venn diagram of partition with A in middle','Venn diagram presuming A has occured']])

P(B_1),P(B_2),...  P(A|B_1),P(A|B_2),...  Numerical Results on Graphs?  [removed] [removed] [removed] [removed] [removed]

reset()