📚 The CoCalc Library - books, templates and other resources

Project: 📚 The Library - Shared Public Version

Path: cocalc-examples / think-stats-2ed / solutions / chap11soln.ipynb

Views: ⁹⁶¹⁴⁰
License: OTHER

Kernel: Python 3

Examples and Exercises from Think Stats, 2nd Edition

MIT License: https://opensource.org/licenses/MIT

In [1]:

%matplotlib inline

import numpy as np
import pandas as pd

import thinkstats2
import thinkplot

Multiple regression

Let's load up the NSFG data again.

In [2]:

import first

live, firsts, others = first.MakeFrames()

Here's birth weight as a function of mother's age (which we saw in the previous chapter).

In [3]:

import statsmodels.formula.api as smf

formula = 'totalwgt_lb ~ agepreg'
model = smf.ols(formula, data=live)
results = model.fit()
results.summary()

OLS Regression Results
Dep. Variable:	totalwgt_lb	R-squared:	0.005
Model:	OLS	Adj. R-squared:	0.005
Method:	Least Squares	F-statistic:	43.02
Date:	Thu, 28 Feb 2019	Prob (F-statistic):	5.72e-11
Time:	09:58:53	Log-Likelihood:	-15897.
No. Observations:	9038	AIC:	3.180e+04
Df Residuals:	9036	BIC:	3.181e+04
Df Model:	1
Covariance Type:	nonrobust

	coef	std err	t	P>\|t\|	[0.025	0.975]
Intercept	6.8304	0.068	100.470	0.000	6.697	6.964
agepreg	0.0175	0.003	6.559	0.000	0.012	0.023

Omnibus:	1024.052	Durbin-Watson:	1.618
Prob(Omnibus):	0.000	Jarque-Bera (JB):	3081.833
Skew:	-0.601	Prob(JB):	0.00
Kurtosis:	5.596	Cond. No.	118.

Warnings:
[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.

We can extract the parameters.

In [4]:

inter = results.params['Intercept']
slope = results.params['agepreg']
inter, slope

(6.830396973311047, 0.017453851471802877)

And the p-value of the slope estimate.

In [5]:

slope_pvalue = results.pvalues['agepreg']
slope_pvalue

5.722947107312786e-11

And the coefficient of determination.

In [6]:

results.rsquared

0.004738115474710369

The difference in birth weight between first babies and others.

In [7]:

diff_weight = firsts.totalwgt_lb.mean() - others.totalwgt_lb.mean()
diff_weight

-0.12476118453549034

The difference in age between mothers of first babies and others.

In [8]:

diff_age = firsts.agepreg.mean() - others.agepreg.mean()
diff_age

-3.5864347661500275

The age difference plausibly explains about half of the difference in weight.

In [9]:

slope * diff_age

-0.06259709972169267

Running a single regression with a categorical variable, isfirst:

In [10]:

live['isfirst'] = live.birthord == 1
formula = 'totalwgt_lb ~ isfirst'
results = smf.ols(formula, data=live).fit()
results.summary()

OLS Regression Results
Dep. Variable:	totalwgt_lb	R-squared:	0.002
Model:	OLS	Adj. R-squared:	0.002
Method:	Least Squares	F-statistic:	17.74
Date:	Thu, 28 Feb 2019	Prob (F-statistic):	2.55e-05
Time:	09:58:53	Log-Likelihood:	-15909.
No. Observations:	9038	AIC:	3.182e+04
Df Residuals:	9036	BIC:	3.184e+04
Df Model:	1
Covariance Type:	nonrobust

	coef	std err	t	P>\|t\|	[0.025	0.975]
Intercept	7.3259	0.021	356.007	0.000	7.286	7.366
isfirst[T.True]	-0.1248	0.030	-4.212	0.000	-0.183	-0.067

Omnibus:	988.919	Durbin-Watson:	1.613
Prob(Omnibus):	0.000	Jarque-Bera (JB):	2897.107
Skew:	-0.589	Prob(JB):	0.00
Kurtosis:	5.511	Cond. No.	2.58

Warnings:
[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.

Now finally running a multiple regression:

In [11]:

formula = 'totalwgt_lb ~ isfirst + agepreg'
results = smf.ols(formula, data=live).fit()
results.summary()

OLS Regression Results
Dep. Variable:	totalwgt_lb	R-squared:	0.005
Model:	OLS	Adj. R-squared:	0.005
Method:	Least Squares	F-statistic:	24.02
Date:	Thu, 28 Feb 2019	Prob (F-statistic):	3.95e-11
Time:	09:58:53	Log-Likelihood:	-15894.
No. Observations:	9038	AIC:	3.179e+04
Df Residuals:	9035	BIC:	3.182e+04
Df Model:	2
Covariance Type:	nonrobust

	coef	std err	t	P>\|t\|	[0.025	0.975]
Intercept	6.9142	0.078	89.073	0.000	6.762	7.066
isfirst[T.True]	-0.0698	0.031	-2.236	0.025	-0.131	-0.009
agepreg	0.0154	0.003	5.499	0.000	0.010	0.021

Omnibus:	1019.945	Durbin-Watson:	1.618
Prob(Omnibus):	0.000	Jarque-Bera (JB):	3063.682
Skew:	-0.599	Prob(JB):	0.00
Kurtosis:	5.588	Cond. No.	137.

Warnings:
[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.

As expected, when we control for mother's age, the apparent difference due to isfirst is cut in half.

If we add age squared, we can control for a quadratic relationship between age and weight.

In [12]:

live['agepreg2'] = live.agepreg**2
formula = 'totalwgt_lb ~ isfirst + agepreg + agepreg2'
results = smf.ols(formula, data=live).fit()
results.summary()

OLS Regression Results
Dep. Variable:	totalwgt_lb	R-squared:	0.007
Model:	OLS	Adj. R-squared:	0.007
Method:	Least Squares	F-statistic:	22.64
Date:	Thu, 28 Feb 2019	Prob (F-statistic):	1.35e-14
Time:	09:58:54	Log-Likelihood:	-15884.
No. Observations:	9038	AIC:	3.178e+04
Df Residuals:	9034	BIC:	3.181e+04
Df Model:	3
Covariance Type:	nonrobust

	coef	std err	t	P>\|t\|	[0.025	0.975]
Intercept	5.6923	0.286	19.937	0.000	5.133	6.252
isfirst[T.True]	-0.0504	0.031	-1.602	0.109	-0.112	0.011
agepreg	0.1124	0.022	5.113	0.000	0.069	0.155
agepreg2	-0.0018	0.000	-4.447	0.000	-0.003	-0.001

Omnibus:	1007.149	Durbin-Watson:	1.616
Prob(Omnibus):	0.000	Jarque-Bera (JB):	3003.343
Skew:	-0.594	Prob(JB):	0.00
Kurtosis:	5.562	Cond. No.	1.39e+04

Warnings:
[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.
[2] The condition number is large, 1.39e+04. This might indicate that there are
strong multicollinearity or other numerical problems.

When we do that, the apparent effect of isfirst gets even smaller, and is no longer statistically significant.

These results suggest that the apparent difference in weight between first babies and others might be explained by difference in mothers' ages, at least in part.

Data Mining

We can use join to combine variables from the preganancy and respondent tables.

In [13]:

import nsfg

live = live[live.prglngth>30]
resp = nsfg.ReadFemResp()
resp.index = resp.caseid
join = live.join(resp, on='caseid', rsuffix='_r')

And we can search for variables with explanatory power.

Because we don't clean most of the variables, we are probably missing some good ones.

In [14]:

import patsy

def GoMining(df):
    """Searches for variables that predict birth weight.

    df: DataFrame of pregnancy records

    returns: list of (rsquared, variable name) pairs
    """
    variables = []
    for name in df.columns:
        try:
            if df[name].var() < 1e-7:
                continue

            formula = 'totalwgt_lb ~ agepreg + ' + name
            formula = formula.encode('ascii')

            model = smf.ols(formula, data=df)
            if model.nobs < len(df)/2:
                continue

            results = model.fit()
        except (ValueError, TypeError, patsy.PatsyError):
            continue
        
        variables.append((results.rsquared, name))

    return variables

In [15]:

variables = GoMining(join)

The following functions report the variables with the highest values of $R^2$ .

In [16]:

import re

def ReadVariables():
    """Reads Stata dictionary files for NSFG data.

    returns: DataFrame that maps variables names to descriptions
    """
    vars1 = thinkstats2.ReadStataDct('2002FemPreg.dct').variables
    vars2 = thinkstats2.ReadStataDct('2002FemResp.dct').variables

    all_vars = vars1.append(vars2)
    all_vars.index = all_vars.name
    return all_vars

def MiningReport(variables, n=30):
    """Prints variables with the highest R^2.

    t: list of (R^2, variable name) pairs
    n: number of pairs to print
    """
    all_vars = ReadVariables()

    variables.sort(reverse=True)
    for r2, name in variables[:n]:
        key = re.sub('_r$', '', name)
        try:
            desc = all_vars.loc[key].desc
            if isinstance(desc, pd.Series):
                desc = desc[0]
            print(name, r2, desc)
        except (KeyError, IndexError):
            print(name, r2)

Some of the variables that do well are not useful for prediction because they are not known ahead of time.

In [17]:

MiningReport(variables)

Combining the variables that seem to have the most explanatory power.

In [18]:

formula = ('totalwgt_lb ~ agepreg + C(race) + babysex==1 + '
               'nbrnaliv>1 + paydu==1 + totincr')
results = smf.ols(formula, data=join).fit()
results.summary()

OLS Regression Results
Dep. Variable:	totalwgt_lb	R-squared:	0.060
Model:	OLS	Adj. R-squared:	0.059
Method:	Least Squares	F-statistic:	79.98
Date:	Thu, 28 Feb 2019	Prob (F-statistic):	4.86e-113
Time:	09:59:13	Log-Likelihood:	-14295.
No. Observations:	8781	AIC:	2.861e+04
Df Residuals:	8773	BIC:	2.866e+04
Df Model:	7
Covariance Type:	nonrobust

	coef	std err	t	P>\|t\|	[0.025	0.975]
Intercept	6.6303	0.065	102.223	0.000	6.503	6.757
C(race)[T.2]	0.3570	0.032	11.215	0.000	0.295	0.419
C(race)[T.3]	0.2665	0.051	5.175	0.000	0.166	0.367
babysex == 1[T.True]	0.2952	0.026	11.216	0.000	0.244	0.347
nbrnaliv > 1[T.True]	-1.3783	0.108	-12.771	0.000	-1.590	-1.167
paydu == 1[T.True]	0.1196	0.031	3.861	0.000	0.059	0.180
agepreg	0.0074	0.003	2.921	0.004	0.002	0.012
totincr	0.0122	0.004	3.110	0.002	0.005	0.020

Omnibus:	398.813	Durbin-Watson:	1.604
Prob(Omnibus):	0.000	Jarque-Bera (JB):	1388.362
Skew:	-0.037	Prob(JB):	3.32e-302
Kurtosis:	4.947	Cond. No.	221.

Warnings:
[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.

Logistic regression

Example: suppose we are trying to predict y using explanatory variables x1 and x2.

In [19]:

y = np.array([0, 1, 0, 1])
x1 = np.array([0, 0, 0, 1])
x2 = np.array([0, 1, 1, 1])

According to the logit model the log odds for the $i$ th element of $y$ is

$\log o = \beta_0 + \beta_1 x_1 + \beta_2 x_2$

So let's start with an arbitrary guess about the elements of $\beta$ :

In [20]:

beta = [-1.5, 2.8, 1.1]

Plugging in the model, we get log odds.

In [21]:

log_o = beta[0] + beta[1] * x1 + beta[2] * x2
log_o

array([-1.5, -0.4, -0.4,  2.4])

Which we can convert to odds.

In [22]:

o = np.exp(log_o)
o

array([ 0.22313016,  0.67032005,  0.67032005, 11.02317638])

And then convert to probabilities.

In [23]:

p = o / (o+1)
p

array([0.18242552, 0.40131234, 0.40131234, 0.9168273 ])

The likelihoods of the actual outcomes are $p$ where $y$ is 1 and $1-p$ where $y$ is 0.

In [24]:

likes = np.where(y, p, 1-p)
likes

array([0.81757448, 0.40131234, 0.59868766, 0.9168273 ])

The likelihood of $y$ given $\beta$ is the product of likes:

In [25]:

like = np.prod(likes)
like

0.1800933529673034

Logistic regression works by searching for the values in $\beta$ that maximize like.

Here's an example using variables in the NSFG respondent file to predict whether a baby will be a boy or a girl.

In [26]:

import first
live, firsts, others = first.MakeFrames()
live = live[live.prglngth>30]
live['boy'] = (live.babysex==1).astype(int)

The mother's age seems to have a small effect.

In [27]:

model = smf.logit('boy ~ agepreg', data=live)
results = model.fit()
results.summary()

Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3

Logit Regression Results
Dep. Variable:	boy	No. Observations:	8884
Model:	Logit	Df Residuals:	8882
Method:	MLE	Df Model:	1
Date:	Thu, 28 Feb 2019	Pseudo R-squ.:	6.144e-06
Time:	09:59:16	Log-Likelihood:	-6156.7
converged:	True	LL-Null:	-6156.8
		LLR p-value:	0.7833

	coef	std err	z	P>\|z\|	[0.025	0.975]
Intercept	0.0058	0.098	0.059	0.953	-0.185	0.197
agepreg	0.0010	0.004	0.275	0.783	-0.006	0.009

Here are the variables that seemed most promising.

In [28]:

formula = 'boy ~ agepreg + hpagelb + birthord + C(race)'
model = smf.logit(formula, data=live)
results = model.fit()
results.summary()

Optimization terminated successfully.
         Current function value: 0.692944
         Iterations 3

Logit Regression Results
Dep. Variable:	boy	No. Observations:	8782
Model:	Logit	Df Residuals:	8776
Method:	MLE	Df Model:	5
Date:	Thu, 28 Feb 2019	Pseudo R-squ.:	0.0001440
Time:	09:59:16	Log-Likelihood:	-6085.4
converged:	True	LL-Null:	-6086.3
		LLR p-value:	0.8822

	coef	std err	z	P>\|z\|	[0.025	0.975]
Intercept	-0.0301	0.104	-0.290	0.772	-0.234	0.173
C(race)[T.2]	-0.0224	0.051	-0.439	0.660	-0.122	0.077
C(race)[T.3]	-0.0005	0.083	-0.005	0.996	-0.163	0.162
agepreg	-0.0027	0.006	-0.484	0.629	-0.014	0.008
hpagelb	0.0047	0.004	1.112	0.266	-0.004	0.013
birthord	0.0050	0.022	0.227	0.821	-0.038	0.048

To make a prediction, we have to extract the exogenous and endogenous variables.

In [29]:

endog = pd.DataFrame(model.endog, columns=[model.endog_names])
exog = pd.DataFrame(model.exog, columns=model.exog_names)

The baseline prediction strategy is to guess "boy". In that case, we're right almost 51% of the time.

In [30]:

actual = endog['boy']
baseline = actual.mean()
baseline

0.507173764518333

If we use the previous model, we can compute the number of predictions we get right.

In [31]:

predict = (results.predict() >= 0.5)
true_pos = predict * actual
true_neg = (1 - predict) * (1 - actual)
sum(true_pos), sum(true_neg)

(3944.0, 548.0)

And the accuracy, which is slightly higher than the baseline.

In [32]:

acc = (sum(true_pos) + sum(true_neg)) / len(actual)
acc

0.5115007970849464

To make a prediction for an individual, we have to get their information into a DataFrame.

In [33]:

columns = ['agepreg', 'hpagelb', 'birthord', 'race']
new = pd.DataFrame([[35, 39, 3, 2]], columns=columns)
y = results.predict(new)
y

0    0.513091
dtype: float64

This person has a 51% chance of having a boy (according to the model).

Exercises

Exercise: Suppose one of your co-workers is expecting a baby and you are participating in an office pool to predict the date of birth. Assuming that bets are placed during the 30th week of pregnancy, what variables could you use to make the best prediction? You should limit yourself to variables that are known before the birth, and likely to be available to the people in the pool.

In [34]:

import first
live, firsts, others = first.MakeFrames()
live = live[live.prglngth>30]

The following are the only variables I found that have a statistically significant effect on pregnancy length.

In [35]:

import statsmodels.formula.api as smf
model = smf.ols('prglngth ~ birthord==1 + race==2 + nbrnaliv>1', data=live)
results = model.fit()
results.summary()

OLS Regression Results
Dep. Variable:	prglngth	R-squared:	0.011
Model:	OLS	Adj. R-squared:	0.011
Method:	Least Squares	F-statistic:	34.28
Date:	Thu, 28 Feb 2019	Prob (F-statistic):	5.09e-22
Time:	09:59:18	Log-Likelihood:	-18247.
No. Observations:	8884	AIC:	3.650e+04
Df Residuals:	8880	BIC:	3.653e+04
Df Model:	3
Covariance Type:	nonrobust

	coef	std err	t	P>\|t\|	[0.025	0.975]
Intercept	38.7617	0.039	1006.410	0.000	38.686	38.837
birthord == 1[T.True]	0.1015	0.040	2.528	0.011	0.023	0.180
race == 2[T.True]	0.1390	0.042	3.311	0.001	0.057	0.221
nbrnaliv > 1[T.True]	-1.4944	0.164	-9.086	0.000	-1.817	-1.172

Omnibus:	1587.470	Durbin-Watson:	1.619
Prob(Omnibus):	0.000	Jarque-Bera (JB):	6160.751
Skew:	-0.852	Prob(JB):	0.00
Kurtosis:	6.707	Cond. No.	10.9

Warnings:
[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.

Exercise: The Trivers-Willard hypothesis suggests that for many mammals the sex ratio depends on “maternal condition”; that is, factors like the mother’s age, size, health, and social status. See https://en.wikipedia.org/wiki/Trivers-Willard_hypothesis

Some studies have shown this effect among humans, but results are mixed. In this chapter we tested some variables related to these factors, but didn’t find any with a statistically significant effect on sex ratio.

As an exercise, use a data mining approach to test the other variables in the pregnancy and respondent files. Can you find any factors with a substantial effect?

In [36]:

# Solution

def GoMining(df):
    """Searches for variables that predict birth weight.

    df: DataFrame of pregnancy records

    returns: list of (rsquared, variable name) pairs
    """
    df['boy'] = (df.babysex==1).astype(int)
    variables = []
    for name in df.columns:
        try:
            if df[name].var() < 1e-7:
                continue

            formula='boy ~ agepreg + ' + name
            model = smf.logit(formula, data=df)
            nobs = len(model.endog)
            if nobs < len(df)/2:
                continue

            results = model.fit()
        except:
            continue

        variables.append((results.prsquared, name))

    return variables

variables = GoMining(join)

Optimization terminated successfully.
         Current function value: 0.692991
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692961
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692849
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692996
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692903
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692724
         Iterations 6
Optimization terminated successfully.
         Current function value: 0.693014
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692992
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693010
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692985
         Iterations 5
Optimization terminated successfully.
         Current function value: 0.692577
         Iterations 6
Optimization terminated successfully.
         Current function value: 0.686599
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693022
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692903
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693031
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692963
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692964
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692905
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693092
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693107
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692589
         Iterations 6
Optimization terminated successfully.
         Current function value: 0.693006
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693014
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692774
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692843
         Iterations 5
Optimization terminated successfully.
         Current function value: 0.692612
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692947
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692817
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692947
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693023
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692967
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692995
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692645
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692555
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693021
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693066
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692911
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692917
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693003
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692861
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692859
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692814
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692389
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692517
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692452
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692701
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692290
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692930
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693003
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692926
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693004
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692930
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692861
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692866
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692866
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692806
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692930
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692989
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692987
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693008
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692985
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693006
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692993
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692871
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692994
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692985
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693001
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692942
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692922
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692986
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692573
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692988
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692817
         Iterations 5
Optimization terminated successfully.
         Current function value: 0.692588
         Iterations 5
Optimization terminated successfully.
         Current function value: 0.692877
         Iterations 5
Optimization terminated successfully.
         Current function value: 0.692652
         Iterations 5
Optimization terminated successfully.
         Current function value: 0.692991
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692918
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692984
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692952
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693008
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692893
         Iterations 5
Warning: Maximum number of iterations has been exceeded.
         Current function value: 0.692776
         Iterations: 35
Optimization terminated successfully.
         Current function value: 0.692638
         Iterations 6
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3

/home/downey/anaconda3/envs/ThinkStats2/lib/python3.7/site-packages/statsmodels/base/model.py:508: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
  "Check mle_retvals", ConvergenceWarning)

Optimization terminated successfully.
         Current function value: 0.692838
         Iterations 5
Optimization terminated successfully.
         Current function value: 0.692971
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692985
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692971
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693003
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692998
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692973
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692973
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692810
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693014
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693003
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692995
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692979
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693014
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693003
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.686305
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693006
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692959
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692991
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692984
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692928
         Iterations 5
Optimization terminated successfully.
         Current function value: 0.692910
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693007
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693008
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692881
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692866
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692861
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692866
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692930
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692806
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692808
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692986
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693004
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692985
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693039
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692886
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692957
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692888
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692989
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693006
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692921
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692996
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692907
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692439
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692916
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692951
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693014
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692950
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693037
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693021
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693018
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692850
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693056
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692885
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693007
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692963
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692994
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692959
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692319
         Iterations 6
Optimization terminated successfully.
         Current function value: 0.692984
         Iterations 5
Optimization terminated successfully.
         Current function value: 0.692841
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693006
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693014
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692879
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693007
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692871
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692993
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692997
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693013
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692956
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693010
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692780
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693055
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693104
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693022
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693069
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693052
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692988
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693037
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693063
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693052
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693078
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693078
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692964
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692801
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693074
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692959
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692995
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693013
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693004
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692911
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693014
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692833
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693025
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692823
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692970
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692945
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693014
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693014
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693013
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693055
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692908
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692703
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692853
         Iterations 5
Optimization terminated successfully.
         Current function value: 0.692981
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693053
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692991
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692962
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692928
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693013
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693009
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692999
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692943
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693049
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692901
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692951
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692998
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692843
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692834
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692783
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692799
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692850
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692841
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693012
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692986
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693118
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693128
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692958
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692898
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693003
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693005
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692829
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692999
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692972
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692901
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692929
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692894
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692820
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692973
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692960
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692999
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693007
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692993
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692986
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693002
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692968
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692995
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693028
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692992
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693010
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692733
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692645
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693029
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692842
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692821
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692986
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693012
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692721
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693054
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692742
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693012
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693007
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693013
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692958
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692975
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692848
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692942
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692953
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692915
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692931
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692973
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692940
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692992
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693006
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693013
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693000
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693008
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693012
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693012
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693013
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693013
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693014
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692999
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692979
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693000
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692984
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692978
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692930
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692829
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692992
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692980
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692971
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693010
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693014
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693000
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693000
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692979
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692996
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693008
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693006
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692985
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693010
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693003
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692990
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693012
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693012
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692950
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693037
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693039
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693091
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693065
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692992
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692998
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692987
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692840
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693013
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692984
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693008
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693002
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693023
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693001
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692998
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693016
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693006
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692970
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693006
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692980
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692989
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693007
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693014
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692985
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693003
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692994
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692990
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693003
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692985
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692965
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692994
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692818
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692980
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692680
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692674
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692884
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692865
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693007
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692883
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692994
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692941
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692998
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692948
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693008
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692917
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692993
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692999
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692968
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692850
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692971
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692844
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692963
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692980
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692984
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692937
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692960
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692970
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693001
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692911
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693002
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692929
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692995
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692845
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692981
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692718
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692973
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692575
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692985
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692673
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692988
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692778
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693018
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692804
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693023
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692752
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693023
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692671
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692986
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692709
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693046
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692650
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693063
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692875
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693098
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692806
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693074
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693058
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693024
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692892
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692998
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693013
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692959
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692853
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692615
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692979
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693046
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692917
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692977
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692985
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693004
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692985
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692932
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692619
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692779
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692886
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692739
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692662
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692621
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692792
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692926
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692879
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692805
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693009
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692673
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692940
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692903
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693005
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693006
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692994
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692992
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692980
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693001
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692999
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693002
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692941
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692972
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692952
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692977
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692920
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692856
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692955
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692852
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692573
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692976
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692879
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693069
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692955
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692868
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693003
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693018
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693027
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692949
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693012
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692945
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692962
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693013
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693014
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692971
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693012
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692579
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692949
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692801
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692987
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693094
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693036
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692917
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693013
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692971
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693006
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692824
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692862
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692862
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692976
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693009
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693009
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692965
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692967
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692973
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693014
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693008
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692998
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692916
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693005
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692951
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692962
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692985
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692952
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692946
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692995
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693013
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692974
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692992
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693012
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692995
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692994
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692866
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692806
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692989
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692987
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692985
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693008
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693006
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693013
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693005
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693007
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693008
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692610
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692904
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692998
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692859
         Iterations 5
Optimization terminated successfully.
         Current function value: 0.692712
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692948
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692993
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692871
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692883
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692801
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692998
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693001
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692994
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692722
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692909
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692561
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692803
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692870
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692755
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692702
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693008
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692963
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692879
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692855
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692837
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692729
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693001
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692990
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692875
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692989
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692813
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692475
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693001
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692833
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692529
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693013
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692898
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692640
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692784
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692875
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692782
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692938
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692871
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692973
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692973
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692973
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692973
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692973
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692939
         Iterations 18
Optimization terminated successfully.
         Current function value: 0.693009
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693005
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692976
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693012
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692995
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692958
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693002
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692983
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693010
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693014
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692955
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692967
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693009
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692859
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693010
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692925
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692969
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692983
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692998
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692998
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692894
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692874
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692859
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693010
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693014
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692924
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692982
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692914
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692969
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692983
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692998
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692998
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692695
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693014
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692955
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692946
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693009
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692859
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693010
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692925
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692969
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692983
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692998
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692998
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692955
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692889
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692823
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692968
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692859
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692998
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692925
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692969
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692983
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692998
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692998
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692989
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692997
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693014
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693014
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693010
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692923
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692990
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692986
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692929
         Iterations 5
Optimization terminated successfully.
         Current function value: 0.693001
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693005
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693014
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692939
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692990
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692986
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692929
         Iterations 5
Optimization terminated successfully.
         Current function value: 0.692991
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692966
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692941
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693013
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692978
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692986
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692929
         Iterations 5
Optimization terminated successfully.
         Current function value: 0.692973
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692973
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692974
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692973
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692930
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692929
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692933
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693045
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693078
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692739
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693012
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692997
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692750
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692991
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692873
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692830
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692998
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693082
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692636
         Iterations 3
Warning: Maximum number of iterations has been exceeded.
         Current function value: 0.692709
         Iterations: 35
Optimization terminated successfully.
         Current function value: 0.692863
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692926
         Iterations 3

/home/downey/anaconda3/envs/ThinkStats2/lib/python3.7/site-packages/statsmodels/base/model.py:508: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
  "Check mle_retvals", ConvergenceWarning)

Optimization terminated successfully.
         Current function value: 0.692726
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692774
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692999
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692861
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692705
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692723
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692803
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692956
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692786
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693007
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692989
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692993
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693013
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692952
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693012
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692970
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693014
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693009
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 4
Warning: Maximum number of iterations has been exceeded.
         Current function value: 0.692862
         Iterations: 35
Optimization terminated successfully.
         Current function value: 0.692861
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693014
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3

/home/downey/anaconda3/envs/ThinkStats2/lib/python3.7/site-packages/statsmodels/base/model.py:508: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
  "Check mle_retvals", ConvergenceWarning)

Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693010
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692658
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692789
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693008
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692855
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692855
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693013
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692749
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692862
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692862
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692821
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692772
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692687
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692947
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692771
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692755
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693006
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692998
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693012
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692949
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692912
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692879
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692950
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692991
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692929
         Iterations 4
Warning: Maximum number of iterations has been exceeded.
         Current function value: 0.692960
         Iterations: 35
Optimization terminated successfully.
         Current function value: 0.693007
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692981
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692975
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692961
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693005
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692995
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693001
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693014
         Iterations 3

/home/downey/anaconda3/envs/ThinkStats2/lib/python3.7/site-packages/statsmodels/base/model.py:508: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
  "Check mle_retvals", ConvergenceWarning)

Optimization terminated successfully.
         Current function value: 0.692957
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692983
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692258
         Iterations 5
Warning: Maximum number of iterations has been exceeded.
         Current function value: 0.692696
         Iterations: 35
Optimization terminated successfully.
         Current function value: 0.692993
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693009
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692853
         Iterations 3

/home/downey/anaconda3/envs/ThinkStats2/lib/python3.7/site-packages/statsmodels/base/model.py:508: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
  "Check mle_retvals", ConvergenceWarning)

Optimization terminated successfully.
         Current function value: 0.692971
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692639
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692917
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692760
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693013
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692832
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693028
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693014
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692888
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692770
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692992
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692976
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692866
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692992
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692976
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692866
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693010
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692896
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692758
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693081
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692986
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692927
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692827
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693008
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692890
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692941
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692920
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692920
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692982
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692998
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692998
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692998
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693001
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693014
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692997
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693008
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693008
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692998
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693005
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692956
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692931
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692996
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692996
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692730
         Iterations 5
Optimization terminated successfully.
         Current function value: 0.693014
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692898
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692879
         Iterations 5
Optimization terminated successfully.
         Current function value: 0.692887
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692987
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692986
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692995
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693013
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692980
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692849
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692998
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692974
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692987
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692957
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693009
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693002
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693013
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692980
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692849
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692998
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692974
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692987
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692957
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693009
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693002
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692969
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693005
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692946
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692862
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692905
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692962
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692977
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693010
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693000
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692998
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693012
         Iterations 3
Warning: Maximum number of iterations has been exceeded.
         Current function value: 0.692939
         Iterations: 35
Optimization terminated successfully.
         Current function value: 0.693003
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692831
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692999
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693003
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692997
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692795
         Iterations 4

/home/downey/anaconda3/envs/ThinkStats2/lib/python3.7/site-packages/statsmodels/base/model.py:508: ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals
  "Check mle_retvals", ConvergenceWarning)

Optimization terminated successfully.
         Current function value: 0.692693
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692457
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692815
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693002
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692989
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693008
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692673
         Iterations 5
Optimization terminated successfully.
         Current function value: 0.692982
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692985
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692986
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692922
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692973
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693011
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693015
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692942
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692921
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693002
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693014
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693014
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692810
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.693003
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692995
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692979
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693014
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.693003
         Iterations 3
Optimization terminated successfully.
         Current function value: 0.692996
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692996
         Iterations 4
Optimization terminated successfully.
         Current function value: 0.692971
         Iterations 3

In [37]:

# Solution

#Here are the 30 variables that yield the highest pseudo-R^2 values.

MiningReport(variables)

totalwgt_lb 0.009696855253233383
birthwgt_lb 0.009274460080281988 BD-3 BIRTHWEIGHT IN POUNDS - 1ST BABY FROM THIS PREGNANCY
constat3 0.0010985419170438382 3RD PRIORITY CODE FOR CURRENT CONTRACEPTIVE STATUS
lbw1 0.0010519527860076705 LOW BIRTHWEIGHT - BABY 1
nplaced 0.001010368752280555 # OF R'S BIO CHILDREN SHE PLACED FOR ADOPTION (BASED ON BPA)
fmarout5 0.0009096579032891183 FORMAL MARITAL STATUS AT PREGNANCY OUTCOME
rmarout6 0.000818252143711895 INFORMAL MARITAL STATUS AT PREGNANCY OUTCOME - 6 CATEGORIES
infever 0.0008115919859909004 EVER USED INFERTILITY SERVICES OF ANY KIND
frsteatd 0.0007675331422082321 AGE (IN MOS) WHEN 1ST SUPPLEMENTED - 1ST FROM THIS PREG
splstwk1 0.0007334122339932581 IF-1 H/P DOING WHAT LAST WEEK (EMPLOYMENT STATUS) 1ST MENTION
pmarpreg 0.0007245809157658822 WHETHER PREGNANCY ENDED BEFORE R'S 1ST MARRIAGE (PREMARITALLY)
usefstp 0.0007122387685902787 EF-3 USE METHOD AT FIRST SEX WITH 1ST PARTNER IN PAST 12 MONTHS?
outcom02 0.0007015744602576479 OUTCOME OF PREGNANCY - 2ND
nummult34 0.0006606172426639745 NUMBER OF METHODS REPORTED IN (OCT 2001)
coh1dur 0.0006550110146368304 DURATION (IN MONTHS) OF R'S FIRST COHABITATION
brnout_r 0.0006438582787924307
brnout 0.0006438582787924307
bpa_bdscheck1 0.0006384992730465999 WHETHER 1ST LIVEBORN BABY FROM THIS PREGNANCY WAS BPA OR BDS
nummult41 0.000624852395994635 NUMBER OF METHODS REPORTED IN (MAY 2002)
agepreg_i 0.0006221097878729154 AGEPREG IMPUTATION FLAG
educmom 0.0005903653895802385 MOTHER'S (OR MOTHER-FIGURE'S) EDUCATION
marout03 0.0005883792792801268 FORMAL MARITAL STATUS WHEN PREGNANCY ENDED - 3RD
abort12 0.0005779076671296179 FA-3B RECEIVED ABORTION LAST 12 MONTHS
p1yhsage 0.0005631662500977797 CI-6 PARTNER'S AGE AT 1ST SEX-1ST REPORTED PARTNER IN LAST 12 MOS
numfirsm1 0.0005601782001567468 TOTAL NUMBER OF RESPONSES IN EB-1 FIRSMETH - PRESCRIPTION METHODS
lbw1_i 0.0005499451546340239 LBW1 IMPUTATION FLAG
agecon_i 0.0005303754891523571 AGECON IMPUTATION FLAG
mar1con1_i 0.0005217496825273837 MAR1CON1 IMPUTATION FLAG
nummult35 0.0005188253090530059 NUMBER OF METHODS REPORTED IN (NOV 2001)
nummult39 0.0005146121683266003 NUMBER OF METHODS REPORTED IN (MAR 2002)

In [38]:

# Solution

# Eliminating variables that are not known during pregnancy and 
# others that are fishy for various reasons, here's the best model I could find:

formula='boy ~ agepreg + fmarout5==5 + infever==1'
model = smf.logit(formula, data=join)
results = model.fit()
results.summary()

Optimization terminated successfully.
         Current function value: 0.691874
         Iterations 4

Logit Regression Results
Dep. Variable:	boy	No. Observations:	8884
Model:	Logit	Df Residuals:	8880
Method:	MLE	Df Model:	3
Date:	Thu, 28 Feb 2019	Pseudo R-squ.:	0.001653
Time:	09:59:57	Log-Likelihood:	-6146.6
converged:	True	LL-Null:	-6156.8
		LLR p-value:	0.0001432

	coef	std err	z	P>\|z\|	[0.025	0.975]
Intercept	-0.1805	0.118	-1.534	0.125	-0.411	0.050
fmarout5 == 5[T.True]	0.1582	0.049	3.217	0.001	0.062	0.255
infever == 1[T.True]	0.2194	0.065	3.374	0.001	0.092	0.347
agepreg	0.0050	0.004	1.172	0.241	-0.003	0.013

Exercise: If the quantity you want to predict is a count, you can use Poisson regression, which is implemented in StatsModels with a function called poisson. It works the same way as ols and logit. As an exercise, let’s use it to predict how many children a woman has born; in the NSFG dataset, this variable is called numbabes.

Suppose you meet a woman who is 35 years old, black, and a college graduate whose annual household income exceeds $75,000. How many children would you predict she has born?

In [39]:

# Solution

# I used a nonlinear model of age.

join.numbabes.replace([97], np.nan, inplace=True)
join['age2'] = join.age_r**2

In [40]:

# Solution

formula='numbabes ~ age_r + age2 + age3 + C(race) + totincr + educat'
formula='numbabes ~ age_r + age2 + C(race) + totincr + educat'
model = smf.poisson(formula, data=join)
results = model.fit()
results.summary()

Optimization terminated successfully.
         Current function value: 1.677002
         Iterations 7

Poisson Regression Results
Dep. Variable:	numbabes	No. Observations:	8884
Model:	Poisson	Df Residuals:	8877
Method:	MLE	Df Model:	6
Date:	Thu, 28 Feb 2019	Pseudo R-squ.:	0.03686
Time:	09:59:57	Log-Likelihood:	-14898.
converged:	True	LL-Null:	-15469.
		LLR p-value:	3.681e-243

	coef	std err	z	P>\|z\|	[0.025	0.975]
Intercept	-1.0324	0.169	-6.098	0.000	-1.364	-0.701
C(race)[T.2]	-0.1401	0.015	-9.479	0.000	-0.169	-0.111
C(race)[T.3]	-0.0991	0.025	-4.029	0.000	-0.147	-0.051
age_r	0.1556	0.010	15.006	0.000	0.135	0.176
age2	-0.0020	0.000	-13.102	0.000	-0.002	-0.002
totincr	-0.0187	0.002	-9.830	0.000	-0.022	-0.015
educat	-0.0471	0.003	-16.076	0.000	-0.053	-0.041

Now we can predict the number of children for a woman who is 35 years old, black, and a college graduate whose annual household income exceeds $75,000

In [41]:

# Solution

columns = ['age_r', 'age2', 'age3', 'race', 'totincr', 'educat']
new = pd.DataFrame([[35, 35**2, 35**3, 1, 14, 16]], columns=columns)
results.predict(new)

0    2.496802
dtype: float64

Exercise: If the quantity you want to predict is categorical, you can use multinomial logistic regression, which is implemented in StatsModels with a function called mnlogit. As an exercise, let’s use it to guess whether a woman is married, cohabitating, widowed, divorced, separated, or never married; in the NSFG dataset, marital status is encoded in a variable called rmarital.

Suppose you meet a woman who is 25 years old, white, and a high school graduate whose annual household income is about $45,000. What is the probability that she is married, cohabitating, etc?

In [42]:

# Solution

# Here's the best model I could find.

formula='rmarital ~ age_r + age2 + C(race) + totincr + educat'
model = smf.mnlogit(formula, data=join)
results = model.fit()
results.summary()

Optimization terminated successfully.
         Current function value: 1.084053
         Iterations 8

MNLogit Regression Results
Dep. Variable:	rmarital	No. Observations:	8884
Model:	MNLogit	Df Residuals:	8849
Method:	MLE	Df Model:	30
Date:	Thu, 28 Feb 2019	Pseudo R-squ.:	0.1682
Time:	09:59:58	Log-Likelihood:	-9630.7
converged:	True	LL-Null:	-11579.
		LLR p-value:	0.000

rmarital=2	coef	std err	z	P>\|z\|	[0.025	0.975]
Intercept	9.0156	0.805	11.199	0.000	7.438	10.593
C(race)[T.2]	-0.9237	0.089	-10.418	0.000	-1.097	-0.750
C(race)[T.3]	-0.6179	0.136	-4.536	0.000	-0.885	-0.351
age_r	-0.3635	0.051	-7.150	0.000	-0.463	-0.264
age2	0.0048	0.001	6.103	0.000	0.003	0.006
totincr	-0.1310	0.012	-11.337	0.000	-0.154	-0.108
educat	-0.1953	0.019	-10.424	0.000	-0.232	-0.159
rmarital=3	coef	std err	z	P>\|z\|	[0.025	0.975]
Intercept	2.9570	3.020	0.979	0.328	-2.963	8.877
C(race)[T.2]	-0.4411	0.237	-1.863	0.062	-0.905	0.023
C(race)[T.3]	0.0591	0.336	0.176	0.860	-0.600	0.718
age_r	-0.3177	0.177	-1.798	0.072	-0.664	0.029
age2	0.0064	0.003	2.528	0.011	0.001	0.011
totincr	-0.3258	0.032	-10.175	0.000	-0.389	-0.263
educat	-0.0991	0.048	-2.050	0.040	-0.194	-0.004
rmarital=4	coef	std err	z	P>\|z\|	[0.025	0.975]
Intercept	-3.5238	1.205	-2.924	0.003	-5.886	-1.162
C(race)[T.2]	-0.3213	0.093	-3.445	0.001	-0.504	-0.139
C(race)[T.3]	-0.7706	0.171	-4.509	0.000	-1.106	-0.436
age_r	0.1155	0.071	1.626	0.104	-0.024	0.255
age2	-0.0007	0.001	-0.701	0.483	-0.003	0.001
totincr	-0.2276	0.012	-19.621	0.000	-0.250	-0.205
educat	0.0667	0.017	3.995	0.000	0.034	0.099
rmarital=5	coef	std err	z	P>\|z\|	[0.025	0.975]
Intercept	-2.8963	1.305	-2.220	0.026	-5.453	-0.339
C(race)[T.2]	-1.0407	0.104	-10.038	0.000	-1.244	-0.837
C(race)[T.3]	-0.5661	0.156	-3.635	0.000	-0.871	-0.261
age_r	0.2411	0.079	3.038	0.002	0.086	0.397
age2	-0.0035	0.001	-2.977	0.003	-0.006	-0.001
totincr	-0.2932	0.015	-20.159	0.000	-0.322	-0.265
educat	-0.0174	0.021	-0.813	0.416	-0.059	0.025
rmarital=6	coef	std err	z	P>\|z\|	[0.025	0.975]
Intercept	8.0533	0.814	9.890	0.000	6.457	9.649
C(race)[T.2]	-2.1871	0.080	-27.211	0.000	-2.345	-2.030
C(race)[T.3]	-1.9611	0.138	-14.188	0.000	-2.232	-1.690
age_r	-0.2127	0.052	-4.122	0.000	-0.314	-0.112
age2	0.0019	0.001	2.321	0.020	0.000	0.003
totincr	-0.2945	0.012	-25.320	0.000	-0.317	-0.272
educat	-0.0742	0.018	-4.169	0.000	-0.109	-0.039

Make a prediction for a woman who is 25 years old, white, and a high school graduate whose annual household income is about $45,000.

In [43]:

# Solution

# This person has a 75% chance of being currently married, 
# a 13% chance of being "not married but living with opposite 
# sex partner", etc.

columns = ['age_r', 'age2', 'race', 'totincr', 'educat']
new = pd.DataFrame([[25, 25**2, 2, 11, 12]], columns=columns)
results.predict(new)

	0	1	2	3	4	5
0	0.750028	0.126397	0.001564	0.033403	0.021485	0.067122

In [ ]:

Examples and Exercises from Think Stats, 2nd Edition

Multiple regression

Data Mining

Logistic regression

Exercises

Product

Resources

Company