CoCalc Shared FilesCDS-102 / Lab Week 10 - Statistical inference of North Carolina birth records / CDS-102 Lab Week 10 Workbook.ipynbOpen in CoCalc with one click!

1

In [2]:

# Run this code block to load the Tidyverse package .libPaths(new = "~/Rlibs") library(tidyverse) # Load inference() function from file "inference.RData" load("inference.RData")

2

Loading tidyverse: ggplot2
Loading tidyverse: tibble
Loading tidyverse: tidyr
Loading tidyverse: readr
Loading tidyverse: purrr
Loading tidyverse: dplyr
Conflicts with tidy packages ---------------------------------------------------
filter(): dplyr, stats
lag(): dplyr, stats

In [3]:

nc.data<-read.csv("nc.csv")

3

The code below generates a full summary statistics report by running the summary() function on the dataset.

4

In [4]:

summary(nc.data)

5

fage mage mature weeks premie
Min. :14.00 Min. :13 mature mom :133 Min. :20.00 full term:846
1st Qu.:25.00 1st Qu.:22 younger mom:867 1st Qu.:37.00 premie :152
Median :30.00 Median :27 Median :39.00 NA's : 2
Mean :30.26 Mean :27 Mean :38.33
3rd Qu.:35.00 3rd Qu.:32 3rd Qu.:40.00
Max. :55.00 Max. :50 Max. :45.00
NA's :171 NA's :2
visits marital gained weight
Min. : 0.0 married :386 Min. : 0.00 Min. : 1.000
1st Qu.:10.0 not married:613 1st Qu.:20.00 1st Qu.: 6.380
Median :12.0 NA's : 1 Median :30.00 Median : 7.310
Mean :12.1 Mean :30.33 Mean : 7.101
3rd Qu.:15.0 3rd Qu.:38.00 3rd Qu.: 8.060
Max. :30.0 Max. :85.00 Max. :11.750
NA's :9 NA's :27
lowbirthweight gender habit whitemom
low :111 female:503 nonsmoker:873 not white:284
not low:889 male :497 smoker :126 white :714
NA's : 1 NA's : 2

The code below filters out entries in the data set with 'NA' values for the 'habit' variable.

6

In [5]:

nc.data<-filter(nc.data,habit!='NA') nc.data.nonsmoker<-filter(nc.data,habit=='nonsmoker') nc.data.smoker<-filter(nc.data,habit=='smoker')

7

The code below plots a (frequency) histogram of the habit and weight variables in the dataset. The code plots both histograms on the same chart using the geom_histogram() function including inputs position="identity" and alpha=0.3 so that the plot is readable.

The code saves the histogram to a .png file using the ggsave() function.

8

In [6]:

options(repr.plot.width = 9, repr.plot.height = 4) both_hist<-ggplot(nc.data) + geom_histogram(mapping = aes(x = weight, y = ..density.., fill = habit), binwidth = 1, position = "identity", alpha = 0.3) ggsave("both_hist.png", plot = both_hist, device="png", scale=1, width=5, height=4) both_hist

9

The code below uses the summary() function on each of the 'habit' subsets to calculate the mean and standard deviation of birth weights for the group of non-smokers and the group of smokers. The smoking group does have a slightly lower mean birth weight. This may suggest a correlation between smoking and birth weight. We will find if this difference is statistically significant in later steps.

10

In [7]:

stat.table.nonsmoker<-summarise(nc.data.nonsmoker, mean=mean(weight), sd=sd(weight)) stat.table.nonsmoker stat.table.smoker<-summarise(nc.data.smoker, mean=mean(weight), sd=sd(weight)) stat.table.smoker

11

mean | sd |
---|---|

7.144273 | 1.518681 |

mean | sd |
---|---|

6.82873 | 1.38618 |

For this task, the null and alternative hypotheses are specified for testing if the average weights of babies born to smoking and non-smoking mothers are different. They are shown below.

12

H0: There is no difference in the mean birth weights between babies born to smokers and babies born to non-smokers.

Ha: There is a difference in the mean birth weights between babies born to smokers and babies born to non-smokers.

13

The code below uses the inference() function to test the hypotheses specified in the previous step. The inference() function simplifies the hypothesis testing procedure, effectively hiding the manual computational work needed to perform them. For the code below, the mean birth weight is estimated for both smoking and non-smoking mothers and the type of test is specified as a 'hypothesis test'. The test is based on the Central Limit Theorem which is specified by the parameter 'method' and setting it to 'theoretical'.

14

In [8]:

inference(y = weight, x = habit, data = nc.data, statistic = "mean", type = "ht", null = 0, alternative = "twosided", method = "theoretical", order = c("smoker", "nonsmoker"))

15

Response variable: numerical
Explanatory variable: categorical (2 levels)
n_smoker = 126, y_bar_smoker = 6.8287, s_smoker = 1.3862
n_nonsmoker = 873, y_bar_nonsmoker = 7.1443, s_nonsmoker = 1.5187
H0: mu_smoker = mu_nonsmoker
HA: mu_smoker != mu_nonsmoker
t = -2.359, df = 125
p_value = 0.0199

The results of the hypothesis test produce a p-value that is less than the significance level which is .05, meaning that the null hypothesis, that there is no statistically significant difference between smokers and nonsmokers, can be rejected.

16

The code below uses the inference() function again but this time the 'type' parameter is changed to 'ci' to produce a confidence interval. The 'null' and 'alternative' inputs are removed becaue the confidence interval will indicate a non-rejection of the null hypothesis if the value 0 is included within the confidence interval (this suggests that there is literally zero difference between the true means of both populations). The confidence interval indicates the range of values in which the difference between the true means of both populations lies.

17

In [9]:

inference(y = weight, x = habit, data = nc.data, statistic = "mean", type = "ci", method = "theoretical", order = c("smoker", "nonsmoker"))

18

Response variable: numerical, Explanatory variable: categorical (2 levels)
n_smoker = 126, y_bar_smoker = 6.8287, s_smoker = 1.3862
n_nonsmoker = 873, y_bar_nonsmoker = 7.1443, s_nonsmoker = 1.5187
95% CI (smoker - nonsmoker): (-0.5803 , -0.0508)

In this example, both values of the confidence interval are negative, indicating that difference of the second mean from the first mean is negative, meaning that the second population (nonsmokers) have a higher mean birth weight.

19

The code below calculates a 95% confidence interval for the average length of pregnancies (weeks variable). Rather than finding a difference between true means of true populations, this confidence interval just finds the range of values in which the true mean of the variable for the population of interest lies (in this case it is the true mean of the length of pregnancy in weeks for the entire population of women in North Carolina).

20

In [10]:

inference(y = weeks, data = nc.data, statistic = "mean", type = "ci", method = "theoretical", order = c("smoker", "nonsmoker"))

21

Single numerical variable
n = 998, y-bar = 38.3347, s = 2.9316
95% CI: (38.1526 , 38.5168)

The results indicate that the true mean of the length of pregnancy for women in North Carolina is somewhere between 38.1 weeks and 38.5 weeks.

22

The code below calculates a new confidence interval for the same parameter at the 90% confidence level. The confidence level is changed by adding a new argument to the function: conf_level = 0.90.

23

In [11]:

inference(y = weeks, data = nc.data, statistic = "mean", type = "ci", method = "theoretical", order = c("smoker", "nonsmoker"),conf_level=.90)

24

Single numerical variable
n = 998, y-bar = 38.3347, s = 2.9316
90% CI: (38.1819 , 38.4874)

This change in confidence level narrows the range of values containing the true mean of the average length of pregnancy for women in North Carolina.

25

The code below tests the hypothesis that there is no difference in mean number of hospital visits for pregnant married women and pregnant unmarried women. The code below uses the inference() function to perform a hypothesis test and a confidence interval using an Î± level of .05. This defines both the p-value threshold for statistical significance and the size of the confidence interval.

26

27

In [12]:

inference(y = visits, x = marital, data = nc.data, statistic = "mean", type = "ht", null = 0, alternative = "twosided", method = "theoretical", order = c("married", "unmarried"))

28

Response variable: numerical
Explanatory variable: categorical (2 levels)
n_married = 380, y_bar_married = 10.9553, s_married = 4.2408
n_not married = 611, y_bar_not married = 12.82, s_not married = 3.5883
H0: mu_married = mu_not married
HA: mu_married != mu_not married
t = -7.1298, df = 379
p_value = < 0.0001

In [13]:

inference(y = visits, x = marital, data = nc.data, statistic = "mean", type = "ci", method = "theoretical", order = c("married", "unmarried"))

29

Response variable: numerical, Explanatory variable: categorical (2 levels)
n_married = 380, y_bar_married = 10.9553, s_married = 4.2408
n_not married = 611, y_bar_not married = 12.82, s_not married = 3.5883
95% CI (married - not married): (-2.3789 , -1.3505)

In [ ]:

30