| Hosted by CoCalc | Download
Kernel: R (SageMath)

Homework 2

CDS-101 (Spring 2017)

Name: Muaad Kholi

# Set the location for R packages
.libPaths(new = "~/Rlibs")
# Load the Tidyverse packages
library(tidyverse)
Loading tidyverse: ggplot2 Loading tidyverse: tibble Loading tidyverse: tidyr Loading tidyverse: readr Loading tidyverse: purrr Loading tidyverse: dplyr Conflicts with tidy packages --------------------------------------------------- filter(): dplyr, stats lag(): dplyr, stats 
# Load the nycflights dataset
flights <- read_csv("nycflights13.csv")
Parsed with column specification: cols( year = col_integer(), month = col_integer(), day = col_integer(), dep_time = col_integer(), sched_dep_time = col_integer(), dep_delay = col_double(), arr_time = col_integer(), sched_arr_time = col_integer(), arr_delay = col_double(), carrier = col_character(), flight = col_integer(), tailnum = col_character(), origin = col_character(), dest = col_character(), air_time = col_double(), distance = col_double(), hour = col_double(), minute = col_double(), time_hour = col_datetime(format = "") )

Question 1:

Vector of numbers between 1 and 100 that are indivisible by 2, 3 and 7: (Spring 2017)

For this task I created three variables:

  1. Variable v1 that contained a vector of all the numbers between 1 and 100.

  2. The second variable "indx" contained all the numbers between 1 and 100 that are divisible by 2, or divisible by 3, or divisible by 7.

  3. The third variable, "others", had v1 without the indx variable, leaving us with all the numbers between 1 and 100 that are indivisible by 2, 3 or 7.

Code looks as follows:

v1 <- 1:100
 indx <- v1 %% 3 & v1 %% 2 & v1 %% 7
 (others <- v1[!!indx])

The code below cretes variable "v1" which is a vector from 1 to 100. Then it creates variable "indx" and assigns it the values of MOD(v1, 3) AND MOD(v1, 2) AND MODA(v1, 7) The code then creates variable "others and assigns it all the values of variable v1, EXCLUDING the values that are in the indx variable.

v1 <- 1:100
indx <- v1 %% 3 & v1 %% 2 & v1 %% 7
(others <- v1[!!indx])
[1] 1 5 11 13 17 19 23 25 29 31 37 41 43 47 53 55 59 61 65 67 71 73 79 83 85 [26] 89 95 97

Question 2:

Find flights that:

  1. Flew to Houston (IAH or HOU)

  2. Were operated by United, American, or Delta.

  3. Were delayed by at least an hour, but made up over 30 minutes in flight.

  4. Departed between midnight and 6am (inclusive).

Answer to Question 2.1.

I'm creating a variable "Houston_Flights" and assigning the subset of data to it.

Houston_Flights <- (filter(flights, dest == "IAH"| dest == "HOU"))

The next line, I printed the variable to confirm the code.

print(Houston_Flights)
# A tibble: 9,313 × 19 year month day dep_time sched_dep_time dep_delay arr_time sched_arr_time <int> <int> <int> <int> <int> <dbl> <int> <int> 1 2013 1 1 517 515 2 830 819 2 2013 1 1 533 529 4 850 830 3 2013 1 1 623 627 -4 933 932 4 2013 1 1 728 732 -4 1041 1038 5 2013 1 1 739 739 0 1104 1038 6 2013 1 1 908 908 0 1228 1219 7 2013 1 1 1028 1026 2 1350 1339 8 2013 1 1 1044 1045 -1 1352 1351 9 2013 1 1 1114 900 134 1447 1222 10 2013 1 1 1205 1200 5 1503 1505 # ... with 9,303 more rows, and 11 more variables: arr_delay <dbl>, # carrier <chr>, flight <int>, tailnum <chr>, origin <chr>, dest <chr>, # air_time <dbl>, distance <dbl>, hour <dbl>, minute <dbl>, time_hour <dttm>

Answer to Question 2.2.

I'm creating a variable "UAD" and assigning the subset of data to it that has flights by united, American, and Delta.

The code below creates variable UAD, and assigns it a a filtered set of the dataset with all the records that have "AA" OR "UA" OR "DL" in teh carrier column.

UAD <- filter(flights, carrier == "AA" | carrier == "UA" | carrier == "DL")

Answer to Question 2.3.

I'm creating a variable "Del_but_made" and assigning the subset of data to it of flights that were delayed by at least an hour but made over 30 minutes in flight.

The code below creates variable "Del_but_made" and assigns it a filtered version of the dataset that contains flights with a dep_delay value bigger than 59 (delayed by at least 60 minutes) AND arr_delay value larger than 30 minutes.

Del_but_made <- filter(flights, dep_delay > 59 & arr_delay > 30)

Answer to Question 2.4

To answer this question, I'm creating a variable " mid_6" with flights that departed between midnight and 6am (inclusive).

The code below creates variable "mid_6" and assigns it a subset of the data that is filtered with only flights that have a departure time of 600 or below (flights departed between midnight and 600 in the morning.

mid_6 <- filter(flights, dep_time <= 600)

Question 3:

Currently dep_time and sched_dep_time are convenient to look at, but hard to compute with because they’re not really continuous numbers. Convert them to the more convenient representation of number of minutes since midnight.

The code below create variable "dep_time_in_minutes" and assigns it a "transmuted" flights dataset with a column "dep_time" that contains the value of "hour" multiplied by 60 and added to the value of minute.

DTIM <- mutate(flights, min_dep_time = dep_time*60)

Times_Parsed <- mutate(flights,
  dep_time,
  hour = dep_time %/% 100,
  minute = dep_time %% 100
)

Dep_time_in_minutes <- transmute(flights, dep_time = hour*60 + minute)

The code below creates variable "times_Parsed" which contains the dataset with two new columns; sched_hour and sched_minute.

Times_Parsed <- mutate(flights,
  sched_dep_time,
  sched_hour = sched_dep_time %/% 100,
  sched_minute = sched_dep_time %% 100
)

The code below creates variable "sched_dep_time_in_minutes" and assigns it a dataset with mutated column "sched_dep_time_minutes" which contains the value of "sched_hour" multiplied by 60, and added to the value of "sched_minute" which calculate the scheudled departure time in minutes total since midnight.

sched_dep_time_in_minutes <- mutate(Times_Parsed, sched_dep_time_minutes = sched_hour*60 + sched_minute)

Question #4:

Compare air_time with the difference arr_time − dep_time. What did you expect to see? What do you need to do to fix it? Implement the fix.

Answer to question 4:

After examining the data, 3 issues were identified with the dep_time - arr_time discrepancy when compared with airtime:

  1. when substracting the dep_time from the arr_time, R is treatign these values as if they were number, and not time. To solve this issue, the columns must be formated in time format, or converted to minutes before they are substracted.

  2. The departure time is clocked at the city of departure, while the arrival time is clocked at the city of arrival, which in many cases resides in different time zones. The solution for this issue is to convert both times to a standard time, such as Zulu time, based on thier respective time zones.

  3. The third issue is that the arrival and departure time may include time on the runway, as long as the plane is moving using its own engines for the purpose of departing or embarking. Airtime is the time in the air only which only counts the time from when the wheels leave the runway and until they touch the runway again when landing. This can only be solved by using different datapoints.

Question 5:

Consider the number of canceled flights per day in the dataset.

  1. Review the dataset and determine what would be reasonable definition of a flight cancella- tion. Filter the dataset so that only the canceled flights remain. Note that the Boolean test (is.na(dep_delay) | is.na(arr_delay)) is not the best possible definition.

  2. Calculate the number of canceled flights per day using the filtered dataset. Is there a pattern? Is the proportion of canceled flights related to the average delay?

Answer for question 5.1:

After reviewing the data, the following citeria woul best fit cancelled flights: flights with NA for departure time OR flights with NA for arrival time OR flights with NA for airtime. Dataset filtered below.

Answer for question 5.2:

To answer this quesiton, I calculated the MEAN of departure delay time for calncelled flights, and the MEAN departure delay for all the flights in the dataset (calculations below.) The calculation indicates that calcelled flights's mean delay time was 36 minutes, while teh entire dataset had a mean of 12.6, less than half that of cancelled flights, which means that flight cancellation is related to the average dealy.

The code below creates variable "cancellations" and assigns it to a filtered dataset that includes only records with NA as thier departure time OR NA forarrival time OR NA for air time.

cancellations <- (filter(flights, is.na(dep_time) | is.na(arr_time)| is.na(air_time)))

The code below calculates and displays the mean of the dep_delay column (excluding NA values) within the "cancellations" variable, which contains only cancelled flights.

summarise(cancellations, delay = mean(dep_delay, na.rm = TRUE))
delay 1 36.01702

The code below calculates and displays the mean of the dep_delay column (excluding NA values) within the entire dataset

summarise(flights, delay = mean(dep_delay, na.rm = TRUE))
delay 1 12.63907

Question #6:

What time of day should you fly if you want to avoid delays as much as possible?

Answer to question #6:

To answer this quesiton, I created a variable (by_time) thich contained the dataset grouped by scheduled departure time. Then I summarized that the data within the new variable by the mean of its departure delay value. From the results, the best time to schedule a flight is at 548, when the mean delay value is closest to 0, as low as 0.07692308.

The code below creates variable "by_time" and assigns it a table with all the unique values for "sched_dep_time" with the mean value for their respective "dep_delay".

by_time <- group_by(flights, sched_dep_time)
summarise(by_time, delay = mean(dep_delay, na.rm = TRUE))
sched_dep_time delay 1 106 NaN 2 500 -3.13823529 3 501 -3.00000000 4 505 2.00000000 5 510 1.00000000 6 515 1.53140097 7 516 -5.00000000 8 517 5.11111111 9 520 -2.57142857 10 525 -0.70270270 11 527 10.00000000 12 528 2.00000000 13 529 4.42857143 14 530 2.08823529 15 534 2.80000000 16 535 -2.00000000 17 536 0.89655172 18 537 6.00000000 19 538 -1.25000000 20 539 -2.50000000 21 540 1.24324324 22 545 -0.34269663 23 548 0.07692308 24 549 10.30000000 25 550 1.73504274 26 551 9.10526316 27 555 -2.00000000 28 557 -1.00000000 29 558 50.00000000 30 559 9.38372093 ⋮ ⋮ ⋮ 992 2207 110.666667 993 2208 -3.000000 994 2210 21.000000 995 2215 21.782609 996 2219 18.846154 997 2220 20.602410 998 2225 28.492754 999 2227 44.000000 1000 2229 10.080000 1001 2230 15.363636 1002 2231 38.229508 1003 2241 17.500000 1004 2245 22.251741 1005 2246 4.608696 1006 2249 19.947368 1007 2250 16.846966 1008 2251 17.244444 1009 2253 11.684211 1010 2255 9.370277 1011 2258 2.058824 1012 2300 27.285714 1013 2305 33.622951 1014 2315 30.000000 1015 2330 18.714286 1016 2339 80.000000 1017 2345 17.000000 1018 2352 -0.187500 1019 2355 9.849315 1020 2358 13.613636 1021 2359 12.698529