ANL252: Construct A Python Program To Store The Above Data In A NumPy Array: Python For Data Analytics Assignment, SUSS, Singapore, Singapore

Question 1

Given the following data which contain 20 rows and 3 columns: X1, X2, and Y.

(a) Construct a Python program to store the above data in a NumPy array.

(b) Suppose a linear regression was fitted on these data. The estimated model is
𝑌𝑌 = 2 − 0.5𝑋𝑋1 + 2.5𝑋𝑋2,

where Ŷ is the predicted (or expected) value of Y, X1 and X2 are the observed values of the columns X1 and X2. Design a Python program to compute Ŷ for every row of the array and store the results in a separate NumPy array as well.

where Y is the actual value stored in the original NumPy array and Ŷ is the predicted value of Y computed in (b). Use a Python program to compute ê for every row of the
array and store the results in a separate NumPy array.

(d) One of the main assumptions for linear regression is that the residuals must be normally distributed with zero mean and constant variance. Create a histogram of the tmatplotlib package. Adjust the parameters of the chart so that the ticks on the x-axis can be read clearly, a title is given to the chart, and both the axes are labeled. Eventually, discuss whether you agree that the normality assumption with zero means (the checking of constant variance is not required here) is valid based on this histogram.

(e) The constant variance assumption can be checked by a scatter plot in which the x-axis represents the values of the predicted values Ŷ and the y-axis represents the residuals ê. If the scatter plot does not show any pattern and the values of all the data points are more or less on the same level. Write a Python program to create such a scatter plot for checking the constant variance assumption. Adjust the parameters of the chart so that the ticks on both axes can be read clearly, a title is given to the chart, and both the axes are labeled. Eventually, discuss whether you agree that the constant variance
the assumption is valid based on this scatter plot.

The data of 19 students in a secondary school class are stored in a .csv data file named “class.csv”. Gender, age, height, and weight are the features of the students that have been recorded. Employ your Python programming skills to carry out the tasks below.

Include your Python program code in the answers and show them in the “Consolas” or “Courier New” fonts (size 12). Make a screenshot of the program output if required.
(a) Prepare a Python program to read in and to convert the data from a .csv text file into a pandas DataFrame. Check the existing missing data in the dataset and adjust the reader accordingly.

(b) The data should be sorted by the age of the students in descending order and then by their gender in ascending order. Employ the corresponding Python syntax to carry
out this task.

(c) Identify the location of the missing values in the DataFrame. Report the rows and columns where the missing data are found.

(d) If missing values are detected in the DataFrame, they have to be treated according to the columns they belong to. Here are the instructions of how we should deal with the missing data in each column:

Gender – replace missing values by the gender with the highest frequency

Age – replace missing values by the median age

Height – replace missing values by the mean height

Weight – replace missing values by the mean weight
Design your own Python program to determine the corresponding statistics for each column to replace the missing values in it.