How is the penalty level of the final selected model in (e) determined?

There are 20 points in this assignment. The minimum increment is 0.5 point. Solve them and fill the answers in the blank space.

 

  1. 1. Continue question 2 of homework assignment 2.csv contains the data with the selected predictors (no missing values but the predictors have not been standardized or coded). Allocate 20% of the data to the test partition. Choose VACATION_No, SW_No, SLOT_Free, GATE_Constrained as the redundant dummies to drop.
  2. Use the remaining 80% of the data as the training partition to build two LASSO regression models with the following pre-specified penalty levels respectively: alpha=0.01 and alpha=0.1. Organize the estimated coefficients and the intercept of both models into the following tables respectively. (2 points)
Pre-specified penalty level: alpha=0.01
Predictor Estimated coefficient Predictor Estimated coefficient
VACATION_Yes   E_POP  
SW_Yes   SLOT_Controlled  
S_INCOME   GATE_Free  
E_INCOME   DISTANCE  
S_POP   Intercept  

 

Pre-specified penalty level: alpha=0.1
Predictor Estimated coefficient Predictor Estimated coefficient
VACATION_Yes   E_POP  
SW_Yes   SLOT_Controlled  
S_INCOME   GATE_Free  
E_INCOME   DISTANCE  
S_POP   Intercept  

 

  1. Do you see any zero estimated coefficient in either of the models in (a)? For each model, if you do not, please explain why there is no zero estimated coefficient. If you do, please explain why those estimated coefficients are zero. (2 points)
Model Yes/No Why
Model with alpha=0.01    
Model with alpha=0.1    

 

  1. Conduct the necessary analysis to rank the predictors based on their importance. If one predictor is the n-th most important one, write number n in its column Rank of Importance. (1 point)
Predictor Rank of importance Predictor Rank of importance
VACATION_Yes   E_POP  
SW_Yes   SLOT_Controlled  
S_INCOME   GATE_Free  
E_INCOME   DISTANCE  
S_POP    

 

  1. Find the numerical predictors among all the predictors and rank them based on how strongly they are correlated with the dependent variable (note that the correlation can be either positive or negative). For example, 1 means the most strongly correlated and 2 means the second most strongly correlated. Compare the ranking based on the importance in (c) and the ranking based on the degree of correlation in (d). Are two rankings consistent with each other within the numerical predictors? (2 points)
Numerical predictor Rank of correlation Numerical predictor Rank of correlation
       
       
       
       
       

 

  1. Use cross-validation to select the model based on predictive performance. Set the number of folds for cross-validation to 3. Organize the estimated coefficients and the intercept as well as the corresponding penalty level alpha of the final selected model in the following table. (2 points)
Predictor Estimated coefficient Predictor Estimated coefficient
VACATION_Yes   E_POP  
SW_Yes   SLOT_Controlled  
S_INCOME   GATE_Free  
E_INCOME   DISTANCE  
S_POP   Intercept  
Penalty level alpha:

 

  1. How is the penalty level of the final selected model in (e) determined? (1 point)
  2. Fill the answers in the following table. Among all three models, which one has the lowest ASE over the test partition? Why does that model have the lowest ASE over the test partition? (2 points)
Model ASE over the test partition
Model in (e)  
Model with alpha=0.01 in (a)  
Model with alpha=0.1 in (a)  

 

  1. Use the final selected model in (e) to predict the average fare on a route with the following characteristics. The prediction outcome should be in its unstandardized value. Show the calculation and explain your answer. (4 points)

VACATION = No, SW = No, S_INCOME = $28,760, E_INCOME = $27,664, S_POP = 4,557,004, E_POP = 3,195,503, SLOT = Free, GATE = Free, DISTANCE = 1976 miles.

  1. Southwest Airline announces that it will serve the same route. Based on the final selected model in (e), do you expect the average ticket fare of the major airline for that route (not counting Southwest) to increase or decrease? How much do you expect the average fare to increase or decrease? Show your calculations. (2 points)
  2. Now we change our way to code the categorical variables by choosing a different redundant dummy to drop. Specifically, instead of dropping SLOT_Free, we drop SLOT_Controlled. Fill the answers in the following table. (2 points)
Questions Yes/No
Will it affect the estimated coefficients or the intercept of the final selected model?  
Will it change the penalty level of the final selected model?  
Will it affect the average squared error over the test partition?  
Will it affect the predicted value for the route in (h)?  

 

Submit your Python code with the filename [DM2022] HW3_YOURFULLNAME.ipynb

Calculate the price of your order

550 words
We'll send you the first draft for approval by September 11, 2018 at 10:52 AM
Total price:
$26
The price is based on these factors:
Academic level
Number of pages
Urgency
Basic features
  • Free title page and bibliography
  • Unlimited revisions
  • Plagiarism-free guarantee
  • Money-back guarantee
  • 24/7 support
On-demand options
  • Writer’s samples
  • Part-by-part delivery
  • Overnight delivery
  • Copies of used sources
  • Expert Proofreading
Paper format
  • 275 words per page
  • 12 pt Arial/Times New Roman
  • Double line spacing
  • Any citation style (APA, MLA, Chicago/Turabian, Harvard)

Our guarantees

Delivering a high-quality product at a reasonable price is not enough anymore.
That’s why we have developed 5 beneficial guarantees that will make your experience with our service enjoyable, easy, and safe.

Money-back guarantee

You have to be 100% sure of the quality of your product to give a money-back guarantee. This describes us perfectly. Make sure that this guarantee is totally transparent.

Read more

Zero-plagiarism guarantee

Each paper is composed from scratch, according to your instructions. It is then checked by our plagiarism-detection software. There is no gap where plagiarism could squeeze in.

Read more

Free-revision policy

Thanks to our free revisions, there is no way for you to be unsatisfied. We will work on your paper until you are completely happy with the result.

Read more

Privacy policy

Your email is safe, as we store it according to international data protection rules. Your bank details are secure, as we use only reliable payment systems.

Read more

Fair-cooperation guarantee

By sending us your money, you buy the service we provide. Check out our terms and conditions if you prefer business talks to be laid out in official language.

Read more
error: Content is protected !!