You are just hired as a data analyst (Note1) in the newly formed analytics department of RiiiD, which is a leading AI startup company specializing in providing learning resources and adaptive practice

You are just hired as a data analyst (Note1) in the newly formed analytics department of RiiiD, which is a leading AI startup company specializing in providing learning resources and adaptive practices to English learners in South Korea. On the first day of your job, you are invited to attend a meeting with the business operation team in which you are briefed on the company’s platform and then you are handed over a dataset (‘EdNet’)  that was collected from this platform over the last two years from over 700K users . This dataset logged detailed user activities while they were interacting with systems.  Intrigued by the sheer amount of data collected, your manager is interested in how the analytics department can help to support the company’s missions to reimagine the learner’s experience using AI/ML/data analytics techniques. You are asked to spend some time to look into the data and prepare a brief to your manager. Specifically, your manager is looking for answers to the following questions.

1. (25 points) Who are the users?  To answer this question, you will need to compile a user profile table (Table 1) with information about users including

Create a few plots to illustrate the information in the tables. Feel free to choose the appropriate type of plots you think is appropriate.

2.               (25 points) What are the questions/items? To answer this question, you will need to compile a question profile table (Table 2)  with information including

a.               Question ID

b.               Question Type

c.                Number of times being practiced

d.               Number of times answered correctly

Create a few plots to illustrate the information in the tables. Feel free to choose the appropriate type of plots you think is appropriate.

3.               (10 points) Design a modified metric of “accuracy” to fairly describe users’ ability by taking into account the difficulty level as derived from Table 2. Describe the procedure to compute the metrics.  Be sure to be specific so that interns can use your pseudo code to implement the metrics without much trouble.

BONUS (10 points), implement the proposed metrics and plot a histogram of the metrics across all users (or subset of users of your choices).

4.               (30 points) Pick a user with a reasonable amount of activity (you will define the “reasonableness” and specify the selection criteria) and create a dashboard that consists of a series of plots to tell a story of this user’s activity patterns. For inspiration, you may look at the user dashboard for fitness tracker such as Fitbit.

5.               (10 points) Propose two tasks for your interns to work on. The first task is of unsupervised/descriptive type and a second one is of supervised/predictive task. Please provide clear specification of the tasks so that your interns can start to work right away. You should try to propose tasks that are not attempted in the existing work with this dataset.

BONUS (10 points): propose a reinforcement learning task

Deliverable:

• A google slide deck summarizing the above findings in the format of plots or tables or other contents as requested by your manager. Please label clearly on the slide which question you are answering. There are no lower/upper limits of the number of slides. Always keep the message concise and effective and keep your audience in mind. In this case, it is your manager. Please change the slides permission to editable, we will make comments on your slides.
• Two summary tables, Table 1 and 2
• Please deposit the above items into your own google folder you are asked to create, under this folder, create a folder named homework1. Please deposit the files there.
• Please don’t deposit your finalized product earlier than October 4th. After you finish, please post a link to your homework google folder on blackboard submission.

Dataset and Background Readings:

Please download the dataset from the following link. For this homework, you may only use KT4 (uncompressed size 6.4GB). But you may need to download other small lookup tables for the purpose of this assignment. Note: it may take a few hours to download/unzip the data files, please make sure you plan ahead.

https://github.com/riiid/ednet

Please refer to this paper for details of dataset (mainly Section 1 and 2, up to page 7)

https://arxiv.org/abs/1912.03072

Software and Tools

You are free to use any software tools you feel comfortable with, which include but are not limited to Python, R, WEKA or Tableau.