Apply all required pre-processing steps to make the data ready for both clustering algorithms.

Learning Goal: I’m working on a python exercise and need an explanation and answer to help me learn.

Note: The main objective of this assignment is to apply the two clustering algorithms (K-means, DBSCAN) on a given dataset. Obviously, preprocessing steps must be done on the data.

In banks, customer loyalty is important since acquiring a new customer is much costlier than retaining an existing customer. Therefore, usually banks would like to predict customer churns. Customer churn refers to the loss of existing clients or customers. These predictions can help banks identify customers who are more likely to close their account and leave the bank.

Given the dataset which shows bank customer’s information, we want to cluster their information and learn from it.

In our assignment we will pretend that we don’t know their types (Targets: Attrited or Existing) and would like to cluster them using K-means and DBSCAN, apply dimension reduction and validate our clustering algorithms.

Your Tasks:

• Apply all required pre-processing steps to make the data ready for both clustering algorithms.
• Apply both clustering algorithms on the dataset using your choice of number of clusters, epsilon and minPts.
• For the current clusters generated by k-means and DBSCAN, find the silhouette average for all of them and explain which is best.
• Run both clustering algorithms again but this time, find the best number of clusters using elbow method (if applicable) and silhouette average (plot the elbow and sillouette average like we saw in the lab)
• Explain why this is the best clustering number
• Apply PCA on the original dataset and show the variance ratio for all features (like in we saw in lab) and explain them.
• After seeing the variance ratio for all features, explain how many dimensions should we use or not use to explain the data.
• Plot the dataset in 3D (using top 3 important features). Can you visualize how many clusters are there? Explain.