Credit Risk Prediction using customer information and financial transactions.

Risk Analytics is one of the key areas of data science and business intelligence in finance. With Risk analytics and management, a company is able to take strategic decisions, increase the trustworthiness and security of the company by identifying, assessing, and controlling threats to an organization’s capital and earnings. Risk Analytics is the application of data science knowledge to manage risk in a way that helps in revenue growth with minimal losses associated with fraudulent activities by bad actors.

This was my first project related to Data Science applied to Finance. We obtain the data from the [lending club dataset](https://www.lendingclub.com/investing/peer-to-peer).

> “ I was so amazed by how difficult was to put the ML pipeline in production.”

Visit this [repository](https://github.com/octadelsueldo/lending_club) and check the notebook out to find more information about the steps of the project and the results. A summary here below:

**Objective**

The objective of this exercise was to classify our current and old customers in order to predict if the new clients will pay the loan or not.

**Target**

The target we want to predict is the \`loan_status\` variable.

**Data**

Initially, we have 4 datasets with the information of our clients, to proceed with the data analysis we merged them into one dataset. The shape of this data is 884876 rows × 151 columns.

**EDA**

After performing an Exploratory Data Analysis and selecting the best variables we got 425015 rows × 51 columns dataset.

**Processing data**

In order to perform the next algorithms, we processed the data (scaling, one hot encoding...) and created our TRAIN, VALIDATION, and TEST datasets.

**Predictions**

Algorithms            |Accuracy|ROC-AUC
:-----------------------|:--------|:--------
Logistic Regression    | 76.11 %| 0.72
Support Vector Machine | 58.89 %| 0.51
Random Forest          | 76.25 %| 0.73
XGBoost                | 76.58 %| 0.73
K-Nearest Neighbors    | 73.80 %| 0.59
XGBoost Tuned          | 73.42 %| 0.73

**Conclusions**

The best model is the XGboost tuned, because it has the best accuracy, precisions, recalls, and f1-score. But as we can see the differences between the metrics are very low, so if we have to conclude which is the best one for production would be the Logistic Regression because it is clearly interpretable and faster than the XGboost or Random Forest Classifier.

**Members of the group**

Octavio del Sueldo: <hugo.delsueldo@cunef.edu>

Jose Lopez: <jose.lopez@cunef.edu>


This is the private repository where I store my personal strategies in the stock market.

I use code to apply automated analysis in the US market, which allows me to check the value and the quality of the companies that I am analyzing. Furthermore, I'm able to automate price alarms depending on the valuation of the companies. Last but not least, I used the Shiny for Python framework to create a web application to visualize those companies of my interest.

> If you are an interested to know more information about the techniques just let me know by email.

![](/images/Screen%20Shot%202022-09-02%20at%2020.49.38.png)



Algorithmic Trading

Clustering of drivers using 6 years of telematic data of drivers + crash prediction + Power BI Dashboard.

**Project:** <https://github.com/octadelsueldo/TFM_Driver_Profiling>

**Authors:**

Octavio del Sueldo 

Jose Lopez

**Supervisor**

Juan Manuel López Zafra

**Abstract:** Traffic accidents represent a high cost for insurance companies as well as for society, in
economic and social terms, because in all cases the costs include medical and rehabilitation ex-
penses, legal and emergency services, property damage and production losses. Thanks to the use of
telematics and data science we may be able to find patterns of behavior that explain the claims.
During this research we will work with a database of more than 95,000 drivers that includes infor-
mation collected over 6 years; for this, we have performed an important work of cleaning and engi-
neering of variables, for finally clustering the drivers through a PAM, being the most representative
variables the intensity of use of the vehicle and the driving experience. In addition, we have made
a prediction based on whether or not they have suffered a crash using a decision tree, obtaining a
72.25% accuracy rate.

> “We highlight Intensity of use and experience as the most decisive variables when it comes to explaining crashes”

1.  **Introduction.**

Insurance is an effective way of protecting individuals against the consequences of
risks. It is based on transferring the risks to an insurer who is responsible for compensat-
ing all or part of the damage caused by the occurrence of an event. A fair price will accu-
rately reflect the actual risk of the insured, otherwise the business may have problems in
the client portfolio, since those clients who have less risks end up supporting those who
are riskier, which causes a client ́s churn with lower loss ratio and an entry of those with
high loss ratio.... \[1-4]

This directly harms insurance companies, policyholders and, consequently, society.
Therefore, actuaries use risk-related information, i.e., they use those factors that model
risk for insurance premium calculation, so that they can construct league tables based on
expected losses.

In the case of automobiles, the traditional variables for determining the risk profile
are personal characteristics, claims history and vehicle characteristics. However, premi-
ums are often inaccurate in practice, as these factors do not have a direct causal relation-
ship with actual driving risk.

Thanks to the development of networks, connectivity, IoT... UBI (Usage Based Insur-
ance) products are increasingly popular within insurance companies. This insurance
product model is based on schemes known as "pay-as-you-drive" (PAYD) and "pay-how-
you-drive" (PHYD); the determination of the premium is based on variables determined
from the actual data of the driver, such as driving time, distance traveled, type of roads
traveled, speeds.... \[5-7]

Thanks to the use of these technologies, the benefit is twofold, since the insured ob-
tains a price that is more in line with his behavior, so that drivers with lower accident
rates will not be penalized, and, on the other hand, insurance companies can effectively
improve the accuracy of insurance prices. In addition, UBI products encourage drivers to drive less or improve their driving habits, as they can benefit from the use of dynamic premiums.

This coupled with the use of data science makes it possible to mine the information
collected by the devices, so that insurers can fine-tune their risk models. Insurers can
model driver behavior and therefore able to predict future claims based on what has hap-
pened in the past.

According to studies on the subject, such as Ryan et al. (1998) \[8-11], young drivers
are those with the highest accident rates, which is why we will focus on this profile of
drivers, specifically, drivers between 18 and 30 years of age.

As we have already mentioned, the main objective of our research is to segment
young drivers based on their driving characteristics, driver information, such as age and
vehicle qualities (power, weight, etc.). This segmentation (unsupervised algorithm) will
help to classify drivers, using a decision tree (supervised algorithm), based on their acci-
dent rate, for which we will use the proxy of crashes (decelerations of more than 4G).

**2. Goals of the Project**

• Database cleaning

• Identification of essential variables.

• Identification of groups of drivers and behavior patterns

• Prediction by classification tree.

• Proposal for a scorecard.

**3. Analysis**

**3.1. Exploratory Data Analysis**

First, we perform an exploratory analysis of the database in order to find possible data quality failures.

**3.2. Feature Engineering**

Once this phase is done, we start with the feature engineering. The first thing will be to perform an imputation of null values ​​(each variable was studied separately). We have also carried out a transformation of variables changing the units of measurement, for example from meters to kilometers. Likewise, we have deleted variables that we do not consider relevant for the analysis, either due to knowledge of the business or due to multicollinearity with other variables. In addition, we have created new variables that have been essential for the explanation of our unsupervised clustering model, such as the intensity of vehicle use. In this section we have also defined the target variable as a dummy variable that takes the value zero if there is no crash and the value one if there is a crash. Finally, given the objectives of the work, we have filtered our database, keeping only those young drivers between 18 and 30 years old, as well as those drivers who had a minimum of 60 registrations and 30 days in the product.

**3.3. Relevant variables**

In a later step, we have carried out an exploratory analysis of the data already filtered by the engineering phase with the aim of obtaining insights on the behavior of these drivers. Thanks to this, we were able to determine that the intensity of vehicle use is related to the number of crashes. Being those drivers with the highest intensity of use the ones that present the greatest crashes. In addition, the experience behind the wheel is another of the variables that explain the number of crashes, so that the less experienced drivers are the ones who present the greatest crashes. After these analyses, we have studied the gender variable and we can conclude that men have a higher accident rate than women.

**3.4. Clustering**

Once the most relevant variables have been determined, we have carried out a clustering through a PAM using the Euclidean distance. To do this, we have determined the optimal number of groups through the elbow rule, resulting in two as the optimal number of groups. Segmenting the dataset, we have obtained a silhouette of 0.25 and two groups that are well differentiated by their characteristics. The first of these corresponds to young drivers who have an average age of less than 25 years and experience behind the wheel of close to 5 years. These drivers use the vehicle more times a day, so they are more exposed to risks, which causes crashes in this group to be higher. On the other hand, the second cluster is made up of the most experienced drivers with an average age close to 29 years and an average experience of more than 9 years. In this case, drivers have fewer crashes due to the learning factor and use the vehicle less times per day, so they are less exposed to crashes.

**3.5. Prediction using a decision tree**

This is where we will take advantage of the groups of drivers and their qualities to predict whether or not they will crash. We have used a decision tree as it is one of the easiest models to interpret for people without technical knowledge of algorithms. Once our environment is initialized in PyCaret, we have defined the number of initial leaves with which we should work the decision tree in such a way that it allows us to obtain the highest Accuracy possible. Likewise, we have tuned our model so that it allows us to better classify those drivers that had crashes within our dataset since, as the classes were unbalanced, our model classified the zeros very well, but left aside the ones of our model, so what we did was assign different weights to each of the classifiers according to the needs of the business. This is where we have assigned a profit of EUR 4000 for those true positives, that is, those that we predict as crash and are effectively crash since if we can identify them we can charge a higher premium to these drivers. On the other hand, we will allocate a reward of EUR 2,500 for those drivers who are identified as not crashing and are effectively not crashing as we will reward them for their good driving, but at the same time we will save capital as they will not give us additional costs for have good driving habits. The false negatives will have a cost of EUR 3,000 since they would be drivers that we have identified as negative when they were positive, that is, we are going to charge them less and they will also crash, so they are the drivers we want to avoid. As for the false positives, we will put a cost of EUR 1000 on them since we will be charging them more than they should and they will also have no incidents driving so it is likely that they can find other alternatives on the market and go to the competition what we would lead to stop charging those premiums.
Once this was done, we achieved superior results compared to our first model. We have decreased about 4 percentage points in Accuracy and also in Precision but we have improved the Recall and the F1 Score since now we are capturing more ones in our model and we go from a Recall of 0.02 to 0.219 while the F1 Score goes from 0.04 to 0.2738 . This leads us to the conclusion that by adding the costs to each of the classifiers according to the relative needs of the business we can outperform our base model.

**4. Conclusions**

After conducting exhaustive research on more than 95,000 drivers and 45 variables, from more than 54 million trips between 2007 and 2013, which determine driving behavior, vehicle characteristics and driver attributes, such as age , experience or sex, we can conclude that there are variables that explain the accident rate for drivers between 18 and 30 years old. Among these variables we highlight the experience behind the wheel or the intensity of use and the fact is that the greater the intensity of use, the greater the accident rate, this makes sense since the insured is more exposed to accidents, so by probability he should have a greater number of crashes, Experience is also decisive when it comes to explaining crashes, and the lesser the experience, the greater the number of crashes. In both cases, the relationship between sex was also studied, and men are always more prone to accidents than women.
Thanks to the identification of these variables, we have carried out a segmentation of the conductors using a Partitioning Around Medoids (PAM). After studying the optimal number of groups, we determined that 2 is the one that maximizes the silhouette of the cluster using the elbow method. Using the Euclidean distance we obtain a mean silhouette of 0.25 and two groups of similar size:

• First group: They are younger and less qualified drivers; the average and maximum speed are slightly lower than those of the first group; however, the intensity of use is higher than average, almost 3 trips a day. Lastly, they have a higher than average accident rate

• Second group: These are the most experienced and older drivers, with higher average and maximum speeds; a possible explanation comes from the fact that they tend to drive on open roads with a higher intensity than in the urban cycle. The intensity of use (defined as the number of times the driver uses the car per day, as stated above) is slightly lower than that of the other group. Finally, the accident rate (measured in terms of collisions) is lower in this group than in the second.

Using the information provided by the unsupervised algorithm, we make a classificatory prediction of users who will or will not crash. For this, we use a decision tree, this is because it is one of the easiest models to interpret by people without technical knowledge. After optimizing the algorithm we have obtained the following metrics

• Accuracy: 72.25

• AUC: 61.64

• Recall: 21.9

• Accuracy: 36.40

• F1Score: 27.38

To achieve these results, we have assigned different weights to each of the possible classifiers in our algorithm, that is, the False Negative will not have the same weight as the True Positive, since they will be those that we treat as low-risk drivers, charging them a low premium when they will crash in the future. Beyond the results, one of the most important conclusions, which reinforces our initial hypotheses, is that the most significant variable for the tree is the intensity of vehicle use.

**References**

1\. Zhang, H., Xu, L., Cheng, X., Chen, W., & Zhao, X. (2017). Big Data Research on Driving Behavior Model and Auto Insurance Pricing Factors Based on UBI. Lecture Notes in Electrical Engineering, 404–411. <https://doi.org/10.1007/978-981-10-7521-6_49>

2\. Ma, Y. L., Zhu, X., Hu, X., & Chiu, Y. C. (2018). The use of context-sensitive insurance telematics data in auto insurance rate making. Transportation Research Part A: Policy and Practice, 113, 243–258. <https://doi.org/10.1016/j.tra.2018.04.013>

3\. Sarabia, J. M., Prieto, F., Jordá, V., & Sperlich, S. (2020). A Note on Combining Machine Learning with Statistical Modeling for Financial Data Analysis. Risks, 8(2), 32. <https://doi.org/10.3390/risks8020032>

4\. Guillen, M., & Pesantez-Narvaez, J. MACHINE LEARNING AND PREDICTIVE MODELING FOR AUTOMOBILE INSURANCE PRICING MACHINE LEARNING Y MODELIZACIÓN PREDICTIVA PARA LA TARIFICACIÓN EN EL SEGURO DE AUTOMÓVILES.

5\. Huang, Y., & Meng, S. (2019). Automobile insurance classification ratemaking based on telematics driving data. Decision Support Systems, 127, 113156. <https://doi.org/10.1016/j.dss.2019.113156>

6\. Bordoff, J., & Noel, P. (2010). Pay-as-you-drive auto insurance. Issues of the Day: 100 Commentaries on Climate, Energy, the Environment, Transportation, and Public Health Policy, 150.

7\. Bolderdijk, J., Knockaert, J., Steg, E., & Verhoef, E. (2011). Effects of Pay-As-You-Drive vehicle insurance on young drivers’ speed choice: Results of a Dutch field experiment. Accident Analysis & Prevention, 43(3), 1181–1186. <https://doi.org/10.1016/j.aap.2010.12.032>

8\. Ryan, G., Legge, M., & Rosman, D. (1998). Age related changes in drivers’ crash risk and crash type. Accident Analysis & Prevention, 30(3), 379–387. [https://doi.org/10.1016/s0001-4575(97)00098-5](https://doi.org/10.1016/s0001-4575%2897%2900098-5)

9\. Lifestyle and accidents among young drivers. (1994, 1 junio). ScienceDirect. <https://linkinghub.elsevier.com/retrieve/pii/0001457594900035>

10\. Machin, M. A., & Sankey, K. S. (2008). Relationships between young drivers’ personality characteristics, risk perceptions, and driving behaviour. Accident Analysis & Prevention, 40(2), 541–547. <https://doi.org/10.1016/j.aap.2007.08.010>

11\. Tränkle, U., Gelau, C., & Metker, T. (1990). Risk perception and age-specific accidents of young drivers. Accident Analysis & Prevention, 22(2), 119–125. [https://doi.org/10.1016/0001-4575(90)90063-q](https://doi.org/10.1016/0001-4575%2890%2990063-q)

12\. Patil, P. (2018, July 7). *What is Exploratory Data Analysis? — Towards Data Science*. Medium. <https://towardsdatascience.com/exploratory-data-analysis-8fc1cb20fd15>

13\. Reid Turner, C., Fuggetta, A., Lavazza, L., & Wolf, A. L. (1999b). A conceptual basis for feature engineering. Journal of Systems and Software, 49(1), 3–15. [https://doi.org/10.1016/s0164-1212(99)00062-x](https://doi.org/10.1016/s0164-1212%2899%2900062-x)

14\. Rençberoğlu, E. (2019, 3 abril). Fundamental Techniques of Feature Engineering for Machine Learning. Medium. <https://towardsdatascience.com/feature-engineering-for-machine-learning-3a5e293a5114>

15\. Kalsoom, R., & Halim, Z. (2013). Clustering the driving features based on data streams. INMIC. Published. <https://doi.org/10.1109/inmic.2013.6731330>

16\. Higgs, B., & Abbas, M. (2013, October). A two-step segmentation algorithm for behavioral clustering of naturalistic driving styles. In 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013) (pp. 857–862). IEEE.

17\. Hanafy, M., & Ming, R. (2021). Machine learning approaches for auto insurance big data. Risks, 9(2), 42.

18\. Silhouettes: A graphical aid to the interpretation and validation of cluster analysis. (1987, 1 november). ScienceDirect. <https://www.sciencedirect.com/science/article/pii/0377042787901257>

19\. Model-based evaluation of clustering validation measures. (2007, 1 march). ScienceDirect. <https://www.sciencedirect.com/science/article/abs/pii/S0031320306003104>

20\. Arora, P., Deepali, & Varshney, S. (2016, 1 January). Analysis of K-Means and K-Medoids Algorithm For Big Data. ScienceDirect, 78. <https://www.sciencedirect.com/science/article/pii/S1877050916000971>

21\. Chakure, A. (2020, 6 November). Decision Tree Classification — The Startup. Medium. <https://medium.com/swlh/decision-tree-classification-de64fc4d5aac>

22\. Moral-García, S., Castellano, J. G., Mantas, C. J., Montella, A., & Abellán, J. (2019). Decision tree ensemble method for analyzing traffic accidents of novice drivers in urban areas. Entropy, 21(4), 360.

23\. Galarnyk, M. (2021, January). Understanding Decision Trees for Classification (Python). Medium. <https://towardsdatascience.com/understanding-decision-trees-for-classification-python-9663d683c952>

24\. Rane, S. (2019, 9 November). SHAP: A reliable way to analyze model interpretability. Medium. <https://towardsdatascience.com/shap-a-reliable-way-to-analyze-your-model-interpretability-874294d30af6>

25\. Dataman. (2021, 2 May). Explain Your Model with the SHAP Values — Towards Data Science. Medium. <https://towardsdatascience.com/explain-your-model-with-the-shap-values-bc36aac4de3d>


Algorithms	Accuracy	ROC-AUC
Logistic Regression	76.11 %	0.72
Support Vector Machine	58.89 %	0.51
Random Forest	76.25 %	0.73
XGBoost	76.58 %	0.73
K-Nearest Neighbors	73.80 %	0.59
XGBoost Tuned	73.42 %	0.73