Code for "Entity-Specific Cyber Risk Assessment Using Insurtech-Empowered Risk Factors" Jiayi Guo, Linfeng Zhang, Zhiyu Quan. Variance, Vol. 19, 2026. https://doi.org/10.66573/001c.155715
Cyber incident records are scarce and organizations are reluctant to disclose incidents. This study enriches publicly collected cyber incident data with proprietary InsurTech entity-level features and builds machine learning models for organization-level cyber risk assessment. For each firm-year, we model
- incident occurrence as a multi-label classification problem, and
- incident frequency as a multi-output regression problem,
over five incident types (Data Breach, Privacy Violation, Extortion/Fraud, IT Error, Other), and examine
which entity-specific features drive the predictions.
The data are not publicly available. The entity-level features come from a proprietary InsurTech platform and cannot be redistributed, and the merged firm-year dataset is derived from them. This repository therefore contains code only, together with tuned hyper-parameters and aggregate results (feature-importance tables and figures). No raw or processed data, and no row-level model outputs, are included.
| Source | Content |
|---|---|
| Cyber incident data (public domain) | 66,215 incident records (2000–2018 retained), assembled from public incident databases and NLP-extracted news reports; incident types follow the Kesan & Zhang (2021) taxonomy; at most one incident of the same type per organization per day |
| InsurTech data (proprietary) | 527 entity-level features for 31,782 companies: business identity (geography, enterprise structure, industry classification, licensing, disclosed risk characteristics) and operational signals (operating status, open hours, shareholding, customer reviews, firmographics) |
| Merged dataset | 39,636 firm-year observations, joined on COMPANY_ID; d = 528 features (527 InsurTech features + ACCIDENT_YEAR), q = 5 incident types |
Base learners are Random Forest and LightGBM (scikit-learn / LightGBM implementations). Label-specific decision thresholds (classification), the label order (chain models) and the hyper-parameters are searched jointly with Optuna (TPE sampler, 100 trials, 5-fold cross-validation, seed 42).
| Task | Method | Code prefix | Implementation |
|---|---|---|---|
| Multi-label classification (occurrence) | Binary Relevance (BR) | MOC |
MultiOutputClassifier |
| Classifier Chain (CC) | CC |
ClassifierChain |
|
| Multi-label Classification Trees (MCTs) | MLRF |
single Random Forest over all labels | |
| Multi-output regression (frequency) | Multi-output Regressor (MOR) | MOR |
MultiOutputRegressor |
| Regressor Chain (RC) | RC |
RegressorChain |
|
| Multi-output Regression Trees (MRTs) | MORF |
multi-output RandomForestRegressor |
Evaluation metrics. Classification: Weighted-F1, Macro-F1, Micro-F1, Sample-F1, Jaccard index, Hamming loss. Regression: average MSE (aMSE), average RMSE (aRMSE), average correlation coefficient (aCC), average relative RMSE (aRRMSE), and global Euclidean distance (EU_DIST).
Feature importance. Impurity-based (gain and split for LightGBM; Gini and entropy for classification forests; MSE-based impurity for regression forests), permutation importance (ROC-AUC for classification, MSE for regression), and TreeSHAP: 20 model–method combinations for classification and 15 for regression.
Script names follow <METHOD>_<LEARNER>_<IMPORTANCE>.py, e.g. CC_RF_shap.py = Classifier Chain + Random Forest +
TreeSHAP; models are named LGBM_MOC, RF_CC, MLRF, LGBM_MOR, RF_RC, MORF, etc.
.
├── preprocessing/ # data preparation
│ ├── raw_cleaning/ # raw JSON -> CSV, cleaning and flattening of the entity-level records
│ │ ├── data_clean_cyber.py
│ │ ├── data_flatten_cyber.py
│ │ └── data_loader.py
│ ├── data_processing.py
│ ├── split_classification.py
│ └── split_regression.py
├── classification/ # multi-label classification (incident occurrence)
│ ├── data/ # place data here (contents git-ignored)
│ │ └── README.md
│ ├── baselines/ # untuned reference models
│ │ ├── CC_lgbm_original.py
│ │ ├── CC_rf_original.py
│ │ └── MOC_original.py
│ ├── tuning/ # Optuna search (hyper-parameters, thresholds, label order) + SLURM scripts
│ │ ├── CC_LGBM_ORDER.py
│ │ ├── CC_RF_ORDER.py
│ │ ├── MLRF_TUNE.py
│ │ ├── MOC_LGBM_TUNE.py
│ │ ├── MOC_RF_TUNE.py
│ │ ├── run_CC_LGBM_ORDER.sh
│ │ ├── run_CC_RF_ORDER.sh
│ │ ├── run_MLRF_TUNE.sh
│ │ ├── run_MOC_LGBM_TUNE.sh
│ │ └── run_MOC_RF_TUNE.sh
│ ├── feature_importance/ # one script per (method, learner, importance technique)
│ │ ├── CC_LGBM_feature_gain.py
│ │ ├── CC_LGBM_feature_split.py
│ │ ├── CC_LGBM_permutation.py
│ │ ├── CC_LGBM_shap.py
│ │ ├── CC_RF_feature_entropy.py
│ │ ├── CC_RF_feature_gini.py
│ │ ├── CC_RF_permutation.py
│ │ ├── CC_RF_shap.py
│ │ ├── MLRF_feature_entropy.py
│ │ ├── MLRF_feature_gini.py
│ │ ├── MLRF_permutation.py
│ │ ├── MLRF_shap.py
│ │ ├── MOC_LGBM_feature_gain.py
│ │ ├── MOC_LGBM_feature_split.py
│ │ ├── MOC_LGBM_permutation.py
│ │ ├── MOC_LGBM_shap.py
│ │ ├── MOC_RF_feature_entropy.py
│ │ ├── MOC_RF_feature_gini.py
│ │ ├── MOC_RF_permutation.py
│ │ └── MOC_RF_shap.py
│ ├── notebooks/ # model performance and feature-importance heatmaps
│ │ ├── feature_importance_heatmaps.ipynb
│ │ └── model_performance.ipynb
│ └── results/ # tuned hyper-parameters and aggregate results
│ ├── CC/ # Classifier Chain: hyper-parameters, label order, thresholds (JSON)
│ │ ├── LGBM_CC_best_chain_order.json
│ │ ├── LGBM_CC_best_model_params.json
│ │ ├── LGBM_CC_best_thresholds.json
│ │ ├── RF_CC_best_chain_order.json
│ │ ├── RF_CC_best_model_params.json
│ │ └── RF_CC_best_thresholds.json
│ ├── MOC/ # Binary Relevance: hyper-parameters, thresholds (JSON)
│ │ ├── LGBM_MOC_best_model_params.json
│ │ ├── LGBM_MOC_best_thresholds.json
│ │ ├── RF_best_model_params.json
│ │ ├── RF_best_thresholds.json
│ │ ├── RF_MOC_best_model_params.json
│ │ └── RF_MOC_best_thresholds.json
│ ├── feature/ # feature-importance results
│ │ ├── per_model/ # one table (all features) and one top-20 plot per model and technique
│ │ │ ├── *.csv # 20 tables
│ │ │ └── *.png # 20 figures
│ │ └── summary/ # summary heatmaps across models
│ │ └── *.png # 6 figures
│ └── figures/ # model performance heatmaps
│ └── *.png # 5 figures
├── regression/ # multi-output regression (incident frequency)
│ ├── data/ # place data here (contents git-ignored)
│ │ └── README.md
│ ├── baselines/ # untuned reference models (RF, kernel ridge, SVR, multi-output SVR)
│ │ ├── MOR_KRR_original.py
│ │ ├── MOR_SVR_original.py
│ │ ├── MORF_original.py
│ │ └── MSVR_original.py
│ ├── tuning/ # Optuna search (hyper-parameters, chain order) + SLURM scripts
│ │ ├── MOR_LGBM_optimized.py
│ │ ├── MOR_RF_optimized.py
│ │ ├── MORF_optimized.py
│ │ ├── RC_LGBM_ORDER.py
│ │ ├── RC_RF_ORDER.py
│ │ ├── run_MOR_LGBM_TUNE.sh
│ │ ├── run_MOR_RF_TUNE.sh
│ │ ├── run_RC_LGBM_ORDER.sh
│ │ ├── run_RC_RF_ORDER.sh
│ │ └── run_RF_TUNE.sh
│ ├── feature_importance/ # one script per (method, learner, importance technique)
│ │ ├── MOR_LGBM_feature_gain.py
│ │ ├── MOR_LGBM_feature_split.py
│ │ ├── MOR_LGBM_permutation.py
│ │ ├── MOR_LGBM_shap.py
│ │ ├── MOR_RF_feature_impurity.py
│ │ ├── MOR_RF_permutation.py
│ │ ├── MOR_RF_shap.py
│ │ ├── MORF_feature_impurity.py
│ │ ├── MORF_permutation.py
│ │ ├── MORF_shap.py
│ │ ├── RC_LGBM_feature_gain.py
│ │ ├── RC_LGBM_feature_split.py
│ │ ├── RC_LGBM_permutation.py
│ │ ├── RC_LGBM_shap.py
│ │ ├── RC_RF_feature_impurity.py
│ │ ├── RC_RF_permutation.py
│ │ └── RC_RF_shap.py
│ ├── notebooks/ # model performance and feature-importance heatmaps
│ │ ├── feature_importance_heatmaps.ipynb
│ │ └── model_performance.ipynb
│ └── results/ # tuned hyper-parameters and aggregate results
│ ├── MOR/ # Multi-output Regressor: hyper-parameters (JSON)
│ │ ├── LGBM_MOR_best_params.json
│ │ ├── RF_best_params.json
│ │ └── RF_MOR_best_params.json
│ ├── RC/ # Regressor Chain: hyper-parameters, chain order (JSON)
│ │ ├── LGBM_RC_best_chain_order.json
│ │ ├── LGBM_RC_best_params.json
│ │ ├── RF_RC_best_chain_order.json
│ │ └── RF_RC_best_params.json
│ ├── feature/ # feature-importance results
│ │ ├── per_model/ # one table (all features) and one top-20 plot per model and technique
│ │ │ ├── *.csv # 15 tables
│ │ │ └── *.png # 16 figures
│ │ └── summary/ # summary heatmaps across models
│ │ └── *.png # 5 figures
│ └── figures/ # model performance heatmaps
│ └── *.png # 2 figures
├── README.md
├── requirements.txt
├── LICENSE
├── CITATION.cff
└── .gitignore
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txtReproduction requires access to the data described above. Without it, the code can be read and the aggregate
results in */results/ inspected, but the pipeline cannot be run.
1. Preprocessing (run from the repository root)
data_processing.py expects two input files in classification/data/, label_w_year.csv (incident labels) and
cyber_cleaned.csv (entity-level features), derived from the raw data with preprocessing/raw_cleaning/ plus
manual cleaning steps.
python preprocessing/data_processing.py # -> classification/data/label_cyber_year.csv
python preprocessing/split_classification.py # -> classification/data/{X,Y}_{train,test}.csv
cp classification/data/label_cyber_year.csv regression/data/
python preprocessing/split_regression.py # -> regression/data/{X,Y}_{train,test}.csv2. Hyper-parameter tuning (run from the task directory)
cd classification
python tuning/MOC_LGBM_TUNE.py # also: MOC_RF_TUNE, CC_LGBM_ORDER, CC_RF_ORDER (label order is searched too), MLRF_TUNE
sbatch tuning/run_MOC_LGBM_TUNE.sh # SLURM wrapper (edit the partition / module / conda env for your cluster)
cd ../regression
python tuning/MOR_LGBM_optimized.py # also: MOR_RF_optimized, MORF_optimized, RC_LGBM_ORDER, RC_RF_ORDEREach tuning script writes best parameters, metrics, predictions and the Optuna study database under results/
(MOC and MLRF: results/MOC; CC and RC chain models: results/ORDER; MOR and MORF: results/LGBM or results/RF).
The tuned parameters used in the paper are provided in results/{CC,MOC} (classification) and results/{MOR,RC}
(regression). To use re-tuned parameters, copy the JSON files into those folders (names as in the provided files).
The chain-model importance scripts (CC_*, RC_*) hard-code the chain order in their CONFIG; update it if you
re-tune the order (it is stored in the corresponding *_best_chain_order.json).
3. Feature importance (run from the task directory; paths in the scripts are relative to it)
cd classification
python feature_importance/CC_RF_shap.py # writes results/feature/per_model/RF_CC_shap_importance.csv + RF_CC_shap_top20.png
python feature_importance/MOC_LGBM_permutation.pyAvailable techniques per learner — LightGBM: feature_gain, feature_split, shap, permutation;
Random Forest: feature_gini/feature_entropy (classification) or feature_impurity (regression), shap, permutation.
Hyper-parameters are read from results/<METHOD>/*.json.
Outputs in results/feature/per_model/ follow <MODEL>_<technique>_importance.csv (all features, ranked) and
<MODEL>_<technique>_top20.png (top-20 plot), with <MODEL> in LGBM_MOC, RF_MOC, LGBM_CC, RF_CC, MLRF
(classification) or LGBM_MOR, RF_MOR, LGBM_RC, RF_RC, MORF (regression) and <technique> in gain,
split, gini, entropy, impurity, shap, permutation.
4. Figures and tables — open the notebooks in */notebooks/ with Jupyter started from the task directory
(cd classification && jupyter lab), since they read results/ and data/ via relative paths.
feature_importance_heatmaps.ipynb reads results/feature/per_model/*.csv and writes the summary heatmaps to
results/feature/summary/; model_performance.ipynb writes the performance heatmaps to results/figures/.
If you use this code, please cite:
@article{guo2026entity,
title = {Entity-Specific Cyber Risk Assessment Using Insurtech-Empowered Risk Factors},
author = {Guo, Jiayi and Zhang, Linfeng and Quan, Zhiyu},
journal = {Variance},
volume = {19},
year = {2026},
doi = {10.66573/001c.155715}
}See also CITATION.cff.
MIT — see LICENSE. The license covers the code only, not the data.