Evaluation¶
This module provides evaluation methods for classification, regression and clustering. Available metrics include:
- AUC: Compute AUC for binary classification.
- KS: Compute Kolmogorov-Smirnov for binary classification.
- LIFT: Compute lift of binary classification.
- PRECISION: Compute the precision for binary and multi-classification
- RECALL: Compute the recall for binary and multi-classification
- ACCURACY: Compute the accuracy for binary and multi-classification
- EXPLAINED_VARIANCE: Compute explain variance for regression tasks
- MEAN_ABSOLUTE_ERROR: Compute mean absolute error for regression tasks
- MEAN_SQUARED_ERROR: Compute mean square error for regression tasks
- MEAN_SQUARED_LOG_ERROR: Compute mean squared logarithmic error for regression tasks
- MEDIAN_ABSOLUTE_ERROR: Compute median absolute error for regression tasks
- R2_SCORE: Compute R^2 (coefficient of determination) score for regression tasks
- ROOT_MEAN_SQUARED_ERROR: Compute the root of mean square error for regression tasks
- JACCARD_SIMILARITY_SCORE:Compute Jaccard similarity score for clustering tasks (labels are needed)
- ADJUSTED_RAND_SCORE:Compute adjusted rand score for clustering tasks (labels are needed)
- FOWLKES_MALLOWS_SCORE:Compute Fowlkes Mallows score for clustering tasks (labels are needed)
- DAVIES_BOULDIN_INDEX:Compute Davies Bouldin index for clustering tasks
- DISTANCE_MEASURE:Compute cluster information in clustering algorithms
- CONTINGENCY_MATRIX:Compute contingency matrix for clustering tasks (labels are needed)
- PSI: Compute Population Stability Index.
- F1-Score: Compute F1-Score for binary tasks.
Param¶
evaluation_param
¶
Attributes¶
Classes¶
EvaluateParam(eval_type='binary', pos_label=1, need_run=True, metrics=None, run_clustering_arbiter_metric=False, unfold_multi_result=False)
¶
Bases: BaseParam
Define the evaluation method of binary/multiple classification and regression
Parameters:
Name | Type | Description | Default |
---|---|---|---|
eval_type |
support 'binary' for HomoLR, HeteroLR and Secureboosting, support 'regression' for Secureboosting, 'multi' is not support these version |
'binary'
|
|
unfold_multi_result |
bool
|
unfold multi result and get several one-vs-rest binary classification results |
False
|
pos_label |
int or float or str
|
specify positive label type, depend on the data's label. this parameter effective only for 'binary' |
1
|
need_run |
Indicate if this module needed to be run |
True
|
Source code in python/federatedml/param/evaluation_param.py
41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 |
|
Attributes¶
eval_type = eval_type
instance-attribute
¶pos_label = pos_label
instance-attribute
¶need_run = need_run
instance-attribute
¶metrics = metrics
instance-attribute
¶unfold_multi_result = unfold_multi_result
instance-attribute
¶run_clustering_arbiter_metric = run_clustering_arbiter_metric
instance-attribute
¶default_metrics = {consts.BINARY: consts.ALL_BINARY_METRICS, consts.MULTY: consts.ALL_MULTI_METRICS, consts.REGRESSION: consts.ALL_REGRESSION_METRICS, consts.CLUSTERING: consts.ALL_CLUSTER_METRICS}
instance-attribute
¶allowed_metrics = {consts.BINARY: consts.ALL_BINARY_METRICS, consts.MULTY: consts.ALL_MULTI_METRICS, consts.REGRESSION: consts.ALL_REGRESSION_METRICS, consts.CLUSTERING: consts.ALL_CLUSTER_METRICS}
instance-attribute
¶Functions¶
check()
¶Source code in python/federatedml/param/evaluation_param.py
115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 |
|
check_single_value_default_metric()
¶Source code in python/federatedml/param/evaluation_param.py
143 144 145 146 147 148 149 150 151 152 153 154 155 |
|
Last update:
2021-11-15