Пороговые значения в двоичных классификаторах

Question

Я пытаюсь понять, как используются decision_function и predict_proba в двоичных классификаторах, и натолкнулся на пороговые значения в precision_recall_curve

Теперь получено, что decision_function вычисляет расстояние до гиперплоскость и predict_proba дают вероятность того, что точка данных принадлежит определенной группе.

* precision_recall_curve возвращает массив порогов с различными значениями порогов.

Если пороги Какова вероятность классификации этих точек данных, тогда как пороги принимают отрицательные значения или значения меньше 0 или больше 1.

Кроме того, что мы используем для тонкой настройки нашего двоичного классификатора? decision_function или predict_proba?

Пример:

from sklearn.metrics import precision_recall_curve

precision, recall, thresholds = precision_recall_curve(y_test, y_scores_lr)
closest_zero = np.argmin(np.abs(thresholds))
closest_zero_p = precision[closest_zero]
closest_zero_r = recall[closest_zero]

print('Thresholds are',thresholds)

Здесь пороги имеют значения как

Thresholds are [ -4.04847662  -3.93819545  -3.48628627  -3.44776445  -3.33892603
  -2.5783356   -2.37746137  -2.34718536  -2.30446832  -2.15792885
  -2.03386685  -1.87131487  -1.7495844   -1.72691524  -1.68712543
  -1.47668716  -1.33979401  -1.3051061   -1.08033549  -0.57099832
   0.13088342   0.17583273   0.47631823   0.6418365    1.00422797
   1.33670725   1.68203683   1.69861005   1.87908244   2.18989765
   2.43420944   2.55168221   3.71752409   3.80620565   4.21070117
   4.25093438   4.30966876   4.31558393   4.55321241   4.57143325
   4.93002949   5.23271557   5.73378353   6.12856799   6.55341039
   6.86404167   6.92400179   7.22184672   7.37403798   7.80959453
   8.26212674   8.3930213    8.45858117   9.84572083   9.87342932
  10.201736    11.20681116  11.4821926   11.55476419  11.68009017
  13.26095216  14.73832302  16.02811865]

Так что, если они являются значениями вероятности, как они не находятся в диапазоне от 0 до 1, это значения решения_функции или что-то еще?

Answer 1

precision_recall_curve дает вам значения точности и возврата для двоичных классификаторов при определенных пороговых значениях. Это предполагает, что вы смотрите на вероятности для определенного класса. После подбора вы получаете вероятности с помощью функции predict_proba(self, X). Одна вероятность для каждого класса. Конечно, для двоичного классификатора это будет два класса. Это отличается от predict(self, X), который, по сути, позволяет узнать, равна ли вероятность для какого-либо класса > 0.5, и затем возвращает этот класс. Я думаю, что вы хотите сделать, это выбрать этот порог (0.5 по умолчанию) идеальным способом для оптимизации f-счета, отзыва или точности. Это может быть достигнуто с помощью вышеупомянутой функции precision_recall_curve.

В следующем примере показано, как это делается.

import numpy as np
from sklearn.linear_model import LogisticRegression
from sklearn.datasets import load_iris
from sklearn.metrics import precision_recall_curve

X, y = load_iris(return_X_y=True)
# reduce multiclass to binary problem, i.e. class 0 or 1 (class 2 starts at index 100)
X = X[0:100]
y = y[0:100]

lr = LogisticRegression(random_state=0).fit(X, y)

y_test_hat = lr.predict_proba(X)

# just look at probabilities for class 1
y_test_hat_class_1 = y_test_hat[:,1]

precisions, recalls, thresholds = precision_recall_curve(y, y_test_hat_class_1)
f_scores = np.nan_to_num((2 * precisions * recalls) / (precisions + recalls))

for p, r, f, t in zip(precisions, recalls, f_scores, thresholds):
    print('Using threshold={} as decision boundary, we reach '
          'precision={}, recall={}, and f-score={}'.format(t, p, r, f))

f_max_index = np.argmax(f_scores)
max_f_score = f_scores[f_max_index]
max_f_score_threshold = thresholds[f_max_index]

print('The threshold for the max f-score is {}'.format(max_f_score_threshold))

Приводит к:

Using threshold=0.8628645363798557 as decision boundary, we reach precision=1.0, recall=1.0, and f-score=1.0
Using threshold=0.9218669507660147 as decision boundary, we reach precision=1.0, recall=0.98, and f-score=0.98989898989899
Using threshold=0.93066642297958 as decision boundary, we reach precision=1.0, recall=0.96, and f-score=0.9795918367346939
Using threshold=0.9332685743944795 as decision boundary, we reach precision=1.0, recall=0.94, and f-score=0.9690721649484536
Using threshold=0.9395382533408563 as decision boundary, we reach precision=1.0, recall=0.92, and f-score=0.9583333333333334
Using threshold=0.9640718757241656 as decision boundary, we reach precision=1.0, recall=0.9, and f-score=0.9473684210526316
Using threshold=0.9670374623286897 as decision boundary, we reach precision=1.0, recall=0.88, and f-score=0.9361702127659575
Using threshold=0.9687934720210198 as decision boundary, we reach precision=1.0, recall=0.86, and f-score=0.924731182795699
Using threshold=0.9726392263137621 as decision boundary, we reach precision=1.0, recall=0.84, and f-score=0.9130434782608696
Using threshold=0.973775627114333 as decision boundary, we reach precision=1.0, recall=0.82, and f-score=0.9010989010989011
Using threshold=0.9740474969329987 as decision boundary, we reach precision=1.0, recall=0.8, and f-score=0.888888888888889
Using threshold=0.9741603105458991 as decision boundary, we reach precision=1.0, recall=0.78, and f-score=0.8764044943820225
Using threshold=0.9747085542467909 as decision boundary, we reach precision=1.0, recall=0.76, and f-score=0.8636363636363636
Using threshold=0.974749494774799 as decision boundary, we reach precision=1.0, recall=0.74, and f-score=0.8505747126436781
Using threshold=0.9769993303678443 as decision boundary, we reach precision=1.0, recall=0.72, and f-score=0.8372093023255813
Using threshold=0.9770140294088295 as decision boundary, we reach precision=1.0, recall=0.7, and f-score=0.8235294117647058
Using threshold=0.9785921201646789 as decision boundary, we reach precision=1.0, recall=0.68, and f-score=0.8095238095238095
Using threshold=0.9786461690308931 as decision boundary, we reach precision=1.0, recall=0.66, and f-score=0.7951807228915663
Using threshold=0.9789411518223052 as decision boundary, we reach precision=1.0, recall=0.64, and f-score=0.7804878048780487
Using threshold=0.9796555988114017 as decision boundary, we reach precision=1.0, recall=0.62, and f-score=0.7654320987654321
Using threshold=0.9801649093623934 as decision boundary, we reach precision=1.0, recall=0.6, and f-score=0.7499999999999999
Using threshold=0.9805566289582609 as decision boundary, we reach precision=1.0, recall=0.58, and f-score=0.7341772151898733
Using threshold=0.9808560894443067 as decision boundary, we reach precision=1.0, recall=0.56, and f-score=0.717948717948718
Using threshold=0.982400866419342 as decision boundary, we reach precision=1.0, recall=0.54, and f-score=0.7012987012987013
Using threshold=0.9828790909959155 as decision boundary, we reach precision=1.0, recall=0.52, and f-score=0.6842105263157895
Using threshold=0.9828854909335458 as decision boundary, we reach precision=1.0, recall=0.5, and f-score=0.6666666666666666
Using threshold=0.9839851081942663 as decision boundary, we reach precision=1.0, recall=0.48, and f-score=0.6486486486486487
Using threshold=0.9845312460821358 as decision boundary, we reach precision=1.0, recall=0.46, and f-score=0.6301369863013699
Using threshold=0.9857012993403023 as decision boundary, we reach precision=1.0, recall=0.44, and f-score=0.6111111111111112
Using threshold=0.9879940756602601 as decision boundary, we reach precision=1.0, recall=0.42, and f-score=0.5915492957746479
Using threshold=0.9882223190984861 as decision boundary, we reach precision=1.0, recall=0.4, and f-score=0.5714285714285715
Using threshold=0.9889482842475497 as decision boundary, we reach precision=1.0, recall=0.38, and f-score=0.5507246376811594
Using threshold=0.9892545856218082 as decision boundary, we reach precision=1.0, recall=0.36, and f-score=0.5294117647058824
Using threshold=0.9899303560728386 as decision boundary, we reach precision=1.0, recall=0.34, and f-score=0.5074626865671642
Using threshold=0.9905455482163618 as decision boundary, we reach precision=1.0, recall=0.32, and f-score=0.48484848484848486
Using threshold=0.9907019104721698 as decision boundary, we reach precision=1.0, recall=0.3, and f-score=0.4615384615384615
Using threshold=0.9911493537429485 as decision boundary, we reach precision=1.0, recall=0.28, and f-score=0.43750000000000006
Using threshold=0.9914230947944308 as decision boundary, we reach precision=1.0, recall=0.26, and f-score=0.41269841269841273
Using threshold=0.9915673581329265 as decision boundary, we reach precision=1.0, recall=0.24, and f-score=0.3870967741935484
Using threshold=0.9919835313724615 as decision boundary, we reach precision=1.0, recall=0.22, and f-score=0.36065573770491804
Using threshold=0.9925274516087134 as decision boundary, we reach precision=1.0, recall=0.2, and f-score=0.33333333333333337
Using threshold=0.9926276253093826 as decision boundary, we reach precision=1.0, recall=0.18, and f-score=0.3050847457627119
Using threshold=0.9930234956465036 as decision boundary, we reach precision=1.0, recall=0.16, and f-score=0.2758620689655173
Using threshold=0.9931758599517743 as decision boundary, we reach precision=1.0, recall=0.14, and f-score=0.24561403508771928
Using threshold=0.9935881899997894 as decision boundary, we reach precision=1.0, recall=0.12, and f-score=0.21428571428571425
Using threshold=0.9946684285206863 as decision boundary, we reach precision=1.0, recall=0.1, and f-score=0.18181818181818182
Using threshold=0.9960976336416663 as decision boundary, we reach precision=1.0, recall=0.08, and f-score=0.14814814814814814
Using threshold=0.996289803123931 as decision boundary, we reach precision=1.0, recall=0.06, and f-score=0.11320754716981131
Using threshold=0.9975518299472802 as decision boundary, we reach precision=1.0, recall=0.04, and f-score=0.07692307692307693
Using threshold=0.998322588642525 as decision boundary, we reach precision=1.0, recall=0.02, and f-score=0.0392156862745098
The threshold for the max f-score is 0.8628645363798557

В этом примере также вычисляется порог, который вы использовали бы для максимизации f-показателя. .

Для получения дополнительной информации о decision_function см. этот ответ более на статистику.