Пользовательский критерий и "make_scorer"

Question

1. Нужно ли передавать параметры (y_true, y_pred) в функцию «make_scorer»?Если так, как они передаются ..?Если можно пример.
2. Как установить пользовательский критерий в «скоринге»?
3. Результат каждой итерации - результат обучения или теста?

---

_scorer = make_scorer(f1_score,pos_label=0)

grid_searcher = GridSearchCV(clf, parameter_grid, verbose=200, scoring=_scorer)
grid_searcher.fit(X_train, y_train)
clf_best = grid_searcher.best_estimator_

• Каждая итерация, когда она производится в процессе:

---

[CV] class_weight=balanced, max_depth=10, n_estimators=100 ........... 
[CV] class_weight=balanced, max_depth=10, n_estimators=100, score=0.4419706300331596, total= 16.4s 
[Parallel(n_jobs=1)]: Done 12 out of 12 | elapsed: 1.7min remaining: 0.0s 
[CV] class_weight=balanced, max_depth=10, n_estimators=150 > – user287629 47 mins ago  

---

y_pred = clf.predict (X_test) 
r = np.sum (y_pred == 0) & (y_pred == y_test) 
s = np.sum (y_pred == 1) & (y_pred! = y_test) 
z = r / s #I need to get a z 

Answer 1

Попробуйте это:

def T_scorer(y_true, y_pred, **kwargs):
    epsilon = 1e-8  # think of the "computational stability" ("ZeroDivisionError")
    r = np.sum((y_pred == 0) & (y_pred == y_true))
    s = np.sum((y_pred == 1) & (y_pred != y_true))
    z = r / (s + epsilon)  # I need to get a z
    return z

_scorer = make_scorer(T_scorer)

grid_searcher = GridSearchCV(clf, parameter_grid, verbose=2, scoring=_scorer)

Answer 2

Попробовал так, не работает (мало опыта):

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.5, random_state=42)
def T_scorer(clf, X_test,y_test):
    y_pred = clf.predict(X_test)
    r = np.sum(y_pred == 0) & (y_pred == y_test)
    s = np.sum(y_pred == 1) & (y_pred!= y_test)
    z = r / s  # I need to get a z
    return z
clf = RandomForestClassifier()

grid_searcher = GridSearchCV(clf, parameter_grid, verbose=2, scoring=T_scorer)
grid_searcher.fit(X_test, y_test)

clf_best = grid_searcher.best_estimator_
print('Best params = ', clf_best.get_params())