何喆379
2022-02-05 22:20
采纳率: 100%
浏览 21

sklearn集成学习 关于 算法准确率的两个问题

在学习sklearn中的集成学习中遇到了两个问题:

  1. 集成学习中soft voting的准确率低于hard voting。
    代码如下:
import numpy as np
import matplotlib.pyplot as plt
from sklearn import datasets
x,y = datasets.make_moons(n_samples = 500,noise = 0.3, random_state = 42)
from sklearn.model_selection import train_test_split
x_train,x_test,y_train,y_test = train_test_split(x,y,random_state = 42)
from sklearn.ensemble import VotingClassifier
voting_clf = VotingClassifier(estimators = [
    ('log_clf',LogisticRegression()),
    ('svm_clf',SVC()),
    ('dt_clf',DecisionTreeClassifier())
],voting = 'hard')
voting_clf.fit(x_train,y_train)
voting_clf.score(x_test,y_test)
voting_clf2 = VotingClassifier(estimators = [
    ('log_clf',LogisticRegression()),
    ('svm_clf',SVC(probability = True)), #修改SVC参数
    ('dt_clf',DecisionTreeClassifier(random_state = 666))],voting = 'soft')
voting_clf2.fit(x_train,y_train)
voting_clf2.score(x_test,y_test)# soft 与hard 的结果都是0.904 很奇怪

  1. 集成学习中采用决策树的数量增多,准确率并没有提高。
    代码如下:
    import numpy as np
    import matplotlib.pyplot as plt
    from sklearn import datasets
    x,y = datasets.make_moons(n_samples = 500,noise = 0.3, random_state = 42)
    from sklearn.model_selection import train_test_split
    x_train,x_test,y_train,y_test = train_test_split(x,y,random_state = 42)
    bagging_clf = BaggingClassifier(DecisionTreeClassifier(),
                                n_estimators =500, max_samples = 100,bootstrap = True)
    # n_estimator 多少个子模型 max_samples看多少样本 bootstrap是否放回
    %%time
    bagging_clf.fit(x_train,y_train)# 500个决策树
    bagging_clf.score(x_test,y_test)
    single_dec_tree = DecisionTreeClassifier()
    single_dec_tree.fit(x_train,y_train)# 1个决策树
    single_dec_tree.score(x_test,y_test)
    bagging_clf5000 = BaggingClassifier(DecisionTreeClassifier(),
                                    n_estimators = 5000,max_samples = 100,bootstrap = True)
    # 5000个决策树
    %%time
    bagging_clf5000.fit(x_train,y_train)
    bagging_clf5000.score(x_test,y_test)
    # 单个决策树的准确率为0.88,500个是0.928,5000个是0.912
    

这两个问题类似,因为从算法的原理上讲,soft的结果至少不会比hard的差,为啥结果会这样呢?
第二个问题中,子模型的数目增多,一定会使得整体的准确率提高,但是为啥决策树的数目增多,准确率不升反降?
难道存在一些子模型的准确率低于平均准确率吗?很奇怪

  • 写回答
  • 好问题 提建议
  • 追加酬金
  • 关注问题
  • 邀请回答

1条回答 默认 最新

相关推荐 更多相似问题