在学习sklearn中的集成学习中遇到了两个问题:
- 集成学习中soft voting的准确率低于hard voting。
代码如下:
import numpy as np
import matplotlib.pyplot as plt
from sklearn import datasets
x,y = datasets.make_moons(n_samples = 500,noise = 0.3, random_state = 42)
from sklearn.model_selection import train_test_split
x_train,x_test,y_train,y_test = train_test_split(x,y,random_state = 42)
from sklearn.ensemble import VotingClassifier
voting_clf = VotingClassifier(estimators = [
('log_clf',LogisticRegression()),
('svm_clf',SVC()),
('dt_clf',DecisionTreeClassifier())
],voting = 'hard')
voting_clf.fit(x_train,y_train)
voting_clf.score(x_test,y_test)
voting_clf2 = VotingClassifier(estimators = [
('log_clf',LogisticRegression()),
('svm_clf',SVC(probability = True)), #修改SVC参数
('dt_clf',DecisionTreeClassifier(random_state = 666))],voting = 'soft')
voting_clf2.fit(x_train,y_train)
voting_clf2.score(x_test,y_test)# soft 与hard 的结果都是0.904 很奇怪
- 集成学习中采用决策树的数量增多,准确率并没有提高。
代码如下:import numpy as np import matplotlib.pyplot as plt from sklearn import datasets x,y = datasets.make_moons(n_samples = 500,noise = 0.3, random_state = 42) from sklearn.model_selection import train_test_split x_train,x_test,y_train,y_test = train_test_split(x,y,random_state = 42) bagging_clf = BaggingClassifier(DecisionTreeClassifier(), n_estimators =500, max_samples = 100,bootstrap = True) # n_estimator 多少个子模型 max_samples看多少样本 bootstrap是否放回 %%time bagging_clf.fit(x_train,y_train)# 500个决策树 bagging_clf.score(x_test,y_test) single_dec_tree = DecisionTreeClassifier() single_dec_tree.fit(x_train,y_train)# 1个决策树 single_dec_tree.score(x_test,y_test) bagging_clf5000 = BaggingClassifier(DecisionTreeClassifier(), n_estimators = 5000,max_samples = 100,bootstrap = True) # 5000个决策树 %%time bagging_clf5000.fit(x_train,y_train) bagging_clf5000.score(x_test,y_test) # 单个决策树的准确率为0.88,500个是0.928,5000个是0.912
这两个问题类似,因为从算法的原理上讲,soft的结果至少不会比hard的差,为啥结果会这样呢?
第二个问题中,子模型的数目增多,一定会使得整体的准确率提高,但是为啥决策树的数目增多,准确率不升反降?
难道存在一些子模型的准确率低于平均准确率吗?很奇怪