作者:ouyan1985_998 | 来源:互联网 | 2023-01-28 10:46
我正在尝试根据本教程的指导为Python sklearn管道创建自定义转换器:http ://danielhnyk.cz/creating-your-own-estimator-scikit-learn/
现在,我的自定义类/变压器看起来像这样:
class SelectBestPercFeats(BaseEstimator, TransformerMixin):
def __init__(self, model=RandomForestRegressor(), percent=0.8,
random_state=52):
self.model = model
self.percent = percent
self.random_state = random_state
def fit(self, X, y, **fit_params):
"""
Find features with best predictive power for the model, and
have cumulative importance value less than self.percent
"""
# Check parameters
if not isinstance(self.percent, float):
print("SelectBestPercFeats.percent is not a float, it should be...")
elif not isinstance(self.random_state, int):
print("SelectBestPercFeats.random_state is not a int, it should be...")
# If checks are good proceed with fitting...
else:
try:
self.model.fit(X, y)
except:
print("Error fitting model inside SelectBestPercFeats object")
return self
# Get feature importance
try:
feat_imp = list(self.model.feature_importances_)
feat_imp_cum = pd.Series(feat_imp, index=X.columns) \
.sort_values(ascending=False).cumsum()
# Get features whose cumulative importance is <= `percent`
n_feats = len(feat_imp_cum[feat_imp_cum <= self.percent].index) + 1
self.bestcolumns_ = list(feat_imp_cum.index)[:n_feats]
except:
print ("ERROR: SelectBestPercFeats can only be used with models with"\
" .feature_importances_ parameter")
return self
def transform(self, X, y=None, **fit_params):
"""
Filter out only the important features (based on percent threshold)
for the model supplied.
:param X: Dataframe with features to be down selected
"""
if self.bestcolumns_ is None:
print("Must call fit function on SelectBestPercFeats object before transforming")
else:
return X[self.bestcolumns_]
我将这个类集成到这样的sklearn管道中:
# Define feature selection and model pipeline components
rf_simp = RandomForestRegressor(criterion='mse', n_jobs=-1,
n_estimators=600)
bestfeat = SelectBestPercFeats(rf_simp, feat_perc)
rf = RandomForestRegressor(n_jobs=-1,
criterion='mse',
n_estimators=200,
max_features=0.4,
)
# Build Pipeline
master_model = Pipeline([('feat_sel', bestfeat), ('rf', rf)])
# define GridSearchCV parameter space to search,
# only listing one parameter to simplify troubleshooting
param_grid = {
'feat_select__percent': [0.8],
}
# Fit pipeline model
grid = GridSearchCV(master_model, cv=3, n_jobs=-1,
param_grid=param_grid)
# Search grid using CV, and get the best estimator
grid.fit(X_train, y_train)
每当我运行最后一行代码(grid.fit(X_train, y_train)
)时,我都会得到以下“ PicklingError”。谁能在我的代码中看到导致此问题的原因?
编辑:
或者,我的Python设置中是否有错误的内容?可能我缺少软件包或类似的东西吗?我刚刚检查我可以import pickle
成功
追溯(最近一次通话最新):文件“”,第5行,文件“ C:\ Users \ jjaaae \ AppData \ Local \ Programs \ Python \ Python36 \ lib \ site-packages \ sklearn \ model_selection_search.py”,行945 ,以适合的方式返回self._fit(X,y,groups,ParameterGrid(self.param_grid))文件“ C:\ Users \ jjaaae \ AppData \ Local \ Programs \ Python \ Python36 \ lib \ site-packages \ sklearn \ model_selection_search。 py”,第564行,在_fit中,用于parameter_iterable文件“ C:\ Users \ jjaaae \ AppData \ Local \ Programs \ Python \ Python36 \ lib \ site-packages \ sklearn \ externals \ joblib \ parallel.py”中的参数,行768 ,通话中
self.retrieve()文件“ C:\ Users \ jjaaae \ AppData \ Local \ Programs \ Python \ Python36 \ lib \ site-packages \ sklearn \ externals \ joblib \ parallel.py”,在第719行中,检索引发异常文件“ C:\ Users \ jjaaae \ AppData \ Local \ Programs \ Python \ Python36 \ lib \ site-packages \ sklearn \ externals \ joblib \ parallel.py“,第682行,用于检索self._output.extend(job.get(timeout = self.timeout))文件“ C:\ Users \ jjaaae \ AppData \ Local \ Programs \ Python \ Python36 \ lib \ multiprocessing \ pool.py”,行608,在中引发self._value文件“ C:\ Users \ jjaaae \ AppData \ Local \ Programs \ Python \ Python36 \ lib \ multiprocessing \ pool.py”,第385行,位于_handle_tasks放置(任务)文件“ C:\ Users \ jjaaae \ AppData \ Local \ Programs \ Python \ Python36 \ lib \ site-packages \ sklearn \ externals \ joblib \ pool.py”,第371行,在发送CustomizablePickler(buffer,self._reducers).dump(obj)_pickle.PicklingError:无法腌制:内建属性查找SelectBestPercFeats失败
Jed..
7
pickle程序包需要在另一个模块中定义自定义类,然后将其导入。因此,创建另一个python包文件(例如transformation.py
),然后像这样导入它from transformation import SelectBestPercFeats
。这样可以解决酸洗错误。
1> Jed..:
pickle程序包需要在另一个模块中定义自定义类,然后将其导入。因此,创建另一个python包文件(例如transformation.py
),然后像这样导入它from transformation import SelectBestPercFeats
。这样可以解决酸洗错误。