作者:手机用户2502885997 | 来源:互联网 | 2023-01-29 21:44
我有以下列的以下熊猫数据框
user_id user_agent_id requests
所有列均包含整数。我不想对它们执行某些操作,而无法使用dask数据框运行它们。这就是我的工作。
user_profile = cache_records_dataframe[['user_id', 'user_agent_id', 'requests']] \
.groupby(['user_id', 'user_agent_id']) \
.size().to_frame(name='appearances') \
.reset_index() # I am not sure I can run this on dask dataframe
user_profile_ddf = df.from_pandas(user_profile, npartitiOns=4)
user_profile_ddf['percent'] = user_profile_ddf.groupby('user_id')['appearances'] \
.apply(lambda x: x / x.sum(), meta=float) #Percentage of appearance for each user group
但我收到以下错误
raise ValueError("Not all divisions are known, can't align "
ValueError: Not all divisions are known, can't align partitions. Please use `set_index` to set the index.
难道我做错了什么?在纯熊猫中,它的效果很好,但是对于许多行来说它变慢了(尽管它们适合内存),所以我想并行化计算。