如何让dask中的所有其他工作节点都可以访问一个8 GB的大文件?我已经在chunksize和client.scatter上尝试过pd.read_csv(),但这需要很长时间。我正在macOS上运行它。
这是我的代码:
import time
import pandas as pd
import dask as dask
import dask.distributed as distributed
import dask.dataframe as dd
import dask.delayed as delayed
from dask.distributed import Client, pr
Q: What is the largest possible size of an ext3 filesystem and of files on ext3?
Ext3 can support files up to 1TB. With a 2.4 kernel the filesystem size is limited by the maximal block device size, which is 2TB. In 2.6 the maximum (32-bit CPU) limit is of block devices is 16TB, but ext3 supports on