ielearn.batch_io module

A collection of utility functions for file i/o.

class ielearn.batch_io.BatchFileHandler(file_ext, batch_size)

Bases: object

DEFAULT_BATCH_SIZE = 5000

JOIN_FUNCS = {'.csv': <function concat>, '.pkl': <function concat>}

SUPPORTED_EXTENSIONS = ('.pkl', '.csv')

verify_path(fn)

Parameters:fn –

class ielearn.batch_io.BatchFileIO

Bases: ielearn.batch_io.Factory

Factory to return a batch reader/writer.

class ielearn.batch_io.BatchFileReader(fns, batch_size=None)

Bases: ielearn.batch_io.BatchFileHandler

Reads files into a DataFrame in a batched fashion.

READ_FUNCS = {'.csv': <function _make_parser_function.<locals>.parser_f>, '.pkl': <function read_pickle>}

read()

class ielearn.batch_io.BatchFileWriter(fn, batch_size=None)

Bases: ielearn.batch_io.BatchFileHandler

Writes a DataFrame to files in a batched fashion.

WRITE_FUNCS = {'.csv': <function DataFrame.to_csv>, '.pkl': <function NDFrame.to_pickle>}

write(df)

Parameters:df –

class ielearn.batch_io.Factory

Bases: object

Abstract factory class to dynamically return a specific internal instance. Subclasses of this class should define the internal classes supported.

ielearn.batch_io.demo()

ielearn.batch_io.extension(fn)

Parameters:fn –