dlk package

Subpackages

Submodules

dlk.online module

dlk.predict module

class dlk.predict.Predict(config: Union[str, dict], checkpoint: str)[source]

Bases: object

Config Example:
>>> {
>>>     "_focus": {
>>>     },
>>>     "_link": {},
>>>     "_search": {},
>>>     "config": {
>>>         "save_dir": "*@*",  # must be provided
>>>         "data_path": "*@*",  # must be provided
>>>     },
>>>     "task": {
>>>         "_name": task_name
>>>         ...
>>>     }
>>> }
get_data(config)[source]

get the data decided by config

Parameters

config – {“config”: {“data_path”: ‘..’}}

Returns

loaded data

get_datamodule(config, data)[source]

get the datamodule decided by config, and fit the data to datamodule

Parameters

  • config – {“task”: {“datamodule”: ‘..’}}

  • data – {“train”: ‘..’, ‘valid’: ‘..’, ..}

Returns

datamodule

get_imodel(config, data)[source]

get the imodel decided by config

Parameters

  • config – {“task”: {“imodel”: ‘..’}}

  • data – {“train”: ‘..’, ‘valid’: ‘..’, ..}

Returns

imodel

get_manager(config, name)[source]

get the tranin/predict manager decided by config

Parameters

  • config – {“task”: {“manager”: ‘..’}, “config”: {“save_dir”}}

  • name – the predict progress name

Returns

manager

predict(data=None, save_condition=False)[source]

init the model, datamodule, manager then predict the predict_dataloader

Parameters

data – if provide will not load from data_path

Returns

None

trace()[source]

trace the model to torchscript :returns: TODO

dlk.process module

class dlk.process.Processor(config: Union[str, Dict])[source]

Bases: object

fit(data: Dict[str, Any], stage='train')[source]

Process the data and return the processed data

Parameters

  • data – {“train”: .., ‘valid’: ..}

  • stage – “train”/ ‘predict’, etc.

Returns

processed data

dlk.train module

class dlk.train.Train(config: Union[str, Dict], ckpt: str = '')[source]

Bases: object

Trainer

Config Example:
>>> {
>>>     "_focus": {
>>>     },
>>>     "_link": {},
>>>     "_search": {},
>>>     "config": {
>>>         "save_dir": "*@*",  # must be provided
>>>         "data_path": "*@*",  # must be provided
>>>     },
>>>     "task": {
>>>         "_name": task_name
>>>         ...
>>>     }
>>> }
dump_config(config: Dict, name: str)[source]

dump the config and change the log file path to config[‘config’][‘save_dir’]+name

Parameters

  • config – {“config”: {“save_dir”: ‘..’}}

  • name – config name

Returns

None

get_data(config)[source]

get the data decided by config

Parameters

config – {“config”: {“data_path”: ‘..’}}

Returns

loaded data

get_datamodule(config, data)[source]

get the datamodule decided by config, and fit the data to datamodule

Parameters

  • config – {“task”: {“datamodule”: ‘..’}}

  • data – {“train”: ‘..’, ‘valid’: ‘..’, ..}

Returns

datamodule

get_imodel(config, data)[source]

get the imodel decided by config, and inject the origin test and valid data

Parameters

  • config – {“task”: {“imodel”: ‘..’}}

  • data – {“train”: ‘..’, ‘valid’: ‘..’, ..}

Returns

imodel

get_manager(config, name)[source]

get the tranin/predict manager decided by config

Parameters

  • config – {“task”: {“manager”: ‘..’}, “config”: {“save_dir”}}

  • name – the predict progress name

Returns

manager

run()[source]

run for all configs

Returns

None

run_oneturn(config, name)[source]

run this config

Parameters

  • config – {“root”: ‘…’}

  • name – config name

Returns

None

Module contents