opendataval package

Subpackages

• opendataval.dataloader package
  • Subpackages
  • Submodules
  • opendataval.dataloader.fetcher module
  • opendataval.dataloader.noisify module
  • opendataval.dataloader.register module
  • opendataval.dataloader.util module
  • Module contents
• opendataval.dataval package
  • Subpackages
  • Submodules
  • opendataval.dataval.api module
  • Module contents
• opendataval.experiment package
  • Submodules
  • opendataval.experiment.api module
  • opendataval.experiment.exper_methods module
  • opendataval.experiment.util module
  • Module contents
• opendataval.model package
  • Submodules
  • opendataval.model.api module
  • opendataval.model.bert module
  • opendataval.model.grad module
  • opendataval.model.lenet module
  • opendataval.model.logistic_regression module
  • opendataval.model.mlp module
  • Module contents

Submodules

opendataval.metrics module

class opendataval.metrics.Metrics(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: FuncEnum

ACCURACY = accuracy

NEG_L2 = neg_l2

NEG_MSE = neg_mse

opendataval.metrics.accuracy(a: Tensor, b: Tensor) → float

Compute accuracy of two one-hot encoding tensors.

opendataval.metrics.neg_l2(a: Tensor, b: Tensor) → float

opendataval.metrics.neg_mse(a: Tensor, b: Tensor)

opendataval.util module

class opendataval.util.FuncEnum(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: StrEnum

Creating a Enum of functions identifiable by a string.

static wrap(func: Callable[[X, ...], Y], name: str | None = None) → wrapper[X, Y]

Function wrapper: class functions are seen as methods and str conversion.

class opendataval.util.MeanStdTime(input_data: list[float], elapsed_time: float = 0.0)

Bases: object

Formats Mean and standard time.

class opendataval.util.ParamSweep(pred_model, evaluator, fetcher, samples: int = 10)

Bases: object

sweep(**kwargs_list) → dict[str, MeanStdTime]

class opendataval.util.ReprMixin(*args, **kwargs)

Bases: object

Gives unique repr of object based on class name and arguments

class opendataval.util.StrEnum(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: str, Enum

StrEnum is not implemented in Python3.9.

opendataval.util.batched(it, n=1)

opendataval.util.get_name(func: Callable) → str

Gets name from function.

opendataval.util.load_mediator_output(filepath: str)

Loads output of Pandas DataFrame csv generated by ExperimentMediator.

opendataval.util.set_random_state(random_state: RandomState | None = None) → RandomState

Set the random state of opendataval, useful for recreation of results.

class opendataval.util.wrapper(function: Callable[[X, ...], Y], name: str | None = None)

Bases: str, Generic[X, Y]

Module contents

Framework with data sets, experiments, and evaluators to quantify the worth of data.

opendataval

opendataval provides a framework to evaluate the worth of data. The framework is easily extendable via adding/registering new datasets via DataFetcher + Register, creating your own DataEvaluator via inheritance, or creating new experiments to be run by ExperimentMediator. The framework provides a robust and replicable way of loading data, selecting a model, training (several) data evaluators, and running an experiment to determine performance on all of them.

Modules

dataloader	Create data sets and loads with DataFetcher.
dataval	Create DataEvaluator to quantify the value of data.
model	Prediction models to be trained, predict, and evaluated.
experiment	Run experiments on DataEvaluator.

Utils

set_random_state([random_state])	Set the random state of opendataval, useful for recreation of results.
load_mediator_output(filepath)	Loads output of Pandas DataFrame csv generated by ExperimentMediator.
__version__	Version release number.