repast4py.logging module

The Logging module contains classes and functions for logging data produced by a repast4py simulation to a file.

class repast4py.logging.DCDataSource(data_class, field_name, ds_name=None)

Bases: object

A DCDataSource implements the repast4py.logging.DataSource protocol for Python dataclasses.dataclass objects. Each DCDataSource gets its data to log from a dataclass field.

The constructor creates a DCDataSource that will log the specified field of the specified dataclass. By default, the field name will be used as the data source name, but an optional data source name can be supplied. The data source name will become the column header in the logged tabular data.

Parameters:

  • data_class (dataclass) – the dataclass containing the values to log

  • field_name (str) – the name of the field in the dataclass to log

  • ds_name (str) – an optional name that will be used as the column header if supplied, otherwise the field_name will be used.

property dtype

Gets the numpy dtype of this DCDataSource.

Returns:

The numpy dtype of this DCDataSource.

property name: str

Gets the name of this DCDataSource.

Returns:

The name of this DataSource.

property value: float

Gets the value of this DCDataSource.

Returns:

The value of this DataSource.

class repast4py.logging.DataSource(*args, **kwargs)

Bases: Protocol

Protocol class for objects that can be used during logging as a source of data. Such objects must have a name, a type, and be able return the data to log via a “value” property.

property dtype: dtype

Gets the numpy dtype of this DataSource.

Returns:

The numpy dtype of this DataSource.

property name: str

Gets the name of this DataSource.

Returns:

The name of this DataSource.

property value: float

Gets the value (i.e., the data to log) of this DataSource.

Returns:

The value (i.e., the data to log) of this DataSource.

class repast4py.logging.ReducingDataLogger(data_source, op, rank)

Bases: object

Creates a ReducingDataRecorder that gets its data from the specified source, reduces that data using the specified op and runs on the specified rank.

Parameters:

  • data_source (DataSource) – the source of the data to reduce.

  • op – an mpi reduction operator, e.g. MPI.SUM

  • rank (int) – the rank of this ReducingDataLogger

property dtype

Gets the numpy dtype of this ReducingDataLogger. This is forwarded from the data source.

Returns:

The numpy dtype of this ReducingDataLogger.

log()

Logs the current value of this ReducingDataRecorder’s data source.

property name: str

Gets the name of this ReducingDataLogger.

This is forwarded from the data source.

Returns:

The name of this ReducingDataLogger.

reduce(comm)

Reduces the values on all processes in the specified communicator to single values using the op specified in the constructor. The reduction is performed on each logged value at which point, the logged values are discarded.

Parameters:

comm (Intracomm) – the communicator over whose ranks the reduction is performed.

Returns:

A numpy array containing the reduced values.

Return type:

array

property size: int

Gets the number of values this logger currently holds. This is set back to 0 on a reduce.

Returns:

The number of values this logger currently holds.

class repast4py.logging.ReducingDataSet(data_loggers, comm, fpath, delimiter=',', buffer_size=1000)

Bases: object

A ReducingDataSet represents a tabular data set where each column is produced by a ReducingDataLogger and where the name of each logger is the name of the column. The produced tabular data is periodically written to a file.

The constructor creates a ReducingDataSet whose columns are produced from the specified data loggers which are reduced across the specified communicator. The data_loggers can be created using the repast4py.logging.create_loggers() function.

Parameters:

  • data_loggers (List[ReducingDataLogger]) – a list of ReducingDataLoggers that will produce the tabular data, one data_logger per column.

  • comm (Comm) – the communicator to reduce over

  • fpath (str) – the file to write the data to

  • delimiter (str) – the delimiter to use to separate the column values

  • buffer_size (int) – the number of values to log before writing to a file.

close()

Closes this ReducingDataSet, writing any remaining data to the file.

log(tick)

Logs the data for the specified tick, by calling log on each data logger contained in this ReducingDataSet.

Parameters:

tick (float) – the tick (timestamp) at which the data is logged.

write()

Writes any currently logged, but not yet reduced data to this ReducingDataSet’s file, by reducing and then writing the resulting data to the file.

class repast4py.logging.TabularLogger(comm, fpath, headers, delimiter=',')

Bases: object

Logs arbitrary values by row in a delimited tabular format. All the rows logged by each rank are concatenated on a write into multiple rows.

Parameters:

  • comm (Comm) – the communicator to reduce over

  • fpath (str) – the file to write the data to

  • headers (List[str]) – the header values for each column

  • delimiter (str) – the seperator to use to seperate the column values

close()

Closes this TabularLogger, writing any rows of data to the file.

log_row(*args)

Logs the specified values as a row in the tabular output.

Each value in the argument list is written to a column in the order they appear in the argument list.

Parameters:

args – variable length argument list containing the values to log. The order of the values should correspond to the headers argument in the constructor.

Examples

The following will log the value of the tick, person_id variables, and 12, and 24 as a row in the tabular data.

>>> logger.log_row(tick, person_id, 12, 24)
write()

Writes all the currently logged rows to a file by gathering all the rows from all ranks, concatenating them, and writing the total collection of rows to the file specified in the constructor. This is a collective operation and must be called by all ranks in the communicator.

repast4py.logging.create_loggers(data_class, op, rank, names=None)

Creates ReducingDataLogger-s from the fields in a dataclasses.dataclass, optionally constraining the loggers to log only from specified fields. By default the names argument is None and all the dataclass fields will be logged.

Parameters:

  • data_class (dataclass) – the dataclass providing the values to log

  • op – an mpi reduction operator (e.g., MPI.SUM)

  • rank (int) – the rank of this process

  • names (Optional[Dict[str, str]]) – a Python dict where the keys are the names of the dataclass fields to log, and the values are the names of the column header in the output tabular data. If value is None, then the dataclass field name will be used as the data source column name.

Returns:

A list of ReducingDataLoggers that can be added to a ReducingDataSet for logging.

Return type:

List[ReducingDataLogger]

Examples

Creating multiple different loggers from the same data source, assigning each a different reduction operation. We append the new loggers to the original list with +=.

>>> meet_log = MeetLog()
>>> meet_log
MeetLog(total_meets=0, min_meets=0, max_meets=0)
>>> loggers = logging.create_loggers(meet_log, op=MPI.SUM, names={'total_meets': 'total'}, rank=rank)
>>> loggers += logging.create_loggers(meet_log, op=MPI.MIN, names={'min_meets': 'min'}, rank=rank)
>>> loggers += logging.create_loggers(meet_log, op=MPI.MAX, names={'max_meets': 'max'}, rank=rank)