Log Config

This config script sets up the logging and structlog modules for message logging to console and file. It simply needs to be imported into any module where logging is needed. In order to capture exceptions and crashes main() should be in a try/except block.

Usage:

if __name__ == '__main__':
    from utils import log_config
    log = log_config.log(log_level='INFO')
    log.info(f"Example info text {[1, 2, 3]}", example_key=[1, 2, 3])
    try:
        main()
    except Exception as err:
        log.exception("Example crashed", exception=err)

notes:

Setting up structlog is very tricky, but after it is done it should just work (or so they promise). Regardless, the comments herein should help explain how this works should we ever need to update it.

See documentation. https://docs.python.org/3/library/logging.handlers.html#timedrotatingfilehandler https://www.structlog.org/en/stable/standard-library.html https://www.structlog.org/en/stable/processors.html#chains

class src.utils.log_config.VerboseLogger

Bases: BoundLogger

Wrapper class that adds a new logging method verbose between debug and info levels.

__init__(*args, **kwargs)

verbose(*args, **kwargs)

src.utils.log_config.add_logging_level(level_name, level_num, method_name=None)

Modified from https://stackoverflow.com/a/35804945

Comprehensively adds a new logging level to the logging module and the currently configured logging class.

levelName becomes an attribute of the logging module with the value levelNum. methodName becomes a convenience method for both logging itself and the class returned by logging.getLoggerClass() (usually just logging.Logger). If methodName is not specified, levelName.lower() is used.

To avoid accidental clobberings of existing attributes, this method will raise an AttributeError if the level name is already an attribute of the logging module or if the method name is already present

Example

>>> add_logging_level('TRACE', logging.DEBUG - 5)
>>> logging.getLogger(__name__).setLevel("TRACE")
>>> logging.getLogger(__name__).trace('that worked')
>>> logging.trace('so did this')
>>> logging.TRACE
5

src.utils.log_config.format_floats(_, __, event_dict)

Truncate all floating-point fields to three decimal places.

This is done only for ConsoleRenderer to reduce the size of logs in the screen when running the radar.

src.utils.log_config.log(

console_log_level=None,

logfile_log_level=None,

aggregator_log_level=None,

console=None,

logfile=None,

aggregator=None,

json_to_console=False,

)

Parameters:

• console_log_level (str | int) – Logging threshold for console renderer [CRITICAL, ERROR, WARNING, INFO, VERBOSE, DEBUG, NOTSET]

• logfile_log_level (str | int) – Logging threshold for logfile renderer [CRITICAL, ERROR, WARNING, INFO, VERBOSE, DEBUG, NOTSET]

• aggregator_log_level (str | int) – Logging threshold for aggregator renderer [CRITICAL, ERROR, WARNING, VERBOSE, INFO, DEBUG, NOTSET]

• console (bool) – Enable (True) or Disable (False) console logging override

• logfile (bool) – Enable (True) or Disable (False) JSON file logging override

• aggregator (bool) – Enable (True) or Disable (False) aggregator log forwarding override

• json_to_console (bool) – Enable (True) or Disable (False) a logger that converts JSON logs to console-style

Note

There are three parts to logging: processors, renderers, and handlers.

• processors: modify, add, or clean up the log message dict

• renderers: make the log message a string, json, dict, etc. with fancy styling

• handlers: print the rendered data to stdout, file, stream, etc.

src.utils.log_config.swap_logger_name(_, __, event_dict)

Swaps the kw logger_name value with the module_name and func_name values then removes them from event_dict.

This is done to hack ConsoleRenderer to somewhat match our past format. This is intended only to be used for console rendering and not JSON rendering (prints nice but does not appear in file).

src.utils.log_config.VERBOSE = 15

New logging level, set between DEBUG and INFO.