Generator that produces Moving Symbol video frames.
This class manages a physical environment in which symbols move around. It also handles rendering of the current physical state. Renders are returned as PIL images, either in RGB or L (8-bit grayscale) mode. It implements the native Python generator interface.
The physical state is initialized based on the parameters given to the constructor (default values are supplied by DEFAULT_PARAMS). Below are the key-value pairs that can be specified:
- data_dir, str: Path to the dataset directory. The dataset directory should contain one folder for each split, e.g. "training" and "testing", and each split directory should contain folders for each class (e.g. 0, ..., 9 for MNIST digits).
- split, str: Name of the data split to sample from
- num_symbols, int: How many symbols should appear in the video
- video_size, (int, int): The resolution of the video as (width, height)
- color_output, bool: Whether to produce "RGB" color images or "L" grayscale images
- symbol_labels, Sequence: The labels for the symbol classes. These must be strings or ints (or any object with str implemented) that match the names of the folders in each split directory
- scale_limits, (float, float): The minimum and maximum scale of an object relative to its original size
- scale_period_limits, (float, float) or list of (float, float): The minimum and maximum duration of a full scale cycle in number of frames
- rotation_speed_limits, (float, float) or list of (float, float): The minimum and maximum angular speed, in radians per frame
- position_speed_limits, (float, float) or list of (float, float): The minimum and maximum translational speed, in pixels per frame
- interacting_symbols, bool: Whether symbols will bounce off each other
- scale_function_type, str: The class of function used to define the scale of each symbol at each time step. Supported options are:
- "sine": Scale is determined by a sine wave
- "triangle": Scale is determined by a triangle wave (i.e. scaling speed is constant, but switches directions if the digit gets too big or small)
- "constant": The symbols do not change scale. Initial scale is randomly sampled from within scale_limits
- rotate_at_start, bool: Whether symbols can start at a rotated angle
- rescale_at_start, bool: Whether symbols can start at any scale in the specified range. If not. the scale of all symbols is initialized to the middle of the allowed scale range.
- lateral_motion_at_start, bool: Whether symbols can only translate left/right/up/down to start. If this is True, symbols can only move non-laterally if they bounce off of other symbols.
- background_data_dir, str: Path to the background directory. This directory should contain one folder for each split, e.g. "training" and "testing", and each split directory should contain folders for each class. If you do not set both this and
background_labels, then all videos will have solid black backgrounds.
- background_labels, list of str: The labels for the background classes. If you do not set both this and
background_data_dir, then all videos will have solid black backgrounds.
A MovingSymbolsEnvironment object also publishes messages that describe each symbol's initialization and state; subscribers can be added with add_subscriber(). A subscriber must implement the process_message method that takes exactly one argument, a dict containing the published message. All messages have the following key-value pairs:
- step, int: The time step that the message describes. For initialization messages, this is -1. The first rendered frame corresponds to step 0.
- type, str: An identifier for what kind of message this is. This can be used to filter out messages and determine the structure of the meta-information without probing it.
- meta, dict: The actual contents of the message.
Below is a list of key-value pairs associated with the meta dict for each message type:
- params: This is a copy of the
params argument passed in to the MovingSymbolsEnvironment object's constructor.
- debug_options: This is a copy of the
debug_options argument passed in to the MovingSymbolsEnvironment object's constructor.
- background
- label: The name of the background class
- image: An np.array containing the full background image of the video
- image_path: Path to the source image
- symbol_init
- symbol_id: The
id field of the relevant Symbol
- label: The
label field of the relevant Symbol
- image: An np.array of the Symbol's full image (dimensions H x W x 4)
- image_path: Path to the Symbol's source image (uncropped)
- vertices: The PyMunk coordinates defining the Symbol's hitbox as a V x 2 np.float array
- symbol_state
- symbol_id: The
id for the relevant Symbol
- position: The Symbol's PyGame coordinates as an np.array
- angle: The Symbol's PyGame angle
- scale: The Symbol's
scale
- velocity: The Symbol's PyGame velocity as an np.array
- angular_velocity: The Symbol's
angular_velocity
- scale_velocity: The Symbol's scale velocity
- start_overlap
- symbol_ids: The
ids of the Symbols that have started overlapping at this time step (as a list)
- end_overlap
- symbol_ids: The
ids of the Symbols that have stopped overlapping at this time step (as a list)
- hit_wall
- symbol_id: The
id of the Symbol that hit a wall at this time step
- wall_label: A label for the wall that was hit at this time step. Can be "left", "right", "top", or "bottom".
1.8.11