Smurf File Emulator

The Smurf File Emulator agent creates fake pysmurf and g3 files using the same directory structure that we’re currently archiving on simons1. This is for DAQ end-to-end and bookbinder tests.

It has several tasks/processes that will exist in the ocs-pysmurf-controller, and writes out fake files based on current examples of smurf anciliary data that’s present on simons1.

usage: python3 agent.py [-h] --stream-id STREAM_ID --base-dir BASE_DIR
                        [--file-duration FILE_DURATION] [--nchans NCHANS]
                        [--sample-rate SAMPLE_RATE] [--frame-len FRAME_LEN]
                        [--drop-chance DROP_CHANCE]

Agent Options

--stream-id

Stream ID for fake smurf stream

--base-dir

Base directory where data should be written

--file-duration

Time in sec before rotating g3 files

Default: 600

--nchans

Number of channels to stream from

Default: 1024

--sample-rate

Sample rate for streaming data

Default: 200

--frame-len

Time per G3 data frame (seconds)

Default: 2

--drop-chance

Fractional chance to drop samples

Default: 0

Configuration File Examples

Below are configuration examples for the ocs config file and for running the Agent in a docker container.

OCS Site Config

Here is an example of the file-emulator site-configuration block:

{'agent-class': 'SmurfFileEmulator',
 'instance-id': 'smurf-file-emulator',
 'arguments': [[
    '--stream-id', 'emulator',
    '--base-dir', '/path/to/fake/data/directory',
    '--file-duration', 60,
    '--nchans', 1024,
    '--sample-rate', 200,
    '--frame-len', 2
 ]]}

Note that if this is running in a docker container, the base-dir must be the in-container path, so if you’re mounting a directory to /data, the base-dir should just be /data.

Docker Compose

This agent doesn’t really need to run in a docker container, but if you’re so inclined an example config entry is:

ocs-smurf-file-emulator:
  image: simonsobs/socs:latest
  hostname: ocs-docker
  environment:
    - INSTANCE_ID=smurf-file-emulator
  volumes:
    - ${OCS_CONFIG_DIR}:/config:ro
    - /path/to/fake/data/dir:/data

Agent API

class socs.agents.smurf_file_emulator.agent.SmurfFileEmulator(agent, args)

OCS Agent for emulating file creation for the smurf system.

uxm_setup(sleep=True)

Task - Emulates files that might come from a general tune dets function. These are some of the files found on simons1 registered when running the following ops with a single band:

1. Find-freq

2. setup_notches

3. tracking_setup

4. short g3 stream

Parameters:

sleep (bool, optional) – If True, will sleep for 1 sec after creating the tunefile, which is required for preventing filename collisions in end-to-end testing.

take_noise(tag=None)

Task - Takes a short noise timestream

Parameters:

tag (str, optional) – User tag to add to the g3 stream.

uxm_relock()

Task - Normally this wouldn’t involve a full find-freq, but for emulation purposes it’s ok if this is the same as uxm_setup.

take_iv(wait=True, tag=None)

Task - Creates files generated associated with iv taking / analysis

Parameters:

• wait (bool, optional) – If true, will wait for the 5 seconds where fake IV data is generated

• tag (str, optional) – User tag to add to the g3 stream.

take_bias_steps(wait=True, tag=None)

Task - Creates files associated with taking bias steps

Parameters:

• wait (bool, optional) – If true, will wait for the 5 seconds where fake data is generated

• tag (str, optional) – User tag to add to the g3 stream.

take_bgmap(wait=True, tag=None)

Task - Creates files associated with taking a bias group mapping.

Parameters:

• wait (bool, optional) – If true, will wait for the 5 seconds where fake data is generated

• tag (str, optional) – User tag to add to the g3 stream.

bias_dets()

Task - Creates files associated with biasing dets, which is none.

stream(duration=None, use_stream_between=False, start_offset=0, tag=None)

Process - Generates example fake-files organized in the same way as they would be a regular smurf-stream. For end-to-end testing, we want an example of a pysmurf-ancilliary file, and then regular g3 that rotate at regular intervals. The content of the files here don’t match what actual G3 or pysmurf files look like, however the directory structure is the same.

Parameters:

• duration (float, optional) – If set, will stop stream after specified amount of time (sec).

• use_stream_between (bool, optional) – If True, will use the DataStreamer’s stream_between function instead of writing frames one at a time. This allows you to write an entire timestream for the specified duration without waiting.

• start_offset (float, optional) – If set, this will add an offset to the start time passed to the stream_between function, allowing you to create offsets between streams taken at the same time.

• tag (str, optional) – User tag to add to the g3 stream.

Supporting API

class socs.agents.smurf_file_emulator.agent.Tune(nchans=1720)

Helper class for generating tunes

encode_band(band)

Encodes band-information in the format of pysmurf tunefiles. This has the same structure as pysmurf tunefiles, but contains just enough information for indexing.

encode_tune()

Encodes a full tune dictionary in the format of pysmurf tunefiles.

write_tune(basedir='')

Writes tune to disk.

Parameters:

basedir (str) – Directory where tune should be written.

write_channel_assignments(bands=None, basedir='')

Writes channel assignment files to disk.

Parameters:

• bands (optional, int, list[int]) – Bands to write to disk. Defaults to all that are present in the tune.

• basedir (str) – Directory where files should be written

class socs.agents.smurf_file_emulator.agent.DataStreamer(stream_id, sample_rate, tune, timestreamdir, file_duration, frame_len, action=None, action_time=None, drop_chance=0, tag='')

Helper class for streaming G3 data

end_file()

Ends the current file by sending a G3EndProcessing Frame.

write_next()

Writes the next data frame to disk. Will rotate files based on the current file start time and the file duration. This sleep wait for the frame-duration before writing the G3Frame to disk.

stream_between(start, stop, wait=False)

This function will create a new observation and “stream” data between a specified start and stop time. This function will by default generate and write the data without sleeping for the specified amount of time. To avoid confusion, this will not rotate G3Files since that gets kind of complicated when you’re not running in real time.

Parameters:

• start (float) – Start time of data

• stop (float) – Stop time of data

• wait (bool) – If True, will sleep for the correct amount of time between each written frame. Defaults to False.

class socs.agents.smurf_file_emulator.agent.G3FrameGenerator(stream_id, sample_rate, tune, action=None, action_time=None, quantize=True, drop_chance=0)

Helper class for generating G3 Streams.