remage package¶

Submodules¶

remage.cli module¶

remage.cli._cleanup_tmp_files(ipc_info)¶

Remove temporary files created by the C++ application, that might not have been cleaned up.

Parameters:: ipc_info (IpcResult)
Return type:: None

remage.cli._run_remage_cpp(args=None, is_cli=False, num_procs=0)¶

run the remage-cpp executable and return the exit code as seen in bash.

Parameters:

args (Sequence[str] | None)
is_cli (bool)
num_procs (int | None)

Return type:

tuple[list[int], list[Signals], IpcResult]

remage.cli.remage_cli()¶

Return type:: int

remage.cli.remage_run(macros=(), *, gdml_files=(), output=None, threads=1, procs=1, overwrite_output=False, merge_output_files=False, flat_output=False, time_window=None, macro_substitutions=None, log_level=None, raise_on_error=True, raise_on_warning=False)¶

Run the remage simulation utility with the provided args.

This is the main entry point for users wanting to run remage from Python code.

Parameters:

macros (Sequence[str] | str) – one or more remage/Geant4 macro command listings to execute.
gdml_files (Sequence[str | Path] | str | Path) – supply one or more GDML files describing the experimental geometry.
output (str | Path | None) – output file for detector hits.
threads (int) – set the number of threads used by remage. This cannot be combined with procs.
procs (int) – set the number of processes used by remage. This cannot be combined with threads.
overwrite_output (bool) – overwrite existing output files.
merge_output_files (bool) – merge output files created by individual remage threads.
flat_output (bool) – if False, perform a reshaping of the output files so that each row in the output table contains data about all steps in a physical interaction in a detector, based on the time-window. This results in each column being a VectorOfVectors. If True, the output table will be flat with each row holding information about a single Geant4 step.
time_window (float | None) – time window to group together steps into hits, in microseconds.
macro_substitutions (Mapping[str, str] | None) – key-value-pairs that will be substituted in macros as Geant4 aliases.
log_level (str | None) – logging level. One of debug_event, debug, detail, summary, warning, error, fatal, nothing.
raise_on_error (bool) – raise a RuntimeError when an error in the C++ application occurs. This applies to non-fatal errors being logged as well as fatal errors. If false, the function only returns the error code, the Python-based post-processing will be skipped in any case.
raise_on_warning (bool) – raise a RuntimeError when a warning (or error) is logged in the C++ application. If false, warnings are only logged and the python-based post-processing will be run normally.

Return type:

tuple[int, IpcResult]

remage.cli.remage_run_from_args(args=None, *, raise_on_error=True, raise_on_warning=False)¶

Run the remage simulation utility with the provided args.

Parameters:

args (list[str] | None) – argument list, as passed to the remage CLI utility.
raise_on_error (bool) – see remage_run()
raise_on_warning (bool) – see remage_run()

Return type:

tuple[int, IpcResult]

remage.find_remage module¶

remage.find_remage._find_remage_from_config()¶

Return type:: tuple[Path, str] | None

remage.find_remage.find_remage_cpp()¶

Find the remage executable, either by using the config stored into the package at build-time or by using the system PATH.

Return type:: Path

remage.ipc module¶

IPC message receiver implementation for remage-cpp.

Note

The C++ IPC sender implementation can be found in RMGIpc.

Binary message format¶

Messages are encoded as UTF-8 strings; transmitting binary (non-string) data with this IPC mechanism is not possible.

Message parts are separated using ASCII control characters. Each message ends with GS (group separator, 0x1D) which may be optionally preceded by ENQ (enquiry, 0x05) to indicate that the C++ process expects an acknowledgement with a POSIX signal before continuing.

Records within a message are delimited by RS (record separator, 0x1E). Each message must contain at least three records. The first is the process index, for typical (non-multiprocessing) uses, this is always 0. The second is treated as the message’s key, whereas the second one is the associated value:

proc_id

key

RS

value

[ENQ]

GS

More values can follow afterwards, again delimited by RS.

Each record may contain multiple units split by US (unit separator, 0x1F). Records with more then one unit are returned as tuples on the python side.

Example: A message

proc_id

key

RS

value0

RS

value1

US

value2

[ENQ]

GS

would be decoded to ["value0", ("value1", "value2")].

Blocking messages¶

Blocking messages need to be acknowledged by sending ASCII ACK over the second pipe (per-process file descriptor) to the remage-cpp process, after performing the associated action (example: checking version equality of python and C++ IPC sides, pre-processing files).

Transmitting additional response data with the acknowledgement is not possible.

class remage.ipc.IpcResult(ipc_info)¶

Bases: object

Storage structure for the IPC messages returned by remage-cpp.

get(name, expected_len=1)¶

Return all messages of a given key name from the IPC message list.

Parameters:

expected_len (int) – only return messages that have the expected number of records. Other messages are silently skipped.
name (str)

Return type:

list[str]

get_single(name, default)¶

Return the single single value for the key name or default if not present.

Note

if more then one value for the key had been submitted, this function will throw.

Parameters:

name (str)
default (str)

Return type:

str

remove(name)¶

Remove all messages with the given key name from the IPC info.

Parameters:: name (str)
Return type:: None

set(name, values)¶

Replace existing stored messages for they key name with values.

Note

This will create a single-record message for each value in values. Setting more complex messages is not implemented yet.

Parameters:

name (str)
values (list[str])

Return type:

None

remage.ipc.handle_ipc_message(msg, proc)¶

Parse a already UTF-8 decoded IPC message from remage-cpp.

This function should directly handle all known blocking IPC messages, which will not be returned for subsequent processing.

Parameters:

msg (str) – The raw message bytes decoded to UTF-8, still including the trailing separator(s).
proc (list[Popen])

Returns:

(is_blocking, parsed_message, is_fatal) where is_blocking is True when the sender waits for a reply, parsed_message is the decoded message or None if it was consumed internally, and is_fatal signals that the application should terminate. proc_num is the index of the subprocess.

Return type:

tuple[bool, list, bool, int]

remage.ipc.ipc_thread_fn(pipe_r, pipes_o_w, proc, unhandled_ipc_messages)¶

Read and handle IPC messages coming from remage-cpp.

Important

This function runs in a dedicated thread in remage.cli._run_remage_cpp() and should not be called directly by the user. The parameter unhandled_ipc_messages acts as a return value, as thread functions cannot directly return.

The function reads from the pipe file descriptor pipe_r and dispatches each complete IPC message to handle_ipc_message() for parsing and handling of the associated action. Any message parts will be decoded as UTF-8 before parsing.

Blocking messages are acknowledged by sending ASCII ACK over the second pipe (per-process file descriptor).

Parameters:

pipe_r (int) – File descriptor for the read end of the IPC pipe.
proc (list[Popen]) – The subprocess(es) running remage-cpp.
pipes_o_w (list[int]) – File descriptor(s) for the write end of the IPC pipe(s) to the subprocess(es).
unhandled_ipc_messages (list) – List that will receive messages which were not directly handled (i.e., for further processing)

Return type:

None

remage.logging module¶

remage.logging.set_logging_level(logger, rmg_log_level)¶

remage.logging.setup_log()¶

Setup a colored logger for this package.

Return type:: Logger

remage.logging.supports_color()¶

Return type:: bool

remage.post_proc module¶

remage.post_proc.copy_links(original_file, output_files, lh5_links_group_name)¶

Parameters:

original_file (str)
output_files (str | list[str])
lh5_links_group_name (str)

Return type:

None

remage.post_proc.deduplicate_table(file, table_name, unique_col, to_struct)¶

Parameters:

file (str)
table_name (str)
unique_col (str)
to_struct (bool)

Return type:

None

remage.post_proc.get_reboost_config(reshape_table_list, other_table_list, *, time_window=10)¶

Get the config file to run reboost.

Parameters:

reshape_table_list (Sequence[str]) – a list of the table in the remage file that need to be reshaped (i.e. Germanium or Scintillator output)
other_table_list (Sequence[str]) – other tables in the file.
time_window (float) – time window to use for building hits (in us).

Return type:

config file as a dictionary.

remage.post_proc.make_tmp(files)¶

Hide files.

Prepend a . to their name and rename them on disk.

Parameters:: files (list[str] | str)
Return type:: list[str]

remage.post_proc.post_proc(ipc_info, flat_output, merge_output_files, time_window_in_us)¶

Parameters:

ipc_info (IpcResult)
flat_output (bool)
merge_output_files (bool)
time_window_in_us (float)

Return type:

None

remage.post_proc.tmp_renamed_files(remage_files)¶: Temporarily rename files, restoring originals on error and deleting temps on success.

remage.post_proc.un_make_tmp(files)¶

Un-hide files.

Remove . from name and rename on disk.

Parameters:: files (list[str] | str)
Return type:: list[str]

remage.utils module¶

remage.utils._to_list(thing)¶

remage.utils.sanitize_macro_cmds(text)¶

Parameters:: text (str | Iterable[str])
Return type:: list[str]