µSR Automatic Run Control


Introduction

The automatic run control operates as a separate process, distinct from other data acquisition front-ends and back-ends, including the run-control user-interface. It should be started automatically, but if it is not, then one can execute $MUSR_DIR/musr_midas/mui/autoruns.tcl. The autorun server is controlled by a few parameters in the ODB, under /autorun, but it is mostly controlled by the auto-run plan file, which it monitors continuously for changes.

(The ODB is the Midas data acquisition system's central storage of experiment parameters. It can be explored and edited using the odbedit program or a web browser [when the http server is running], but all the ODB parameters mentioned in this document are accessible from the µSR run control graphic interface.)

Control Panel

The run-control program has a small panel for controlling the auto-runs, accessed through the menu selection “Extras > Auto-Run” (or Alt-E  A). There you can set the relevant ODB parameters, including the name of the plan file.

In the future, there will be a customized run-plan editor built into the data acquisition user-interface, but for now, one must construct the plan file using a regular text editor. You can use the “Edit Plan” button on the auto-run control panel, but it is also effective to run a text editor by hand. The plan file is (re)loaded whenever it is modified, as long as autoruns are enabled.

If there are problems with the syntax of a plan file, the autorun processor usually only reports the first error it encounters, usually in a pop-up window. If the autorun processor is connected to a different display, however, the error message might only appear in the run-status window. If you edit the plan file via the “Edit Plan” button, you will edit a scratch copy of the plan file in a text editor. This file is automatically checked for errors before being copied to the actual plan file (after prompting: “Ready to use it?”). The plan may also be checked for errors using the “Check” button. There is also a stand-alone command, “checkplan” (executing $MUSR_DIR/musr_midas/mui/checkplan.tcl; usage “checkplan  filename  [first_run]”) that performs checks on a plan file, useful for verifying a hand-edited plan. Nevertheless, the autorun processor may encounter some errors that the preliminary scan does not detect, so these checks will not completely replace a good proofread.

Auto-run Control Parameters

There are a few parameters to set on the autorun control panel. Most of these are passed to the autorun processor through the Midas ODB (in the area “/autorun”).

Operation  Enable/Disable
ODB:  /autorun/enable,  boolean (y/n)
Tell whether the autorun process operates at all. It does not actually shut down when disabled, but it checks only this parameter, waiting to be enabled again.

State  Autoruns status read-back
ODB:  /autorun/state,   int   interpreted as:

    0   disabled    Operation is not enabled
    1   idle    Waiting for a plan
    2   acquiring    Run is in progress and active
    3   paused    Run is paused
    4   ending    Ending a run
    5   stopped    Run has ended
    6   setting    Setting Camp variables
    7   changing    Waiting for “requirements”
    8   starting    Starting a new run
    9   reload    Trigger reload of plan file

Most state values are set by the autorun server, strictly for monitoring its progress, but a value of “reload” (9) may be set by other clients to force the autorun server to re-read the plan file, and perhaps resume processing after being idle (1); the Apply button will trigger reloading this way. (The other way to force a reload is to modify the plan file itself, perhaps like touch plan.dat.)

Plan file  Full file specification for the run plan
ODB:  /autorun/plan file,   string
The autorun processor watches this file for changes, and if any occur, it re-reads the run plan. Note that, if you type the plan file name in the entry field, you must apply the change by pressing Enter or clicking Apply.

Pausing  Enable/Disable run pausing
ODB:  /autorun/enable pausing,   boolean (y/n)
Tells the autoruns processor whether to keep checking requirements after starting a run, and to pause the run while requirements are not satisfied. See below for “Requirements” in an autorun plan. (A Camp alarm with a hardware veto gives better response.)

Refresh Period  Cycle time of autorun processor
ODB:  /autorun/refresh seconds,   int
Tells how often the autorun processor checks status and looks for a new plan. Sensible values range from 5 to 30 seconds.

Target Counts  Desired total counts for TD-µSR runs
ODB:  /autorun/target counts,   int
Tells how many events to accumulate in histograms before ending the run. This number is reset for every run, according to the run plan setting (if that run has a “counts” setting in the plan), but it can be changed during the run to take more or less data.

Note that the target-counts setting is used by the run-status display for a progress bar, even when autoruns are disabled or idle. To make use of the progress bar, set the target counts as desired; to disable the feature, set the counts to zero.

Count Histogram  What “Target Counts” applies to
ODB:  /autorun/count histogram,   int
The “Target Counts” may be compared with the counts in any particular histogram or with the total of all histograms, as indicated by the “Count Histogram” setting. The first histogram is number 1. Enter zero or negative to use the total of all histograms.

Run Time Limit  Maximum time before ending a run
ODB:  /autorun/time limit (minutes),   float
You may enter the time limit in various formats, but the value will be displayed and saved as a number of minutes. Use a very high value or zero if there is to be no time limit. Between runs, this value may be reset by any specifications in the run plan. It can be manually changed during a run without editing the plan. If your run seems to end prematurely, check this parameter!

The following control panel features are not passed through the ODB.

Editor  Preferred editor for editing plan file
There will be a GUI page for creating and editing run plans, but only after we have more experience with using auto-runs control. For now, there is a choice of various plain text editors.

Edit Plan  Invoke the editor on the Run plan file.
The editor actually works on a temporary file, which is synchronized with the plan file, and when the temporary file is saved it will be checked for errors before being copied over the actual plan file. You will get confused if you open multiple plan editors simultaneously, of if you change the plan file name while editing the file, so exit the editor when finished a group of changes, and re-use the same editor window when one is already present.

Check  Check the plan for errors. This happens automatically when you save a plan file from the editor, but the “Check” button checks the real plan file, not the temporary copy.

Renumber  Use this when you have a plan file where the run numbers do not match the actual runs. It alters the run numbers in the plan file to count consecutively beginning from what will be the number of the next run.

Help  Display this documentation.

Apply  Apply the settings. If autoruns are enabled, but idle, then force reloading the plan, even if it hasn't changed. Pressing the Enter key also invokes the Apply function, as does selecting Enable or Disable. The Apply button shows red when there are unapplied changes.

Revert  Undo any changes that have not been applied.

Quit  Close the control window without applying.

Exit  Apply settings, if there are any changes pending, and close the control window.

Run plan file: commands and syntax

The run plan consists of a series of commands, in plain text, grouped by run number - all commands in a group apply to a particular run. Each command consists of a keyword, optionally followed by a colon ( : ), and white space before any values. Besides the trailing colon, embeded underscore characters (_) are ignored, as is capitalization of the command. Although they are ignored, capitalization, underscores, or colons can make a plan file more readable, especially when there are continued lines. In this documentation, the optional capitalization is usually used, and the colon used occasionally.

Commands may be broken across multiple lines by ending the partial line(s) with “\”.

Lines that begin with certain characters (!#%;) are comments, and are ignored, as are blank lines.

In the command syntax below, bold indicates a literal keyword, italics indicate some sort of value, “|” separates multiple possibilities, and “[ ]” brackets indicate some optional parameter. The “|”, “[”, and “]” characters should not be typed into a run plan!

One command, Run, is absolutely required, and it comes first:

Run  number
Run  next
Next  run

The Run command begins the commands for that particular run; all ensuing commands apply to that run, until the next Run command. The specified run number must be consecutive with the previous. You can use the next keyword instead of a number, but the first run must be explicitly numbered. (The reason the numbering must be defined is so that the plan can be edited and reloaded in the middle, without having to remove all the runs that have already been performed.)

If there are settings to be made after the last run has completed, use

Finally

to separate those settings from the preceding run; it is like a Run command, but no run will be performed.

If it is necessary to switch experiment type (TD-μSR or I-μSR), use the command:

muSRType  I-muSR  |  TD-muSR

Only the first letter of the type matters (I or T). This command is optional, and, if omitted, the experiment type remains the same as previous. If the muSRType command is given, it should immediately follow the Run command (at least, there must be no intervening commands that might depend on the run type.)

The remaining commands can be in any order.

In the case of I-µSR runs, a sweep range is required:

SweepRange  from  delim  to  delim  step

The delimiter delim signifies white-space and/or punctuation characters or even any of the letters in “toby”;   e.g.,  “10,100:2”  or  “5000 7000 500”  or  “1 to 10 by 1”. People might try to use  “200-300:10”  for a range, but “-” is not a legal delimiter since the dangers of misinterpreting a minus sign are too great. All the values (from, to, step) are integers.

Other commands (all the commands listed below) are strictly optional, and, if omitted, they retain the previous settings; that is, the values will not be actively set to any values, but will be retained by the respective Midas or Camp systems.

There are three possible commands to control when the run should be ended:

Counts  number [ M ]  [ hist_number ]    ( for TD )
Time_limit  elapsed_time     ( for TD or I )
Sweeps  number_of_sweeps     ( for I )

Typically, one should use Counts to tell the total number of events to accumulate in a TD-µSR run; it can be given in “real” number form, and with an optional “M” suffix to indicate “millions” (e.g., 3200000, 32e5, and 3.2M are all the same). This parameter will replace the value in the ODB at the beginning of a run, but a user may manually change it in the ODB while the run is in progress, which is the best way to extend an automatic run in progress. If no Counts command is given for a run, the existing value of “/autorun/total counts” from the ODB is kept. If the optional hist_number is specified, then the counts are checked for that histogram only, rather than the total of all histograms.

The Time_limit setting (with alias “Elapsed”) tells the longest time that may be spent on a run. The elapsed_time can be in many formats, and a simple number is interpreted as minutes; all of the following are equal:  1:30,  5400 s,  90 min,  1.5hr,  01:30:00,  90.  Units are any strings that begin with “s”, “m”, or “h”. Notice that “1:30” means 1.5 hours, not 1.5 minutes! For 1.5 minutes you would need “0:01:30” (or “1.5 min” or “90s”). You can specify both counts and time limits, in which case the run ends when either is satisfied. Note that using an elapsed-time limit may result in runs with no data if the beam is off for a long time.

The Sweeps command is for I-µSR runs only, and tells how many sweeps to make. It updates the corresponding parameter in the ODB: “/Equipment/MUSR_I_acq/settings/input/num sweeps”, which applies to any run, not only auto-runs.

Arrange to have autorun error and warning messages forwarded by email with

Email  addr  [ , addr ... ]

providing a list of email addresses, separated by commas. When there are problems, the corresponding messages will be emailed to these addresses. If you have an address that gets forwarded to a cell phone or pager, that would be good to use.

Several commands apply to the run headers, corresponding to the Midas “edit on start” variables.

Sample   ...
Orientation   ...
Operator   ...
Experiment   number
Temperature   temperature  |  Camp_var
Field     field  |  Camp_var

Note that temperature and field may be entered as numbers (with implicit units of Kelvin and Gauss), as numbers with units (provided the conversion to Kelvin or Gauss is known to the program), or as a the full Camp path for a temperature or field variable.

Title:   ...

The run title should only be provided for TD-µSR and β-NMR runs (not for I-µSR). It allows automatic insertion of other header fields using the tags <Sample>, <Operator>, <Experiment>, <Temperature>, <Field>, and <Orientation> (yes, you type the “angle brackets” around the tag name). For example:

Title: <Sample> annealed, T=<Temperature>, B=<Field> <Orientation>, p=+2%

The autorun system will replace these tags with the appropriate header values, both at the beginning of the run and at intervals throughout the run; this ensures that automatic Temperature and Field headers, as well as any manual settings, are propagated to the run title. The system will STOP updating the title if the user manually sets it to something different.

For I-µSR (not TD) there are the header info settings:

Comment1   ...
Comment2   ...
Other   ...

The “Other” value is used for constructing the I-µSR run title; think of it as a subtitle. Do not enter a title for I-µSR runs, as it will be overwritten anyway.

For time-integral runs, there is also the percentage tolerance for the beam normalization:

Tolerance   number   [ % ]

(The percent sign is optional, and does not affect the interpretation of the number.)

Other acquisition control parameters have no “personalized” commands, but should be set by loading a previously-saved configuration:

Mode  saved_TDmuSR_mode_name
Setup  saved_ImuSR_setup

However, any Odb parameter can be set using the SetOdb command:

SetOdb  odb_variable   value

where odb_variable is the full Odb path to the variable, enclosed in quotes if the name contains any spaces. The value should also be quoted if it contains spaces (use double-quotes, not apostrophes).

You can control the beamline using the commands

SetEpics  Epics_var   value
LoadTune  Tune_name   [ MoveSlits ]   [ Argon=keyword ]
MoveSlits  Tune_name
TuneBeam  Script_name   [ Tune_name ]
SaveTune  Tune_name

The SetEpics command (with spelling variants) sets a particular beam element to a value. Epics variables are always in upper case, and contain colons (“:”) as separators; they usually begin with the beamline or system name. To find the Epics_var, go to the beamline Epics control window and click the middle mouse button on the entry field (blue text) where you would type the value. The Epics_var will be displayed in a small pop-up window (the text is “selected”, so you can immediately “paste” it into the plan file). The full Epics_var may be abbreviated for some elements commonly adjusted in autoruns, especially for turning elements on or off. When switching an element on or off, the value should be “on” or “off” for clarity; thus the examples

SetEpics M20:EXPT:CUR 215      set magnet to 215 A
Setepics EXPT 215      abbreviated set magnet to 215 A
Setepics M20:EXPT:DRVOFF 1      confusing turn off magnet
Setepics EXPT on      simplified turn on magnet again

The LoadTune command (alias RestoreTune, plus variants like load_tune) reloads a saved tune identified by the Tune_name, which is the base name of the tune file, without directories or extension. This does not change any slits settings unless the optional keyword “MoveSlits” is given (or, really, any word containing “slits”). The separate MoveSlits command is an alias for “LoadTune Tune_name  MoveSlits”, so it restores the magnet settings and moves the slits. Argon for the separators will typically be turned on automatically if necessary for the tune's spin-rotator settings (but not turned off), and the behaviour can be selected by providing the optional Argon=keyword argument, with options:

    onEnsure argon flow is on
    offEnsure argon flow is turned off
    safeEnsure argon flow is on if the tune voltage is high
    autoSet argon on or off depending on separator voltage
    none   Do no setting (any other keyword)

TuneBeam (or autotune or multiplet_tune) should not be required often. It runs a “multiplet” tuning script to optimize the beamline settings, where the Script_name is the full file name of the tuning script, but without a directory path. After tuning, the resulting settings may be saved as the (optional) Tune_name; two copies will be saved: Tune_name.snap and time-stamp_Tune_name.snap. (SaveTune skips the autotuning and saves the current beamline settings, if that could ever be useful.)

Examples are:

LoadTune e1234TF
autotune tuneend.scr
load_tune HelLF20kG moveslits Argon=off

Make Camp settings using either or both of:

SetCamp  Camp_var   value
Camp_cmd  Camp_command

(SetCamp has aliases set_camp, camp_set and CampSet, because we all forget.)

The Camp_var argument is a complete Camp variable path, including all slashes, such as “/Helios/mag_field”.

The value argument should be numeric or a text string, as appropriate for the particular Camp_var. Numeric values may be given as arithmetic expressions. For selection variables, one must use the exact same string as used by Camp, including capitalization, or an integer index, counting from zero. As a special shortcut, abbreviations of the form <Camp_var> (with explicit angle-brackets) may be used as (or in) the value argument, indicating that the value of another Camp variable should be substituted.

The Camp_command is any command (or inline Tcl script) as documented in Appendix B of the Camp Software Manual.

Examples are:

SetCamp: /Magnet/mag_field 0.25
SetCamp: /Sample/control_set 15
SetCamp: /Diffuser/control_set </Sample/control_set> - 0.5
SetCamp: /field_control/setpoint </Hall/field>
SetCamp: /Diffuser/heat_range MED
camp_cmd: insLoad /defibrulator medical.kit

The autorun controller will perform all those Camp and Epics commands at once, as soon as the preceding run has ended. This doesn't meet the full needs for setting Camp devices, particularly in cases such as the field control example above, where one would like to reference the /Hall/field value after the magnet has stabilized. One way to handle this is to schedule some settings after a fixed time delay:

After elapsed_time :  SetCamp  . . .  |  camp_cmd  . . .  |  SetEpics  . . .  |  AutoTune  . . .  |

where the forms of elapsed_time is similar to the run time-limit, except that a bare number is interpreted as seconds (six minutes could be given by any of 6m, 0:06, 0.1h, 360s, 360). This is the first of two composite commands, where a required colon (:) must separate the “After” declaration from the ensuing, delayed, command. Note that only some commands can be deferred with After; any others give a syntax error (because they aren't allowed).

Before a run can begin, we must know that the experimental conditions have responded to the device settings, so we specify “requirements”

 
 
blank 
    stable    at     number     [ within  error ]   [ for  elapsed_time ]
      equal  Variable  
Require    Variable   
    above      number
  [ for  elapsed_time ]
      below     number
   
  is    string

The run will not begin until every requirement is satisfied simultaneously, and there are no Camp alarms active. (Thus, you can enforce many typical constraints by setting alarms in Camp, rather than giving requirement commands.)

Each Variable is the full path specification of a Camp variable, such as “/Sample/sample_read”, or an Epics variable, such as “M20:S1:HVNEG:VOL” (distinguished by the presense of / or :).

For numeric variables, one should usually use the “stable” form of the command. The autorun system will collect values of the primary Variable, retaining those over the latest elapsed_time. To satisfy the requirement, all retained values must be within error of either: the latest value (blank comparison fields), or the given number (“at”), or the current value of the comparison variable (“equal”). (Notice that  Require /foo/bar stable  is the same as  Require /foo/bar stable equal /foo/bar.)

The default elapsed_time, if none was specified, is 1 second (almost immediately). The default units, if a bare number was given, is seconds.

Numeric variables can also be tested to be “above” or “below” some value.

For string or selection variables, use the “is” form for the requirement. If the string contains spaces, then it should be quoted.

Examples:

Require /diffuser/control_read stable equal /diffuser/control_set within 0.5
Require /sample/sample_read stable within 0.5 for 2 m
Require /Hall_Probe/field stable at 1.0 for 30
Require /shield/sample_read below 100
Require M20:EXPT:RDCUR stable equal M20:EXPT:CUR within 2
Require /Magnet/ramp_status is Persistent

(Note that the last example specifies “Persistent”; “persistent” would not work because the actual value in Camp is capitalized. Make sure the requirement string exactly matches what Camp displays.)

Here is another example, part of a temperature scan, where the desired temperature is specified just once, on the diffuser set-point.

Run 1234
Title <Sample>, <Temperature>, automatically
Temperature /sample/sample_read
Require /sample/sample_read stable equal /diffuser/control_set within 3
Require /sample/sample_read stable within 0.5 for 2m
Setcamp /diffuser/control_set 22

The run will begin when the sample temperature levels off (within 0.5 K) for 2 minutes, and within 3 K of the diffuser setpoint (i.e., between 19 and 25 K). Whatever the average sample temperature is, over the course of the run, it will be recorded in the temperature field of the run header, and thence the run title of the data file. Note that, in this example as always, the Setcamp command is performed before ever checking requirements; entering the Setcamp declaration after the requirements does not imply that the settings are performed after requirements are satisfied. For that feature, see the When command.

Max_wait  elapsed_time

In case of conflicting requirements, or some other problem where the requirements are never satisfied, there is a Max_wait command. After waiting for elapsed_time, the autorun sequencer will start the next run even if the requirements are not satisfied. The various formats for elapsed_time are that same as on the Time_limit command, with default units of minutes (see above).

For greater control over the timing of Camp settings than is provided by the After command, one can schedule them the same way as specifying a requirement:

When  requirement_specification  :  SetCamp   ...  |  camp_cmd   ...  |  SetEpics   ...  |  AutoTune   ...  |  After   ...  |  blank

The requirement_specification is any valid parameter list for the Require command (above), and is followed by a mandatory colon separator. Only the device setting commands for Camp and Epics (SetCamp, SetEpics, camp_cmd, and their aliases) and the After command may be scheduled this way. You can also specify a null action with no scheduled command, which behaves like a simple Require command except...

Conditions specified by When must all be satisfied at some point before a run can begin, but they need not remain satisfied until the run begins. This makes a When command with a null action subtly different from the same Require command: all Require conditions must be simultaneously satisfied when the run begins, whereas all When actions (or null actions) must have been performed sometime before the run begins.

If a Camp setting must be delayed until after the run begins, then you must use an After command, either alone or scheduled by a When command, like

When  /sample/sample_read stable within .5 : After 5m: setCamp /sample/setup/P 20

A typical use is to start controlling a variable after a previous setting is complete:

When /Hall/field stable within .5: setCamp /field_cont/setpoint </Hall/field>
When /Hall/field stable within .5: setCamp /field_cont/function 2
When /sample/sample_read below 8: setCamp /sample/heat_range LOW
When /diffuser/sample_read stable equal /diffuser/control_set within 1 for 1.5m: \
        After 3m: setCamp /nv_cont/function 2

(Note the use of \ to join multiple lines when commands get too long, and the substitution for the current value of the field readback.) This example turns on (feedback) field control when the magnet current reaches its setpoint, and turns on needle-valve control three minutes after the cryostat diffuser temperature gets close to its setpoint. Notably, it also specifies the same requirements for beginning the run; thus, presuming the (diffuser) temperature is the last to stabilize, the needle-valve control will begin about three minutes after the run begins.

In the example (and in practice) note the use of multiple When commands having the same requirement but different actions. To save typing and computer processing, these can be written as a block

When  requirement_specification   do  
   .  .  .
enddo
                       
When  requirement_specification   {  
   .  .  .
}

where multiple camp or epics setting commands are surrounded by “do...enddo”, or their equivalents, curly braces “{...}”, and controlled by a single When command. For example,

When  /Hall/field stable within .5 {
  setCamp /field_cont/setpoint </Hall/field>
  setCamp /field_cont/function 2
}

Note which commands go on separate lines in this example. You cannot combine those multiple lines onto one.

That completes the definition of the plan file. Please relate your experiences, frustrations, and suggestions to Donald.