Simulation plan tutorial

A simulation plan is the definition of a set of simulations where, for each simulation, certain parameters are given different values in order to elucidate a problem. There are two main categories of simulation plans.

  • Independent simulations, where the order in which the simulations are run has no effect and, therefore, the simulations may be run in parallel on different computers. The values of the parameters for each simulation are predetermined.
  • Sequential simulations, where the parameters of each simulation depend on the results from the previous simulation.

Independent simulations (e.g. for sensitivity analysis)

The first step in setting up a simulation plan for independent simulations is to define the parameters that will be varied and the values that they will take. These are then organized into an appropriate experimental design (e.g. full factorial, fractional factorial), which is defined as a matrix. The sample script described here for setting up a plan for independent simulations may be modified by the user and users are free to create their own scripts if the existing scripts are not suitable for their purpose. This script requires two types of data files, the experimental design matrix (matrix.txt) and the parameter files (parametername.txt).

Experimental design matrix

The experimental design matrix has a row for each simulation run and a column for each parameter to be varied. The values of the parameters are represented by integer keys (e.g. -1, 0 and 1, for lower value, normal value and higher value). The matrix is defined in a .txt file, without column or row headers, with the keys separated by semicolons.

Parameter files

There is a parameter file for each parameter to be varied. This specifies the correspondence between the keys and the values of the parameter. The parameter may be a value, a list of values or an expression.

If the experimental design specifies two keys, -1 and 1, for a parameter then the parameter file will look like

Parameter defined by a list
-1=0.5;0.8;0.9;0.11
1=0.5;0.11;0.15;0.41
Parameter defined by an expression
-1=if(condition = true) return 5 ; else return 3 ;
1=if(condition = true) return 7 ; else return 3 ;
Parameter defined by a single value
-1=0.8
1=1.2
Rules

The parameters for rules can also be modified. The parameter file for the rule must define all the values for the public parameters of the rule (see file amp.txt in the sample files http://www.isis-fish.org/v4/downloads/Exemple_directory.zips). For some rules, it may be necessary to define the population, the fishing gear, the zone, etc. targeted by the rule. In this case the name of the object cannot be used and the internal code for the object is required. This is of the form

fr.ifremer.isisfish.entities.Zone#1169028645767#0.37798185123822536

To determine the code, run a simulation with the rule correctly parameterized. You will find the code in the debug.txt file (in the same directory as the .bat or .sh file used to run ISIS-Fish) against the parameter name at the point where the rule is initialized.

N.B. the syntax in the simulation plan files is strict.

  • No spaces
  • No semicolon at the end of the line
  • No leading + sign for positive keys (+1 is not allowed, use 1)

Sample files

These sample files are for a sensitivity analysis for the parameters describing growth, accessibility, selectivity and closure period for a regulated fishing area.

sample matrix.txt and parametername.txt files

The matrix.txt and parametername.txt files should be placed in single directory, preferably a sub-directory of the directory with the .bat or .sh file used to run ISIS-Fish.

sample simulation plan script

Simulation plan scripts can be written or modified using the script editor. If you do not use the script editor, the sample script should be copied to isis-fish-4isis-databasesimulationplans in your user or home directory. This script sets the values of the parameters to be varied for each simulation using the experimental design.

This script has all the methods for reading the matrix.txt and parametername.txt files. The directory with the matrix.txt and parametername.txt files should be declared in the script (around line 117).

public String param_directory = "Sample_directory/"

The parameters to be varied, their order in the matrix and the filenames are defined by the declarations in the script around lines 104-108 and 158

static private final String MATRIX = "matrix";
...
static private final String AMP = "amp";
Arrays.asList(new String[] { VBGF, CATCHABILITY, SELECTIVITY, AMP }

The script also has code for modifying the parameters in the database (see the API documentation (http://www.isis-fish.org/v4/user/API.html) for the methods that can be used to modify the database).

Sequential simulations (e.g. for calibrating the model using a simplex method)

For sequential simulations the values of the parameters for a simulation depend on the results of the previous simulations. The simulation plan script must, therefore calculate the parameter values using the results of the previous simulations and set the new values of the parameters.

Sample script

Calibration script

This simulation plan uses a variable step simplex method for calibrating two parameters (the accessibility for large and small fish) to match quarterly landings.

Running a simulation plan

The simulation should be set up using the simulation launcher as normal except for the rules. If a rule is modified by the simulation plan script, then this should be removed from the list of rules in the simulation launcher Parameters tab, as it will be added by the script after the rules defined for the simulation.

Check Use simulation plan and the Simulation plan tab will be enabled. Select the simulation plan script in the dropdown list and click Add. A dialogue box with the simulation plan parameters will pop up. Set any parameters required (they can be modified later in the Simulation plan tab), return to the Parameters tab and click Simulate.

N.B. Use the checklist to ensure that the simulation is fully set up – it would be a pity to do 500 simulations and find that you had forgotten to initialize the populations!

Bibliography

  • Drouineau, H., Mahévas, S., Pelletier, D. and Beliaeff, B. 2006. Assessing the impact of different management options using ISIS-Fish: the French Hake-Nephrops mixed fishery of the Bay of Biscay. Aquatic living resource, 19 : 15-29.
  • Saltelli, A., Tarantola, S., Campolongo, F. and Ratto, M. 2004. Sensitivity Analysis in Practise. A guide to Assessing Scientific Models. J.W.&. Sons. pp.
  • Kleijnen, J.P.C. 1998. Experimental Design for Sensitivity Analysis, Optimization, and Validation of Simulations Models. In Handbook of simulation. Principles, Methodology, Advances, Applications and Practise, pp. 173-224. Ed. by Banks, J. Wiley, New York. Engeneering and Management Press. 864 pp.
  • Walters, F.H., Parker, L.R., Morgan, S.L. and Deming, S.N. 1991. Sequential Simplex optimization: a technique for improving quality and productivity in research, development, and manufacturing (Chemometrics series). B.R. CRC Press LLC. 402 pp.
  • Nocedal, J., and Wright, S.J. 2006. Numerical Optimization. Mikosch, T. V., Resnick, S. I. , and Robinson, S. M. (Eds.). 2nd Ed., Springer Series in Operations Research, New York.