SUMMARY

all.csv contains the results of the main stage of the evaluation of
solvers for satisfiability of propositional LTL in [1].

FORMAT

Fields are separated by ';'.

The first line contains column headers. The first header is
"benchmark". The next headers consist of a configuration descriptor
according to Sect. 3 of [1] concatenated with "_time", "_mem", _res",
or "_len_wit".

Each subsequent line is for one version of a benchmark instance. The
first field of such a line is the instance name with "___f" for
non-negated and "___t" for negated versions appended. Then follow
groups of 4 fields per configuration as follows:

- If the configuration timed out on that instance version, then all
  four fields are "to".

- If the configuration memed out on that instance version, then all
  four fields are "mo".

- If the configuration is buggy on that instance version, then all
  four fields are "er".

- Otherwise:

  - The first field is the run time in seconds.

  - The second field is the memory usage in MB.

  - The third field is "s" if the instance version was found
    satisfiable. It is "u" if the instance version was found
    unsatisfiable.

  - The fourth field is the length of the model in states for
    satisfiable instances and configurations with model construction
    enabled. It is "na" otherwise.

REFERENCES

[1] V. Schuppan, L. Darmawan: Evaluating LTL Satisfiability
    Solvers. In: Ninth International Symposium on Automated Technology
    for Verification and Analysis (ATVA 2011), Taipei, Taiwan, October
    11-14, 2011.
