This directory contains examples, log files, and scripts from the
experimental evaluation in [1].

The columns of the tables of results (all.raw.{csv,txt}) are as follows:
t           run time [seconds]
m           memory [MB]
sti         size of syntax tree of input formula [# nodes]
stj         size of syntax tree of input formula after possible nnf-ization [# nodes]
sts         size of syntax tree of simplified input formula [# nodes]
stc         size of syntax tree of uc with algorithm proof [# nodes]
stp         size of syntax tree of uc with algorithm proof [# nodes]
stpd        size of syntax tree of uc with algorithm proofdeletion [# nodes]
std         size of syntax tree of uc with algorithm deletion [# nodes]
clj         # SNF clauses obtained from input formula
cls         # SNF clauses obtained from input formula, simplified
clc         # SNF clauses of uc
pv          peak number of vertices in resolution graph
pe          peak number of edges in resolution graph
pve         peak number of (vertices + edges) in resolution graph
it          # iterations in deletion-based uc minimization
itu         # iterations in deletion-based uc minimization with unsat result
its         # iterations in deletion-based uc minimization with sat result

Entries have the following meaning:
to          time out
mo          memory out
na          not available
si          simplified to false during translation from LTL to SNF
---         example not run (only for subfamilies of increasing difficulty; at
            least two less difficult examples of subfamily timed or memed out)

File (type)               Content
--------------------------------------------------------------------------------
all.raw.csv               table of results as csv
all.raw.txt               table of results as txt
config_examples.bash      list of examples for run_benchmarks.bash
Config_examples.pm        list of examples for generate_tab.pl
generate_tab.pl           script used to generate tables of results
README                    this file
run_benchmarks.bash       script used to run examples (note: the version of
                          run used is customized to allow configuring
                          the rate of reporting)

                          For the following files we have
- ${e} is the name of an example
- ${m} is whether uc extraction was enabled and which optimizations
  were not used:
  - none: no uc
  - proof: (non-minimal) uc extracted from resolution graph
  - proofdeletion: uc extracted from resolution graph + deletion-based minimization
  - deletion: uc by deletion-based minimization
  - proof_aug2: include premise of aug2 in resolution graph
  - proof_bfsloopitinitcg: include premise 1 of BFS-loop-it-init-c in
    resolution graph
  - proof_bfsloopitinitce: include premise 2 of BFS-loop-it-init-c in
    resolution graph
  - proof_bfsloopconclusion2e: include premise 2 of
    BFS-loop-conclusion2 in resolution graph
  - proof_nopruneloopits: don't immediately prune failed loop search
    iterations from resolution graph
  - proof_noprunemain: don't prune vertices and edges unreachable from
    the main partition between loop searches from resolution graph
  - proof_all: all above optimizations disabled
  - sizenonnf: only used to compute size of input formula w/o
    transforming it into NNF
- ${f} is the format of the uc: internal, snf, ltlc, ltl

${e}.trp                  example
${e}.${m}.log             log file
${e}.trp.simple.${m}.${f} core file

Viktor Schuppan
Füssen, 27 September 2013

[1] V. Schuppan: Extracting Unsatisfiable Cores for LTL via Temporal
    Resolution. In: C. Sanchez, B. Venable, E. Zimanyi (eds.):
    Proceedings of the 20th International Symposium on Temporal
    Representation and Reasoning, TIME 2013, Pensacola, FL, USA,
    September 26-28, 2013, pp. 54-61. IEEE Computer Society, 2013.
