# Generate differs in 10 and 100 runs
- [x] New "new" format
- [x] Init must check if both files are the same; if so, new format; if not, new new format.
- [x] `Validation` needs pairs of files (Generated/Reference); assume `short` is always present, long is optional
- [x] "update" can read the Validation directly
- [x] If the generated file doesn't exist in the long run, ask again
- [x] In "init", update the reference (this should be done inside `Generator` )\
# 0.20 Documentation
- Rundir
- Reduction
- Small-Fail/Large-Fail
- What is the absolute and relative differences
# Not reducing
- ===THIS MAY BE UNNECESSARY WITH THE NEW CONFIG FORMAT===
- `caseSetupDict.static` , maybe?
- When there is no reduction, there is no need to build the short/long run things
- The old format supports that, we just need to figure out when there are no reductions in `init` to generate the old format.
- There is also a suggestion to change the run-dir when there is no reduction, which I'm not sure if possible due the other of "Expander"/"Reducer"...
- Also, would need to have, in meta information, to have both (one option) rundirs.
- This would help the other point of keeping the run-dir static for init, update and run.
- Expanding all run-dirs at the start would save a few runs too.
- In the pipeline, the sequence is Expander, Reducer, Loader.
- The Expander doesn't know if the case can be reduced or not, and can't make a judgement if we need 1 or 2 run-dirs.
- Maybe Reducer could have a "reducer probability" kinda of thing that Expander could call to find out the number of run-dirs.
- Function returns: caseSetupDicts that can be reduced but are not continuations, caseSetupDicts that are continuations and caseSetupDicts that can't be reduced. I'd like to make it an iterator, but I guess returning a Vec is good enough (specially since we need to sort the files in the file system to find out the continuations).
- Problem: Variations need to be sorta-kinda expanded, 'cause they have more/replace caseSetupDicts
- So Reducer needs to know about variations and the merge space from the Expander, and Expander needs to know about reductability from the Reducer.
- (All this to know how many run-dirs we need...)
- Default doesn't have long run (all caseSetupDicts are static); variation have short and long run (at least oneCaseSetupDict is reducible).
- Rename the run-dir after expansion, if found anything that can be reduced?
- Expand the short and long run-dirs always, and then just ignore/delete the long one if the case isn't reducible?
- We need to remove it if it doesn't have a long validation either.
- Should we follow the same idea for controlDicts?
- **Need to document this properly** [[#0.20 Documentation]]
# system/validationDict
Idea for a Foam-formated file for Verify
```c++
// List of variables to be checked; optional, since MD5 validation does not
// use any variables.
variables ("phi" "meanT");
// Execution is run in sequence: If the first run ("quick") fails, then the
// second run ("short") is run, and if that fails, the third is run ("long"),
// and so on. On the other hand, the execution of the runs stops on the first
// full success.
//
// This example is a bit of a stretch -- why would be want to run the same case
// 4 different times? -- but it shows that, if that necessity appears to run
// any different number of runs in the future, we can support it. It also shows
// that one can have mixed reduced and non-reduce runs (e.g., there could be
// just one run without reduction, or just one reduced run, and so on.)
// We could also add support in Verify to only execute runs with a specific
// name, allowing to run all examples in their "infinite" (non-reduced) mode
// (as long as they all name their runs with no steps "infinite", that is).
//
// Names are free form, and we could use them to defined the run-dir, e.g.,
// "short" validation will be run in ".run-short", "long" in ".run-long" and
// so on.
runs
(
// This is a quick test: The example is reduced to just 2 timesteps, and
// we check the resulting file against its MD5 hash.
quick {
steps 2;
generatedFile "postProcessing/2/blah";
// because the validation is in MD5 format, there is no need to have
// a referenceFile
failIf {
md5Differs "123123";
}
}
// In case the MD5 fails, the case is run again, but this time reduced to
// 10 timesteps. This run also includes continuations, by changing the
// listed files to make one follow the timesteps of the previous. The run
// will fail if relative or absolute differences are above 0.
short {
steps 10;
generatedFile "postProcessing/10/blah";
referenceFile "verification/10/blah";
// No tolerances mean "it will warn if the values are not the same"
// This makes the continuations explicit, by specificing files that
// form a single run.
continuations (
"system/caseSetupDict.initial"
"system/caseSetupDict.continuation1"
"system/caseSetupDict.continuation2"
);
}
// If in 10 timestemps the values do not match the reference file, a 100
// timesteps run is done. There is no continuation, and for this run to
// fail, both absolute and relative tolerances must be above the designed
// threshold.
long {
steps 100;
generatedFile "postProcessing/100/blah";
referenceFile "verification/100/blah";
failIf {
absolute 10;
relative 10;
// will fail is the absolute difference is above 10 AND the
// relative difference is above 10;
operator and;
}
}
// If the 100 timesteps fails, then we run the example a 4th time, this
// time without any reductions (the "steps" property is not set in this
// run). There are no continations either.
infinite {
// With no steps, there is no reduction.
// (maybe we need something to tell that there is no reduction
// explicitly?)
generatedFile "postProcessing/20000/blah;"
referenceFile "verification/20000/blah";
failIf {
absolute 20;
relative 20;
// no operator means OR, so example will fail if the absolute
// different is above 20 OR the relative difference is above 20
}
}
);
// This is used for filtering.
tags ("GIB" "AES" "compressible");
// Only present if the example can't be run in some platform.
unstable {
operatingSystem windows; // valid values: "windows", "linux", "all"
reason "Allrun uses Python, and Python isn't usually available on Windows";
}
```
Pest:
```pest
foam = { SOI ~ entry ~ EOI }
// general stuff
quote = _{ "\"" }
semicolon = _{ ";" }
braces_open = _{ "{" }
braces_close = _ { "}" }
parentheses_open = _{ "(" }
parentheses_close = _{ ")" }
specials = { semicolon | braces_open | braces_close | parentheses_open | parentheses_close | WHITESPACE }
WHITESPACE = _{ " " | "\t" | NEWLINE }
COMMENT = _{ "//" ~ (!NEWLINE ~ ANY)+ }
// somewhat complex structures
single_word = @{ (!specials ~ ANY)+ }
quoted_word = @{ quote ~ (!quote ~ ANY)* ~ quote }
keyword = { quoted_word | single_word }
value = { keyword }
// main foam stuff
entry = { (dictionary | list | attribution)* }
attribution = { keyword ~ value+ ~ semicolon }
dictionary = { keyword ~ braces_open ~ entry ~ braces_close }
list = { keyword ~ parentheses_open ~ value+ ~ parentheses_close ~ semicolon }
```
Parsed:
```
- foam
- entry
- list
- keyword > single_word: "variables"
- value > keyword > quoted_word: "\"phi\""
- value > keyword > quoted_word: "\"meanT\""
- list
- keyword > single_word: "runs"
- dictionary
- keyword > single_word: "quick"
- entry
- attribution
- keyword > single_word: "steps"
- value > keyword > single_word: "2"
- attribution
- keyword > single_word: "generatedFile"
- value > keyword > quoted_word: "\"postProcessing/2/blah\""
- dictionary
- keyword > single_word: "failIf"
- entry > attribution
- keyword > single_word: "md5Differs"
- value > keyword > quoted_word: "\"123123\""
- dictionary
- keyword > single_word: "short"
- entry
- attribution
- keyword > single_word: "steps"
- value > keyword > single_word: "10"
- attribution
- keyword > single_word: "generatedFile"
- value > keyword > quoted_word: "\"postProcessing/10/blah\""
- attribution
- keyword > single_word: "referenceFile"
- value > keyword > quoted_word: "\"verification/10/blah\""
- list
- keyword > single_word: "continuations"
- value > keyword > quoted_word: "\"system/caseSetupDict.initial\""
- value > keyword > quoted_word: "\"system/caseSetupDict.continuation1\""
- value > keyword > quoted_word: "\"system/caseSetupDict.continuation2\""
- dictionary
- keyword > single_word: "long"
- entry
- attribution
- keyword > single_word: "steps"
- value > keyword > single_word: "100"
- attribution
- keyword > single_word: "generatedFile"
- value > keyword > quoted_word: "\"postProcessing/100/blah\""
- attribution
- keyword > single_word: "referenceFile"
- value > keyword > quoted_word: "\"verification/100/blah\""
- dictionary
- keyword > single_word: "failIf"
- entry
- attribution
- keyword > single_word: "absolute"
- value > keyword > single_word: "10"
- attribution
- keyword > single_word: "relative"
- value > keyword > single_word: "10"
- attribution
- keyword > single_word: "operator"
- value > keyword > single_word: "and"
- dictionary
- keyword > single_word: "infinite"
- entry
- attribution
- keyword > single_word: "generatedFile"
- value > keyword > quoted_word: "\"postProcessing/20000/blah;\""
- value > keyword > single_word: "referenceFile"
- value > keyword > quoted_word: "\"verification/20000/blah\""
- dictionary
- keyword > single_word: "failIf"
- entry
- attribution
- keyword > single_word: "absolute"
- value > keyword > single_word: "20"
- attribution
- keyword > single_word: "relative"
- value > keyword > single_word: "20"
- list
- keyword > single_word: "tags"
- value > keyword > quoted_word: "\"GIB\""
- value > keyword > quoted_word: "\"AES\""
- value > keyword > quoted_word: "\"compressible\""
- dictionary
- keyword > single_word: "unstable"
- entry
- attribution
- keyword > single_word: "operatingSystem"
- value > keyword > single_word: "windows"
- attribution
- keyword > single_word: "reason"
- value > keyword > quoted_word: "\"Allrun uses Python, and Python isn't usually available on Windows\""
- EOI: ""
```
Migration plans?
- Side pipeline?
- Current pipeline still exists, a new pipeline is build next to it with the new actors, with the new structures and "finder" sends requests depending on found files.
- What about reporter?
- No actors?
- New design is in early stages and I'm not sure how long till they are both feature compatible.
- Also, how about the current tests?
Intermediate format?
- We have nothing
- We find a dictionary
- That's a run, so we need to keep this in a list
- In the run, we need to capture each content
- It is "steps"? Int
- It is 'failIf'? process dic
- Now how it is "run"?
- How do we fill the blanks while loading?
