# Generate differs in 10 and 100 runs
- [x] New "new" format
- [x] Init must check if both files are the same; if so, new format; if not, new new format.
- [x] `Validation` needs pairs of files (Generated/Reference); assume `short` is always present, long is optional
- [x] "update" can read the Validation directly
- [x] If the generated file doesn't exist in the long run, ask again
- [x] In "init", update the reference (this should be done inside `Generator` )\
# 0.20 Documentation
- Rundir
- Reduction
- Small-Fail/Large-Fail
- What is the absolute and relative differences
# Not reducing
- ===THIS MAY BE UNNECESSARY WITH THE NEW CONFIG FORMAT===
- `caseSetupDict.static` , maybe?
- When there is no reduction, there is no need to build the short/long run things
- The old format supports that, we just need to figure out when there are no reductions in `init` to generate the old format.
- There is also a suggestion to change the run-dir when there is no reduction, which I'm not sure if possible due the other of "Expander"/"Reducer"...
- Also, would need to have, in meta information, to have both (one option) rundirs.
- This would help the other point of keeping the run-dir static for init, update and run.
- Expanding all run-dirs at the start would save a few runs too.
- In the pipeline, the sequence is Expander, Reducer, Loader.
- The Expander doesn't know if the case can be reduced or not, and can't make a judgement if we need 1 or 2 run-dirs.
- Maybe Reducer could have a "reducer probability" kinda of thing that Expander could call to find out the number of run-dirs.
- Function returns: caseSetupDicts that can be reduced but are not continuations, caseSetupDicts that are continuations and caseSetupDicts that can't be reduced. I'd like to make it an iterator, but I guess returning a Vec is good enough (specially since we need to sort the files in the file system to find out the continuations).
- Problem: Variations need to be sorta-kinda expanded, 'cause they have more/replace caseSetupDicts
- So Reducer needs to know about variations and the merge space from the Expander, and Expander needs to know about reductability from the Reducer.
- (All this to know how many run-dirs we need...)
- Default doesn't have long run (all caseSetupDicts are static); variation have short and long run (at least oneCaseSetupDict is reducible).
- Rename the run-dir after expansion, if found anything that can be reduced?
- Expand the short and long run-dirs always, and then just ignore/delete the long one if the case isn't reducible?
- We need to remove it if it doesn't have a long validation either.
- Should we follow the same idea for controlDicts?
- **Need to document this properly** [[#0.20 Documentation]]
# system/validationDict
Idea for a Foam-formated file for Verify
```c++
// Execution is run in sequence: If the first run ("quick") fails, then the
// second run ("short") is run, and if that fails, the third is run ("long"),
// and so on. On the other hand, the execution of the runs stops on the first
// full success.
//
// This example is a bit of a stretch -- why would be want to run the same case
// 4 different times? -- but it shows that, if that necessity appears to run
// any different number of runs in the future, we can support it. It also shows
// that one can have mixed reduced and non-reduce runs (e.g., there could be
// just one run without reduction, or just one reduced run, and so on.)
// We could also add support in Verify to only execute runs with a specific
// name, allowing to run all examples in their "infinite" (non-reduced) mode
// (as long as they all name their runs with no steps "infinite", that is).
//
// Names are free form, and we could use them to defined the run-dir, e.g.,
// "short" validation will be run in ".run-short", "long" in ".run-long" and
// so on.
runs
// This is a quick test: The example is reduced to just 2 timesteps, and
// we check the resulting file against its MD5 hash.
steps 2;
// Any other key name that it is not reserved ("steps", "continuations")
// represents a file to be compared.
"postProcessing/2/qux" {
md5 "123123";
}
}
// In case the MD5 fails, the case is run again, but this time reduced to
// 10 timesteps. This run also includes continuations, by changing the
// listed files to make one follow the timesteps of the previous. The run
// will fail if relative or absolute differences are above 0.
short {
steps 10;
// This makes the continuations explicit, by specificing files that
// form a single run.
continuations (
"system/caseSetupDict.initial"
"system/caseSetupDict.continuation1"
"system/caseSetupDict.continuation2"
);
"postProcessing/10/foo" {
reference "verification/10/foo";
variables "phi" "Tmean";
// No tolerances mean "it will fail if the values are not the same".
}
// This is a completely different file from the above, with different
// keys
"postProcessing/10/bar" {
reference "verification/10/bar";
variables "Tmin" "Tmax";
}
}
// If in 10 timestemps the values do not match the reference file, a 100
// timesteps run is done. There is no continuation, and for this run to
// fail, both absolute and relative tolerances must be above the designed
// threshold.
long {
steps 100;
// This file only appears in the 100 runs, so we can't have a global
// list of files not attached to steps (although it is the same "foo"
// from 10 steps).
"postProcessing/100/foo" {
reference "verification/100/foo";
variables "phi" "Tmean";
}
"postProcessing/100/bar" {
reference "verification/100/bar";
variables "Tmin" "Tmax";
}
}
// If the 100 timesteps fails, then we run the example a 4th time, this
// time without any reductions (the "steps" property is not set in this
// run). There are no continations either.
"run till completion" {
// With no steps, there is no reduction.
"postProcessing/20000/baz" {
reference "verification/20000/baz";
variables "integral";
absolute 20;
relative 20;
// no operator means OR, so example will fail if the
// absolute different is above 20 OR the relative
// difference is above 20.
}
}
);
// This is used for filtering.
tags ("GIB" "AES" "compressible");
// Only present if the example can't be run in some platform.
unstable {
operatingSystem windows; // valid values: "windows", "linux", "all"
reason "Allrun uses Python, and Python isn't usually available on Windows";
}
```
Pest:
```pest
foam = { SOI ~ entry ~ EOI }
// general stuff
quote = _{ "\"" }
semicolon = _{ ";" }
braces_open = _{ "{" }
braces_close = _ { "}" }
parentheses_open = _{ "(" }
parentheses_close = _{ ")" }
specials = { semicolon | braces_open | braces_close | parentheses_open | parentheses_close | WHITESPACE }
WHITESPACE = _{ " " | "\t" | NEWLINE }
COMMENT = _{ "//" ~ (!NEWLINE ~ ANY)+ }
// somewhat complex structures
single_word = @{ (!specials ~ ANY)+ }
quoted_word = @{ quote ~ (!quote ~ ANY)* ~ quote }
keyword = { quoted_word | single_word }
value = { keyword }
// main foam stuff
entry = { (dictionary | list | attribution)* }
attribution = { keyword ~ value+ ~ semicolon }
dictionary = { keyword ~ braces_open ~ entry ~ braces_close }
list = { keyword ~ parentheses_open ~ value+ ~ parentheses_close ~ semicolon }
```
Parsed:
```
- foam
- entry
- list
- keyword > single_word: "variables"
- value > keyword > quoted_word: "\"phi\""
- value > keyword > quoted_word: "\"meanT\""
- list
- keyword > single_word: "runs"
- dictionary
- keyword > single_word: "quick"
- entry
- attribution
- keyword > single_word: "steps"
- value > keyword > single_word: "2"
- attribution
- keyword > single_word: "generatedFile"
- value > keyword > quoted_word: "\"postProcessing/2/blah\""
- dictionary
- keyword > single_word: "failIf"
- entry > attribution
- keyword > single_word: "md5Differs"
- value > keyword > quoted_word: "\"123123\""
- dictionary
- keyword > single_word: "short"
- entry
- attribution
- keyword > single_word: "steps"
- value > keyword > single_word: "10"
- attribution
- keyword > single_word: "generatedFile"
- value > keyword > quoted_word: "\"postProcessing/10/blah\""
- attribution
- keyword > single_word: "referenceFile"
- value > keyword > quoted_word: "\"verification/10/blah\""
- list
- keyword > single_word: "continuations"
- value > keyword > quoted_word: "\"system/caseSetupDict.initial\""
- value > keyword > quoted_word: "\"system/caseSetupDict.continuation1\""
- value > keyword > quoted_word: "\"system/caseSetupDict.continuation2\""
- dictionary
- keyword > single_word: "long"
- entry
- attribution
- keyword > single_word: "steps"
- value > keyword > single_word: "100"
- attribution
- keyword > single_word: "generatedFile"
- value > keyword > quoted_word: "\"postProcessing/100/blah\""
- attribution
- keyword > single_word: "referenceFile"
- value > keyword > quoted_word: "\"verification/100/blah\""
- dictionary
- keyword > single_word: "failIf"
- entry
- attribution
- keyword > single_word: "absolute"
- value > keyword > single_word: "10"
- attribution
- keyword > single_word: "relative"
- value > keyword > single_word: "10"
- attribution
- keyword > single_word: "operator"
- value > keyword > single_word: "and"
- dictionary
- keyword > single_word: "infinite"
- entry
- attribution
- keyword > single_word: "generatedFile"
- value > keyword > quoted_word: "\"postProcessing/20000/blah;\""
- value > keyword > single_word: "referenceFile"
- value > keyword > quoted_word: "\"verification/20000/blah\""
- dictionary
- keyword > single_word: "failIf"
- entry
- attribution
- keyword > single_word: "absolute"
- value > keyword > single_word: "20"
- attribution
- keyword > single_word: "relative"
- value > keyword > single_word: "20"
- list
- keyword > single_word: "tags"
- value > keyword > quoted_word: "\"GIB\""
- value > keyword > quoted_word: "\"AES\""
- value > keyword > quoted_word: "\"compressible\""
- dictionary
- keyword > single_word: "unstable"
- entry
- attribution
- keyword > single_word: "operatingSystem"
- value > keyword > single_word: "windows"
- attribution
- keyword > single_word: "reason"
- value > keyword > quoted_word: "\"Allrun uses Python, and Python isn't usually available on Windows\""
- EOI: ""
```
Migration plans?
- Side pipeline?
- Current pipeline still exists, a new pipeline is build next to it with the new actors, with the new structures and "finder" sends requests depending on found files.
- What about reporter?
- No actors?
- New design is in early stages and I'm not sure how long till they are both feature compatible.
- Also, how about the current tests?
Intermediate format?
- We have nothing
- We find a dictionary
- That's a run, so we need to keep this in a list
- In the run, we need to capture each content
- It is "steps"? Int
- It is 'failIf'? process dic
- Now how it is "run"?
- How do we fill the blanks while loading?
# Logos
```rust
use logos::Logos;
#[allow(dead_code)]
#[derive(Logos, Debug)]
#[logos(skip r"[ \t\n\r]")]
pub(crate) enum Token< 'a> {
#[regex(r#"\/\*[^\/\*]*\*\/"#, |lex| lex.slice())]
MultilineComment(& 'a str),
#[regex(r#""[^"]+""#, |lex| lex.slice().trim_start_matches('"').trim_end_matches('"'))]
#[regex("[a-zA-Z0-9]+", |lex| lex.slice())]
Keyword(& 'a str),
#[regex(r#"//[^\n]*"#, |lex| lex.slice())]
Comment(& 'a str),
#[token(";")]
End,
#[token("{")]
DictStart,
#[token("}")]
DictEnd,
#[token("(")]
ListStart,
#[token(")")]
ListEnd,
}
```
```
variable "value with quotes";
```
Keyword("variable") => Keyword("value with quotes) => End
```
dict { dict { var value; }}
```
Keyword("dict") => DictStart => Keyword("dict) => DictStart => Keyword("var") => Keyword("value) => End => DictEnd => DictEnd
So:
- Keyword (always)
- if next is:
- DictStart: Start dict
- ListStart: Start List
- Keyword: Grab keyword, make valuelist
loop {
- Grab keyword (key)
- Grab next element:
- Comment? None
- MultilineComment: None
- ListStart: process_list (consume token, anyway)
- ListEnd: ERROR
- DictStart: process_dictionary (consume token, anyway)
- DictEnd: Empty dictionary, ERROR
- Keyword: assume values, start Vec, add keyword, consume till "End"
- End: ERROR
- Grab next element:
- None: Complete
- DictEnd: This dictionary is complete, return it
- Anything else:
}
# Post lexer work
- [ ] grep filter on expansion
- [ ] Runner with comparer
- Spawn tasks? How to join?
- Also, since the executor follows one run, and the comparer follow that run, does it matter using tasks?
- [x] Build "rundir" when loading Verification
- [ ] Variations?!?!
- Expand the whole case at the start.
- Load verification from expanded dirs.
- This is what the "Loader" was doing before, which I inadvertently merged with the verification loader.
- Order is mixed: The whole example can be expanded into their runs, but the execution is per non-expanded content (default, variations)
- Need to send, into the pipeline, the default, variations and the replicated Verification for each.
- Problem: The variation may contain different steps, anyway. So need to "expand" without having the verification info beforehand.
- What if:
- Expand default run into `.run-default`
- Expand variations into `.run-[variationname]`
- Inside each of those, expand the case (default won't expand, but still....)
- We could say it is not possible to add a new verificationDict into variations, but I do like the idea of having a different content for it.
- Ok, ignoring this for now 'cause it fucks up my thinking.
- Expand by run on default, send variations down the pipeline
