| Description |
The replicability crisis in experimental science is fueled by a lack of transparent and explicit discussion of experimental designs in published work. An experimental design is a description of experimental factors and how to map those factors onto a sequence of trials such that researchers can draw statistically valid conclusions. This thesis introduces SweetPea, a SAT-sampler aided language that facilitates creating understandable, reproducible, and statistically robust experimental designs. SweetPea consists of (1) a high-level language to declaratively describe an experimental design, and (2) a runtime to generate unbiased sequences of trials given satisfiable constraints. The high-level language provides primitives that closely match natural descriptions of experimental designs. To ensure statistically significant results, every possible sequence of trials which satisfies the design must have an equal likelihood of being chosen for the experiment. This requirement is computationally intractable; SweetPea gains traction by approximately sampling the solutions using the SAT-sampler Unigen. This allows scientists to easily create sharable experimental designs in terms familiar to their domain which produce unbiased sequences of trials. |
