Contents
How are statistical techniques used in modeling and simulation validation?
This briefing provides an overview of some statistical design and analysis methods that can help support the characterization of the accuracy of the model by providing quantitative comparisons of M&S data to reference data gathered from live testing. The briefing also describes a simulation study used to
How are observed values different from simulated values?
The observed values are outside of the 95% confidence intervals of the simulated dataset so it seems pretty obvious that there is a significant difference between the observed and the simulated; but in some cases, the difference might not be so obvious.
How to compare observed data to simulated data?
A possible approach is to look at deviations from the mean of the simulations. You could use Euclidean distance, or Euclidean distance on the square-roots (which has variance-stabilizing properties), or Mahalanobis distance, or anything else you come up with.
When to use regression analysis in a simulation?
The briefing also describes a simulation study used to obtain recommendations for which techniques to use in different situations. For example, if a designed experiment was executed, more advanced and sensitive statistical methods such as regression analysis should be considered. This page intentionally left blank.
Which is the best method to compare live and simulated data?
Experimental design and corresponding statistical analysis techniques for comparing live and simulated data will be discussed and compared. A simulation study shows that regression analysis is the most powerful comparison when experimental
Which is the best comparison between simulation and experimental design?
A simulation study shows that regression analysis is the most powerful comparison when experimental design techniques are used, while more robust non-parametric techniques provide widely applicable solutions for the comparison. Acknowledgments Technical review was performed by Dean Thomas and William G. Gardner from the Operational Evaluation