Some Questions
William Q. Meeker
Iowa State University

The following are possible questions that the customer for a statistical study (e.g., a manager) might want to ask. Not all questions will pertain to all situations. Of course, it goes without saying that these are also questions that an analyst should be prepared to answer!

1. How did you define the population (or process) for this problem?

2. How did you (or will you) choose the experimental units (or sample units) that you used in your study? To what extent are these experimental units representative to the question at hand?

3. How do you know that your data are (or will be) relevant for answering the question at hand?

4. How did you (or will you) choose your sample size?

5. What is the uncertainty in your estimate (or prediction)?

6. What are the important sources of variability in your process output?

7. Do the data from your study adequately represent all relevant sources of variability in the process of interest? If not, which ones are missing and how will the missingness affect your conclusions?

8. Is it possible to plan a study that will indicate possible methods for reducing variability in the process of interest?

9. How did you choose your model? What are the implications for your conclusions if your model does not adequately describe your process (or population)?

10. What are the other important assumptions involved in the conclusions that you have drawn from your study?

11. What do the data say about these assumptions?

12. What are the implications if your assumptions do not adequately describe your process (or sampling method)?

13. Is is possible to do sensitivity analyses to check the sensitivity of your conclusions to your model and other assumptions?

14. Predictions (or other conclusions about a process) generally suppose that the process will behave in the future as it has in the past, or as it has been modeled using past data. What assurances do you have that this will be the case?