Two-Stage Multiple-Comparison Procedures For Steady-State Simulations

Halim Damerdji and Marvin K. Nakayama

Halim Damerdji
Dept. of Industrial Engineering
North Carolina State University
Raleigh, NC 27695-7906
Tel.:(919) 515-3598
Fax: (919) 515-5281
damerdji@eos.ncsu.edu

Marvin K. Nakayama
Dept. of Computer and Information Science
New Jersey Institute of Technology
Newark, NJ 07102
Tel.:(973) 596-3398
Fax: (973) 596-5777
marvin@cis.njit.edu

ACM Transactions on Modeling and Computer Simulation
vol. 9, no. 1 (January 1998)

Paper (PostScript 325 KB)
Paper (GZipped PostScript 117 KB)
Papers only available to TOMACS subscribers and others authorized to access the ACM Digital Library.

Abstract

Procedures for multiple comparisons with the best are investigated in the context of steady-state simulation, whereby a number k of different systems (stochastic processes) are compared based upon their (asymptotic) means µ_i (i = 1, 2, …, k). The variances of these (asymptotically stationary) processes are assumed to be unknown and possibly unequal. We consider the problem of constructing simultaneous confidence intervals for µ_i - max_{j ¬= i} µ_j (i = 1, 2, …, k), which is known as multiple comparisons with the best (MCB). Our intervals will be constrained to contain 0, and so they are so-called constrained MCB intervals. In particular, two-stage procedures for construction of absolute- and relative-width confidence intervals are presented. Their validity is addressed by showing that the confidence intervals cover the parameters with probability at least some user-specified threshold value, as the confidence intervals' width parameter shrinks down to 0. The general assumption on the processes is that they satisfy a functional central limit theorem. The simulation output analysis procedures used are based on the method of standardized time series, of which the method of batch means is a special case. The techniques developed here extend to other multiple-comparison procedures such as unconstrained MCB, multiple comparisons with a control and all-pairwise comparisons. Although simulation is the context in this paper, the results naturally apply to (asymptotically) stationary time series.

General Terms

Algorithms, Experimentation, Performance, Theory.

Categories and Subject Descriptors

G.3 Probability and Statistics [probabilistic algorithms (including Monte Carlo); statistical computing]
I.6.6 Simulation and Modeling [Simulation Output Analysis]

Additional Keywords and Phrases

Stochastic simulation; steady-state output analysis; standardized time series; multiple comparisons; two-stage procedures.