A response spectrum superposition technique is often used to study the structural response to strong earthquake excitation. However, in its present form this technique can provide only the highest peak of the response at various levels of a multi-story structure. For better understanding of the progressing damage, as the structure is subjected to successive excursions beyond the design level, and to estimate the number of times certain responses may be exceeded, it is essential to know all the peaks of the response, not just the highest peak. In the present work, a probabilistic theory has been developed, using order statistics, to find the expected, the most probable, or with any desired confidence level, the amplitudes of all the local maxima in the random response functions are defined in terms of the parameters a sub rms (root-mean-square amplitude), ε (a measure of the width of energy spectrum), and N (total number of peak of response), all of which have been defined in terms of the modal properties of the structure and the Fourier and the response spectrum amplitudes of the input accelerograms. Theoretical relations have been formulated to study the response peaks of multi-story structures under excitations by a single translational component of ground acceleration, by all the three translational components of motion acting simultaneously, by torsional ground acceleration, and by the rocking acceleration of the ground. Application to a simple hypothetical three story structure shows good agreement with the time history solutions for all orders of peaks.