We have looked in some detail at the frequency response in the vicinity of a single, isolated peak. Next, we can learn something interesting by looking the response with two peaks, far enough apart that their modal overlap is low. We start from the general expression for the frequency response of a mechanical system, specifically for the case of admittance (velocity response per unit force):
$$Y(x,y,\omega) = i \omega \sum_n \dfrac{u_n(x) u_n(y)}{\omega_n^2+2i\omega \omega_n \zeta_n-\omega^2} \tag{1}$$
where the sum is over modes labelled by $n$. In this formula, the $n$th mode is assumed to have mode shape $u_n$, resonant frequency $\omega_n$ and damping ratio $\zeta_n$. The damping ratio is related to the modal loss factor by $\eta_n = 2 \zeta_n$, and to the modal Q-factor by $Q_n=1/\eta_n$. The notation $u_n(x)$ is a shorthand to denote “the amplitude of the mode shape at the excitation position $x$, in the relevant direction of the applied driving force”. In a similar way, $u_n(y)$ denotes the corresponding modal amplitude at the position and in the direction of the measuring sensor. The mode shapes must be normalised in a particular way in order for this formula to work: see section 2.2.5 for the details.
We will consider the case with just two terms in the summation. We will first show some computed results, then use an approximate analysis due to Skudrzyk [1] to obtain mathematical expressions for the response and learn an important extra detail. For brevity, we can write
$$K_n=u_n(x) u_n(y), \mathrm{~~~~~} n=1,2 . \tag{2}$$
There is an important difference in the behaviour of the admittance between the two peaks, depending on whether $K_1$ and $K_2$ have the same sign or opposite signs. Figure 1 illustrates this, for a particular case. The red curve shows the magnitude of the admittance when $K_1=K_2=1$, while the blue curve shows the case with $K_1=1, K_2=-1$. In the red curve we see an antiresonance between the two peaks: a sharp dip that looks rather like one of the peaks, upside down. However, in the blue curve we see no such sharp dip, only a shallow valley.

It is easy to see how this behaviour arises, in qualitative terms, by plotting the separate complex contributions to equation (1). Figure 2 shows the result, for the case when the two coefficients have the same sign. The imaginary parts of the two terms are plotted in solid lines, while the real parts are plotted in dashed lines. The two dashed lines are positive at all frequencies, while the two solid lines show a pattern with positive values below the resonance frequency, switching to negative values above the resonance. (The change of sign corresponds to the familiar switch from stiffness-like behaviour of an oscillator below resonance to mass-like behaviour above resonance.)

The result of this pattern is that the two solid lines have opposite signs in the frequency range between the two resonances. Now think what happens when we add the two complex functions together to give the combined admittance. As we already knew from section 10.5.1, the imaginary part of each separate contribution is always significantly larger than the real part for any frequency well away from the resonance. This means that the magnitude of the complex sum is dominated by the imaginary parts. These have opposite signs, with one increasing while the other decreases. It follow that there must be a single frequency where the two terms cancel exactly. This cancellation creates the antiresonance dip: the complex sum then consists only of the rather small value given by the sum of the two real parts.
Contrast this with the case plotted in Fig. 3, where the two modal coefficients have opposite signs. This time, the two imaginary parts have the same sign between the resonances (negative for the case plotted). There can be no cancellation. The real parts now have opposite signs, and cancel at some frequency, but this makes little difference to the magnitude of the complex sum. The result is the shallow valley seen in the blue curve of Fig. 1.

To obtain a mathematical approximation to the combined admittance with the two terms, it is convenient to make use of the partial-fraction expansion derived in section 10.5.1, and the approximation of keeping only the pole at positive frequency:
$$Y \approx – \dfrac{i+\zeta}{2} \left[ \dfrac{K_1}{\omega – \omega_1(1 + i \zeta)} + \dfrac{K_2}{\omega – \omega_2(1 + i \zeta)} \right] \tag{3}$$
where for simplicity it has been assumed that both modes have the same damping ratio $\zeta$, and as usual we will assume small damping so that $\zeta \ll 1$.
For the case $K_1 = K_2 = 1$ as in the red curve of Fig. 1, this can be simplified to
$$Y \approx – i~ \dfrac{\omega – (\omega_1+\omega_2)(1+i \zeta)/2}{(\omega -\omega_1)(\omega – \omega_2)} \tag{4}$$
where terms involving $\zeta$ have been ignored where they are added to terms that are much larger. The denominator of this expression is a parabola with zeros at $\omega_1$ and $\omega_2$, and reaching a minimum at $\omega = (\omega_1+\omega_2)/2$ with the value $-(\omega_2 – \omega_1)^2/4$. At that same frequency, the real part of the numerator expression goes to zero, leaving only the term involving $\zeta$. The result is an estimate for the lowest value of $|Y|$:
$$|Y|_{min} \approx \dfrac{2 \zeta (\omega_1 + \omega_2)}{(\omega_2 – \omega_1)^2} \tag{5}$$
at $\omega = (\omega_1+\omega_2)/2$.
Performing a similar analysis for the case $K_1 = 1, K_2 = -1$, as in the blue curve of Fig. 1, gives
$$Y \approx \dfrac{i}{2}~ \dfrac{\omega_2 – \omega_1}{(\omega -\omega_1)(\omega – \omega_2)} . \tag{6}$$
The minimum value of $|Y|$ again occurs near $\omega = (\omega_1+\omega_2)/2$, and this time the minimum value is
$$|Y|_{min} \approx \dfrac{2 i}{(\omega_2 – \omega_1)} . \tag{7}$$
The accuracy of these approximations is confirmed by Fig. 4, which is a repeat of Fig. 1 with the addition of the approximate curves from equation (3) as dashed green lines, and the estimates of the minimum positions from equations (5) and (7) shown by stars.

Looking back at equation (3), if we take just the first term and look at the value at $\omega = \omega_1$, this gives an estimate of the maximum value of $|Y|$ at the peak:
$$|Y|_{peak} \approx \dfrac{K_1}{2 \omega_1 \zeta} . \tag{8}$$
Combining this with the estimates in equations (5) and (7), we see the essence of the behaviour of peaks, antiresonances and shallow valleys. The peak height is proportional to $1/\zeta$, the antiresonance depth is proportional to $\zeta$, and the depth of the shallow valley is independent of $\zeta$. So antiresonances really do behave like “upside down resonances” — if the damping is reduced, the peaks get higher, while the antiresonances get correspondingly deeper.
[1] E. Skudrzyk: The mean-value method of predicting the dynamic response of complex vibrators. Journal of the Acoustical Society of America 67, 1105–1135 (1980).