The modes of linear time invariant mechanical systems can be estimated from output-only vibration measurements under ambient excitation conditions with subspace-based system identification methods. In the presence of additional unmeasured periodic excitation, for example due to rotating machinery, the measurements can be described by a state-space model where the periodic input dynamics appear as a subsystem in addition to the structural system of interest. While subspace identification is still consistent in this case, the periodic input may render the modal parameter estimation difficult, and periodic modes often disturb the estimation of close structural modes. The aim of this work is to develop a subspace identification method for the estimation of the structural parameters while rejecting the influence of the periodic input. In the proposed approach, the periodic information is estimated from the data with a non-steady state Kalman filter, and then removed from the original output signal by an orthogonal projection. Consequently, the parameters of the periodic subsystem are rejected from the estimates, and it is shown that the modes of the structural system are consistently estimated. Furthermore, standard data analysis procedures, like the stabilization diagram, are easier to interpret. The proposed method is validated on Monte Carlo simulations and applied to both a laboratory example and a full-scale structure in operation.