Due to their three-dimensional nature, damages have different effects on each vibration mode of a frame structure, depending on the direction of deflection of its structural elements. In this study, a Multi-mode Multi-directional Damage Index (MMDI) method is proposed to benefit from the recognition of the multi-directional effects of damages in order to reduce damage detection errors. The novelty of the method resides in the use of specifically developed modal combination factors for the identification of the vibration modes more relevant to the damage detection and for the assessment of the directionality of the damage effects. The detection capability of the MMDI method is experimentally investigated on a bridge testbed consisting of a two-column moment resisting frame. Individual and combined damages are simulated into columns and beams of the testbed through steel parts removal. The dynamic characteristics of the system in its undamaged and damaged configurations are identified upon ambient-like accelerations from a low dense sensor network. The effectiveness of the MMDI method is assessed for its ability of identifying damages without any pre-selection of vibration modes. The accuracy of the damage assessment is measured through four indicators targeting identification, false detection, localization, and severity estimation of the damages.