This scoping review presents an overview of physical fitness parameters in rhythmic gymnastics as well as the association of fitness with gymnasts' performance, competitive level, and age. PubMed, Scopus, and Sport Discus databases were searched. Of the 586 records retrieved, 41 studies met the inclusion criteria (n = 1915 participants). The included studies examined flexibility, aerobic capacity, muscle power, muscle endurance, muscle strength, sprint speed, agility, balance, and coordination. Performance was associated with flexibility, aerobic capacity, lower-limb muscle power, agility, muscular endurance, balance, and coordination from a young age. Flexibility, aerobic capacity, and muscle power were, in general, higher in high-level gymnasts than in low-level gymnasts or controls. Older rhythmic gymnasts demonstrated higher scores than the younger ones in flexibility, aerobic capacity, balance, and sport-specific coordination but not in muscle endurance, while some studies reported a decline in muscle power with age. Supplementary physical fitness training improved all physical abilities irrespective of the gymnasts' level. Rhythmic gymnastics training alone improved muscle power, agility, speed, muscular endurance, and balance to a lesser extent than targeted fitness training. Muscular strength, speed, and agility are largely under-researched in rhythmic gymnastics. Emphasis should be given to targeted strength and power training due to the high mechanical loads placed on skeletally immature athletes.