Objectives: Physical activity (PA) is important for the prevention and management of numerous diseases and may have a positive effect on ability to work. The study aimed to assess the level of occupational and leisure time PA of soldiers during the COVID-19 pandemic and to explore whether there was a relationship between PA and perceived work ability. Material and Methods: The study involved 305 men and 68 women who were territorial army soldiers aged 18–55 (M±SD 32.9±9.01). The Seven-Day Physical Activity Recall (SDPAR) and the Work Ability Index (WAI) were used. Results: The level of self-reported PA for the studied soldiers was relatively high; 80% of them met the recommendations of the WHO and were characterized by having a good (60%) or excellent (20%) WAI status. The level of occupational PA of male soldiers was higher than the level of leisure time PA, and they indicated higher levels of occupational PA and leisure time PA during the workweek than the weekend. Conclusions: Current work ability in comparison to the best in life and work ability related to the physical requirements of the work were positively correlated with leisure time PA. Work ability related to physical requirements was also positively correlated with occupational and total PA, and work ability related to mental requirements was positively correlated with total PA. The study supports the relationship between PA and several aspects of work ability.