This longitudinal study examined associations of bone mass with physical activity and vitamin D level over more than 6 years through puberty. A total of 663 participants (320 boys) with mean age 9.6 years at baseline (10–17 years at follow-up), underwent dual energy X-ray absorptiometry, anthropometry and blood samples for vitamin D at least twice during the study period (with three possible time-points). Physical activity was assessed using accelerometers at follow-up. A positive association was found between percent time spent at vigorous physical activity and total-body less head bone mineral content (β = 5.8, p = 0.002). The magnitude of this association increased with maturational development; thus physical activity may have a greater influence on bone mass in the more mature participants. The vitamin D levels were also positively associated with bone mass. A high degree of tracking was observed with changes in anthropometric Z scores predictive of deviation from tracking. No environmental factor predicted deviation from tracking.