This study explored the importance of using relevant measures when evaluating the effectiveness of early childhood interventions. Data from the federally-funded evaluation of the Comprehensive Child Development Program were used to examine whether the behavior measure, the Child Behavior Checklist (CBCL), was an appropriate tool for the diverse community-based sample of young, low-income children. Results demonstrated no confirmation of the CBCL syndromes composing the Externalizing and Internalizing behavioral dimensions used to determine program impacts. Exploratory analyses revealed that two-thirds of the clinical behavior problems included in the CBCL were of very low prevalence in this community-based sample. These findings stress the importance of measurement fit in national evaluations of early childhood programs serving these vulnerable children.