Species of Elaeocarpaceae vary, but complete chloroplast genome data and systematic comparisons across the family are rarely reported. To understand the variation in chloroplast sequence size and structure in Elaeocarpaceae, the chloroplast genomes of 9 species were sequenced using the Illumina HiSeq 2000 platform and further assembled and annotated with Elaeocarpus japonicus and Sloanea sinensis (family Elaeocarpaceae) as references. A phylogenomic tree was constructed based on the complete chloroplast genomes of the 11 species representing 5 genera of Elaeocarpaceae. Chloroplast genome characteristics were examined by using Circoletto and IRscope software. The results revealed the following: (a) The 11 sequenced chloroplast genomes ranged in size from 157,546 bp to 159,400 bp. (b) The chloroplast genomes of Elaeocarpus, Sloanea, Crinodendron and Vallea lacked the rpl32 gene in the small single-copy (SSC) region. The large single-copy (LSC) region of the chloroplast genomes lacked the ndhK gene in Elaeocarpus, Vallea stipularis, and Aristotelia fruticosa. The LSC region of the chloroplast genomes lacked the infA gene in Elaeocarpus and Crinodendron patagua. (c) Through inverted repeat (IR) expansion and contraction analysis, a significant difference was found between the LSC/IRB and IRA/LSC boundaries among these species. Rps3 was detected in the neighboring regions of the LSC and IRb regions in Elaeocarpus. (d) Phylogenomic analysis revealed that the genus Elaeocarpus is closely related to Crinodendron patagua on an independent branch and Aristotelia fruticosa is closely related to Vallea stipularis, forming a clade with the genus Sloanea. Structural comparisons showed that Elaeocarpaceae diverged at 60 Mya, the genus Elaeocarpus diverged 53 Mya and that the genus Sloanea diverged 0.44 Mya. These results provide new insight into the evolution of the Elaeocarpaceae.