There is limited knowledge on how the association of trees with different mycorrhizal types shapes soil microbial communities in the context of changing tree diversity levels. We used arbuscular (AM) and ectomycorrhizal (EcM) tree species as con- and heterospecific tree species pairs (TSPs), which were established in plots of three tree diversity levels including monocultures, two-species mixtures, and multi-tree species mixtures in a tree diversity experiment in subtropical China. We found that the tree mycorrhizal type had a significant effect on fungal but not bacterial alpha diversity. Furthermore, only EcM but not AM TSPs fungal alpha diversity increased with tree diversity, and the differences between AM and EcM TSPs disappeared in multi-species mixtures. Tree mycorrhizal type, tree diversity and their interaction had significant effects on fungal community composition. Neither fungi nor bacteria showed any significant compositional variation in TSPs located in multi-species mixtures. Accordingly, the most influential taxa driving the tree mycorrhizal differences at low tree diversity were not significant in multi-tree species mixtures. Collectively, our results indicate that tree mycorrhizal type is an important factor determining the diversity and community composition of soil microbes, and higher tree diversity levels promote convergence of the soil microbial communities.