To optimize the layout of rural settlements in hilly areas, 10 indicators were selected from three aspects of nature, location, and infrastructure based on the research on the rural settlements in Miaoqian Town, Dangyang City where the development potential was evaluated and graded. The result shows that the areas of rural residential areas in the first-, second-, third- and fourth-level development potential areas were 342.23, 337.84, 172.66, and 155.91 hm², respectively, accounting for 33.93, 33.49, 17.12, and 15.46%. According to the level of the potential of comprehensive development, the residential areas were divided into four categories: scale agglomeration, renovation and improvement, control and expansion, and consolidation and relocation. Corresponding optimization measures were proposed in this study for each category. The study result provides a reference for the optimization of the spatial pattern of residential areas in Miaoqian Town and also a research idea for the optimization of the same type of residential areas.