Limited information is available on long-term effects of metal(loid)s pollution incidents. Here, we analyze the distribution characteristics and quantification of elements in the Southwest China Keda mining site, which is one of the most populated sites and enables human health and ecological risk assessments of elemental pollution. The results on modified degree of contamination indicated that the soil and sediment were highly contaminated near Dahu Lake. The health risk of children was almost 2.5 times that of adults in surface water, and 7.1 times in soil, respectively. Moreover, Tl and As were the main health risk contributors in surface water and soil, respectively, and As posed the highest ecological risk both in soil and sediment. These results indicated the potential impact of toxic metal(loid)s on the health of residents and environment. Hence, more scientific attention and proper management need to be paid to this environmental challenge in the future.