According to the National Weather Service, the Lynchburg Water Resources Department reported 4 to 6 inches of rain earlier Thursday evening caused the College Lake Dam to fill beyond its capacity.
However, National Weather Service officials warned that if a complete dam failure occurs, "the water depth in Lynchburg could exceed 17 feet in seven minutes".
A major road for the city crosses over top of the dam, which is located right next to the University of Lynchburg.
Water from the College Lake Dam in the city of Lynchburg, Virginia, was overflowing uncontrollably and the dam was being monitored for structural damage, an NWS bulletin issued at 2:59 a.m. local time said - warning of resulting flash floods.
The Weather Service said that College Lake was at "near capacity" after up to six inches of rain fell Thursday evening.
Local fire and police sent boats to rescue people to area shelters, including E.C.Glass High School and Sandusky Middle School.
And as of 10:01 p.m., officials said 12 to 18 inches of water were flowing over Lakeside Drive into Blackwater Creek.
Most at risk were some 124 houses in the city of 80,000.
The dam was built in the 1930s by the Virginia Department of Highways and is now listed among "high hazard dams" in need of fix in Virginia.
The warning comes as rain has saturated parts of the East for days, with flash-flood watches in effect Friday morning for millions from Georgia to Vermont.
Evacuees from the Sandusky Apartments are being taken to the Salvation Army, the City of Lynchburg said in a Facebook post.
Turner Perrow, a member of the Lynchburg City Council who is also a civil engineer, tells NPR that the dam presents "a complicated problem".