The community started as a number of small factories deriving power from a dam on the Unadilla River during the first decade of the 19th century. These were known locally as the shops and included a scythe and hoe factory, a blacksmith shop, a grist mill, a saw mill, a horse rake factory and wagon shop, and a foundry and machine shop.
As with many small communities, Leonardsville was given its name by the Post Office Department, which in this case named if after Reuben Leonard who, in his early years, ran a local grocery and dry goods business that became a convenient location to drop off mail for local residents.
In 1856, the grist mill, saw mill and agricultural implement factory were destroyed by fire, but were rebuilt immediately. Soon after that, the shops were purchased by the Babcock family who operated them until the 1930s, employing at their peak over 100 workers. Until the 1950s, when the manufacturing shops closed, Leonardsville was a stop on the Unadilla Valley Railroad, had a milk station (now a recycling center) and a feed store. None of the manufacturing buildings remain, but the former Crandall Department Store still stands and is now the regionally known Horned Dorset Restaurant.
Leonardsville had its own kindergarten through 12th grade central school until 1969, when the district merged with Bridgewater and West Winfield, creating the Mount Markham Central School district. The building was one of the new district's elementary schools until 2001, but now is used for specialized educational programs.