The town was settled between 1710 and 1713 by Palatine Germans.
During the American Revolution, most of the buildings in the town were destroyed by British raiders and their native allies. The Old Stone Fort Museum still stands as a historical landmark of the time period, and the television show Ghost Hunters investigated the property in an episode that aired December 9, 2010. It was also considered a bread basket because of the amount of wheat produced during the war.
Schoharie became a district in Albany County before the formation of Schoharie County. As a town formed in Albany County in 1788, it became the founding town of the newly created Schoharie County upon the county's formation in 1795. In 1797, part of the town was used to form the Towns of Blenheim, Broome, Cobleskill, and Middleburgh. The Towns of Esperance and Wright were removed from Schoharie in 1846.
George Westinghouse was born at Central Bridge in 1846 and went on to invent the railroad air-brake and help develop Tesla's AC motor and promote its use over the rival DC power supply system. Author Chris Hedges grew up in Schoharie, where his father was the pastor of a Presbyterian church. He writes about the town in Losing Moses on the Freeway: The 10 Commandments in America (2005). NYS Assemblyman Pete Lopez is a long-time resident of Schoharie. He was the Supervisor of the Town of Schoharie for years before becoming County Clerk.
On August 28, 2011, the town of Schoharie was flooded by Hurricane Irene. The Schoharie Creek rose to record levels and was categorized as the 500 year flood causing massive destruction of roads, homes, and businesses within the town. Due to the devastation in the area, federal agencies such as FEMA and The National Guard were called in to assess damages and provide assistance to affected residents. The town of Schoharie is a largely agricultural community. Many farms in the area were devastated due to animals lost in flood waters or drowned, barns deemed unusable, and fall harvest crops ruined.
The Becker Stone House, Becker-Westfall House, The Colyer House, Sternbergh House, and Westheimer Site are listed on the National Register of Historic Places. The Abraham Sternberg House was added in 2010.