This was one of the original areas of the Burnetsfield Patent in the province of New York, where in 1722-1723, Governor Burnet granted Palatine German immigrants leases to purchase land from the Mohawk, one of the powerful nations of the Iroquois Confederacy. It was the first land sold to Europeans west of Schenectady. During the 18th-century warfare in the valley, the village was attacked by French and Iroquois forces during the French and Indian War (Seven Years War), and many women and children were taken to Canada as captives.
In addition to intermittent raids, during the American Revolutionary War, the village was attacked in September 1778 by British forces, as well as Iroquois led by the Mohawk chief Joseph Brant. Residents retreated to Fort Herkimer, which they built on the south bank of the Mohawk River.
The town was set apart in the now defunct Tryon County, which was broken up into Montgomery and other counties after the Revolutionary War. It was established as a town in 1788 before the formation of Herkimer County. Previous to the establishment, the area that was to become the town had also been called German Flatts.
In 1791, when Herkimer County was established, German Flatts was one of the three original towns. Three new towns were created from German Flatts in 1792: Frankfort, Litchfield, and Warren. Another part of German Flatts was used in 1829 to form part of the Town of Little Falls.