The Oneida tribe claim to be the original inhabitants of the area.
The town was settled around 1799 near Brownville village. It was one of the first towns in the county. Its name is derived from the founder and first settler, Jacob Brown, who was a prominent figure as major-general in the United States Army and is considered to be a heroic figure in the War of 1812.
The town was formed in 1802 while still part of Herkimer County.
Many of the northern towns of the county were established, or partly established, from parts of Brownville, including Le Ray (1806), Lyme (1818), Pamelia (1819), Orleans (1821), and Alexandria (1821).
In 1828, the community of Brownville set itself off from the town by incorporating as a village.
In 1855, the community of Dexter set itself off from the town by incorporating as a village.