The town is on the south border of the county, southeast of Binghamton.
The area was first settled around 1788. The Town of Conklin was established in 1824 from the town of Chenango by Nicholas Conklin (1782–1858). The town exchanged territories with adjacent towns before establishing its modern boundaries. In 1831, part of Conklin was used to form the town of Windsor, but Conklin received territory from Windsor in 1851. Another part of Conklin was used in 1859 to form the town of Kirkwood.
Alpheus Corby, a Conklin resident, built a castle-like structure as his home in 1900. The building is the current community center of the town.
In June 2006, the town was devastated by a massive flood that isolated the center of the town and required hundreds of people to be airlifted to safety. The flood destroyed hundreds of homes and businesses and caused tens of millions of dollars in damage.
The town suffered another flood in September 2011.