The town was an original town of the county. It was established in 1789, while still in Ontario County and became part of Yates County when the county was created in 1823. The area around Bluff Point was annexed to the town in 1814.
Jerusalem has been divided to form other towns in the county: in 1803, the Town of Benton was taken from Jerusalem, and itself later divided to make other towns.
Located within the town are a number of properties listed on the National Register of Historic Places including: Peter Bitley House, Thomas Bitley House, Samuel Botsford House, Esperanza, Hampstead, Uriah Hanford House, George Hays House, Ezikial Perry House, St. Luke's Episcopal Church, Sill Tenant House, Abraham Wagener House, H. Allen Wagener House, Jemima Wilkinson House, and Sherman Williams House and Fruit Barn.