Settlement began in 1784. The town was formed in 1819 from the Town of Middletown.
In 1845 in an Anti-rent War, protesters, several hundred dressed as "Calico Indians," shot and killed Under-Sheriff Osman Steele, when he and deputies attempted to collect overdue rent of about $64. Hundreds of protestors were arrested and two were sentenced to death, but their sentences were later commuted.
On December 31, 2003, the incorporated village of Andes ceased to exist. During a special village election on June 3, 2002, the residents approved dissolving the village by a vote of 81 to 63.