Elba claims to be the "Onion Capital of the World." Torrey Farms, one of the largest farms in New York, is located in Elba with about . They also own about of muckland in Potter, New York, where they grow a large number of onions.
Elba is also host to its annual 'Onion Festival' on the first weekend of August. The Onion Festival draws the vast majority of Elba citizens and some people from neighboring towns in Genesee County and some, even, from all over New York. Among the events are the Fireman's Parade, Kiddie Parade, raffling of a car, (usually a new Mustang GT) and the crowning of an "Onion Queen"; 2012's Onion Queen was Jamie Marshall. All profits from the festival go towards the funding of the Elba Fire Department.
The site of the village was first settled in 1804, by John Roraback, a weaver. The village of Elba was incorporated in 1884. Previously, in 1882, an attempt to incorporate "Pine Hill" as the town's name lead to dissension and a lawsuit.
For a short time the town was actually called Pine Hill as derived from a Native American name for the area in which the town resided.
Elba was a stop on the Underground Railroad. There literally was an underground section, as there was a tunnel underneath Main Street between the inn and the home of its owner, Elias Pettibone, which is now collapsed.