The town was first settled around 1665. Local legend has it that after rescuing a Native American Chief's kidnapped daughter, Richard Smith was told that the Chief would grant title to all of the land Smith could encircle in one day - on a bull. Richard Smith chose to ride the bull on the longest day of the year (summer solstice) - to enable him to ride longer "in one day". The land he acquired in this way is said to approximate the current town's location. There is a large anatomically correct statue of Smith's bull, known as Whisper, at the fork of Jericho Turnpike (New York State Route 25) and St. Johnland Road (New York State Route 25A). Smithtown originally was known as "Smithfield."
The border between Smithtown and the Town of Huntington is partially defined by Bread and Cheese Hollow Road (Suffolk County Road 4), so named after Bread and Cheese hollow, which according to legend is where Smith stopped on his ride to have a lunch of bread and cheese. The road is reputed to follow part of his original ride.