The history of the village can be traced as far back as the Roman times based on a site containing a pottery kiln dated around the third century.
According to the Oxford Dictionary of English Place-Names the origin of Elmswell or in its Old English form Elmswella, as referred to in the Domesday Book of 1086 comes from "Spring or Stream where elm-trees grow". The place-name Elmswella is formed by the conjunction of "elm" + "wella", where wella is Old English for stream.
A huge village green – Butten Haugh Green – once formed the centre of Elmswell. However, the arrival of the railway in 1846 and the bacon factory in 1911, meant the green now has houses built on it.
The 1881 census showed that the number of dwellings in the village was 196 and the population was 761. Currently, the number of houses is over 1300 and the number of inhabitants is over 3350.