The village is on the crest of an escarpment. The parish extends mostly north and north-east of the village, in which directions the land rises gently and then descends to the Glyme at Glympton and Wootton about to the north-east. South of the Stonesfield, below the escarpment is the River Evenlode.
Stonesfield is mentioned in the Domesday Book when its toponym was Stunsfeld (meaning "fools field"). This was because of the stony nature of the soil in the area, so the mutation of the name is most appropriate. Thomas Hearne used the spelling "Stunsfield" in 1712 and Akerman spelt it "Stuntesfield" in 1854.
In 1743 a clock was installed in the Church of England parish church of St. James. It was said to have been made for a local manor house in 1543, and transferred to the church after the house was demolished. The clock has since been moved from Stonesfield, rebuilt, and installed at Judd's Garage at Wootton.
Until the 20th century Stonesfield was the source of Stonesfield slate; a type of Cotswold stone slate that is common on the roofs of older buildings in the Cotswolds and Oxfordshire. Many of the older buildings of the University of Oxford have Stonesfield slate roofs.
The first fossil bones to be described as those of a dinosaur of the genus Megalosaurus were found close to Stonesfield and named in 1824 by William Buckland. The slate-mining activities unearthed many finds in succeeding years.
During the 1960s and 1970s new houses were built on the eastern side of the village. Most of these and all of the old cottages and larger modern houses in the original part of the village are now unaffordable to the children of locals, many of whom consequently have moved out of the area.