Charlton House is the birthplace of inventor Henry Bessemer in 1813. The mill-wheel was adapted by his grandfather to power a small foundry. The water-mill was therefore converted to a foundry during the occupancy of the Bessemer family and back to a mill again afterwards. There are remains of a windmill less than half a mile from the Windmill Pub from which it may have taken its name. There was, until the 1970s, also a water-wheel in the mill-race in the yard of Wellhead Farm.
According to an article by Peter Harkness in Vol 1, No 1 of "Old Hitchin Life" the Harkness family's now world-famous rose-nursery was, in the late 19th century, based in Charlton as well as Bedale, in Yorkshire, with Robert Harkness moving into Charlton House (Bessemer's birthplace) in 1895.