Downe is south west of Orpington and south east of Charing Cross. Downe lies on a hill, and much of the centre of the village is unchanged; the former village school now acts as the village hall. The word Downe originates from the Anglosaxon word doon, latterly down, hence the South and North Downs. The village was part of Kent until April 1965 when it (and the remaining part of Orpington Urban District Council) was subsumed into the new London Borough of Bromley.
When Charles Darwin moved there in 1842, the village was known as Down. Its name was changed later to Downe.