Although it is the northernmost settlement in the council area of North Ayrshire it is contiguous with Wemyss Bay, which is in Inverclyde. The dividing line is the Kelly Burn, which flows into the Firth of Clyde just south of the Rothesay ferry terminal. Despite their proximity, the two villages have historically been divided, Skelmorlie in Ayrshire and Wemyss Bay in Renfrewshire.
Skelmorlie itself is divided into two sections, Lower and Upper Skelmorlie. There is one primary school in this village, with secondary age pupils going to Largs Academy in North Ayrshire.
In common with this part of the Clyde foreshore the rich red sandstone is a prominent feature of the landscape and housing in Skelmorlie.