Roch Castle was founded in the 2nd half of the 13th century, although the lordship de Rupe (rock) can be traced back to about 1200. An earlier fortress may have existed here, but the prominent D-shaped tower on this isolated rocky outcrop is thought to have been built by Adam de Rupe. The family had played an important role in the English settlement of Pembrokeshire and owned considerable territory in the northern areas. Roch Castle was doubtless built as one of the outer defences of "Little England" or "Landsker" for it is near the unmarked border for which centuries has separated the English and Welsh areas of Pembrokeshire.
The de Rupe, or Roche, family came to an end in 1420 and the castle changed ownership a number of times until it came into the possession of the Walter family, who owned it when the Civil War broke out in 1642. Although William Walter sat out the war in the safety of London, his castle was garrisoned by the Royalists, and it was involved in much action in 1644 when it was taken by Parliamentarians, recaptured by the Royalists, and then fell once again to Cromwell's forces. Walter did not return to Roch Castle, going instead to Hague, but his daughter Lucy stayed in London and became the mistress of Charles II. Their son was acknowledged by Charles, who made him Duke of Monmouth - the doomed leader of the Monmouth Rebellion against James II.