To the north of the ancient market town of Haddington behind the Garleton hills sits the ruin of Garleton castle, with an intact crow stepped gabled hall-house, a row of modern cottages which incorporate earlier castle material (including 16th century cannon-loops) and the remains of a drum tower with three basement vaults set into an angled wall. L-plan in form, these vaults which suggest the distribution of weight from several levels above are  all that remain of the castle raised in the late 1500's by Sir John Seton (treasurer of the household and Lord of Session under King James VI of Scots 1567-1625) on land originally owned by the Lyndsay family of nearby Byres castle. Locally the Lyndsays also held Luffness castle beside Aberlady and lands around Ormiston which the later signed over to the Cockburn family.

Garleton incorporates fabric from an earlier Lyndsay tower on the site, and rubble taken from Byres castle as it appears to have been dismantled by the English in 1548 to ensure the security of their 'Fortress of Haddingtoun' during the wars of the 'Rough Wooing' (1544-1550) where by the use of castle sacking they hoped to force the marriage of the infant Mary Queen of Scots to the English Prince Edward. The Seton family suffered heavily during these times with the burning of Seton Palace, Seton Collegiate church and the destruction of Winton Castle in 1544. In 1724 Garleton was sold to the Earl of Wemyss. The presence of the modern cottages on site highlights the common practice of Victorian builders to dismantle such ancient towers as Garleton viewing them as ready made quarries failing to appreciate their historical and architectural value.