Upon his accession, the Conqueror gave the town and castle to his half-brother, Robert, Earl of Mortagne in Normandy, whose descendant, William, was deprived of all his possessions, and banished the realm, by Henry I. for rebellion. That monarch granted them to Gilbert de Aquila, in allusion to whose name this district was afterwards styled the Honour of the Eagle.
The outer work of the castle contains many Roman bricks and much herring-bone work. The outer walls, the most ancient part of the fortification, inclose seven acres, and are from twenty to twenty-five feet high. The moat on the south side is still wide and deep; on the other side it has been filled up. The entrance is on the west or land side, between two round towers, over a drawbrige. Within the walls is another and much more modern fortification, approaching a pentagonal form, with nearly five circular towers, moated on the north and west. It is entered from the outer court by a drawbridge on the west side between two towers. The principal barbican, or watch tower, is not at the entrance, but towards the north-east corner. The walls are nine feet thick, and the towers were two or three stories in height. The castle was of great strength: it withstood the attacks of William Rufus's army for six days, protecting Odo, Bishop of Bayeux, who ultimately yielded only for want of provisions; and it afterwards successfully resisted the siege of King Stephen, who personally superintended the attack, but met with so gallant an opposition from Gilbert, Earl of Clare, that he was obliged to withdraw his force, leaving only a small body to blockade it by sea and land. It once more resisted hostile attacks, when it was fruitlessly assailed in 1265, by Simon de Montfort, son of the renowned Earl of Leicester. Again, when Sir John Pelham was in Yorkshire, in 1339, assisting Henry, Duke of Lancaster, to gain the crown, the castle, left under the command of Lady Jane Pelham, was attacked by large bodies of the yeomen of Kent, Surrey, and Sussex, who favoured the deposed King Richard, but was bravely and successfully defended by Lady Jane Pelham.
Pevensey castle remained as a fortress till the reign of Elizabeth: two ancient culverins, one of which bears her initials, are yet preserved; after which its history is not traced till the Parliamentary survey of 1675, when the fortress was in ruins, and the ground within the walls was cultivated as a garden. The demesne and castle are now held by the Cavendish family, under a lease from the Duchy of Lancaster, which was originally granted to the Pelhams by Henry IV., son of the famous John of Gaunt, Duke of Lancaster, to whom the Honour of the Eagle had been given, on his surrender of the great earldom of Richmond.