In 1646 he resigned his command. He was weary and heavy laden with unjust burdens heaped on him by those for whom and with whom he was fighting; he was worn out by the siege; by the characteristic treachery of the King, who, being unable to assist him, could not refrain from sending lying promises instead; and by the malice of his neighbour, George Carteret, Governor of Jersey, who himself made free with the Guernsey supplies, while writing home to the King that Sir Peter has betrayed his trust. Betrayed his trust, indeed, when he and his garrison are reduced to "one biscuit a day and a little porrage for supper," together with limpets and herbs in the best mess they can make; nay, more, when they have pulled up their floors for firewood, and are dying of hunger and want in the stone shell of Castle Cornet for the love of their King. However, circumstances and Sir George Carteret were too much for him, and, at the request of Prince Charles, he resigned his command to Sir Baldwin Wake in May 1646, remaining three years after this date at St. Malo, where he did what he was able to supply the wants of the castle. Sir Baldwin surrendered the castle to Blake in 1650. It was the last fortress to surrender.
In 1649 Sir Peter, finding the promises of reward made by the Prince to be as sincere as those of his father, returned to England, and probably through the intervention of his father-in-law, who was a strict Parliament man, his house and a portion of his estates at Chicksands were restored to him. To these he retired, disappointed in spirit, feeble in health, soon to be bereft of the company of his wife, who died towards the end of 1650, and, but for the constant ministering of his daughter Dorothy, living lonely and forgotten, to see the cause for which he had fought discredited and dead. He died in March 1654, after a long, weary illness. The parish register of Campton describes him as "a friend to the poor, a lover of learning, a maintainer of divine exercises." There is still an inscription to his memory on a marble monument on the north side of the chancel in Campton church.
Sir Peter had seven sons and five daughters. There were only three sons living in 1653; the others died young, one laying down his life for the King at Hartland in Devonshire, in some skirmish, we must now suppose, of which no trace remains. Of those living, Sir John, the eldest son and the first baronet, married his cousin Eleanor Danvers, and lived in Gloucestershire during his father's life. Henry, afterwards knighted, was probably the jealous brother who lived at Chicksands with Dorothy and her father, with whom she had many skirmishes, and who wished in his kind fraternal way to see his sister well--that is to say, wealthily--married. Robert is a younger brother, a year older than Dorothy, who died in September 1653, and who did not apparently live at Chicksands. Dorothy herself was born in 1627; where, it is impossible to say. Sir Peter was presumably at Castle Cornet at that date, but it is doubtful if Lady Osborne ever stayed there, the accommodation within its walls being straitened and primitive even for that day. Dorothy was probably born in England, maybe at Chicksands. Her other sisters had married and settled in various parts of England before 1653. Her eldest sister (not Anne, as Wotton conjectures) married one Sir Thomas Peyton, a Kentish Royalist of some note. What little could be gleaned of his actions from amongst Kentish antiquities and history, and such letters of his as lie entombed in the MSS. of the British Museum, is set down hereafter. He appears to have acted, after her father's death, as Dorothy's guardian, and his name occurs more than once in the pages of her letters.