William Cavendish, first Earl of Devonshire, acquired the Abbey in 1613. The estate that Cavendish purchased included a reasonable sized mansion with the south elevation overlooking grounds that had once been the Abbey precinct. However, Cavendish was the second son of Bess of Hardwick, and an immensely wealthy aristocrat (owning 97,000 acres) and decided to make Leicester Abbey his principle residence. So Cavendish embarked on a massive building programme to extend the mansion. Ranges were added to the south elevation and a large new wing was added to the north, along what had once been the west wall of the halt-way in front of the monastic gateway.
William Cavendish died in 1626 and was succeeded by his son, the second Earl, also a William. The second Earl only survived his father by two years, whereupon he was succeeded by his son, the third Earl, yet another William, who was still a minor.
The Cavendish family had large estates in Derbyshire: in addition to Hardwick House the first Earl had inherited Chatsworth House from his mother, and he and his mother had invested massively in both. So it is not surprising that after the death of the first Earl, the family decided to make Chatsworth their principle residence rather Leicester Abbey. Thereafter, Cavendish House became a secondary residence, used as a stopping off point on the journey between Chatsworth and London. Once the third Earl came of age in 1638 it became the main residence of his mother, the Dowager Countess.