The first record of a settlement at Compton Verney was the late Saxon manor of Compton. It had good communications, being served by the Fosse Way, which ran north–south half a mile from the site and led from the Roman settlements of Cirencester to Leicester. The Domesday Book of 1086 lists Compton as two manors, the largest of which was held by the Count of Meulan and later inherited by the Earls of Warwick.

At some time before 1150 the manor was granted to Robert Murdac when it became known as Compton Murdak, and was inherited by his descenants and thence to the heirs of the Murdak family. In 1370, after two hundred years of ownership by his family, Sir Thomas Murdak surrendered the estate to Alice Perrers, mistress of King Edward III.

In 1435 it was acquired by Richard Verney (1435–1490) with the assistance of his younger brother John Verney, Dean of Lichfield, and Richard Beauchamp, Earl of Warwick. The Verney family had begun acquiring lands in the area of Compton Murdak and the surrounding villages in the 1430s before purchasing the estate. In about 1500 the manor became known as Compton Verney. According to William Dugdale in his Antiquities of Warwickshire (1656) a new manor house was built in about 1442: "Richard Verney Esquire (afterward Knight)… built a great part of the house as it now standeth, wherein besides his own armes with matches, he then set up towards the upper end of the Hall, the armes of King Henry the Sixth".The Manor House was extended by succeeding generations as the family's fortune expanded. In 1695 Richard Verney (1621–1711) established his claim to the Barony of Willoughby de Broke before the House of Lords, following the early death of his fifteen-year-old great-nephew William Verney, 10th Baron Willoughby de Broke, only male descendant of his elder brother Greville Verney, 8th Baron Willoughby de Broke, and moved from his Rutland estate to live at Compton Verney. His son George Verney, 12th Baron Willoughby de Broke (1659–1728) rebuilt the house and landscaped the gardens in the early 18th century. John Peyto-Verney, 14th Baron Willoughby de Broke (1738–1816) completely remodelled the house to the design of Robert Adam and the grounds to the design of Capability Brown.

Adverse financial circumstances forced the family to let the house from 1887 and finally to sell it in 1921 to the Leeds soap magnate Josph Watson (d.1922) (created in 1922 Baron Manton of Compton Verney) who however never resided there having died the following year. His eldest son and heir Miles Watson, 2nd Baron Manton (1899–1968) resided at Compton Verney for a short while, but soon sold it and moved to Plumpton Place in Sussex.

It then belonged to various owners before being requisitioned by the War Office during the Second World War. After the War the house was never lived in again. It now belongs to the Compton Verney House Trust who run it as an Art Gallery.