Somerville has been interested in art. In 2001, she was one of the people who selected the National Portrait Gallery's BP Portrait of the Year. She has also served as a judge for several years on the RIBA Annual Architecture Award Panels. On 18 September 2003, Somerville was appointed Chair of the Advisory Committee of the Government Art Collection.
Somerville has married three times: to Stephen Band (1970 - 1975, no children), Ray Gowdridge ( 1984 - 1992, two children). She currently lives in North London with her third husband, the architect Sir Jeremy Dixon. Somerville and Dixon were interviewed by police in 1996. The people who developed their photograph had raised suspicions, because the couple wanted ot have images of their seven-year-old child developed. These photos showed the child bathing. No caution or charges followed the investigation.