Yes I found Chesterfields letters, thanks for the pointer. However I mis-posted the link, I meant to post
https://archive.org/details/thralesofstreath00hest, which is also interesting and has accounts of children and the school.
Seems The house is in the Manor of Lambeth Wick, owned by the Archbishop of Canterbury. The manor was leased to Sir Stephen Fox in 1701 and the lease stayed with that family until the early 20th century, they were responsible for the development of Brixton.
https://ovalhistory.co.uk/a-guide-to-each-street-in-my-project/brixton-road/Now a new tack - Wiki for Sir Stephen Fox says he married (2nd mar) Christiana Hope (daughter of Rev Francis Hope). The headmaster of the school after William Hancok Roberts was, if I remember correctly was a Rev Hope. I wonder if there'sa connection.