Note of clarification for this thread:
1851
Civil Parish: St. George Hanover Square
Sub-registration District: Belgrave
Enumeration District: 01a
Robert Gann (73) is actually a Chelsea "Out" Pensioner, and his address has no connection with Chelsea, Chelsea Barracks or Chelsea Hospital.
Ancestry page numbers:
Page 17, Robert is listed simply as "Chelsea Pensioner", but, same location:
Page 20, Thos. Mahon/Maton aged 67, is accurately listed as "Chelsea Out Pensioner".
Page 1, Description of Enumeration District: the address is clearly written as Old Barrack Yard (not Old Barracks Garden as Lily kindly suggested: Yd/Gd). Helpfully (and surprisingly!), the Enumerator has sketched a little map of the neighbourhood.
The location is Knightsbridge and Old Barrack Yard still exists. Here is an interesting write-up with old photos, new photos, old map details and historical info, including discussion about the Grenadier Pub.
https://alondoninheritance.com/category/londonpubs/Other Googling suggests that Wellington used to keep his horses there (?).
*****
I wonder if you have the details for Robert in 1841?
If not, then you may be surprised to learn that he is actually living with His Grace the Duke of Wellington (aged 72) and the Duke’s nephew Gerald Wellesley (aged 30, Clergyman).
The Civil Parish is the same as in 1851.
1841
Civil Parish: St. George Hanover Square
Sub-registration District: Mayfair
Enumeration District: 9
Ancestry page number:
Page 7, Robert Gann, aged 64, M.S. (Male Servant).
There are many Female Servants and about 14 Male Servants, but Robert is the oldest (most senior?), being only 8 years younger than the Duke himself!
The address is just listed as Piccadilly, but it is in fact this impressive building:
https://historicengland.org.uk/services-skills/education/educational-images/apsley-house-149-piccadilly-hyde-park-corner-westminster-9657 Robert’s wife, Sarah, is not with him.
You may enjoy correcting the garbled mistranscription for this household on Ancestry? - starting with the Duke, and including Robert.
UKgirl
P.S. I notice that Rosie previously found this 1841 entry, but perhaps didn't realise who the Head of the Household was!