It's possible Amy was his birth mother, and his parents just didn't manage to get married until 1902. But a copy of his birth certificate might be the only way to know for sure.
Trying to trace the rest - it seems that his mother is listed as 'Amy L. Fuller', possibly Amy L. Fuller died 1966 aged 91 in Gosport. And now it gets a little complicated:
probatesearch.service.gov.uk shows that this woman was Amy Lesley or Amy Leslie Fuller, of 21 Deanes Park Road, Fareham Hampshire but died in Lee-on-Solent. Adminsitration (no will) granted to wo married women: Janet Isobel Clifford (born Janet I. L. Fuller in 1913 in Preston, mother's maiden name listed as 'Lever', married 1935, died 1983
https://familysearch.org/ark:/61903/1:1:QVZM-GJG3 ), and Violet Queenie May Smith (b. Violet Q.M. J. Fuller 1907, married 1937, died 1982
https://familysearch.org/ark:/61903/1:1:QVZQ-9MKS ).
http://www.lancashirebmd.org.uk/ shows that the L. in Janet's name was "Leslie"
I was having some trouble finding this family in 1911, until I tried under 'Leslie' - I believe Amy Leslie, and daughters Dorothy and Violet, are living in Bristol in 1911, and are the 'Fuller' family you're looking for, but you would need to pay to get the full info and see who else they are living with.
http://www.britishnewspaperarchive.co.uk/ shows some references to Frederick Leonard Griffin Fuller as a publican in the Portsmouth area in 1903, but also in 1906, a Frederick Griffin Fuller who said he went under the professional name of Percy Leslie, and also Percy Leslie Fuller.
So Harry could have also used the surname Leslie or Lesley, and he may have had three younger sisters or half-sisters whose families you might be able to trace.