If the Victor born 1904 is the correct man, I have a feeling the father Joseph has left Emily and may have possibly married again (bigamously??) in 1906
Marriages Dec 1906
HOYLAND Joseph Walsall 6b 1344
LEADBETTER Amy Walsall 6b 1344
1911 census
Amy Hoyland Wife Married Female 31 1880 - Walsall Staf
Marjorie Hoyland Daughter - Female 2 1909 - Portsmouth
1921 census has Joseph/Amy & Elizabeth.
SS