If this is him, there is a timing problem for the son... but it does match the occupation as given in the marriage snip that Daisypetal posted...
From Leicester Chronicle 08 October 1870
Edited to add: the burial for this fellow at Aldershot Military Burial Ground, Hampshire states he was a Divisional Staff Sargeant, and age 28 (not 29 as the newspapers says).
I don't think this is the right John Yates as it would mean there is no way John could be Edwards father.
A few similarities though??? Are there two John Yates whose occupation is a Clerk

It maybe a possibility to consider that Mary Ann became pregnant after the death of her husband John Yates and entered his details on the birth certificate of her son Edward and giving him the middle name of John

EDIT to add - I have been looking but have not found a William YATES occ Hosier on a census
ADDED - Is it possible that Walton H Bell (Beil) born Quebec Canada might be his father his occupation Book keeper
CLERK (unemployed) on 1881 census - Do you know where Walton was living in 1871?