There are that many workhouse records for them it's hard to keep track, but there is a admission dated 20 Oct 1870 for Edward & the boys (transcribed as Iris) in the Cleveland Street Workhouse, Westminster:
Edward 38 discharged 3 Mar 1871 Own Request
John 14 discharged 29 Oct 1870 to Edmonton
Edward 9 discharged 3 Nov 1870 to Edmonton
William 9 discharged 29 Oct 1870 to Edmonton
Robert 6 discharged 1 Dec 1870 to Edmonton
James 1 1/2 discharged 3 Mar 1871 with Father
Hannah Ivis aged 35 and James Ivis aged 9 months were admitted to the same workhouse on 9 Aug 1869 and discharged on 14th Aug. The age of 9 months would tie in with the James Ives birth you mentioned, registered December quarter 1868, mmn Clockus.
An earlier admission (9 Apr 1868) is for the boys only, plus a 15 year old Hannah Ivis - presumably their sister. This seems to be her birth registration:
IVISS, HANNAH mmn CLOGHER
GRO Reference: 1853 M Quarter in SAINT MARTIN IN THE FIELDS Volume 01A Page 269
In 1881 think James is in the Home for Working & Destitute Lads (Dr Barnardo's Home), Stepney Causeway, aged 10 & born Wylde Street, Covent Garden.