working from the census, Samuel appears to be a little older than you suggest
1881 which you have gives him as 49
1871 28 Garth Rd, Maesteg RG10/5414 folio 23 pg 14
Samuel Grove head mar 36 engine fitter b. Llangyfelach
Mary wife 30 b. Neath
William son 13 b. Llangynwyd
Sarah Jane dau 11 b. Llangynwyd
Gwenllian dau 8 b. Llangynwyd
Catherine dau 6 b. Monmouthshire, Tryfethin
Margaret dau 4 b. Glam, Dowlais
Samuel son 10 m b. Maesteg
1861 36 Brown St, Llangynwyd RG9/4082 folio 16 pg 25
Samuel Grove head mar 29 Engineer b. Llanevelach
Mary wife 24 b. Neath
William son 2 b. Llangynwyd
Sarah dau 4 months b. Llangynwyd
1851 25 Union Street, Maesteg HO107/2462 folio 166 pg 24
Margaret Grove head widow 67 b. Glam, Swansea
Samuel son 19 Engineer b. Llangevelach
John Williams grandson 4 b. Llangynwyd
David Martell lodger 35 labourer b. Pembs, Camrose?
1841 Maesteg HO107/1422/15 folio 7 pg 5
Margaret 50
Ruth 20
Willm 15 hallier
Samuel 11 labourer
all born in county
Which leads to a baptism on 1-5-1831 in Llangyfelach for Samuel Grove son of Margaret Grove and Hopkin Gibbs, father being a farmer, abode Alltwen Clydach. The abode looks to have been transcribed as one place, but perhaps it should be Alltwenn for father & Clydach for mother.
William was baptised 15-3-1826 son of Thomas & Margaret Grove, father a bargeman, abode Clydach. Other children baptised in same parish to sam parents Anne bp 25-4-1823 and Sarah bp 15-9-1817.
Thomas was buried 19-3-1829 aged 50.
You can access transcriptions of the parish records on FreeReg
http://www.freereg.org.uk/The Groves in Llangyfelach are easy enough to spot as most Groves are further west in the county in Gower in parishes like Llangennith, Rhossili, Llanddewi.
There's a possible marriage for Hopkin Gibb 17-6-1836 in Llanguicke to Catherine Grove, groom being of Cadoxton. That Hopkin is a mason on the 1851 census, in Alltwen, but I think it's the same man as I've just found Hopkin's father's (Thomas) will and it says farmer on that.