Perhaps a red herring but here goes ...
Given that we have found the 1894 marriage and that there aren't any deaths for William J or
Martha Harriet between 1894 and 1901, I had a thorough look at the 1901 census ... and come up with this:
RG13/5043 12 16
Bettws, Glamorgan (Bridgend registration district)
Moelgula Fach William J
Parrey, head, 27, Colliery Labourer underground, b. Hatch Parish Somerset
Harriet, wife, 28, b. Somerset Golden Hill
William T, son, 5, b. Somerset Taunton
Mary Annie, dau, 1, b. Cardiff
Their neighbours are called Cross (Thomas age 30 from Somerset and family).
Now I can't see a matching birth registration for a Mary Annie Parrey or Parry in Cardiff, but there is a Mary Ann Paddick June 1899 Cardiff. There's also a William T Paddick registered in Taunton in Dec 1896 (but none called Parry or Parrey). I can't see a matching William Parr(e)y on previous census either.
So: Does anybody think Paddick could have be misunderstood as Parrey?
If so, then this could be a possible match for him in 1881:
http://search.ancestry.co.uk/cgi-bin/sse.dll?db=uki1881&indiv=try&h=18473902