I see his mother died and his father remarried before the 1901 census. None of the children are with their father and his new wife.
Yes Ellen died in childbirth and the child died as well in 1877. Ellen was Joseph's 2nd wife. He'd previously been married to her sister, Mary and she died young leaving him with a number of young children. I can only assume that Ellen helped to look after her nieces and nephews and they ended up getting married.
He then married my great-grandmother Ann Eliza in 1899 and went on to have 3 more children, including my grandad. My dad and his brothers and sisters did not know of these 2 other families until I started doing the family tree. So I can only assume he didn't have anything to do with them or my grandfather never talked about them. He can't have been a heartless man as my great grandmother had a child prior to marriage and he brought up Bertie as his own.