OK, then. It is most unfortunate that you are dealing with such a common name in Brown.
IMHO, the info about Charles being the father appears to be, at this point, heresay and should be treated with caution. That both Nellie and Henry named a Henry as their father may, of course, be Nellie naming her brother, as you pointed out earlier, and Henry making up a name but you should consider that they may have actually remembered who their father was. And Henry also apparently had a name for his mother. I think it would at this point be prudent to explore a family with Henry and Sarah as parents, if only to rule out a Henry as the father. We wouldn't want you to go up the wrong tree.
I'm still a bit confused about Nellie and her relationship (or not) to Henry in that you mentioned you weren't sure but then indicated later that a rellie said they were living together in Chatham at some point? Do you have contact with any descendents of Nellie and Sidney to help corroborate your info?
If you can find the shipping manifest again that would probably be helpful here. I have had a quick look but haven't turned up anything contrete.
This is a tough one, for sure.