Hello Abygail,
I stumbled across this thread as I am researching Henry Thomas FITZGERALD who you mention in your opening post, who died in 1826 in Tasmania. I gather from your subsequent posts that you have probably discounted his son William Henry FITZGERALD as "yours", but just in case, here's what I think I know so far (though I'm still only at the beginning of my researching this family).
Henry Thomas FITZGERALD arrived in Tasmania in 1823, married Harriet DOWNEY on 4 Dec 1824 and had two children:
1825 - Mary Ann (who married John LEE in 1838, very young I know! However, pretty much confirmed in her obituary)
1826 - William Henry (who I think married Harriet GUNYON in 1852)
As you say, Henry Thomas drowned in 1826 "leaving a wife and two infant children wholly unprovided for". Pretty sure Harrriet then marries George WILLIAMS in 1828, convict from the Lady East, who arrived in 1824, convicted in Middlesex, originally from Bath. His convict indent does not paint a great picture of him, repeatedly absent with his wife and keeping a disorderly house. As you say, a Harriet WILLIAMS died in 1881 aged 80, which is around the correct age, however I haven't yet confirmed whether this is the correct Harriet.
The son, William Henry FITZGERALD lived around Gould's Country, Tasmania and died in 1909.
So I'm afraid this doesn't help you much except to probably eliminate someone.