Firstly, I am interested to know how it has been established that Mary Milbourne had a daughter called Sarah.
Could "Sarah", the suggested daughter of Mary Milbourne and John Taylor, be the Sarah who is shown as the wife of James Smith, constable of Cabramatta in the 1828 NSW census. Sarah is shown in the 1828 census as CF (came free) on Friends (the same ship that Mary Taylor arrived on) but the date is shown as 1817 (another convict ship Friendship arrived in 1818). Errors were commonly made in the census though.
I have seen on one site the suggestion that the Sarah Taylor who married Robert Garrard (shown as Garrett in NSW BDM site) was aboriginal. I cannot find Robert Garrard in the NSW 1828 census which would show details of his wife.