I thought that you might have had more information than had made it into your first post!
I suspect that any connection will be in Ireland, before any of them moved to Scotland, let alone Australia.
There's a death of Felix Branigan, aged 71, in Clougharevan, District of Mullaghglass, Newry, on 7 November 1884. The death was registered by his widow, Mary, so I think it likely that he is Nicholas' father.
There are a lot of deaths of Mary Bran(n)igans, as you might guess. You can take a look at them at
https://www.irishgenealogy.ie/en/ and see if you think any might be your Mary Doran. However it will be difficult to be sure because (unlike Scottish ones) Irish death certificates don't tell you the name of the deceased's spouse, let alone their parents.
There seem to be two McGrogan families in Holytown in the 1901 census.
One is the family of Daniel McGrogan and His wife Mary, who are likely to be the Daniel McGrogan and Mary Campbell married in Holytown in 1884. Daniel was 39, a coal miner, born Maryhill. He is therefore probably the Daniel McGrogan born 21 August 1862 in the parish of Barony, parents Patrick McGrogan and Margaret McCabe. This couple also had a daughter Rose, born 4 May 1855. If you think this family is yours, I recommend viewing that birth certificate because it will tell you where Patrick and Margaret were born. They were married in Glasgow in 1852 and the marriage is recorded in both the Roman Catholic register and the Church of Scotland register. Margaret McCabe or McGrogan died in Glasgow in 1870, aged 42. The index does not say what her mother's maiden surname was but the actual certificate will probably tell you the name of her father. There is also a death of a Patrick McGrogan, aged 60, in Glasgow in 1898, mother's maiden surname Tage. He might be the Patrick McGrogan, aged 50, born Ireland, in the 1891 census in Glasgow Calton. His death certificate should tell you the full names of both his parents. Annoyingly, I have yet to find Patrick, Margaret and family in the 1861 census, which would have told me where Margaret was born.
At 32 Baird's Square is John McGrogan, 41, born Glasgow, who, I am willing to bet, is Daniel's older brother, born 10 January 1860 in Glasgow, with wife Margaret, 37, born Holytown, who is probably Margaret O'Neill, married in Glasgow in 1882. You'll need her marriage certificate to find the names of her parents, because there are two possible candidates: daughter of William O'Neill and Mary McGery, born 15 June 1864, and daughter of John O'Neil and Sarah O'Neil, born 8 November 1864. It looks as if Margaret died in Holytown in 1935.
According to the 1871 census, both William O'Neil and his wife Mary were born in Ireland. I have not yet found John O'Neil and Sarah O'Neil in a census. The two Margarets' birth certificates will tell you where and when their respective parents were married.
But so far, there is no coincidence of surnames between the McGrogan and Brannigan families.