Hei Maria,
What date did you find the families in Cumberland and Durham?
Where in Co. Down were the children of Andrew and Mary born? I am just wondering whether Robert might have died so that James was living with Andrew and Mary when he married Rose Rooney. Rooney, to my mind, would be more common in Co. Down than in Belfast.
Directory of Belfast and the Province of Ulster 1852
Belfast
William Clark, general smith, Little York Street
Francis Rooney, mariner, 33, Green Street
James born about 1847 so I will have a look in directories late 1860s to early 1870s to see if I can find Robert and more Rooneys.
All the Best, Chris