Not sure if this will help or not , but I'll post it for other Caldwell hunters as well! My guess is that the James I have is too early for you, probably being born around 1760-1765, therefore having children about the 1780's - 1790's, though if your Margaret was the last in a large number of children its just possible.
The Langtry's shown were from Kilmore, near Lurgan but moved to Belfast at about the time of this tree. There is a Lismoyne Park in Belfast, which is the closest I've found to "Lismoin" that these CALDWELLS come from. They may have had shipping interests out of Belfast, as out family was into shipping between Belfast and Liverpool at that time.
There is a doc at PRONI (T/781/4) that details the Callwell[sic] Family as follows :-
Unknown CALLWELL/CALDWELL parents had children :-
(1) James
(2) Robert of Lismoin
(3) Nathaniel
(4) Mrs Thompson
(5) Mrs Robert Magee
(6) Mrs Langtry (From my notes Frances : b 11/JUN/1772 m 8/AUG/1794 to George LANGTRY, d 8/11/1825)
Children of Robert Callwell of Lismon :-
(2.1) Robert m Benice(?) Harrison (sorry the writing is hard to read)
(2.2) Benice(?) m Mr Irwin
(2.3) William m Elizabeth Marter or Martin "who ran away" (children Emily and Anette)
(2.4) Henry m Miss Martin of Ross, Galway
(2.5) Janet (Unmarried)
On a line that protrudes between (2.2) and (2.3) above, but looking like they belong to Robert(2) are also :
(2.?) "Son in Amerika"
(2.?) do (ditto - I assume another son in America)
(2.?) George of Lismoin
(2.?) William Henry, surgeon at Royal Hospital
Children of Nathaniel (3) are :-
(3.1) jane m Dr Staunton
(3.2) Lelias m Lundy Foot
(3.3) Nathaniel m (1) Miss D'Olier, (2) Mrs English nee Miss Wilson
(3.4) Margaret m George Langtry (From my notes - b 1807 Dublin, m George LANGTRY 2/JAN/1826, d 12/JUN/1876)
(3.5) Joseph m Miss Foot (Archdeacon of Clogher)
(3.6) Catherine m Mr Robert Wilson
(3.7) Elizabeth m Mr Smyley(?) (had children Ellen and Fanny)
(3.8 ) Ellen/Elizabeth m Mr Irwin (had children Henry , unreadable and unreadable))
I think the Henry born to Ellen/Elizabeth and Mr Irwin (3.
might be my great-great-grandfather... he was a “Mr HH Irwin”, with a sister Jane Harriette, but I’ve also seen a record of his father married to an Elizabeth Callwell with 3 younger kids... this looks like a very useful puzzle piece! :-)