Another question: Per census records Caroline Hannah was raised by her maternal grandparents. Herbert, his wife Hannah, and the woman I believe is Herbert's mother do not appear in the 1840 or 1850 census. Would Herbert have left his child in England and taken his mother to Ireland on a military deployment?
I know it's not possible to trace military records without knowing the soldier's regiment, but I'm interested in getting a general idea about what his life would have been like
Herbert may have left his child and perhaps his wife in England. Some army postings were unhealthy or otherwise unsuitable for soldiers' families. Only a small number of wives accompanied their husbands. A certain percentage were "on the strength" of the regiment and would be allowed to stay in married quarters and allocated rations. Married quarters in some barracks were basic, some were part of a room divided by a curtain. A ballot was held if there were more privates who wanted to take their wives than the number allocated .
Herbert's mother was unlikely to have accompanied him.
Was Herbert a private or an officer?
Local newspapers may have reported when a regiment arrived in a town or left.
There are online lists for some years of regimental deployments. They might be from the "Army Gazette". You can look up a year and browse it to see which regiments were on the move and where they went.
I contributed a post to a RootsChat thread which listed regiments in Ireland in 19thC. It might be in the section Ireland Resources on the Ireland board.
Have you looked for Herbert's military records in National Archives Discovery catalogue? I mean National Archives in England.
Where was Caroline baptised and which baptism register was she on? Were there any other children? There's an Overseas Births Register which began 18th century.
Have you found Herbert's widow on a later census? Have you looked for a 2nd marriage? She may have married another soldier. If she remained a widow she may have applied for assistance for herself and children.