Hi,
Welcome to RootsChat

There is a Priscilla with the same details as Allen in the GRO Chaplains Returns Marriages,
Pricilla HASWELL
1825
Trinidad , West Indies
Page: 599
but I don't know if this is the only bride who fits without checking all the 1825 West Indies marriages, 535 of them on FindMyPast

Unless he married two Priscilla's or couldn't marry her until later I agree with Annette that this marriage looks unlikely because of the children's births.
I don't know if you have seen it but there this record,
http://discovery.nationalarchives.gov.uk/details/r/C8862944Wo 23 - Royal Hospital Chelsea: Admission Books, Registers, And Papers 1702-1876
Serjeants
Regiments: 5th Batt
Allen HEDLEY
Date of Admission: 1 Aug 1825
Age: 44
Complaint or cause of discharge: Worn Out
Where born: Elsdon, Northumberland
Height: 5ft 7 1/2
Hair: Brown
Eyes: Blue
Complexion: Fresh
Occupation: Labourer
There are some figures under the "Statement of Service", I think to do with pay, including some in the "West Indies" column.
A woman marrying in the West Indies could be the widow or daughter of another serviceman or just someone who lived there, sorry that's not very helpful in your search.
I looked at the 1851 census that you refer to and Priscilla doesn't have "do do" under her place of birth like the person above her for ditto , Okehampton, Devon, she just has two lines - -. so it isn't clear where she was born.
You could order the marriage cert. from the GRO to see if there are more details about the marriage, more info about that here,
https://www.gro.gov.uk/gro/content/certificates/most_customers_want_to_know.asp#OverseasRegards,
Daisy