The following is very speculative. It needs intensive checking, validating, eliminating. I am offering this info on the offchance that whoever has mentioned Spain may have meant the Spanish Peninsula (Iberia) and that there's some foundation to that oral history.... and that I have counted back sufficient generations to get to the Napoleonic Wars .....
I, stress, I have NOT validated any of the following information which I have read.
It’s source is: Australian Biographical and Genealogical Record Series 1 1788-1841 (Society of Australian Genealogists) (So it is submitted info, submitted back before family history was internet savvy, in an era when research was 'research' so to write.
http://www.sag.org.au/component/content/article/28-basics-on---/79-abgr.html 
PITT, Samuel (1797-1847), soldier, schoolmaster and publican, of Sydney NSW and Hobart VDL; born 01 Dec 1797 Tiverton DEV ENG; baptised 12 March 1798 Tiverton Independent non conformist, Tiverton DEV ENG;

son of Roger Pitt, of Tiverton DEV ENG and Sarah;

education read /write; arrived 26 Nov 1831 Sydney NSW per Surry as Sergeant; enlisted 22 Aug 1826 Bristol ENG 4th Foot Regt Kings Own; rank schoolmaster sergeant; discharged 31 Dec 1834;

land granted 22 acres 08 Feb 1836 Bankstown NSW;

spouse (1) Ann Hutchings (c1803-1848) dau of John Hutchings of Tiverton DEV ENG, married 30 May 1819 St Peter's CE, Tiverton DEV ENG;

issue by (1) Samuel (1820-1831), John (1821-), Roger (1823-1889), Mary Ann (1825-1911) (m Francis),
Charles (1827-1903), Theodore (1835-1916), Caroline Amelia (1838-1879) (m Schott), Mary Jane (1839-1918) (m Walmsley), Samuel (1840-1844), Sarah Ellen (1843-1924) (m Nutt, Hillier).

Samuel enlisted as a private in 4th Kings Own Regiment in 1826. He served in Portugal, Ireland and England. As a sergeant, he sailed with the guard on the transport Surry, arriving in Sydney in 1831.

He became Schoolmaster Sergeant at Norfolk Island in 1833 and was discharged in 1834. From 1835 to 1837 he was parish clerk of St Luke's, Liverpool, NSW and was granted land at Bankstown in 1836.

Samuel held the licence for the Sir William Pitt hotel in Pitt Street, Sydney in 1837. In the same year, he was appointed schoolmaster at Sandy Bay, VDL.
He returned to NSW about 1844 and in 1847 was the licensee of the Black Boy Tavern on the corner of George and King Streets, Sydney.

He died at his Sydney home on 31 Sep 1847 and was buried in the Devonshire Street Cemetery. obit: SMH 01 Oct 1847. (Early Settlement of Sandy Bay, Hobart A. Rowntree).
http://www.sag.org.au/component/content/article/28-basics-on---/79-abgr.html Notice that this chap served in Portugal and of course, Portugal and Spain are close neighbours. Many of the regiments served in the Iberian Peninsula during the Napoleonic Wars.
http://downfallofnapoleonbm.weebly.com/the-spanish-ulcer.html According to the above Bio/Genie Report, among the children of this Samuel PITT is a Charles PITT, 1827 -1903 …. Could he be your Charles?
http://pandora.nla.gov.au/pan/10421/20041220-0000/www.firstfamilies2001.net.au/firstfamily1aed-2.html?id=Pitt4943100010325 Cheers, JM