Well I finally got to the record office for you !!
John Simpson died 23 April 1896 at Burghill Asylum, and
Sarah Ann Simpson (nee Clarke) died 24 August 1898 at Catherine Street in the All Saints district.
Family was living on Catherine St. in 1881 and 1891 censuses.
It was somewhat of a negative result, but more might be possible.
I firstly searching all the MI's for the All Saints graveyard; zilch !
Then I searched all the MI's for the public cemeteries covering families from All Saints District ! Zilch !
Then I widened my search to cover all the other public cemeteries in Hereford for Church of England useage. I DIDN'T cover non conformists, Roman Catholics, etc. I DID find two MI's for other Simpsons, but not for John or Sarah ... I will send you those details in a P.M. just in case they might be useful or relevant. If you know they were RC or non-conformists, then I can search those another time.
I then asked about newspaper obits ... they are in the library, which wasn't where I was, so that'll have to wait for another day.
Finally I asked about asylum records, thinking they might turn up something about John ... but although they DO have those at the RO, you have to apply in writing for a researcher to extract information for you (i.e. one of the RO staff does it) ... even though its over the 100 year period for secrecy of records, and even though the person concerned is no-one to do with me, and indeed, I myself have no rellies in or near Hereford.
So I'll also PM you the address you have to write to in case you want to see any info from the asylum record.
I'll go to the library for you when I'm next in Hereford with time to research, but it probably wont be for a month. The staff at the RO were very helpful, and did suggest that if the man was C of E, and was from an asylum, and didn't appear to have a headstone, its quite possible he was a pauper and was buried in a paupers unmarked grave ...