A few random thoughts.
Joshua and Mary married in Oct 1778, yet their apparent first child, Rachel, wasn’t born until 1 Apr 1781. Where were they in that two year period.
There’s a big gap in the Soham registers in the decades before Joshua and Mary started having children in 1781, where are no Says. I don’t think Joshua was from Soham.
You mention Little Plumstead, just outside Norwich. I think it’s a red herring. A Joshua and Mary baptised a daughter, Ann, there in 1789. Unless it was an adult baptism she can’t have been the daughter of Joshua and Mary Utting who married in Norwich in 1749 and baptised a son, so you say, in Little Plumstead in 1750. It looks more likely that the Joshua and Mary who also baptised two children in Norwich in 1781 and 1783, and couldn’t have been the Soham couple as there are two children born in Jan and May 1783, one to each couple, was the Joshua baptised in Lt Plumstead in 1750. So I would rule them out. Reference to Lt Plumstead and St James Pockthorpe, Norwich parish registers might help in eliminating him.
There is a Joshua Say of Little Plumstead, Norfolk who married Mary Utting 1 Oct 1749 at St Mary in the Marsh, Norwich....They seemed to stay in Wortwell and Joshua senior died there in Wortwell in 1755.......
The thread mentioned below puts the death in Lt Plumstead, not Wortwell
(as does the LDS pilot site) . Thus there's no link between Norwich and Wortwell
You make the point that Joshua seemed keen to name a son John (although it might simply be a case that he wasn’t very imaginative with names), which if you’re right might suggest that his father was John. Have you seen the thread at
http://boards.ancestry.co.uk/surnames.say/42.1.1/mb.ashx where it’s suggested that he might be the son (or grandson?) of John Say who died in 1763 – presumably the same John whose 1763 will Rosie found?
Have you checked Wortwell parish register for baptisms?
I'd start with the 1763 will
David