I agree that William's birth would - if registered - have probably been either Q1 or Q2, more likely Q2. However, I have checked for births in those quarters - and similarly for James - and not found an appropriate one.
As for late baptisms, yes I have checked both London and Surrey on Ancestry, and can't find anything suitable. I have failed in the past due to surnames being transcribed in a very surprising manner, but I think I have looked at the most obvious bus-spellings. I would be happy to be contradicted.
As for the 1851C, James is 13 and William 10, both born in Bermondsey. Both ages are consistent with 1841, thus confirming the birth years.
All other children were baptised.. so I am surprised by the absence of such a record for the two boys. I am wondering if the ambiguity about the family's religious affiliation could explain the absence. Perhaps, the family changed allegiance around about the time of the births. Perhaps, they were only baptised as non-conformists.
If they were baptised as non-conformists that makes life more difficult, as locating such records in my experience is not easy. I have no idea where to look for non-conformist records in the Bermondsey/Southwark area. This is why in another thread, I asked if anyone knew where the records for the Southwark Chapel were deposited, as that is where their father was buried in 1845.
PS Yes I have looked at the baptisms of the other children and sadly they give no clue as to where James came from. The marriage of James and Frances is of no help, just stating that at the time they were both members of the parish of St Mary Abbots in Kensington. The baptisms just confirm that James was a carpenter and the family's residence in Kensington, Newington, Southwark and Bermondsey. I know for sure from later censuses that Frances was from Brampton in Huntingdonshire and have found her baptism and birth family. But James only appears on the 1841C, where of course, it only states that he wasn't born in Surrey.
It is worth noting that Some baptisms were a few years after the birth. So the two boys could have been baptised together a few years later, but if they were I can find no record. I am beginning to think it will require someone with access and knowledge to non-conformist records to unlock the puzzle.