Leic Mercury has a report from March 1860 in which one William Foulds was charged with stealing property from his mother Ann Foulds of Bedford St, Leicester (the case was discharged) -- is this him?
Added - interesting court report from October 1860 which presumably does refer to your William. The headline is "A neglected lad" and William's age is given as 14, occ shoemaker. He was charged with theft but the "Learned Recorder entertained great compassion for him, as he was a poor neglected boy, with no-one to care for him. He was glad to know that there was now opportunity under the new arrangements in the prison, for keeping lads like him away from bad companions; and he hoped he would attend to the lessons and advice that would be given him, so that when he came out of prison, Mr Marshall would give him a good character and some person would be found to take care of him. He should have given him a lighter sentence than he was about to do, but he tbought it would be for his own benefit to get the instruction imparted in the gaol"
William was sentenced to 6 months "with such hard labour as would do him no harm". However "a kind friend" promised that if William served two months extra, he would give him employmnet when he came out, a guarantee that William accepted.
Things must have been really hard for young William if prison was better than whatever his alternative was.
Unfortunately he reappeared in court on charges of theft from someone in Bedford St in Oct 1861 (1 month hard labour), Jan 1862 (he stole a sheet from the lodgin house in which he was staying, again in Bedford St -- 3 mths hard labour), May 1862 ("An incorrigible youth" - remanded until July when he was sentencd to 18 months - age given as 18, occ shoemaker).
So Bedford St seems to have been a bit of a focal point for young William, which would point me towards the earliest theft charge in Bedford St also being him, which would back up the suggestion above that he was son of Ann and John.