Hello,
there is no James in 1851 for Toddington with a Samuel.
I only have access to Folio No's but did have a look at nearby villages. The ones that I thought might possibly be your Sinfields are as follows having looked at them on the IGI. Maybe someone can do a look up for the full entry. Is only an idea as Sinfields are pretty prolific in Beds/Bucks. My husbands grandmother was a 'Sinfield' so I have done a little on them in both Beds and Bucks but no connection to yours that I can see.
1851 census Lidlington
Folio 30
James Sinfield, 8, LID
Joseph Sinfield,11, LID
Ann Sinfield, 23, LID
Elizabeth Sinfield, 22, LID
George Sinfield, 19, LID
Henry Sinfield, 19, LID,
Thomas Sinfield, 28, LID
Ridgmont
Folio 102
Samuel Sinfield 47, LID
Samuel Sinfield, 10, jnr, LID
If you check out Sam jnr on the IGI he is from Samuel/Mary.
Almost all the others in Lidlington also are from Samuel/Mary so I think worth a try.
Good luck, Waterfall