Family mssing from 1861 census

[prozac]

I have an ancestor (and family) totally missing from the 1861, what would be the reason for this?

Joseph SMITH (I know, I know...) born in 1803, Tipton.  Wife is Jane, born in 1807, Tipton.
In 1841, 1851 and 1871, they are living at Horton Street, West Bromwich.  In the electoral register for 1861, Joseph Smith is listed at Horton Street.

He was a Boiler Maker in 1851 and 1871.  He died in 1873, the probate states Joseph Smith, of Newtown, West Bromwich and I believe Horton Street was part of Newtown.  Jane also died in 1873.

I have searched all kinds of combinations, I have considered they were visiting someone, but I just cannot locate them.

[prozac]

I'll also add, I found Horton Street in 1861, and I searched the entire street and they were not there. 

[CaroleW]

I see the 1871 shows a son Josiah b 1850 West Bromwich.  Can't find them in 1861 so assume they were just missed by the enumerator