A few people deliberately avoided enumeration in every census year, but it is much more likely that 'missing' people are actually there, but are hard to find or identify for some reason.
Individuals who are not with their families are a case in point. If the head of household filling in the schedule is a landlord or employer, they are much less likely to know (or even care about) accurate age and birthplace details for everyone under their roof. And of course if you do find your single apprentice, servant, soldier etc, it's much harder to be sure that you have the right one, because they are not with their father William or sister Philadelphia, or whatever you would normally use as a checking point.
Plenty of people today completely fail to follow simple instructions and fill in forms correctly, so why should we expect our Victorian ancestors to be any different? There were misunderstandings over the arrangements for shift workers, and people who were short-term visitors could be missed out because the head assumed that they would be included at their usual address, and so on.
Then the enumerator had to copy up all the schedules, as best he could. Since it is only the enumeration books that we see, we have no way of knowing what the enumerator was copying from, so even though we are pleased to find one whose handwriting is very clear, it won't help at all if he had to decipher something horrible. And that's before Ancestry et al start indexing, but that's another story...
1911 is very interesting, because for the first time we get the information directly from the schedules and cut out the middle man (or woman). The standard of legibility is generally pretty good. On the debit side, 1911 was the first census to be the subject of an organized boycott (by the suffragettes), but you can't win 'em all, I suppose.
Mean_genie