I have Bowers ancestors too.
My great grandmother was Millicent May Bowers. You can read about her and see a photo at
https://a-little-bit-of-this.co.uk/2021/01/16/millicent-may-bowers/. She was the second wife of my great grandfather Joseph Charles Abram and they had three children, including my grandfather my grandfather Louis Bowers Abram.
She can be found on the 1891 census living at Horringer Road, Bury St Edmunds, with her parents Henry and Eliza and four brothers and sisters. Aged 17 she can be found at 18 Eastgate Street, Bury St Edmunds, again with Henry and Eliza. In 1911, she can be found in Woodbridge with her sister.
Looking for Henry, I believe I have found him on the Bury St Edmunds census in 1881, Horningsheath in 1871. I am struggling with 1861, as I believe that Henry's parents were John and Elizabeth or Eliza. I may have located him on the 1851 census, again in Horningsheath but the surname is Bowars not Bowers and because I haven't found the 1851 census, I cannot be sure. Then in 1841, I have located a Henry Bowers, less than one year old, living with an Eliza Bowers and no other family members at Upstreet Green, Great Waldingfield, Sudbury, Suffolk.
I can't quite make out what is going on but maybe the information about my Henry will be useful to you.