These little things such as tribal wars are sent to try us
My ancestor usually noted Hanover on the census as his country of origin, then changed it 1891 to Germany. (He was actually born in a village in what used to be the British Saxon "Kingdom of Hannover"). My aunt told me that the family always called him "the Austrian". He was born in the 1850s and when he was quite young Prussia and Austria organised all the Germanic states to come together under one umberella, if a state demurred then a war was likely to start up and that's what happened up to and during the 1860s when people were eager to leave before a gun was fired. This carried on in the 1870s and eventually got sorted out circa 1902-ish. Many young men who didn't want to be conscripted into the navy or army went AWOL This is what my ancestor's uncle decided to do and he took my six year old ancestor away from his parents and homeland to safety.
1891 census entry = PRUSSIA/GERMANY. Britain recognised the independent state of Prussia, it also then recognised the legal Treaty signed by individual states when they agreed the overall name of their shared country should be Germany. Thus your ancestor could be from the original state of Prussia in Germany or he could have lived in a smaller state under the rule of Prussia in the overall state of Germany.
Two thirds down on this webpage is a map that is colour coded. Underneath the map are the colour coded lists of countries that came under the umbrella of (1) Prussia; (2) Austria), (3) Neutral, and (4) countries with disputed sovereignty.
https://en.wikipedia.org/wiki/Austro-Prussian_WarWhen one looks at the list ruled by the Austrian Empire it's easy to see why my kin called their relative from Hannover, "the Austrian".
As for the Prussian Kingdom and your ancestor's roots; besides the original Prussia itself before Chancellor Bismark took other states under its wing, you have 15 other states to choose from - it even ruled Italy! If you notice the country "Brunswick" - every town in England has a street, or a square or a house named "Brunswick" - that's because the British Duke of Brunswick once used to rule over the principality.
Quite often the surname can give a clue to its origin - such as northern, eastern Germany, etc.
Here's a small sample of surnames originating in eastern Prussia for example
http://surnames.behindthename.com/submit/names/usage/german-east-prussian