Exactly how much trouble can that bureaucrat cause you? With his rules and his clerks, his records and his regulations?
Probably not none. His very existence shows he has some clout.
But in most historical settings, and settings based on historical ones, he has a lot less than you might think.
Bureaucracy is very old, as witness that the oldest writings we have, inscribed onto clay tablets millennia ago are accounting records. What the king, or the temple, got in taxes, or offerings, and what was paid out.
Still, things complicated their ability to complicate your life.
Literacy is important, pretty much essential. In fact, there are legends among some non-state peoples in Southeast Asia that once upon a time, they had writing, but it was either lost or stolen from them. Trying to keep a high standard of records by memory alone doesn't do the trick.
But since until recently, the ability to read and write was expensive to acquire, and writing materials were non-trivial, there were hard limits on records. Bureaucracy is expensive. All the wages. All the paper or clay tablets or what have you. All the filing just to find it -- Minoan Crete required a filing system, as deduced by the order of the clay tablets once Linear B was deciphered.
All the money must come from somewhere. Extensive bureaucracy requires an efficient economy and an efficient tax system. Neither one is commonplace in history.
Knowledge of what to write down is also important. There are theories that civilizations always use grain crops not so much for their superiority as food sources as for the ability of outsiders to evaluate your crop and impose taxes as a consequence. Bureaucrats who can not find out things can not cause trouble on that basis.
Even when things can be easily evaluated by someone on the spot, there's also the matter of time and distance. The farther off something is, the harder it is for any bureaucrat to find out things about it, to determine taxes and regulations. The bureaucrat has problems gathering information merely as an outsider, not just as a bureaucrat (though that does not help).
Spies are either outsiders themselves or regard the bureaucrats as outsiders. Some may work honestly, but others will take your money and lie -- on general principles, or to their own advantage. And there's always the question of how long it takes for their news to get back to you.
There's a vicious circle of requiring bureaucracy to gather the taxes you would need to make a solid bureaucracy.
As for promulgating and enforcing regulations, Edmund Burke observed even in the modern era:
The last cause of this disobedient spirit in the Colonies is hardly less powerful than the rest, as it is not merely moral, but laid deep in the natural constitution of things. Three thousand miles of ocean lie between you and them. No contrivance can prevent the effect of this distance in weakening government. Seas roll, and months pass, between the order and the execution, and the want of a speedy explanation of a single point is enough to defeat a whole system. You have, indeed, winged ministers of vengeance, who carry your bolts in their pounces to the remotest verge of the sea. But there a power steps in that limits the arrogance of raging passions and furious elements, and says, SO FAR SHALL THOU GO, AND NO FARTHER. Who are you, that you should fret and rage, and bite the chains of nature? Nothing worse happens to you than does to all nations who have extensive empire; and it happens in all the forms into which empire can be thrown.
This does also have its downside for the subjects in that knowledge of how to wisely regulate is also impossible, and even detrimental effects of regulations may be impossible to glean. Yes, to this day we have people who insist that the effect of a regulation is its declared purpose, no matter what the evidence that it's doing the exact opposite -- but earlier in history, it was much harder to discern what it was doing, and often considerably harder to get the word to the officials responsible, and frequently impossible to make him care.
Subjects may have to take advantage of their distance to evade it and hope the king does not send troops to punish them.
When adding bureaucrats to a world, consider your technology. Consider what magical substitutes you have for technology your world does not have. And aim your bureaucracy for the level you can reasonably support. (Or, of course, develop your world to have the bureaucracy you want.)
As long as the region has literacy, you can do something. The commonest place to have trouble would be in a localized situation. Coming into port. Passing through a city gate. Setting up to sell at a market. Bureaucracy can bring itself to bear most easily when it is in a location, and the conditions are finite, constrained, and open to regulation.
Larger problems with bureaucrats are something you have to build into the world.