Does your world-building allow your society to replace itself?
Now, I grant you that many stories are not settings conductive to pregnancy and child-rearing. Wresting one's way through a snowstorm in a mountain pass, fighting a fire-breathing dragon, or descending to the otherworldly labyrinth are not places where prudent people introduce children.
In many fantasy words, the proper thing for the hero and heroine to do would be to conclude their adventures, have their wedding, and settle down so they can get a house and all that in order before the baby's born. In a good number, any child appearing in the story setting would be a reasonable grounds for panic.
Not to mention that if you have a family with two parents and ten children, you are already pushing the bounds of how many characters the story can reasonably hold.
If you look at nineteenth century fiction, you often have families of three, four, five children, without any suggestion that children had died. (To be sure, to an extent the writers would expect that most readers would take it as there having been a dead child or two in the family. Charles Dickens, since he was actually depicting the family life between the marriage of Charles and Lucie and the French Revolution, calmly gave them a son as well as a daughter and killed the son off in the only paragraph where he appeared.)
Still, many stories show enough of the world around the edges to make it very questionable. Especially cozy stories where the characters are in a settled place, exactly where people would have and raise children.
What do we find? Characters who are only children, or have only one sibling, when obviously many people are dying without children. Couples who have only one or two children. Worlds in which we are told that miscarriages and stillbirths have vastly increased, but in which no customs are in place to encourage or compel reproduction to make up for it.
There are cultures where a barren woman has been known to raise the bride price for her husband to have a second wife so he won't divorce her. A culture in which the reaction is a shrug is much more unusual. Divorce, or a second wife/concubine, is routine for handling the situation. Many disputes with the Church stemmed in the medieval era when a king, or a nobleman, wanted to repudiate a wife who had no children, or only girls.
If you take advantage of fantasy magic to decrease infant mortality, and to prevent infertility, you can make many changes. What you can not change is that if your setting does not reproduce itself, it will dwindle and die unless importing enough replacements -- which means there has to be a location that not only reproduces itself, but which has surplus enough to overflow. And does overflow.
Countrysides have long been the source of inhabitants for the lethal cities. Country folk really do have more children -- stork nests correlate with fertility for that very reason -- and they do leave for the bright lights of the big city.
As a consequence, if your city is not reproducing itself, there need to be newcomers, and people who have not been settled long, and it needs to be a commonplace that very few city people did not come from somewhere else.
In Britain, in the nineteenth century, the Queen's List annually handed out noble titles among other honors. Yet the numbers of new nobles only replaced families dying out. Some people merely compared society to a lamp, burning from the top but being fed from the bottom, but others investigated and concluded the culprit was the nobles' habit of marrying heiresses for their money. The thing was, the fewer siblings she had, the more stunning the portion she could bring to her marriage. Then she would go on to have as many children as her mother had. If she were an only child -- the best situation for the wealth -- and the child were a daughter, or even if it were a son, and he died with no brother to serve as the spare, that was the end of the line. Old houses might be able to trace back to a cousin, but the cumulative effect was the extinction of noble lines.
Note that this set-up is entirely dependent on people desiring those titles. If your situation is dependent on new blood, your current population has to allow them in, and they have to want to come. And this will influence the size of the group.
I have seen online fringe groups discussing and belittling the idea that most children take after their parents on the grounds that they were mostly different from their parents, and not realizing that was why they were fringe.
Likewise, if most people who live in the Adventure Outpost never have children, their number will be severely limited by the rarity of people who are willing to drop their lives and everyone they know to trek to the outpost and its danger.
Demographers consider it an important transition when most people in a place are born there for good reason.
It's amazing when I consider the number of settings I've seen -- not just fantasy or historical adventure, but modern as well -- where children are never seen or even mentioned. Makes me wonder just how the population replenishes itself in those worlds.
While do not kill off young children in my stories, I do make having families a priority for my characters. All important ones have two or more children - when they get well into their adulthood. Even the evil wizard has multiple children and many grandchildren - with which he uses to advance his diabolical plans.