Gensokyo is essentially a world for youkai. Then why are there humans living in it? It’s simple. That’s because youkai need human fear to keep existing. Without them, they would cease to exist.
Because of that fact, the Hakurei Border was designed to be impassable by ordinary humans. However, the same rules do not apply to the youkai. After all, they form hunting parties for human flesh to the Outside World regularly.
The only way you can go out of Gensokyo if you’re an ordinary human is by asking Reimu or Yukari’s help. In a sense, those two are the “jailers” for ordinary humans in Gensokyo.
Who knows? Maybe there’s a chance for the humans living in the Human Village will want to abandon Gensokyo if they know what lives they could have in the Outside World. The first and major perk is, of course, living without worry of troublesome youkai attacking you ever again.