This really depends on what he's calling his fee. Only licensed brokers are allowed to charge a Broker's Fee which is something like 15% of the yearly rent. And Brokers can't charge a fee if they also own or manage the building.
Landlords can charge security and perhaps a few other fees for pets etc.
If the apartment isn't regulated, and you don't like the LL's deal, maybe it would be better to continue looking. If he's jerking you around before you even move in, he's likely to be worse once you're a tenant.