When you say "other than white," does that mean day-glow red, or perhaps a light, pale or pastel color?
There's nothing in the rent stabilization code or other housing statutes on this. However you should be guided by your lease and common sense.
Some LLs will tell tenants they will withhold the security deposit so they can paint before a new tenant moves in. That is bogus. Painting is considered regular maintenance and covers normal "wear and tear." As such, that is covered by your rent money. It's the same thing for worn carpets and such.
But some tenants will paint room(s) a loud or outrageous color, and that is difficult to cover or return to white, beige or some other light tint without multiple coats. If that's the case, the lease and common sense should tell you that you have some work to do.
If what you did was something not very loud, then there's an argument that what you did might be acceptable.
That the LL never painted the unit while you were there is immaterial in our opinion, but try it as an argument - maybe you'll convince him.
If you chose to paint using a loud color, then you should probably return it to a light color. BTW, when I say light, pale or so on, it does not mean that has to actually be white. Just Google "light paint colors" and you'll find a long list of images of light colors that are not white.
As a guide, put yourself in the place of a new tenant. Would you rent the place with the color that you left it as?
From a practical standpoint, there is no law on this. If you can't come to an agreement, the LL might take the cost of painting out of your deposit (of course inflated beyond reason). Then you can sue him to the unreturned money, at which point it's up to a judge.