How do I convert free users to paid without annoying them?

Make the ask specific and personal. Reference something they actually did with the product, name the upgrade that solves the next problem they will hit, and give them one clear action to take. Generic upgrade emails get ignored. Specific ones get replies.