Remove from Flow won't work, because it only affects a given campaign you're in while you run it.
Instead, I'd modify the "Send Email" flow step to have two choices, and if they were sent the email already, do nothing, otherwise send you message.
Then, you can change the rules so a lead can only qualify once and eventually remove this step once your few leads have passed through.