You could try going into the Members tab and removing them individually using the Change Status to "Not in Program". Otherwise, I might review the triggered smart campaigns that you have adding members to the program because even if you removed them, the smart campaign may be adding them back in each time it runs.
... View more