I agree, to preserve information for reporting as well as dedupe in the future, you do not want to delete them. So what you can do is perform several functions in order to make sure they are not marketed to (ex. blacklisted or marketing suspended). However, they will still remain in the database and count towards your database limit.
Another option is to create a new lead status for these leads so they are easily recognizable.
Here are a couple discussions from the past with a similar question:
Leads deleted in SFDC (this one suggests that "archiving" is available through partitions with a Marketo Enterprise account)