There are a lot of threads on data cleanliness. I suggest you look at those as well as blog posts from RingLead, Openprise, and similar vendors.
You should be able to clean up the data using tools like that w/o deleting all of Marketo. And you should clean up your CRM at the same time, even if your choice is to delete what's in Marketo. If your CRM isn't clean, Marketo will never be clean.
But if the data you have in Marketo isn't that valuable to you, sure, purge it all and reconnect the systems later.