I am admittedly posting this prior to looking for documentation - as the community (especially Sanford) has been really helpful in helping us think of solutions to a few of our problems so far. For today's issue, I present to you the case of the growing database. Let's say for example that we have 950K records in Marketo, and because of the way we are loading records in via API - that number grows quickly to over 1M. Oh no! We now owe Marketo lots of money as we have entered a new payment tier.
Then (totally for example!) we decide a good way to get back below the payment threshold. We are going to run some jobs that delete user records for leads that hard bounce, opt out of our email program, unsubscribe (because of durability), etc. It works! Everything is going smoothly - everyone is chipper. We are below the payment threshold.
Then, an obstacle! We have a client call in and request an audit of a program they were enrolled in. Easy enough right? Until we realize that the data they are requesting has been deleted as a result of our new deletion policy. So, the question becomes:
How can we keep those records in order to have the data necessary for emergency audits, historical reports, etc. - without having them count against our known lead count? We already store some key reporting metrics elsewhere - so our main use case is really based on the ability to look back, audit, and be able to re-create the story of what really happened. Any ideas?