When merging, Marketo also tries to provoque a merge in the CRM. If the CRM merge fails, the Marketo merge merge fails as well.
So, as Dan points out, what is your CRM ?
Are your trying to merge a CRM lead with a CRM contact ? 2 CRM contacts that do not belong to the same account ? These cases are almost 100% guarantee of failure.
To build on what Greg stated, the reason why it's important to know your CRM platform, is because merging of leads is treated differently based on your CRM platform. For example, if you have MS Dynamics, the merge must be done in CRM, not Marketo (vote for this idea here if you have MSD: ). If you have SFDC, you can merge the records in Marketo. But like Greg said, only leads with the same CRM type (lead or contact) can be merged (e.g., you cannot merge a lead and a contact - they must both be leads or contacts).