Marketo will not sync any custom fields for Campaign or Campaign Members, so you won't see any difference in performance from it, but it is always good practice to hide fields from users where they won't be necessary.
Sorry for the long response time. I had opened a case with Marketo on confirmation of this not too long after I posted it and they gave me a different answer initially. After it was escalated to t2 (I had several questions... ), the response I got back is inline with yours:
Custom fields on the campaign member object will not sync to marketo. Only custom fields on the Lead, Opportunity, Account, and Contact object will sync to Marketo for use.
Thanks for confirming what I was seeing!