Hi there,
You cannot delete custom fields but you can hide them. In 'field managemet' (where you create custom fields), you do this under 'Field actions'. The system does not let you delete fields because this leaves you open to accidently deleting data.
The only hazard of keeping an unused field is that it can make your list of fields long and messy. If you or your team can't find the field you want, you are more likely to accidently make duplicates.
A neat list of fields is an easy-to-use list of fields!
Cheers,
Sarah