I would like to know whether we can access Target Account Management without Admin access. I have tried revoking the admin access but it is showing 'You do not have sufficient privileges to perform this action' while entering TAM.
No you don't need to have Admin access to use Target Account Management. You have to assign a license to the user who needs access (and they can be admin or not). In case you've not done that part, here's how to issue a license.
We had some challenges with non-admin users not being able to access TAM despite having the licenses to do so - they were unable to access/use the Account Smart Lists or view shared lists because they kept getting that permissions error. It turned out this was something on Marketo's side that they had to fix for us and now the team has access.
If you've assigned the license(s) and your users are still getting that error, it's worthwhile submitting a support ticket with as screenshots/details of the error so they can troubleshoot (assuming it's the exact same issue as we had).