On our instance we use a custom subdomain for all our lading page like: p.example.com then all the landing pages live off that domain like: p.example.com/some-page.html. Currently if a user goes to the root subdomain: p.example.com they get redirect to our main domain: example.com. I think this is due to the MKTO settings where the Hompage is set to: example.com. My question is can I create a landing page within MKTO with a name of 'index' so its url would be p.example.com/index.html and then set the homepage to p.example.com and have it serve up the page created at index.html?
Tip: don't use domain.com if you mean a dummy domain. That's an actual private domain. Use the actual RFC-prescribed domain example.com or any <tld>.example, like rojas.example. These are guaranteed to never be owned by private entities and so are safe to use in documentation.
Now, to your question.
The reason nonexistent pages are directed to your website root is not because that's the Homepage, it's because you have it as the Fallback Page.
And no, you can't have a page on your site as the Fallback. This makes sense when you consider that Marketo doesn't continually keep track of whether the Fallback Page is actually reachable. If it became unreachable, you'd have an infinite loop. That's bad.
What you should do is have the Fallback Page be identifiable on your website, as in this screenshot:
Then, if you so choose, you can have your web admin (the admin of www.example.com) redirect every request with from=mkto-fallback in the query string back to a specific page like pages.example.com/index.
It's an extra redirect but would get what you want.