I just inserted a table with one column and two rows - put the image in the top cell and the caption in the bottom. Went to table properties and aligned right. That seemed to work best so far. Still looking for other solutions if they are out there.
I believe using <tables> will be your best bet to add captions to images. It will also allow you to cater for mobile responsiveness.
As to "Why don't images in design studio have a caption feature? " I would suggest adding this as an idea in the community.
I think there might be a way to do this without using tables in the email, I tried a few things and added some notes below:
I copied the HTML you posted above into an editable area in a tester email. When I tried using the "text-align" tool on the image itself it actually ends up adding a float to the image (this is 'expected' from the editor toolbar but a really poor solution, specifically for email). When I added the text-align right to the 'caption' (<p> tag) it actually doesn't line up under the image (because the image is floated right). I also had a look at using the "align" dropdown in the Advanced panel and selecting "right" on that one and got similar results -- although it doesn't add the "float" style to the image, it does add an inline attribute called "align" (align="right") and that behaves just like a float in this context, it pushed the image far right and the text lines up to the left of it.
Here the image is pushed to the right side of the 'caption'.
I read that you put your content into a table and aligned the table, and I think that's working b/c you're able to style the table-cells (parent elements) to be aligned right and/or b/c the table rows are keeping the content from displaying on the same line.
Another way you could adjust for this would be to add a style to your parent element that wraps your content -- in your example HTML, that'd be the <div id="picture-container">.
In this example, I've added the "text-align:right;" (by manually typing into the HTML editor -- Green sticker) to the parent element (the <div>) so that everything inside of that will line up to the right. The image AND the paragraph will both inherit the text-align style and show up (one atop the other) on the right side of the editor (see green box in background). Unfortunately, Marketo's editor doesn't "recognize" the parent container, it's more concerned with the content (the image and text, aka "children") so you're not able to modify the wrapping <div> (aka "parent") from the WYSIWYG editor, you'll actually have to add it into the HTML Source Editor.
Let me know if this works out for you, or if you've got any questions on what's going on here.