I noticed this too particularly when I view it on some email clients. I also noticed that the alt text do appear on larger images, but not on small icons. But when I do mouse over the image that's turned off, I see text on the bottom of the browser.
If you specifically set the size and use display:block; (which you should) there is only a limited amount of space to place the text. So keep it short. You can also use title="" that has a hoover effect. That way people could still over a non-loaded image (red cross) and get some insights.