If a lead just has a bounce then no they are not counted as delivered. BUT if a click or open is subsequently logged against these records then they will count in the delivered column in the Email Performance report.
The rules at the bottom of this article are helpful