The new(er) email program is pretty buggy. It can be very annoying to get the control panel to read the correct numbers--I've experienced the same issue where I set my smart list and the total should be 8, and the control panel shows 6,000+
Your best bet is to set your smart list settings and exit the program entirely. And refresh your browser. Then navigate back to the control panel and it should refresh on its own and come up with the correct number. Maybe. It's a very buggy process.
What I would recommend is that you build your target list with a smart list or smart campaign beforehand and only use that as your criteria within the email program. Not only will this save you the hassle of the control panel, but it will also speed up your sends. If you use complex smart list logic within an email program, it will severely slow down your send (I've had an email take an hour after it was scheduled to send to complete because of complex list logic).