Abdicate! Adjudicate!
This shouldn't be hard. This is much like how facebook handles it. The alert icon is highlighted. When you click it it lists all the recent activity it thinks you care about and when you click out of it that info is still there, but the icon is no longer highlighted. I can make an AJAX call when the box UI top box is closed with the X to let the site know to dismiss the new alerts alert.
What a pain in the ass it must be to be popular! Why do I even try?