Favicons are cached rather heavily by the browsers, especially IE. If you have managed to see the correct icon in most browsers, including IE 11, the server is configured properly.
As for the wrong icon showing up after restarting the website in IIS, that's very odd and I have no advice for you about that problem.
The only thing left you could try is to make sure there is no favicon.ico file in the root of the site or, if there is, it is the correct one.
Thanks,
Joe D.