Does the site look different in Chrome or Firefox when you use the IP vs using the Domain?
It sounds like the DNS A record for the Domain isn't pointing to the correct IP, maybe an old site?
What about the site is wrong in IE, does it not have any style at all, or are some styles not applied? Are there no images?
This is a very strange problem so we need more information to be able to help you. Saying the site looks lousy doesn't give us any clues.
Thanks,
Joe D.