It's because certain sites are written using specific MS html code (ie that's the default setting of MS FrontPage) or use weird Java routines. While both conflict with recognized standards, they are none the less still used by some today.
The solution is easy: install FF's extension called IE Tab and it will open an IE windows within FF and if you want it will remember which website require IE Tab and automatically do it whenever you visit them,
As for the initial question: Download and install Firefox, it will coexist with whatever you have now. If you take the time to look at Firefox and compare it to IE you might actually never use IE again (it's faster, safer and more user friendly).