and the thing is, it is not consistently doing so. Sometimes it is just happening in IE and sometimes it happens only on certain posts or only on homepage.

Here are the list of things you can do to fix that.

Step 1. Validate !
This is the most common cause for this issue. Try to validate the page that has this issue, at W3C’s Validator.
Once you fix the issues found by the validator, Refresh the page and see if the problem is gone.
If not, go to step 2.

Step 2. Check your Images Width
The next cause could be, your images are wider than the ‘main content’ div (usually #content in most of the themes).
If it is OK, try to resize the image to fit within the content DIV.
May be, you can give a smaller size image on the page, and the full-size image can be hyperlinked to that small image. This also helps in faster loading of your page.

If the problem persists, go to Step 3.

Step 3. Reduce the size of the ‘Content Area’

You can open up your style.css and look for the ‘width’ definitions for #content and/or #sidebar.
Try to reduce the width by 5px or 10px or 15px or 20px or 25px till you get things right.
(Most Layouts will still look good, after reducing the width by this method.)

Most cases would be solved by now. (If you guys have any other suggestions, please let me know).
Thats all for now, folks !