The Wall Street Journal:
We are tweaking the way we correct our mistakes on Twitter. If you’ve sent a tweet that needs to be deleted, either because of a typo, an incorrect link or a more-serious error requiring a correction, the social-media team asks that you take the following steps.
1. Take a screenshot of the incorrect tweet
2. Delete the incorrect tweet
3. Send a new tweet, with the correct language/information and a link to the story you’re sharing, and attach the screenshot of the previous tweet.
Addendum (8/1/2015): Kudos to Slate for a similar policy.