Joe Davis said:
If you want to "un-publish" a feature from the current page, click Page Settings, select the particular feature and click the Trash Icon.
but he meant
If you want to "un-publish" a feature from the current page, click Edit This Page, select the particular feature and click the Trash Icon.
This is on PageLayout.aspx. Deleting from there with the trash can icon does not really delete it, it just un-publishes it from the page, whereas deleting it from ModuleSettings.aspx or ContentManager.aspx does really delete it and remove it from all pages.
Best,
Joe