To delete a feature you must first delete all instances of the feature.
From the Edit This Page link you can remove a feature from a page but that does not delete the instance, it could be added back to the page from Administration > Content Manager.
There are 2 ways to delete an instance. First by clicking the settings link (assuming you used Module Title control which provides a settings link) from settings there is a delete button to delete the instance. Another way is under Administration > Content Manager, find your instance then click Publish/Delete, then click the delete button at the bottom.
Once all instances are deleted you can delete the feature itself from Administration > Advanced Tools > Feature Installation
Hope that helps,
Joe