Hi,
The core features are configured using xml config files under the /Setup/applications/mojoportal-core/FeatureDefinitions folder.
These files point to the module controls for various features, most of which are located in the /Modules folder. You can potentially remove all of the core features and add a feature definiton for only custom features that you create. If the feature is already installed best to manually un-install using the site admin first before physically removing the files. If you remove the files before setting up the site (by visiting siteroot/Setup/Default.aspx, then they won't get installed. Configuration of features happens by visiting the /Setup/Default.aspx page. It looks for these files and configures the features.
So for a concrete example if you don't want the blog to get installed you can delete:
/Setup/applications/mojoportal-core/FeatureDefinitions/20000_BlogModule.config
/Modules/BlogModule.ascx
/BlogView.aspx
/BlogArchiveView.aspx
/BlogCategoryView.aspx
/BlogCategoryEdit.aspx
/BlogEdit.aspx
or you can uninstall and remove them if the site is already setup. Similar process for other features. First uninstall using site admin features under Administration > Features, then physically remove the files. Feature specific supporting pages have the feature name as part of the name so if you want to remove forums you can remove the module control and pages named Forum*.aspx
I recommend that you at least keep the HtmlContent module but its not required either. If you do remove it you should also look at the file(s) in the /Setup/initialcontent/pages folder. Config files there are used to create pages during installation. Since features can be configured on pages in these files you should make sure there aren't any that use features you are excluding.
Hope it helps,
Joe