The only thing we have that can protect files by role is the Shared Files feature. You could potentially upload the file in a page with Shared Files protected by the same roles as your forum page. Then you could copy and paste the download link from the Shared files into the link in your forum post.
As far as allowed upload extensions, from memory I don't think shared files limits the allowed extensions since it does not store the file on disk with the original extension.
For files uploaded through the editor or file manager it is constrained by settings in Web.config
Hope it helps,
Joe