Only certain roles can access the admin menu, there is nothing you can do to change that and you cannot currently add custom permissions in site settings.
Instead you could make your own custom page for your own custom menu and you can implement your own logic about roles that can access it. you could create your own custom link to your custom admin page and implement your own logic to show and hide the link according to allowed roles that you define. You could embed the link to your custom admin menu in the layout.master page similar as we have for the built in admin menu.