I think it is possible. Assuming the site whose users you want to use is siteid 2 and the first site with only an admin user is siteid 1, then what I would do is go into the mp_Sites table and set the IsServerAdminSite field to 1 on site 2 so that it is also a server admin site like site 1 is.
Since site 1 only has 1 user, the admin, you should make usre it has a different loginname and different email than the admin from site 2.
Then set it to use related sites mode and set the RelatedSiteId = 2
In theory it should work, but I admit I have not tested much where the users were coming from a secondary site. I'd be sure and back up your db before trying it. If it gives problems you could run a query to set all users to have siteid 1 and siteguid from that site, then change it to RelatedSiteId = 1, but I don't really expect you to have problems.
Hope it helps,
Joe