Hi Carole,
Yes by default we're trying not to load extra javascript if it isn't needed. Colorbox is used for the file manager link in the toolbar for authenticated users with file upload permission so that control tells script loader to include colorbox, but if you sign out it isn't doing that.
If you are using colorbox links in content you can add this on <portal:ScriptLoader...IncludeColorBox="true" in layout.master to make it always include colorbox. We don't really have a way of detecting its use from the content so if using it in content you'll need that setting to explicitly tell script loader to include it.
Hope that helps,
Joe