Robots.txt Disallow: /SiteMap.aspx

This forum is only for questions or discussions about working with the mojoPortal source code in Visual Studio, obtaining the source code from the repository, developing custom features, etc. If your question is not along these lines this is not the right forum. Please try to post your question in the appropriate forum.

Please do not post questions about design, CSS, or skinning here. Use the Help With Skins Forum for those questions.

This forum is for discussing mojoPortal development

This forum is only for questions or discussions about working with the mojoPortal source code in Visual Studio, obtaining the source code from the repository, developing custom features, etc. If your question is not along these lines this is not the right forum. Please try to post your question in the appropriate forum.

You can monitor commits to the repository from this page. We also recommend developers to subscribe to email notifications in the developer forum as occasionally important things are announced.

Before posting questions here you might want to review the developer documentation.

Do not post questions about design, CSS, or skinning here. Use the Help With Skins Forum for those questions.
This thread is closed to new posts. You must sign in to post in the forums.
2/1/2013 10:21:09 AM
crippsy
Gravatar
Total Posts 167
View Posts

Robots.txt Disallow: /SiteMap.aspx

Hi I have a SEO contractor performing some SEO on my sites.

A couple of the requirements are that Sitemap.XML is missing and also that Robots disallow SiteMap.aspx

Firstly I would suspect that the sitemap.xml is superseded by the aspx version so should not matter, however I was wondering why the Robots.txt disallows the sitemap.aspx as a default. Surely this is something we would want the search engines to find. Would there be a negative impact in allowing this? what was the reasoning behind disallowing this?

Any thoughts

2/1/2013 11:05:25 AM
Joe Audette
Gravatar
Total Posts 18439
View Posts

Re: Robots.txt Disallow: /SiteMap.aspx

Hi,

SiteMap.aspx is for human use not google use. It has no content and it should not be indexed or crawled by google. In fact it could potentially hurt seo if it was crawled it might be seen as a link farm or having too many internal links. So there is no value to letting google crawl it and potential downsides to doing so.

For the correct urls for site maps that are built to be submitted to google see the article Submitting Your Site Maps to Google and Bing

Hope that helps,

Joe

You must sign in to post in the forums. This thread is closed to new posts.