Custom page for SEO Analysts or Auditors looking for Robots[dot]txt  or sitemap in  www.arjunraghunandanan.com


Sitemap

Disclaimers

Dear visitor,

you or your code may have visited www.arjunraghunandanan.com/robots.txt  or www.arjunraghunandanan.com/sitemap.xml  as a part of usual Web Crawling, Search Engine Visibility Auditing or In-Depth SEO testing purposes resulting in a 404 error. 

I regret to inform you that my personal website, which is hosted on Google Sites, is unable to provide a custom `robots.txt` file or a custom sitemap. This is due to limitations of the Google Sites platform, which does not allow the creation of root-level files or the uploading of custom sitemaps. Google does all of it automatically and I choose to accept it and settle with what it has. 

All Websites hosted in Google Sites Platform is subject to Google's Pre-defined robots file setup which is available at sites.google.com/robots.txt  which is  as given below

User-agent: *

Disallow: /feeds

Allow: /*/_/rsrc/

Allow: /_/atari/*

Disallow: /*/_/

If you are not a developer or SEO analyst and you came to this url by accident, read below:

A `robots.txt` file is used to provide instructions to search engine crawlers about which pages or files on a website should not be indexed. Without a custom `robots.txt` file, I am unable to specify which pages or files should be excluded from search engine results.

See example in New York Times Website: www.nytimes.com/robots.txt

For technical details about sitemap, visit Google's Documentation
https://developers.google.com/search/docs/crawling-indexing/robots/intro?sjid=11328847362566027550-AP


A sitemap is a file that provides information to search engines about the pages and files on a website, including when they were last updated and any alternate language versions. Without the ability to upload a custom sitemap, I am unable to provide this information to search engines.

For technical details about sitemap, visit Google's Documentation
https://developers.google.com/search/docs/crawling-indexing/sitemaps/overview


I understand that this may be an inconvenience for some visitors, particularly those in the SEO or web development industry. 

However, I hope that this explanation provides some clarity on the issue. 

Thank you for your understanding.

Best regards,

Arjun Raghunandanan


this page is under www.arjunraghunandanan.com/robots-sitemaps  and is excluded from normal navigation bar.