Google Sitemaps

Google supports sitemaps:

https://www.google.com/webmasters/sitemaps/docs/en/about.html

It's an XML format for listing pages on a site, it should be fairly easy to generate this with petal, it's a simple format, eg:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.google.com/schemas/sitemap/0.84">
 <url>
    <loc>http://www.yoursite.com/</loc>
    <lastmod>2005-01-01</lastmod>
 </url>
 <url>
    <loc>http://www.yoursite.com/foo/</loc>
    <lastmod>2004-12-23</lastmod>
 </url>
</urlset>

https://www.google.com/webmasters/sitemaps/docs/en/protocol.html

We might as well do this as /.sitemap.xml and perhaps a plain text version would make sense:

https://www.google.com/webmasters/sitemaps/docs/en/faq.html#s9

What is a bit odd is that you have to sumbit the sitemap to google, they don't seem to intend to find them themselves:

https://www.google.com/webmasters/sitemaps/docs/en/faq.html#s4

Sign up

Google Sitemaps