SEO

Discovery

Integrace robots.txt a sitemap.xml pro technickou dohledatelnost webu.

Overview

Discovery module handles technical discoverability endpoints (/robots.txt, /sitemap.xml) for the application. It is opt-in: routes are exposed only when enabled in app configuration.

Minimal configuration

Enable Discovery in app config and define explicit sitemap inputs. Routes are named routes without implicit crawling. Providers are used for dynamic content.

return [
    'discovery' => [
        'robots' => [
            'enabled' => true,
        ],
        'sitemap' => [
            'enabled' => true,
            'base_url' => Env::string('APP_BASE_URL', 'http://localhost'),
            'routes' => [
                'home.index',
                'documentation.index',
                'contact.index',
            ],
            'providers' => [
                \App\Discovery\DocumentationSitemapProvider::class,
            ],
        ],
    ],
];

Routes vs Providers

routes contains explicit named routes. providers contains classes implementing SitemapProviderInterface for dynamic URLs. Framework does not generate sitemap from all routes automatically.

Why explicit input

Not every route is indexable. POST routes do not belong in sitemap. Demo/download/technical endpoints are usually not intended for indexing. Dynamic routes like /documentation/{path:any} should be emitted by a provider from the same source used by the documentation module.

Route-based example

Example for explicitly listed public GET pages in sitemap configuration:

'sitemap' => [
    'routes' => [
        'home.index',
        'typography.index',
        'examples.index',
        'documentation.index',
        'contact.index',
    ],
]

Provider-based example

Example provider for dynamic pages generated from application data source:

final class DocumentationSitemapProvider implements SitemapProviderInterface
{
    public function __construct(
        private readonly DocumentationCatalogInterface $documentation,
    ) {}

    public function urls(): iterable
    {
        foreach ($this->documentation->pages() as $page) {
            $path = trim($page->path(), '/');
            if ($path === '') {
                continue;
            }

            yield SitemapUrl::create(
                loc: '/documentation/' . $path,
                changefreq: 'weekly',
                priority: 0.7,
            );
        }
    }
}

CLI generation

For cache mode or pre-generation use CLI command:

php framework/bin/lemonade discovery:sitemap:generate

Stream mode vs Cache mode

Use stream mode for small or medium sites when sitemap is generated on request. Use cache mode for larger sites and generate files via CLI. XML is written incrementally through XmlStreamWriter, so generation does not require building full XML in memory.