SEO

Discovery

Integrace robots.txt a sitemap.xml pro technickou dohledatelnost webu.

Overview

Discovery module handles technical discoverability endpoints (/robots.txt, /sitemap.xml) for the application. It is opt-in: routes are exposed only when enabled in app configuration.

Minimal configuration

Enable Discovery in app config and define explicit sitemap inputs in app/Config/Discovery.yaml. Routes are named routes without implicit crawling. Providers are used for dynamic content. YAML remains only the application input layer; runtime still uses typed config objects.

example.php
module: discovery
config:
  robots:
    enabled: true
  sitemap:
    enabled: true
    base_url:
      $env: APP_BASE_URL
      type: string
      default: http://localhost
    routes:
      - home.index
      - documentation.index
      - contact.index
    providers:
      - App\Discovery\DocumentationSitemapProvider

Routes vs Providers

routes contains explicit named routes. providers contains classes implementing SitemapProviderInterface for dynamic URLs. Framework does not generate sitemap from all routes automatically.

Why explicit input

Not every route is indexable. POST routes do not belong in sitemap. Demo/download/technical endpoints are usually not intended for indexing. Dynamic routes like /documentation/{path:any} should be emitted by a provider from the same source used by the documentation module.

Route-based example

Example for explicitly listed public GET pages in sitemap configuration:

example.php
module: discovery
config:
  sitemap:
    routes:
      - home.index
      - typography.index
      - examples.index
      - documentation.index
      - contact.index

Provider-based example

Example provider for dynamic pages generated from application data source:

example.php
final class DocumentationSitemapProvider implements SitemapProviderInterface
{
    public function __construct(
        private readonly DocumentationCatalogInterface $documentation,
    ) {}

    public function urls(): iterable
    {
        foreach ($this->documentation->pages() as $page) {
            $path = trim($page->path(), '/');
            if ($path === '') {
                continue;
            }

            yield SitemapUrl::create(
                loc: '/documentation/' . $path,
                changefreq: 'weekly',
                priority: 0.7,
            );
        }
    }
}

CLI generation

For cache mode or pre-generation use CLI command:

example.php
php framework/bin/lemonade discovery:sitemap:generate

Stream mode vs Cache mode

Use stream mode for small or medium sites when sitemap is generated on request. Use cache mode for larger sites and generate files via CLI. XML is written incrementally through XmlStreamWriter, so generation does not require building full XML in memory.