From 85e9441ffae5be36c8253ee81fd126dd6e4f4503 Mon Sep 17 00:00:00 2001 From: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Date: Fri, 27 Mar 2026 21:42:57 +0000 Subject: [PATCH 1/6] docs: document sitemap lastmod timestamps for SEO metadata page Co-Authored-By: bot_apk --- .../docs/pages/changelog/2026-03-27.mdx | 9 +++++++ fern/products/docs/pages/seo/metadata.mdx | 27 ++++++++++++++++++- 2 files changed, 35 insertions(+), 1 deletion(-) create mode 100644 fern/products/docs/pages/changelog/2026-03-27.mdx diff --git a/fern/products/docs/pages/changelog/2026-03-27.mdx b/fern/products/docs/pages/changelog/2026-03-27.mdx new file mode 100644 index 000000000..d1be7c8c8 --- /dev/null +++ b/fern/products/docs/pages/changelog/2026-03-27.mdx @@ -0,0 +1,9 @@ +--- +tags: ["seo"] +--- + +## Sitemap `lastmod` timestamps + +Your sitemap includes a `` timestamp on each page entry, reflecting when the page's content last changed. Search engines use this signal to prioritize crawling recently updated pages. Timestamps update automatically on each publish and ignore trivial formatting changes like whitespace or capitalization differences. + + diff --git a/fern/products/docs/pages/seo/metadata.mdx b/fern/products/docs/pages/seo/metadata.mdx index fbac6cf22..3a26d9373 100644 --- a/fern/products/docs/pages/seo/metadata.mdx +++ b/fern/products/docs/pages/seo/metadata.mdx @@ -1,6 +1,6 @@ --- title: Configure SEO metadata -description: Configure SEO metadata in Fern docs with page-level frontmatter and site-wide settings. Control titles, descriptions, and social media previews. +description: Configure SEO metadata in Fern docs with page-level frontmatter and site-wide settings. Control titles, descriptions, social media previews, and sitemap timestamps. max-toc-depth: 3 --- @@ -279,3 +279,28 @@ Use `noindex` for internal-only pages, staging content, or pages you don't want If `true`, search engines won't follow any links on the page. + +## Sitemap + +Fern automatically generates a `sitemap.xml` for your documentation site and includes a `` timestamp on each page entry. The timestamp reflects when the page's content last changed, not when it was last published. This helps search engines prioritize crawling recently updated pages over unchanged ones. + +No configuration is required — `lastmod` timestamps are computed and updated automatically each time you publish your docs. Trivial formatting changes like whitespace or capitalization differences don't trigger a new timestamp. + + +### Implementation details + +The `lastmod` feature uses two internal database tables: `slugs` and `markdowns`. + +- **`slugs`** stores one row per unique URL slug, keyed by `orgId`, `domain`, `basepath`, and `slug`. +- **`markdowns`** stores one row per markdown source file (`pageId`), linked to a slug via foreign key. Multiple markdown files (e.g., changelog entries) can share a single slug. + +On each docs publish (`finishDocsRegister`), the system: +1. Resolves page IDs to URL slugs from the navigation tree using `NodeCollector`. +2. Computes a SHA-256 hash of each page's normalized markdown content. +3. Compares hashes against stored values — only changed or new pages are upserted, and stale entries are cleaned up. +4. Changelog entries map to the parent `ChangelogNode`'s slug since they render on a single page with hash fragments. + +Normalization (`normalizeMarkdownForHashing`) strips whitespace, copyright symbols, and capitalization before hashing to avoid false-positive change detection from trivial formatting differences. + +The sitemap route handler fetches slug entries from FDR via oRPC endpoints (`POST /slugs` and `POST /markdowns`), builds a `slug → lastUpdated` lookup, and adds `` to each sitemap XML entry. If no slug data exists for a domain (e.g., it hasn't been published yet), the sitemap omits `` timestamps gracefully rather than failing. + From 246129a6a5ab984f8623d93e26ef3d07f545a628 Mon Sep 17 00:00:00 2001 From: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Date: Fri, 27 Mar 2026 21:49:08 +0000 Subject: [PATCH 2/6] docs: reference Google best practices, generalize sitemap description, clean up implementation details Co-Authored-By: bot_apk --- .../docs/pages/changelog/2026-03-27.mdx | 4 ++-- fern/products/docs/pages/seo/metadata.mdx | 19 ++++++++----------- 2 files changed, 10 insertions(+), 13 deletions(-) diff --git a/fern/products/docs/pages/changelog/2026-03-27.mdx b/fern/products/docs/pages/changelog/2026-03-27.mdx index d1be7c8c8..9e3c483d4 100644 --- a/fern/products/docs/pages/changelog/2026-03-27.mdx +++ b/fern/products/docs/pages/changelog/2026-03-27.mdx @@ -2,8 +2,8 @@ tags: ["seo"] --- -## Sitemap `lastmod` timestamps +## Sitemap timestamps -Your sitemap includes a `` timestamp on each page entry, reflecting when the page's content last changed. Search engines use this signal to prioritize crawling recently updated pages. Timestamps update automatically on each publish and ignore trivial formatting changes like whitespace or capitalization differences. +Your sitemap entries include a timestamp reflecting when each page's content last changed. Following [Google's sitemap best practices](https://developers.google.com/search/docs/crawling-indexing/sitemaps/build-sitemap), this helps search engines prioritize crawling recently updated pages. Timestamps update automatically on each publish and ignore trivial formatting changes like whitespace or capitalization differences. diff --git a/fern/products/docs/pages/seo/metadata.mdx b/fern/products/docs/pages/seo/metadata.mdx index 3a26d9373..04bdb1a1e 100644 --- a/fern/products/docs/pages/seo/metadata.mdx +++ b/fern/products/docs/pages/seo/metadata.mdx @@ -282,25 +282,22 @@ Use `noindex` for internal-only pages, staging content, or pages you don't want ## Sitemap -Fern automatically generates a `sitemap.xml` for your documentation site and includes a `` timestamp on each page entry. The timestamp reflects when the page's content last changed, not when it was last published. This helps search engines prioritize crawling recently updated pages over unchanged ones. +Fern automatically generates a `sitemap.xml` for your documentation site. Following [Google's sitemap best practices](https://developers.google.com/search/docs/crawling-indexing/sitemaps/build-sitemap), each page entry includes a timestamp reflecting when its content last changed — not when it was last published. This helps search engines prioritize crawling recently updated pages over unchanged ones. -No configuration is required — `lastmod` timestamps are computed and updated automatically each time you publish your docs. Trivial formatting changes like whitespace or capitalization differences don't trigger a new timestamp. +No configuration is required — timestamps are computed and updated automatically each time you publish your docs. Trivial formatting changes like whitespace or capitalization differences don't trigger a new timestamp. ### Implementation details -The `lastmod` feature uses two internal database tables: `slugs` and `markdowns`. +Fern tracks content changes using two internal tables: one for URL slugs and one for markdown source files. Multiple source files (e.g., changelog entries) can map to a single slug. -- **`slugs`** stores one row per unique URL slug, keyed by `orgId`, `domain`, `basepath`, and `slug`. -- **`markdowns`** stores one row per markdown source file (`pageId`), linked to a slug via foreign key. Multiple markdown files (e.g., changelog entries) can share a single slug. - -On each docs publish (`finishDocsRegister`), the system: -1. Resolves page IDs to URL slugs from the navigation tree using `NodeCollector`. +On each publish, the system: +1. Resolves each page to its URL slug from the navigation tree. 2. Computes a SHA-256 hash of each page's normalized markdown content. 3. Compares hashes against stored values — only changed or new pages are upserted, and stale entries are cleaned up. -4. Changelog entries map to the parent `ChangelogNode`'s slug since they render on a single page with hash fragments. +4. Changelog entries map to their parent changelog page's slug since they render on a single page with hash fragments. -Normalization (`normalizeMarkdownForHashing`) strips whitespace, copyright symbols, and capitalization before hashing to avoid false-positive change detection from trivial formatting differences. +Normalization strips whitespace, copyright symbols, and capitalization before hashing to avoid false-positive change detection from trivial formatting differences. -The sitemap route handler fetches slug entries from FDR via oRPC endpoints (`POST /slugs` and `POST /markdowns`), builds a `slug → lastUpdated` lookup, and adds `` to each sitemap XML entry. If no slug data exists for a domain (e.g., it hasn't been published yet), the sitemap omits `` timestamps gracefully rather than failing. +The sitemap route fetches stored slug data, builds a URL-to-timestamp lookup, and adds the timestamp to each sitemap XML entry. If no data exists for a domain yet, the sitemap omits timestamps gracefully rather than failing. From aba4b0b034a685895a6b2bd58288ae7b77d0dae1 Mon Sep 17 00:00:00 2001 From: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Date: Fri, 27 Mar 2026 21:52:16 +0000 Subject: [PATCH 3/6] docs: connect sitemap timestamps to last-updated frontmatter field Co-Authored-By: bot_apk --- fern/products/docs/pages/seo/metadata.mdx | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fern/products/docs/pages/seo/metadata.mdx b/fern/products/docs/pages/seo/metadata.mdx index 04bdb1a1e..f9fd4c6d6 100644 --- a/fern/products/docs/pages/seo/metadata.mdx +++ b/fern/products/docs/pages/seo/metadata.mdx @@ -286,6 +286,8 @@ Fern automatically generates a `sitemap.xml` for your documentation site. Follow No configuration is required — timestamps are computed and updated automatically each time you publish your docs. Trivial formatting changes like whitespace or capitalization differences don't trigger a new timestamp. +Sitemap timestamps are separate from the [`last-updated` frontmatter field](/learn/docs/configuration/page-level-settings#last-updated), which displays a visible date in the page footer for readers. Sitemap timestamps are invisible to readers and used exclusively by search engines. + ### Implementation details From 6fb4548b56df6c39f86a6803b278429ba35fe60b Mon Sep 17 00:00:00 2001 From: Devin Logan Date: Fri, 27 Mar 2026 18:05:19 -0400 Subject: [PATCH 4/6] clarify text --- .../docs/pages/changelog/2026-03-27.mdx | 2 +- fern/products/docs/pages/seo/metadata.mdx | 24 --------------- fern/products/docs/pages/seo/overview.mdx | 30 ++++++++++++++++++- 3 files changed, 30 insertions(+), 26 deletions(-) diff --git a/fern/products/docs/pages/changelog/2026-03-27.mdx b/fern/products/docs/pages/changelog/2026-03-27.mdx index 9e3c483d4..c7e45e49e 100644 --- a/fern/products/docs/pages/changelog/2026-03-27.mdx +++ b/fern/products/docs/pages/changelog/2026-03-27.mdx @@ -6,4 +6,4 @@ tags: ["seo"] Your sitemap entries include a timestamp reflecting when each page's content last changed. Following [Google's sitemap best practices](https://developers.google.com/search/docs/crawling-indexing/sitemaps/build-sitemap), this helps search engines prioritize crawling recently updated pages. Timestamps update automatically on each publish and ignore trivial formatting changes like whitespace or capitalization differences. - + diff --git a/fern/products/docs/pages/seo/metadata.mdx b/fern/products/docs/pages/seo/metadata.mdx index f9fd4c6d6..64ea99a27 100644 --- a/fern/products/docs/pages/seo/metadata.mdx +++ b/fern/products/docs/pages/seo/metadata.mdx @@ -279,27 +279,3 @@ Use `noindex` for internal-only pages, staging content, or pages you don't want If `true`, search engines won't follow any links on the page. - -## Sitemap - -Fern automatically generates a `sitemap.xml` for your documentation site. Following [Google's sitemap best practices](https://developers.google.com/search/docs/crawling-indexing/sitemaps/build-sitemap), each page entry includes a timestamp reflecting when its content last changed — not when it was last published. This helps search engines prioritize crawling recently updated pages over unchanged ones. - -No configuration is required — timestamps are computed and updated automatically each time you publish your docs. Trivial formatting changes like whitespace or capitalization differences don't trigger a new timestamp. - -Sitemap timestamps are separate from the [`last-updated` frontmatter field](/learn/docs/configuration/page-level-settings#last-updated), which displays a visible date in the page footer for readers. Sitemap timestamps are invisible to readers and used exclusively by search engines. - - -### Implementation details - -Fern tracks content changes using two internal tables: one for URL slugs and one for markdown source files. Multiple source files (e.g., changelog entries) can map to a single slug. - -On each publish, the system: -1. Resolves each page to its URL slug from the navigation tree. -2. Computes a SHA-256 hash of each page's normalized markdown content. -3. Compares hashes against stored values — only changed or new pages are upserted, and stale entries are cleaned up. -4. Changelog entries map to their parent changelog page's slug since they render on a single page with hash fragments. - -Normalization strips whitespace, copyright symbols, and capitalization before hashing to avoid false-positive change detection from trivial formatting differences. - -The sitemap route fetches stored slug data, builds a URL-to-timestamp lookup, and adds the timestamp to each sitemap XML entry. If no data exists for a domain yet, the sitemap omits timestamps gracefully rather than failing. - diff --git a/fern/products/docs/pages/seo/overview.mdx b/fern/products/docs/pages/seo/overview.mdx index bd4398783..fec8f0079 100644 --- a/fern/products/docs/pages/seo/overview.mdx +++ b/fern/products/docs/pages/seo/overview.mdx @@ -5,7 +5,35 @@ description: Understand Fern's built-in features for search engine optimization Fern optimizes your documentation for both traditional search engines and AI-powered tools out of the box. SEO ensures your pages rank well in Google, Bing, and other search engines, while GEO (Generative Engine Optimization) ensures AI tools like ChatGPT, Claude, and Cursor can efficiently consume and reference your content. -Out of the box, Fern generates meta tags, social previews, canonical URLs, and clean slugs for every page — including AI-optimized content via [`llms.txt`](/learn/docs/ai-features/llms-txt). When you want more control, you can customize: +## What Fern handles automatically + +Without any configuration, Fern generates meta tags, social previews, canonical URLs, clean slugs, and AI-optimized content via [`llms.txt`](/learn/docs/ai-features/llms-txt) for every page. + +Fern also generates a `sitemap.xml` for your documentation site that follows [Google's sitemap best practices](https://developers.google.com/search/docs/crawling-indexing/sitemaps/build-sitemap). Each page entry includes a `` timestamp that updates only when the page's content actually changes. Trivial formatting differences like whitespace or capitalization are ignored. These timestamps are invisible to readers and used exclusively by search engines to prioritize crawling recently updated pages. + + + Sitemap timestamps are separate from the [`last-updated` frontmatter field](/learn/docs/configuration/page-level-settings#last-updated), which displays a visible date in the page footer for readers. + + + +### Sitemap implementation details + +Fern tracks content changes using two internal tables: one for URL slugs and one for markdown source files. Multiple source files (e.g., changelog entries) can map to a single slug. + +On each publish, the system: +1. Resolves each page to its URL slug from the navigation tree. +2. Computes a SHA-256 hash of each page's normalized markdown content. +3. Compares hashes against stored values — only changed or new pages are upserted, and stale entries are cleaned up. +4. Changelog entries map to their parent changelog page's slug since they render on a single page with hash fragments. + +Normalization strips whitespace, copyright symbols, and capitalization before hashing to avoid false-positive change detection from trivial formatting differences. + +The sitemap route fetches stored slug data, builds a URL-to-timestamp lookup, and adds the timestamp to each sitemap XML entry. If no data exists for a domain yet, the sitemap omits timestamps gracefully rather than failing. + + +## Customize your SEO + +When you want more control, you can configure: From f2c84aa1bad4047427d6a30aa6b7fa407596cbec Mon Sep 17 00:00:00 2001 From: Devin Logan Date: Fri, 27 Mar 2026 18:56:15 -0400 Subject: [PATCH 5/6] update last updated field --- fern/products/docs/pages/customization/frontmatter.mdx | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fern/products/docs/pages/customization/frontmatter.mdx b/fern/products/docs/pages/customization/frontmatter.mdx index fc69aae96..90b3df9db 100644 --- a/fern/products/docs/pages/customization/frontmatter.mdx +++ b/fern/products/docs/pages/customization/frontmatter.mdx @@ -101,6 +101,8 @@ For example, scroll to the top of this page you're visiting now and you'll see t Displays a "Last updated" timestamp in the page footer. Use this to show readers when the content was last modified. The value is displayed as-is, so you can use any date format you prefer. + + This field is separate from the timestamps in your [sitemap](/learn/docs/seo/overview#what-fern-handles-automatically), which are managed automatically and used exclusively by search engines. From 127af26342539345eac96bd56eedb88785a8dac4 Mon Sep 17 00:00:00 2001 From: Devin Logan Date: Fri, 27 Mar 2026 19:00:14 -0400 Subject: [PATCH 6/6] update changelog wording --- fern/products/docs/pages/changelog/2026-03-27.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fern/products/docs/pages/changelog/2026-03-27.mdx b/fern/products/docs/pages/changelog/2026-03-27.mdx index c7e45e49e..72fb1feef 100644 --- a/fern/products/docs/pages/changelog/2026-03-27.mdx +++ b/fern/products/docs/pages/changelog/2026-03-27.mdx @@ -4,6 +4,6 @@ tags: ["seo"] ## Sitemap timestamps -Your sitemap entries include a timestamp reflecting when each page's content last changed. Following [Google's sitemap best practices](https://developers.google.com/search/docs/crawling-indexing/sitemaps/build-sitemap), this helps search engines prioritize crawling recently updated pages. Timestamps update automatically on each publish and ignore trivial formatting changes like whitespace or capitalization differences. +Your sitemap entries now include a `` timestamp that's invisible to readers and used exclusively by search engines to prioritize crawling recently updated pages. Timestamps update only when a page's content actually changes — trivial formatting differences like whitespace or capitalization are ignored. This follows [Google's sitemap best practices](https://developers.google.com/search/docs/crawling-indexing/sitemaps/build-sitemap).