Feat/pagefind integrate pages config#2880
Conversation
Previously, because we used the glob within pagefind's configuration option which controlled how pagefind discover's HTML files to index, it would only index the files that we declared within the glob. But since we now insert data-pagefind-body into the pages that we want to index, technically pagefind will index all the pages. However, it still seems to index pages that are not searchable, its just that nothing on these pages will be searchable, hence creating sort of a pseudo not indexed page. Currently, the fix is just to take the count of the number of addressable pages that are searchable.
No longer needed since we adding the glob pattern into the pagefind's index dictionary.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #2880 +/- ##
==========================================
- Coverage 71.40% 71.28% -0.12%
==========================================
Files 132 132
Lines 7231 7203 -28
Branches 1712 1611 -101
==========================================
- Hits 5163 5135 -28
Misses 1967 1967
Partials 101 101 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
Hello my friends, its basically 3 am now. Why am I awake you ask? I wonder too. |
|
😁
|
There was a problem hiding this comment.
Pull request overview
Integrates Pagefind page scoping with the existing pages[].searchable configuration by marking searchable pages in the rendered HTML, and removes the previous Pagefind glob-based configuration path.
Changes:
- Remove
pagefind.globconfiguration support and related glob-handling logic/tests. - Inject
data-pagefind-bodyinto rendered pages based onpages[].searchable. - Update documentation and functional test snapshots to reflect the new Pagefind indexing approach.
Reviewed changes
Copilot reviewed 117 out of 117 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/core/test/unit/Site/SiteGenerationManager.test.ts | Updates unit tests for Pagefind indexing behavior and logging; removes glob normalization tests. |
| packages/core/src/Site/SiteGenerationManager.ts | Removes glob-based indexing logic; indexes output directory and computes “searchable count” from addressablePages. |
| packages/core/src/Site/SiteConfig.ts | Removes pagefind.glob from the typed config surface. |
| packages/core/src/Page/page.njk | Adds conditional data-pagefind-body attribute on <body> for searchable pages. |
| packages/core/src/Page/index.ts | Passes pageConfig.searchable into the page template data. |
| packages/cli/test/functional/test_site/expected/testWeb3FormPlugin.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testVariableContainsInclude.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testTree.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testTooltipSpacing.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testThumbnails.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testSourceContainScript.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testSingleAltFrontMatter.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testPopoverTrigger.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testPopovers.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testPlantUML.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testPanelsClosingTransition.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testPanels.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testPanelMarkdownParsing.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testPageNavWithoutTitleAndNavHeadings.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testPageNavWithOnlyTitle.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testPageNavTarget.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testPageNavPrint.html | Updates snapshot <body> to include data-pagefind-body alongside scrollspy attrs. |
| packages/cli/test/functional/test_site/expected/testPageNav.html | Updates snapshot <body> to include data-pagefind-body alongside scrollspy attrs. |
| packages/cli/test/functional/test_site/expected/testOcticonInPage.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testNunjucksPathResolving.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testModals.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testMermaid.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testMath.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testMaterialIconsInPage.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testList.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testLinks.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testLayoutsWithAltFrontMatter.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testLayoutsOverrideWithAltFrontmatter.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testLayoutsOverride.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testLayouts.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testIncludePluginsRendered.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testIncludeMultipleModals.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testIncludeBoilerplate.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testImages.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testIconsInSiteLayout.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testHr.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testGlyphiconInPage.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testFontAwesomeInPage.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testExternalScripts.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testEmptyFrontmatter.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testEmptyAltFrontMatter.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testDates.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testCodeBlocks.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testCenterText.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testBootstrapIconInPage.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testAntiFOUCStyles.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testAnnotate.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testAnchorGeneration.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testAltFrontMatterParsing.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/testAltFrontMatterInvalidKeyValue.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/test_md_fragment.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/sub_site/testNunjucksPathResolving.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/sub_site/nested_sub_site/testNunjucksPathResolving.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/sub_site/nested_sub_site/index.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/sub_site/index.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site/expected/index.html | Updates snapshot <body> to include data-pagefind-body alongside scrollspy attrs. |
| packages/cli/test/functional/test_site/expected/bugs/index.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site_templates/test_project/expected/userGuide/UserGuide.html | Updates template snapshot <body> to include data-pagefind-body alongside scrollspy attrs. |
| packages/cli/test/functional/test_site_templates/test_project/expected/userGuide/QuickStart.html | Updates template snapshot <body> to include data-pagefind-body alongside scrollspy attrs. |
| packages/cli/test/functional/test_site_templates/test_project/expected/userGuide/Features.html | Updates template snapshot <body> to include data-pagefind-body alongside scrollspy attrs. |
| packages/cli/test/functional/test_site_templates/test_project/expected/userGuide/FAQ.html | Updates template snapshot <body> to include data-pagefind-body alongside scrollspy attrs. |
| packages/cli/test/functional/test_site_templates/test_project/expected/team/johndoe.html | Updates template snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site_templates/test_project/expected/team/AboutUs.html | Updates template snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site_templates/test_project/expected/index.html | Updates template snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site_templates/test_project/expected/developerGuide/TracingCode.html | Updates template snapshot <body> to include data-pagefind-body alongside scrollspy attrs. |
| packages/cli/test/functional/test_site_templates/test_project/expected/developerGuide/Testing.html | Updates template snapshot <body> to include data-pagefind-body alongside scrollspy attrs. |
| packages/cli/test/functional/test_site_templates/test_project/expected/developerGuide/SettingUp.html | Updates template snapshot <body> to include data-pagefind-body alongside scrollspy attrs. |
| packages/cli/test/functional/test_site_templates/test_project/expected/developerGuide/Requirements.html | Updates template snapshot <body> to include data-pagefind-body alongside scrollspy attrs. |
| packages/cli/test/functional/test_site_templates/test_project/expected/developerGuide/Implementation.html | Updates template snapshot <body> to include data-pagefind-body alongside scrollspy attrs. |
| packages/cli/test/functional/test_site_templates/test_project/expected/developerGuide/Documentation.html | Updates template snapshot <body> to include data-pagefind-body alongside scrollspy attrs. |
| packages/cli/test/functional/test_site_templates/test_project/expected/developerGuide/DevOps.html | Updates template snapshot <body> to include data-pagefind-body alongside scrollspy attrs. |
| packages/cli/test/functional/test_site_templates/test_project/expected/developerGuide/DeveloperGuide.html | Updates template snapshot <body> to include data-pagefind-body alongside scrollspy attrs. |
| packages/cli/test/functional/test_site_templates/test_project/expected/developerGuide/Design.html | Updates template snapshot <body> to include data-pagefind-body alongside scrollspy attrs. |
| packages/cli/test/functional/test_site_templates/test_project/expected/developerGuide/Configuration.html | Updates template snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site_templates/test_portfolio/expected/index.html | Updates template snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site_templates/test_minimal/expected/index.html | Updates template snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site_templates/test_default/expected/index.html | Updates template snapshot <body> to include data-pagefind-body alongside scrollspy attrs. |
| packages/cli/test/functional/test_site_templates/test_default/expected/contents/topic3b.html | Updates template snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site_templates/test_default/expected/contents/topic3a.html | Updates template snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site_templates/test_default/expected/contents/topic2.html | Updates template snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site_templates/test_default/expected/contents/topic1.html | Updates template snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site_templates/test_default/expected/404.html | Updates template snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site_table_plugin/expected/index.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site_special_tags/expected/index.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site_custom_plugins/expected/index.html | Updates snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site_convert/test_navigation_convert/expected/test_folder/extra_3.html | Updates conversion snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site_convert/test_navigation_convert/expected/test_folder/extra_2.html | Updates conversion snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site_convert/test_navigation_convert/expected/test_folder/extra_1.html | Updates conversion snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site_convert/test_navigation_convert/expected/README.html | Updates conversion snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site_convert/test_navigation_convert/expected/Page-1.html | Updates conversion snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site_convert/test_navigation_convert/expected/index.html | Updates conversion snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site_convert/test_navigation_convert/expected/Home.html | Updates conversion snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site_convert/test_navigation_convert/expected/contents/topic3b.html | Updates conversion snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site_convert/test_navigation_convert/expected/contents/topic3a.html | Updates conversion snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site_convert/test_navigation_convert/expected/contents/topic2.html | Updates conversion snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site_convert/test_navigation_convert/expected/contents/topic1.html | Updates conversion snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site_convert/test_navigation_convert/expected/about.html | Updates conversion snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site_convert/test_navigation_convert/expected/404.html | Updates conversion snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site_convert/test_basic_convert/expected/Page-1.html | Updates conversion snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site_convert/test_basic_convert/expected/index.html | Updates conversion snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site_convert/test_basic_convert/expected/Home.html | Updates conversion snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site_convert/test_basic_convert/expected/contents/topic3b.html | Updates conversion snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site_convert/test_basic_convert/expected/contents/topic3a.html | Updates conversion snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site_convert/test_basic_convert/expected/contents/topic2.html | Updates conversion snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site_convert/test_basic_convert/expected/contents/topic1.html | Updates conversion snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site_convert/test_basic_convert/expected/about.html | Updates conversion snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site_convert/test_basic_convert/expected/404.html | Updates conversion snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site_convert/test_basic_convert/expected/_Sidebar.html | Updates conversion snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site_convert/test_basic_convert/expected/_Footer.html | Updates conversion snapshot <body> to include data-pagefind-body. |
| packages/cli/test/functional/test_site_algolia_plugin/expected/index.html | Updates snapshot <body> to include data-pagefind-body. |
| docs/userGuide/siteJsonFile.md | Updates pages[].searchable documentation to include Pagefind indexing behavior. |
| docs/userGuide/makingTheSiteSearchable.md | Updates Pagefind docs to use pages[].searchable rather than pagefind.glob. |
Comments suppressed due to low confidence (1)
packages/core/src/Site/SiteGenerationManager.ts:910
- The log uses
totalPageCountcomputed fromaddressablePages, but ignoresresult.page_countreturned by Pagefind. This can produce misleading output if Pagefind indexes additional HTML files (e.g., 404 pages, plugin-generated pages) or fails to index some pages; consider logging both counts (indexed vs configured-searchable) or usingresult.page_countfor the "indexed" value.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Docs now clearly document the interactions of searchable with pagefind's native attributes
What is the purpose of this pull request?
Overview of changes:
closes #2874
Updated pagefind glob-based exclusion to align with existing src/glob patterns used within pages option.
Anything you'd like to highlight/discuss:
Instead of adding the glob pattern to pagefind to configure how it html files, instead, we inject
data-pagefind-bodyinto that are searchable through page.njk.Hence, since we are no longer adding a glob pattern into pagefind's index dictionary, there is no longer a need for all the old glob helper methods for processing and validation, so they have all been removed.
Testing instructions:
Yall can try it out by adding some pages within your site to be no searchable :)
Proposed commit message: (wrap lines at 72 characters)
Update pagefind to align with src/glob in pages config options
Checklist: ☑️
Reviewer checklist:
Indicate the SEMVER impact of the PR:
At the end of the review, please label the PR with the appropriate label:
r.Major,r.Minor,r.Patch.Breaking change release note preparation (if applicable):