As in content trees with 10 thousand of directories and more.
A benchmark with the bottle neck code in `helpers.ExtractAndGroupRootPaths`:
```
│ cmp20251125.bench │ fix-extractandgrouproot-14211.bench │
│ sec/op │ sec/op vs base │
ExtractAndGroupRootPaths-10 1282818.8µ ± 8% 493.8µ ± 38% -99.96% (p=0.002 n=6)
│ cmp20251125.bench │ fix-extractandgrouproot-14211.bench │
│ B/op │ B/op vs base │
ExtractAndGroupRootPaths-10 3343.8Ki ± 0% 146.3Ki ± 0% -95.63% (p=0.002 n=6)
│ cmp20251125.bench │ fix-extractandgrouproot-14211.bench │
│ allocs/op │ allocs/op vs base │
ExtractAndGroupRootPaths-10 20.043k ± 0% 2.979k ± 0% -85.14% (p=0.002 n=6)
```
For test project that started this (a 60k directory conent tree), the server startup with no rendering, wen from 1.5 minutes to less than 4 seconds:
```
hugop server --renderSegments none main ✚ ✖ ✱ ◼
Watching for changes in /Users/bep/dev/sites/hugotestsites/60k/content/{section0,section1,section10,section100,section101,section102,section103,section104,section105,section106,...}
Watching for changes in /Users/bep/dev/sites/hugotestsites/60k/layouts/_default
Watching for config changes in /Users/bep/dev/sites/hugotestsites/60k/config.toml
Start building sites …
hugo v0.153.0-DEV-7e27c303904ed8b221d6a5a4fc9a764bb7b2935b darwin/arm64 BuildDate=2025-11-25T15:02:58Z
│ EN
──────────────────┼────
Pages │ 0
Paginator pages │ 0
Non-page files │ 0
Static files │ 0
Processed images │ 0
Aliases │ 0
Cleaned │ 0
Built in 3884 ms
Environment: "development"
Serving pages from disk
Running in Fast Render Mode. For full rebuilds on change: hugo server --disableFastRender
Web Server is available at //localhost:1313/ (bind address 127.0.0.1)
Press Ctrl+C to stop
```
Note that the output may be a little different and a little more verbose than before., but the information is correct and this implementation is significantly faster and simpler.
Fixes#14211
New options:
* `FromHeaders`: Server header matching for redirects
* `FromRe`: Regexp with group support, i.e. it replaces $1, $2 in To with the group matches.
Note that if both `From` and `FromRe` is set, both must match.
Also
* Allow redirects to non HTML URLs as long as the Sec-Fetch-Mode is set to navigate on the request.
* Detect and stop redirect loops.
This was all done while testing out InertiaJS with Hugo. So, after this commit, this setup will support the main parts of the protocol that Inertia uses:
```toml
[server]
[[server.headers]]
for = '/**/inertia.json'
[server.headers.values]
Content-Type = 'text/html'
X-Inertia = 'true'
Vary = 'Accept'
[[server.redirects]]
force = true
from = '/**/'
fromRe = "^/(.*)/$"
fromHeaders = { "X-Inertia" = "true" }
status = 301
to = '/$1/inertia.json'
```
Unfortunately, a provider like Netlify does not support redirects matching by request headers. It should be possible with some edge function, but then again, I'm not sure that InertiaJS is a very good fit with the common Hugo use cases.
But this commit should be generally useful.
The failing test case here is
* A custom search output format defined on the home page, marked as `noAlternative` and not `permalinkable`
* In fast render mode, when making a change to a data source for that search output format, the JSON file was not refreshed.
There are variants of the above, but the gist of it is:
* The change set was correctly determined, but since the search JSON file was not in the recently visited browser stack, we skipped rendering it.
Running with `hugo server --disableFastRender` would be a workaround for the above.
This commit fixes this by:
* Adding a check for the HTTP request header `Sec-Fetch-Mode = navigation` to the condition for if we should track server request as a user navigation (and not e.g. a HTTP request for a linked CSS stylesheet).
* Making sure that we compare against the real relative URL for non-permalinkable output formats.
Fixes#13014
Do not offer filenames to arguments not taking one, complete arguments
of options taking resource kinds, directory names, --logLevel, release
--step, config and new --format.
As an internal refactoring, use higher level functions to set flag
completions. SetAnnotation works, but is more verbose than
alternatives, and uses bash specific wording.
While at it, move setting completions next to flag definitions
consistently.
Remove superfluous --destination completer setting, which is already
set elsewhere.
* Add --pprof flag to server to enable profile debugging.
* Don't cache the resource content, it seem to eat memory on bigger sites.
* Keep --printMemoryUsag running in server
Fixes#11974
There are some breaking changes in this commit, see #11455.
Closes#11455Closes#11549
This fixes a set of bugs (see issue list) and it is also paying some technical debt accumulated over the years. We now build with Staticcheck enabled in the CI build.
The performance should be about the same as before for regular sized Hugo sites, but it should perform and scale much better to larger data sets, as objects that uses lots of memory (e.g. rendered Markdown, big JSON files read into maps with transform.Unmarshal etc.) will now get automatically garbage collected if needed. Performance on partial rebuilds when running the server in fast render mode should be the same, but the change detection should be much more accurate.
A list of the notable new features:
* A new dependency tracker that covers (almost) all of Hugo's API and is used to do fine grained partial rebuilds when running the server.
* A new and simpler tree document store which allows fast lookups and prefix-walking in all dimensions (e.g. language) concurrently.
* You can now configure an upper memory limit allowing for much larger data sets and/or running on lower specced PCs.
We have lifted the "no resources in sub folders" restriction for branch bundles (e.g. sections).
Memory Limit
* Hugos will, by default, set aside a quarter of the total system memory, but you can set this via the OS environment variable HUGO_MEMORYLIMIT (in gigabytes). This is backed by a partitioned LRU cache used throughout Hugo. A cache that gets dynamically resized in low memory situations, allowing Go's Garbage Collector to free the memory.
New Dependency Tracker: Hugo has had a rule based coarse grained approach to server rebuilds that has worked mostly pretty well, but there have been some surprises (e.g. stale content). This is now revamped with a new dependency tracker that can quickly calculate the delta given a changed resource (e.g. a content file, template, JS file etc.). This handles transitive relations, e.g. $page -> js.Build -> JS import, or $page1.Content -> render hook -> site.GetPage -> $page2.Title, or $page1.Content -> shortcode -> partial -> site.RegularPages -> $page2.Content -> shortcode ..., and should also handle changes to aggregated values (e.g. site.Lastmod) effectively.
This covers all of Hugo's API with 2 known exceptions (a list that may not be fully exhaustive):
Changes to files loaded with template func os.ReadFile may not be handled correctly. We recommend loading resources with resources.Get
Changes to Hugo objects (e.g. Page) passed in the template context to lang.Translate may not be detected correctly. We recommend having simple i18n templates without too much data context passed in other than simple types such as strings and numbers.
Note that the cachebuster configuration (when A changes then rebuild B) works well with the above, but we recommend that you revise that configuration, as it in most situations should not be needed. One example where it is still needed is with TailwindCSS and using changes to hugo_stats.json to trigger new CSS rebuilds.
Document Store: Previously, a little simplified, we split the document store (where we store pages and resources) in a tree per language. This worked pretty well, but the structure made some operations harder than they needed to be. We have now restructured it into one Radix tree for all languages. Internally the language is considered to be a dimension of that tree, and the tree can be viewed in all dimensions concurrently. This makes some operations re. language simpler (e.g. finding translations is just a slice range), but the idea is that it should also be relatively inexpensive to add more dimensions if needed (e.g. role).
Fixes#10169Fixes#10364Fixes#10482Fixes#10630Fixes#10656Fixes#10694Fixes#10918Fixes#11262Fixes#11439Fixes#11453Fixes#11457Fixes#11466Fixes#11540Fixes#11551Fixes#11556Fixes#11654Fixes#11661Fixes#11663Fixes#11664Fixes#11669Fixes#11671Fixes#11807Fixes#11808Fixes#11809Fixes#11815Fixes#11840Fixes#11853Fixes#11860Fixes#11883Fixes#11904Fixes#7388Fixes#7425Fixes#7436Fixes#7544Fixes#7882Fixes#7960Fixes#8255Fixes#8307Fixes#8863Fixes#8927Fixes#9192Fixes#9324
* commands: Add TLS/HTTPS support to hugo server
The "auto cert" handling in this PR is backed by mkcert (see link below).
To get this up and running on a new PC, you can:
```
hugo server trust
hugo server --tlsAuto
```
When `--tlsAuto` (or `--tlsCertFile` and `--tlsKeyFile`) is set and no `--baseURL` is provided as a flag, the server is
started with TLS and `https` as the protocol.
Note that you only need to run `hugo server trust` once per PC.
If you already have the key and the cert file (e.g. by using mkcert directly), you can do:
```
hugo server --tlsCertFile mycert.pem --tlsKeyFile mykey.pem
```
See https://github.com/FiloSottile/mkcertFixes#11064
Primary motivation is documentation, but it will also hopefully simplify the code.
Also,
* Lower case the default output format names; this is in line with the custom ones (map keys) and how
it's treated all the places. This avoids doing `stringds.EqualFold` everywhere.
Closes#10896Closes#10620
Bjørn Erik Pedersen