Commit Graph

797 Commits

Author SHA1 Message Date
Bjørn Erik Pedersen ca4025405c Optimize memory allocations for sites matrix vector stores
By

* Caching common sites matrix setups (e.g. the single site in single site setups).
* Adding a fast path to IntSets.HasAnyVector for the common case of single vector input.

```
AssembleDeepSiteWithManySections/depth=3/sectionsPerLevel=2/pagesPerSection=100-10   31.62m ± 46%   30.68m ± 42%  ~ (p=0.310 n=6)

                                                                                   │ master.bench │          perfcommon.bench          │
                                                                                   │     B/op     │     B/op      vs base              │
AssembleDeepSiteWithManySections/depth=3/sectionsPerLevel=2/pagesPerSection=100-10   31.98Mi ± 0%   31.24Mi ± 0%  -2.30% (p=0.002 n=6)

                                                                                   │ master.bench │         perfcommon.bench          │
                                                                                   │  allocs/op   │  allocs/op   vs base              │
AssembleDeepSiteWithManySections/depth=3/sectionsPerLevel=2/pagesPerSection=100-10    460.9k ± 0%   419.9k ± 0%  -8.90% (p=0.002 n=6)
````
2025-11-17 16:01:37 +01:00
Bjørn Erik Pedersen 94a6233aab hugolib: Fix recently introduced data race
This introduces a new thread safe maps.Map type to avoid abusing the exsting maps.Cache for regular map uses.

Fixes #14140
2025-11-15 21:17:42 +01:00
Bjørn Erik Pedersen 26f31ff6ce hugolib: Improve performance of content trees with many sections
Hugo's build process is roughly divided into three steps:

1. Process content (walk file system and insert source nodes into content tree)
2. Assemble content (assemble pages and resources according to sites matrix)
3. Render content

In #13679 we consolidated the page creation logic into one place (the assemble step). This made it much simpler to reason about, but it lost us some performance esp. in big content trees.

This commit re-introduces parallelization in the first step in the assemble step by handling each top level section in its own goroutine. This gives significant performance improvements for content trees with many sections.

Compared to master:

```
AssembleDeepSiteWithManySections/depth=1/sectionsPerLevel=6/pagesPerSection=100-10    19.26m ± ∞ ¹   14.54m ± ∞ ¹  -24.52% (p=0.029 n=4)
AssembleDeepSiteWithManySections/depth=2/sectionsPerLevel=2/pagesPerSection=100-10    19.74m ± ∞ ¹   16.45m ± ∞ ¹  -16.71% (p=0.029 n=4)
AssembleDeepSiteWithManySections/depth=2/sectionsPerLevel=6/pagesPerSection=100-10   106.18m ± ∞ ¹   71.23m ± ∞ ¹  -32.91% (p=0.029 n=4)
AssembleDeepSiteWithManySections/depth=3/sectionsPerLevel=2/pagesPerSection=100-10    38.85m ± ∞ ¹   30.47m ± ∞ ¹  -21.59% (p=0.029 n=4)
```
2025-11-14 11:24:31 +01:00
Bjørn Erik Pedersen 4c7a78f5ca testing: Revise usage of b.N and b.Loop() in benchmarks 2025-11-07 13:33:39 +01:00
Bjørn Erik Pedersen 91eac9e573 all: Fix some benchmarks broken by modernize
It's not possible to use `b.Loop()` when `b.N` is used to prepare test data.

See 264022a75a
See #14107
2025-11-06 10:43:12 +01:00
Bjørn Erik Pedersen 04650ce778 all: Run modernize -fix ./...
Closes #14107
2025-11-05 21:05:42 +01:00
Bjørn Erik Pedersen 264022a75a Add roles and versions as new dimensions (in addition to language)
See the main issue #13776 for details.

Fixes #519
Fixes #13680
Fixes #13663
Fixes #13776
Fixes #13855
Fixes #13648
Fixes #13996
Fixes #14001
Fixes #14031
Fixes #13818
Fixes #13196
2025-11-05 20:39:25 +01:00
Bjørn Erik Pedersen e9bda21ce9 hreflect: Cache reflect method lookups used in collections.Where and others
We already cached the method index on the struct, but caching the resolved `reflect.Method` itself saves us from having to do another lookup on each call, which is escpeciall import in the hot path used by collections.Where and otherrs:

```bash
                                        │ master.bench │    fix-reflectmethodcache.bench     │
                                        │    sec/op    │    sec/op     vs base               │
WhereSliceOfStructPointersWithMethod-10   592.2µ ± ∞ ¹   390.1µ ± ∞ ¹  -34.14% (p=0.029 n=4)
¹ need >= 6 samples for confidence interval at level 0.95

                                        │  master.bench  │     fix-reflectmethodcache.bench     │
                                        │      B/op      │     B/op       vs base               │
WhereSliceOfStructPointersWithMethod-10   205.14Ki ± ∞ ¹   64.52Ki ± ∞ ¹  -68.55% (p=0.029 n=4)
¹ need >= 6 samples for confidence interval at level 0.95

                                        │ master.bench │    fix-reflectmethodcache.bench     │
                                        │  allocs/op   │  allocs/op    vs base               │
WhereSliceOfStructPointersWithMethod-10   9.003k ± ∞ ¹   4.503k ± ∞ ¹  -49.98% (p=0.029 n=4)
````
2025-10-27 15:45:50 +01:00
Bjørn Erik Pedersen 3893e70510 all: Simplify the reflect usage
* Use helper funcs in hreflect package when possible.
* Use hreflect.ConvertIfPossible to handle conversions when possible.
* Move scratch.go from common/maps to common/hstore to clear cyclic import in the next step.
* Move Indirect to hreflect and reimplementing it and adusting the behavior to preserve struct pointers.
* Adjust evaluateSubElem used by where and others making the struct pointer method case slightly faster.
2025-10-26 17:36:58 +01:00
hugoreleaser b95f7d2552 releaser: Prepare repository for 0.153.0-DEV
[ci skip]
2025-10-24 15:44:59 +00:00
hugoreleaser 6abdacad3f releaser: Bump versions for release of 0.152.2
[ci skip]
2025-10-24 15:31:49 +00:00
hugoreleaser 524b98665b releaser: Prepare repository for 0.153.0-DEV
[ci skip]
2025-10-22 19:24:59 +00:00
hugoreleaser 5869cbddd8 releaser: Bump versions for release of 0.152.1
[ci skip]
2025-10-22 19:10:44 +00:00
Bjørn Erik Pedersen e08278d165 Expand the numeric conversions to template funcs/methods
So the example mentioned in #14079 also works for TOML and JSON front matter.

See #14079
2025-10-22 21:04:39 +02:00
Bjørn Erik Pedersen d4c78885ae Fix it so YAML integer types can be used where Go int types are expected
E.g. in date.AddDate.

In Hugo v0.152.0 we moved to a new YAML library (github.com/goccy/go-yaml) which produces uint64 for unsigned integers.

This unfortunately breaks common constructs like:

  .Date.AddDate 0 0 7

when .Date is a time.Time and the integers are unmarshaled from YAML front matter.

This commit adds code to handle conversion from uint64 (and other int types) to the required int types where possible.

Fixes #14079
2025-10-22 21:04:39 +02:00
Bjørn Erik Pedersen 0579afc3c5 Fix "assignment to entry in nil map" on empty YAML config files
Fixes #14074
2025-10-22 13:40:03 +02:00
hugoreleaser 0ccbc63ab8 releaser: Prepare repository for 0.153.0-DEV
[ci skip]
2025-10-21 16:43:50 +00:00
hugoreleaser 06bc2c16c0 releaser: Bump versions for release of 0.152.0
[ci skip]
2025-10-21 16:29:49 +00:00
Bjørn Erik Pedersen a1307700dd config: Clone language map entries before modifying them
Now, with YAML anchor and alias support, these can point to shared data,
which must not be modified in place.

Fixes #14072
2025-10-21 13:34:15 +02:00
Bjørn Erik Pedersen bd50c9c7e7 Misc YAML adjustments
Closes #14067
2025-10-20 16:05:05 +02:00
Bjørn Erik Pedersen a3d9548469 Replace to gopkg.in/yaml with github.com/goccy/go-yaml (note)
This commit also adds validation to prevent the "Billion Laughs" attack (see https://github.com/goccy/go-yaml/issues/461). The limit of non-scalar aliases to the same node is set to 10,000. See benchmarks below.

```                                                            │        sec/op         │
UnmarshalBillionLaughs/Billion_Laughs_no_validation-10                 125.2µ ± ∞ ¹
UnmarshalBillionLaughs/Billion_Laughs_with_validation-10               655.8µ ± ∞ ¹
UnmarshalBillionLaughs/YAML_Front_Matter_no_validation-10              9.223µ ± ∞ ¹
UnmarshalBillionLaughs/YAML_Front_Matter_with_validation-10            9.443µ ± ∞ ¹
geomean                                                                51.71µ
¹ need >= 6 samples for confidence interval at level 0.95

                                                            │ fix-goyaml-8822.bench │
                                                            │         B/op          │
UnmarshalBillionLaughs/Billion_Laughs_no_validation-10                177.0Ki ± ∞ ¹
UnmarshalBillionLaughs/Billion_Laughs_with_validation-10              177.0Ki ± ∞ ¹
UnmarshalBillionLaughs/YAML_Front_Matter_no_validation-10             11.67Ki ± ∞ ¹
UnmarshalBillionLaughs/YAML_Front_Matter_with_validation-10           11.67Ki ± ∞ ¹
geomean                                                               45.45Ki
¹ need >= 6 samples for confidence interval at level 0.95

                                                            │ fix-goyaml-8822.bench │
                                                            │       allocs/op       │
UnmarshalBillionLaughs/Billion_Laughs_no_validation-10                 3.302k ± ∞ ¹
UnmarshalBillionLaughs/Billion_Laughs_with_validation-10               3.305k ± ∞ ¹
UnmarshalBillionLaughs/YAML_Front_Matter_no_validation-10               253.0 ± ∞ ¹
UnmarshalBillionLaughs/YAML_Front_Matter_with_validation-10             253.0 ± ∞ ¹
````

Fixes #8822
Fixes #13043
Fixes #14053
Fixes ##8427
2025-10-18 13:52:22 +02:00
hugoreleaser b76c50ac18 releaser: Prepare repository for 0.152.0-DEV
[ci skip]
2025-10-15 15:06:42 +00:00
hugoreleaser 1cdd17882c releaser: Bump versions for release of 0.151.1
[ci skip]
2025-10-15 14:51:34 +00:00
hugoreleaser c29897fac0 releaser: Prepare repository for 0.152.0-DEV
[ci skip]
2025-10-02 13:45:36 +00:00
hugoreleaser c70ab27ceb releaser: Bump versions for release of 0.151.0
[ci skip]
2025-10-02 13:30:36 +00:00
Bjørn Erik Pedersen ec463c0977 Report OSC 9;4 progress when building
As supported by the Ghostty terminal and others.
2025-10-01 14:27:23 +02:00
Bjørn Erik Pedersen 4d1303512b common/hreflect: Speed up IsTrutfulValue
By caching the calculation of whether a type implements `IsZero`:

```
IsTruthFulVAlue-10    467.95n ± ∞ ¹   79.13n ± ∞ ¹  -83.09% (p=0.029 n=4)
```
2025-09-29 17:06:50 +02:00
hugoreleaser 18b9b6488b releaser: Prepare repository for 0.151.0-DEV
[ci skip]
2025-09-25 10:45:05 +00:00
hugoreleaser ce44a8e835 releaser: Bump versions for release of 0.150.1
[ci skip]
2025-09-25 10:26:04 +00:00
hugoreleaser 0f18cbe990 releaser: Prepare repository for 0.151.0-DEV
[ci skip]
2025-09-08 13:16:24 +00:00
hugoreleaser 3f5473b7d4 releaser: Bump versions for release of 0.150.0
[ci skip]
2025-09-08 13:01:12 +00:00
hugoreleaser 321a66ef19 releaser: Prepare repository for 0.150.0-DEV
[ci skip]
2025-09-04 14:18:57 +00:00
hugoreleaser 57a784e027 releaser: Bump versions for release of 0.149.1
[ci skip]
2025-09-04 14:04:46 +00:00
Bjørn Erik Pedersen 4f2d2b2cc4 Fix nilpointer on ToC heading
Fixes #11843

Co-authored-by: Joe Mooring <joe.mooring@veriphor.com>
2025-09-04 14:01:51 +02:00
hugoreleaser 70d62993ee releaser: Prepare repository for 0.150.0-DEV
[ci skip]
2025-08-27 15:51:44 +00:00
hugoreleaser 66240338f1 releaser: Bump versions for release of 0.149.0
[ci skip]
2025-08-27 15:37:16 +00:00
Bjørn Erik Pedersen 1ba80874e4 tpl/collections: Add collections.D using Vitter's Method D for sequential random sampling 2025-08-26 20:37:08 +02:00
Bjørn Erik Pedersen 186934feb4 common/hcontext: Replace with external package 2025-08-22 09:13:46 +02:00
hugoreleaser 3aa22b0942 releaser: Prepare repository for 0.149.0-DEV
[ci skip]
2025-07-27 12:59:26 +00:00
hugoreleaser 40c3d8233d releaser: Bump versions for release of 0.148.2
[ci skip]
2025-07-27 12:43:24 +00:00
Bjørn Erik Pedersen 3937ab24d0 Revert "hugolib: Honor implicit "page" type during template selection"
This reverts commit cfc8d315b4.

See #13868
2025-07-27 14:39:39 +02:00
hugoreleaser bbcc2a7973 releaser: Prepare repository for 0.149.0-DEV
[ci skip]
2025-07-11 13:13:44 +00:00
hugoreleaser 98ba786f2f releaser: Bump versions for release of 0.148.1
[ci skip]
2025-07-11 12:56:21 +00:00
hugoreleaser 65893efd8d releaser: Prepare repository for 0.149.0-DEV
[ci skip]
2025-07-08 13:51:56 +00:00
hugoreleaser c0d9bebacc releaser: Bump versions for release of 0.148.0
[ci skip]
2025-07-08 13:34:49 +00:00
Joe Mooring cfc8d315b4 hugolib: Honor implicit "page" type during template selection
Closes #13826
2025-07-02 13:43:40 +02:00
hugoreleaser 762417617c releaser: Prepare repository for 0.148.0-DEV
[ci skip]
2025-06-23 08:38:21 +00:00
hugoreleaser 29bdbde19c releaser: Bump versions for release of 0.147.9
[ci skip]
2025-06-23 08:22:20 +00:00
Joe Mooring 4217fee4b0 common/terminal: Enable color output on windows
Closes #8209
2025-06-09 11:44:13 +02:00
hugoreleaser fad57964aa releaser: Prepare repository for 0.148.0-DEV
[ci skip]
2025-06-07 13:13:56 +00:00