oss/forgejo
oss/forgejo
1
0
Fork 0
mirror of https://codeberg.org/forgejo/forgejo.git synced 2025-08-01 17:38:33 +00:00
Code Wiki Activity

feat: replace go-rpmutils library with our own (#7953)

Browse source 
- While doing cross-compiling of Forgejo (with CGO enabled) it was noticed that besides compiling sqlite3, github.com/DataDog/zstd (a CGO wrapper around the reference zstd library) took a long time to compile. Upon investigating why this library was included in the first place I concluded that this library is not even used and compiling this package is a waste of time and CPU cycles.
- https://github.com/sassoftware/go-rpmutils is the library that uses the CGO zstd library, and would use it the pure Go variant (https://github.com/klauspost/compress/zstd) if CGO is enabled (which is the default). It uses zstd to uncompress the payload of a RPM packages. This is a operation that Forgejo does not use in the slightest, hence being unused code.
- It is not possible to force compiling the pure Go variant if CGO is enabled. Therefore forking and removing this code is the only option to avoid compiling the zstd C library. The changes made to the fork can be seen here: 2660c86d57...v1.0.0
- Via [actiongraph](https://github.com/icio/actiongraph) you can precisely see where the Go compiler is spending time, on a beefy machine the compilation takes of the zstd C library takes ~40s. For reference compiling the sqlite3 C library takes ~50s.
- The forgejo binary (build via `make backend`) reduced its size by 835776 bytes (0.8 MiB).

TL;DR forked library to remove unused code to avoid Go spending a considerable amount of time compiling the reference zstd library that in the end is never used.

<!--start release-notes-assistant-->

## Release notes
<!--URL:https://codeberg.org/forgejo/forgejo-->
- Features
  - [PR](https://codeberg.org/forgejo/forgejo/pulls/7953): <!--number 7953 --><!--line 0 --><!--description cmVwbGFjZSBnby1ycG11dGlscyBsaWJyYXJ5IHdpdGggb3VyIG93bg==-->replace go-rpmutils library with our own<!--description-->
<!--end release-notes-assistant-->

Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/7953
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Gusted <postmaster@gusted.xyz>
Co-committed-by: Gusted <postmaster@gusted.xyz>
This commit is contained in:
Gusted 2025-05-25 12:30:04 +02:00 • committed by Earl Warren
parent 006f488577
commit 62ec469394
6 changed files with 14 additions and 22 deletions
Download patch file Download diff file Expand all files Collapse all files

15
assets/go-licenses.json generated
View file

File diff suppressed because one or more lines are too long

5
go.mod
View file

@ -7,6 +7,7 @@ toolchain go1.24.3
require (
code.forgejo.org/f3/gof3/v3 v3.10.6
code.forgejo.org/forgejo-contrib/go-libravatar v0.0.0-20191008002943-06d1c002b251
code.forgejo.org/forgejo/go-rpmutils v1.0.0
code.forgejo.org/forgejo/levelqueue v1.0.0
code.forgejo.org/forgejo/reply v1.0.2
code.forgejo.org/go-chi/binding v1.0.1
@ -87,7 +88,6 @@ require (
github.com/redis/go-redis/v9 v9.8.0
github.com/robfig/cron/v3 v3.0.1
github.com/santhosh-tekuri/jsonschema/v6 v6.0.1
github.com/sassoftware/go-rpmutils v0.4.0
github.com/sergi/go-diff v1.3.2-0.20230802210424-5b0b94c5c0d3
github.com/shurcooL/vfsgen v0.0.0-20230704071429-0000e147ea92
github.com/stretchr/testify v1.10.0
@ -121,7 +121,6 @@ require (
dario.cat/mergo v1.0.0 // indirect
filippo.io/edwards25519 v1.1.0 // indirect
git.sr.ht/~mariusor/go-xsd-duration v0.0.0-20220703122237-02e73435a078 // indirect
github.com/DataDog/zstd v1.5.5 // indirect
github.com/Microsoft/go-winio v0.6.2 // indirect
github.com/RoaringBitmap/roaring/v2 v2.4.5 // indirect
github.com/andybalholm/brotli v1.1.1 // indirect
@ -152,7 +151,7 @@ require (
github.com/caddyserver/zerossl v0.1.3 // indirect
github.com/cention-sany/utf7 v0.0.0-20170124080048-26cad61bd60a // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/cloudflare/circl v1.6.0 // indirect
github.com/cloudflare/circl v1.6.1 // indirect
github.com/cpuguy83/go-md2man/v2 v2.0.5 // indirect
github.com/cyphar/filepath-securejoin v0.3.6 // indirect
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect

10
go.sum
View file

@ -8,6 +8,8 @@ code.forgejo.org/forgejo/act v1.26.0 h1:6mTmoaw7d/WpYiw/Pw6AaypxFdgJog5OFi/PMEgE
code.forgejo.org/forgejo/act v1.26.0/go.mod h1:HFDFrXPrqfM9aH2RCnMiBdo/3ThxDmZjp58InPjGOfo=
code.forgejo.org/forgejo/archiver/v3 v3.5.1 h1:UmmbA7D5550uf71SQjarmrn6yKwOGxtEjb3jaYYtmSE=
code.forgejo.org/forgejo/archiver/v3 v3.5.1/go.mod h1:e3dqJ7H78uzsRSEACH1joayhuSyhnonssnDhppzS1L4=
code.forgejo.org/forgejo/go-rpmutils v1.0.0 h1:RZGGeKt70p/WaIEL97pyT6uiiEIoN8/aLmS5Z6WmX0M=
code.forgejo.org/forgejo/go-rpmutils v1.0.0/go.mod h1:cg+VbgLXfrDPza9T+kBsMb3TVmmzPN4XseT6gDGLSUk=
code.forgejo.org/forgejo/go-xsd-duration v0.0.0-20220703122237-02e73435a078 h1:RArF5AsF9LH4nEoJxqRxcP5r8hhRfWcId84G82YbqzA=
code.forgejo.org/forgejo/go-xsd-duration v0.0.0-20220703122237-02e73435a078/go.mod h1:g/V2Hjas6Z1UHUp4yIx6bATpNzJ7DYtD0FG3+xARWxs=
code.forgejo.org/forgejo/levelqueue v1.0.0 h1:9krYpU6BM+j/1Ntj6m+VCAIu0UNnne1/UfU/XgPpLuE=
@ -46,8 +48,6 @@ github.com/6543/go-version v1.3.1 h1:HvOp+Telns7HWJ2Xo/05YXQSB2bE0WmVgbHqwMPZT4U
github.com/6543/go-version v1.3.1/go.mod h1:oqFAHCwtLVUTLdhQmVZWYvaHXTdsbB4SY85at64SQEo=
github.com/Azure/go-ntlmssp v0.0.0-20221128193559-754e69321358 h1:mFRzDkZVAjdal+s7s0MwaRv9igoPqLRdzOLzw/8Xvq8=
github.com/Azure/go-ntlmssp v0.0.0-20221128193559-754e69321358/go.mod h1:chxPXzSsl7ZWRAuOIE23GDNzjWuZquvFlgA8xmpunjU=
github.com/DataDog/zstd v1.5.5 h1:oWf5W7GtOLgp6bciQYDmhHHjdhYkALu6S/5Ni9ZgSvQ=
github.com/DataDog/zstd v1.5.5/go.mod h1:g4AWEaM3yOg3HYfnJ3YIawPnVdXJh9QME85blwSAmyw=
github.com/Microsoft/go-winio v0.5.2/go.mod h1:WpS1mjBmmwHBEWmogvA2mj8546UReBk4v8QkMxJ6pZY=
github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY=
github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU=
@ -150,8 +150,8 @@ github.com/chromedp/sysutil v1.0.0/go.mod h1:kgWmDdq8fTzXYcKIBqIYvRRTnYb9aNS9moA
github.com/chzyer/logex v1.2.1/go.mod h1:JLbx6lG2kDbNRFnfkgvh4eRJRPX1QCoOIWomwysCBrQ=
github.com/chzyer/readline v1.5.1/go.mod h1:Eh+b79XXUwfKfcPLepksvw2tcLE/Ct21YObkaSkeBlk=
github.com/chzyer/test v1.0.0/go.mod h1:2JlltgoNkt4TW/z9V/IzDdFaMTM2JPIi26O1pF38GC8=
github.com/cloudflare/circl v1.6.0 h1:cr5JKic4HI+LkINy2lg3W2jF8sHCVTBncJr5gIIq7qk=
github.com/cloudflare/circl v1.6.0/go.mod h1:uddAzsPgqdMAYatqJ0lsjX1oECcQLIlRpzZh3pJrofs=
github.com/cloudflare/circl v1.6.1 h1:zqIqSPIndyBh1bjLVVDHMPpVKqp8Su/V+6MeDzzQBQ0=
github.com/cloudflare/circl v1.6.1/go.mod h1:uddAzsPgqdMAYatqJ0lsjX1oECcQLIlRpzZh3pJrofs=
github.com/cpuguy83/go-md2man/v2 v2.0.5 h1:ZtcqGrnekaHpVLArFSe4HK5DoKx1T0rq2DwVB0alcyc=
github.com/cpuguy83/go-md2man/v2 v2.0.5/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
@ -501,8 +501,6 @@ github.com/russross/blackfriday/v2 v2.1.0 h1:JIOH55/0cWyOuilr9/qlrm0BSXldqnqwMsf
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/santhosh-tekuri/jsonschema/v6 v6.0.1 h1:PKK9DyHxif4LZo+uQSgXNqs0jj5+xZwwfKHgph2lxBw=
github.com/santhosh-tekuri/jsonschema/v6 v6.0.1/go.mod h1:JXeL+ps8p7/KNMjDQk3TCwPpBy0wYklyWTfbkIzdIFU=
github.com/sassoftware/go-rpmutils v0.4.0 h1:ojND82NYBxgwrV+mX1CWsd5QJvvEZTKddtCdFLPWhpg=
github.com/sassoftware/go-rpmutils v0.4.0/go.mod h1:3goNWi7PGAT3/dlql2lv3+MSN5jNYPjT5mVcQcIsYzI=
github.com/serenize/snaker v0.0.0-20171204205717-a683aaf2d516/go.mod h1:Yow6lPLSAXx2ifx470yD/nUe22Dv5vBvxK/UK9UUTVs=
github.com/sergi/go-diff v1.3.2-0.20230802210424-5b0b94c5c0d3 h1:n661drycOFuPLCN3Uc8sB6B/s6Z4t2xvBgU1htSHuq8=
github.com/sergi/go-diff v1.3.2-0.20230802210424-5b0b94c5c0d3/go.mod h1:A0bzQcvG0E7Rwjx0REVgAGH58e96+X0MeOfepqsbeW4=

2
modules/packages/rpm/metadata.go
View file

@ -11,7 +11,7 @@ import (
"forgejo.org/modules/timeutil"
"forgejo.org/modules/validation"
"github.com/sassoftware/go-rpmutils"
"code.forgejo.org/forgejo/go-rpmutils"
)
const (

2
services/packages/rpm/repository.go
View file

@ -26,10 +26,10 @@ import (
"forgejo.org/modules/util"
packages_service "forgejo.org/services/packages"
"code.forgejo.org/forgejo/go-rpmutils"
"github.com/ProtonMail/go-crypto/openpgp"
"github.com/ProtonMail/go-crypto/openpgp/armor"
"github.com/ProtonMail/go-crypto/openpgp/packet"
"github.com/sassoftware/go-rpmutils"
)
// GetOrCreateRepositoryVersion gets or creates the internal repository package

2
tests/integration/api_packages_rpm_test.go
View file

@ -24,8 +24,8 @@ import (
"forgejo.org/modules/util"
"forgejo.org/tests"
"code.forgejo.org/forgejo/go-rpmutils"
"github.com/ProtonMail/go-crypto/openpgp"
"github.com/sassoftware/go-rpmutils"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)