mirror of
https://code.forgejo.org/forgejo/runner.git
synced 2025-09-05 18:40:59 +00:00
022d5ad3e7
Uses the `Repo` field as an index during searches of the cache database. Removes unused indexes.
To measure the performance of this change, I created a synthetic test which wrote 10,000 records into the artifact cache DB. Of course, all benchmarks are lies that can't be generalized to real-world usage, but it seems clear from the magnitude of improvement that this fixes a flawed implementation, even if it's not perfect.
- Unmodified performance:
- Write: 196 records/second
- Read: 1 record/second
- With `Repo` index being used for reads, and other indexes being removed:
- Write: 347 records/second
- Read: 22,398 records/second
`Repo` is, I think, the only index that made sense to remain, with an eye on workflow run performance:
- `Key` -- can't be used for index because `findCache` searches for key *prefixes*, not equal values.
- `Version` -- isn't very distinct for different workflow runs (https://code.forgejo.org/actions/cache#cache-version)
- `Complete` - significant portion of the cache DB will be complete, making it the least selective possible index
- `UsedAt` & `CreatedAt` - only used in GC operation, so could remain, but this isn't a performance-sensitive codepath
Closes #874.
<!--start release-notes-assistant-->
<!--URL:https://code.forgejo.org/forgejo/runner-->
- bug fixes
- [PR](https://code.forgejo.org/forgejo/runner/pulls/878): <!--number 878 --><!--line 0 --><!--description Zml4OiBhcnRpZmFjdCBjYWNoZSBEQiBub3QgdXNpbmcgaW5kZXhlcyBmb3Igc2VhcmNoaW5n-->fix: artifact cache DB not using indexes for searching<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://code.forgejo.org/forgejo/runner/pulls/878
Reviewed-by: Michael Kriese <michael.kriese@gmx.de>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
35 lines
821 B
Go
35 lines
821 B
Go
package artifactcache
|
|
|
|
type Request struct {
|
|
Key string `json:"key" `
|
|
Version string `json:"version"`
|
|
Size int64 `json:"cacheSize"`
|
|
}
|
|
|
|
func (c *Request) ToCache() *Cache {
|
|
if c == nil {
|
|
return nil
|
|
}
|
|
ret := &Cache{
|
|
Key: c.Key,
|
|
Version: c.Version,
|
|
Size: c.Size,
|
|
}
|
|
if c.Size == 0 {
|
|
// So the request comes from old versions of actions, like `actions/cache@v2`.
|
|
// It doesn't send cache size. Set it to -1 to indicate that.
|
|
ret.Size = -1
|
|
}
|
|
return ret
|
|
}
|
|
|
|
type Cache struct {
|
|
ID uint64 `json:"id" boltholdKey:"ID"`
|
|
Repo string `json:"repo" boltholdIndex:"Repo"`
|
|
Key string `json:"key"`
|
|
Version string `json:"version"`
|
|
Size int64 `json:"cacheSize"`
|
|
Complete bool `json:"complete"`
|
|
UsedAt int64 `json:"usedAt"`
|
|
CreatedAt int64 `json:"createdAt"`
|
|
}
|