0d8478cff8
- In Go 1.21 the crypto/sha256 [got a massive
improvement](https://go.dev/doc/go1.21#crypto/sha256) by utilizing the
SHA instructions for AMD64 CPUs, which sha256-simd already was doing.
The performance is now on par and I think it's preferable to use the
standard library rather than a package when possible.
```
cpu: AMD Ryzen 5 3600X 6-Core Processor
│ simd.txt │ go.txt │
│ sec/op │ sec/op vs base │
Hash/8Bytes-12 63.25n ± 1% 73.38n ± 1% +16.02% (p=0.002 n=6)
Hash/64Bytes-12 98.73n ± 1% 105.30n ± 1% +6.65% (p=0.002 n=6)
Hash/1K-12 567.2n ± 1% 572.8n ± 1% +0.99% (p=0.002 n=6)
Hash/8K-12 4.062µ ± 1% 4.062µ ± 1% ~ (p=0.396 n=6)
Hash/1M-12 512.1µ ± 0% 510.6µ ± 1% ~ (p=0.485 n=6)
Hash/5M-12 2.556m ± 1% 2.564m ± 0% ~ (p=0.093 n=6)
Hash/10M-12 5.112m ± 0% 5.127m ± 0% ~ (p=0.093 n=6)
geomean 13.82µ 14.27µ +3.28%
│ simd.txt │ go.txt │
│ B/s │ B/s vs base │
Hash/8Bytes-12 120.6Mi ± 1% 104.0Mi ± 1% -13.81% (p=0.002 n=6)
Hash/64Bytes-12 618.2Mi ± 1% 579.8Mi ± 1% -6.22% (p=0.002 n=6)
Hash/1K-12 1.682Gi ± 1% 1.665Gi ± 1% -0.98% (p=0.002 n=6)
Hash/8K-12 1.878Gi ± 1% 1.878Gi ± 1% ~ (p=0.310 n=6)
Hash/1M-12 1.907Gi ± 0% 1.913Gi ± 1% ~ (p=0.485 n=6)
Hash/5M-12 1.911Gi ± 1% 1.904Gi ± 0% ~ (p=0.093 n=6)
Hash/10M-12 1.910Gi ± 0% 1.905Gi ± 0% ~ (p=0.093 n=6)
geomean 1.066Gi 1.032Gi -3.18%
```
(cherry picked from commit abd94ff5b59c86e793fd9bf12187ea6cfd1f3fa1)
(cherry picked from commit 15e81637abf70576a564cf9eecaa9640228afb5b)
Conflicts:
go.mod
https://codeberg.org/forgejo/forgejo/pulls/1581
(cherry picked from commit 5caea2d75aeac78fb306f58a3cf7809d5b70c7f2)
(cherry picked from commit 08da542cce2c1571cedd4183268a903ab581d2e3)
(cherry picked from commit d71a8cc9fb816a3b6562a661286f1d3961821b67)
(cherry picked from commit 63c9fc2bee5b71e6ce3898bbf9b9bce827705acc)
(cherry picked from commit e1db85d48a2de7cff0d438aac81023c4b50cdae4)
(cherry picked from commit
|
||
|---|---|---|
| .. | ||
| alpine | ||
| cargo | ||
| chef | ||
| composer | ||
| conan | ||
| conda | ||
| container | ||
| cran | ||
| debian | ||
| generic | ||
| goproxy | ||
| helm | ||
| helper | ||
| maven | ||
| npm | ||
| nuget | ||
| pub | ||
| pypi | ||
| rpm | ||
| rubygems | ||
| swift | ||
| vagrant | ||
| api.go | ||
| README.md | ||
Gitea Package Registry
This document gives a brief overview how the package registry is organized in code.
Structure
The package registry code is divided into multiple modules to split the functionality and make code reuse possible.
| Module | Description |
|---|---|
models/packages |
Common methods and models used by all registry types |
models/packages/<type> |
Methods used by specific registry type. There should be no need to use type specific models. |
modules/packages |
Common methods and types used by multiple registry types |
modules/packages/<type> |
Registry type specific methods and types (e.g. metadata extraction of package files) |
routers/api/packages |
Route definitions for all registry types |
routers/api/packages/<type> |
Route implementation for a specific registry type |
services/packages |
Helper methods used by registry types to handle common tasks like package creation and deletion in routers |
services/packages/<type> |
Registry type specific methods used by routers and services |
Models
Every package registry implementation uses the same underlaying models:
| Model | Description |
|---|---|
Package |
The root of a package providing values fixed for every version (e.g. the package name) |
PackageVersion |
A version of a package containing metadata (e.g. the package description) |
PackageFile |
A file of a package describing its content (e.g. file name) |
PackageBlob |
The content of a file (may be shared by multiple files) |
PackageProperty |
Additional properties attached to Package, PackageVersion or PackageFile (e.g. used if metadata is needed for routing) |
The following diagram shows the relationship between the models:
Package <1---*> PackageVersion <1---*> PackageFile <*---1> PackageBlob
Adding a new package registry type
Before adding a new package registry type have a look at the existing implementation to get an impression of how it could work.
Most registry types offer endpoints to retrieve the metadata, upload and download package files.
The upload endpoint is often the heavy part because it must validate the uploaded blob, extract metadata and create the models.
The methods to validate and extract the metadata should be added in the modules/packages/<type> package.
If the upload is valid the methods in services/packages allow to store the upload and create the corresponding models.
It depends if the registry type allows multiple files per package version which method should be called:
CreatePackageAndAddFile: error if package version already existsCreatePackageOrAddFileToExisting: error if file already existsAddFileToExistingPackage: error if package version does not exist or file already exists
services/packages also contains helper methods to download a file or to remove a package version.
There are no helper methods for metadata endpoints because they are very type specific.