Fix#31423
(cherry picked from commit f4b8f6fc40ce2869135372a5c6ec6418d27ebfba)
Conflicts:
models/fixtures/comment.yml
comment fixtures have to be shifted because there is one more in Forgejo
Multiple chunks are uploaded with type "block" without using
"appendBlock" and eventually out of order for bigger uploads.
8MB seems to be the chunk size
This change parses the blockList uploaded after all blocks to get the
final artifact size and order them correctly before calculating the
sha256 checksum over all blocks
Fixes#31354
(cherry picked from commit b594cec2bda6f861effedb2e8e0a7ebba191c0e9)
Conflicts:
routers/api/actions/artifactsv4.go
conflict because of Refactor AppURL usage (#30885) 67c1a07285008cc00036a87cef966c3bd519a50c
that was not cherry-picked in Forgejo
the resolution consist of removing the extra ctx argument
- [x] add architecture-specific removal support
- [x] Fix upload competition
- [x] Fix not checking input when downloading
docs: https://codeberg.org/forgejo/docs/pulls/874
### Release notes
- [ ] I do not want this change to show in the release notes.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/5351
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Exploding Dragon <explodingfkl@gmail.com>
Co-committed-by: Exploding Dragon <explodingfkl@gmail.com>
(cherry picked from commit 89742c4913)
A 500 status code was thrown when passing a non-existent target to the
create release API. This snapshot handles this error and instead throws
a 404 status code.
Discovered while working on #31840.
(cherry picked from commit f05d9c98c4cb95e3a8a71bf3e2f8f4529e09f96f)
- The Conan and Container packages use a different type of
authentication. It first authenticates via the regular way (api tokens
or user:password, handled via `auth.Basic`) and then generates a JWT
token that is used by the package software (such as Docker) to do the
action they wanted to do. This JWT token didn't properly propagate the
API scopes that the token was generated for, and thus could lead to a
'scope escalation' within the Conan and Container packages, read
access to write access.
- Store the API scope in the JWT token, so it can be propagated on
subsequent calls that uses that JWT token.
- Integration test added.
- Resolves#5128
- Follow up of #4819
- When no `ssh` executable is present, disable the UI and backend bits
that allow the creation of push mirrors that use SSH authentication. As
this feature requires the usage of the `ssh` binary.
- Integration test added.
It loads the Commit with a temporary open GitRepo. This is incorrect,
the GitRepo should be open as long as the Commit can be used. This
mainly removes the usage of this function as it's not needed.
- Continuation of https://github.com/go-gitea/gitea/pull/18835 (by
@Gusted, so it's fine to change copyright holder to Forgejo).
- Add the option to use SSH for push mirrors, this would allow for the
deploy keys feature to be used and not require tokens to be used which
cannot be limited to a specific repository. The private key is stored
encrypted (via the `keying` module) on the database and NEVER given to
the user, to avoid accidental exposure and misuse.
- CAVEAT: This does require the `ssh` binary to be present, which may
not be available in containerized environments, this could be solved by
adding a SSH client into forgejo itself and use the forgejo binary as
SSH command, but should be done in another PR.
- CAVEAT: Mirroring of LFS content is not supported, this would require
the previous stated problem to be solved due to LFS authentication (an
attempt was made at forgejo/forgejo#2544).
- Integration test added.
- Resolves#4416
This reverts commit 4ed372af13.
This change from Gitea was not considered by the Forgejo UI team and there is a consensus that it feels like a regression.
The test which was added in that commit is kept and modified to test that reviews can successfully be submitted on closed and merged PRs.
Closesforgejo/design#11
ForkRepository performs two different functions:
* The fork itself, if it does not already exist
* Updates and notifications after the fork is performed
The function is split to reflect that and otherwise unmodified.
The two function are given different names to:
* clarify which integration tests provides coverage
* distinguish it from the notification method by the same name
Previous arch package grouping was not well-suited for complex or multi-architecture environments. It now supports the following content:
- Support grouping by any path.
- New support for packages in `xz` format.
- Fix clean up rules
<!--start release-notes-assistant-->
## Draft release notes
<!--URL:https://codeberg.org/forgejo/forgejo-->
- Features
- [PR](https://codeberg.org/forgejo/forgejo/pulls/4903): <!--number 4903 --><!--line 0 --><!--description c3VwcG9ydCBncm91cGluZyBieSBhbnkgcGF0aCBmb3IgYXJjaCBwYWNrYWdl-->support grouping by any path for arch package<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4903
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Exploding Dragon <explodingfkl@gmail.com>
Co-committed-by: Exploding Dragon <explodingfkl@gmail.com>
Now that my colleague just posted a wonderful blog post https://blog.datalad.org/posts/forgejo-runner-podman-deployment/ on forgejo runner, some time I will try to add that damn codespell action to work on CI here ;) meanwhile some typos managed to sneak in and this PR should address them (one change might be functional in a test -- not sure if would cause a fail or not)
### Release notes
- [ ] I do not want this change to show in the release notes.
- [ ] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4857
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Yaroslav Halchenko <debian@onerussian.com>
Co-committed-by: Yaroslav Halchenko <debian@onerussian.com>
An instance-wide actor is required for outgoing signed requests that are
done on behalf of the instance, rather than on behalf of other actors.
Such things include updating profile information, or fetching public
keys.
Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
Fix#31707.
It's split from #31724.
Although #31724 could also fix#31707, it has change a lot so it's not a
good idea to backport it.
(cherry picked from commit 81fa471119a6733d257f63f8c2c1f4acc583d21b)
The previous commit laid out the foundation of the quota engine, this
one builds on top of it, and implements the actual enforcement.
Enforcement happens at the route decoration level, whenever possible. In
case of the API, when over quota, a 413 error is returned, with an
appropriate JSON payload. In case of web routes, a 413 HTML page is
rendered with similar information.
This implementation is for a **soft quota**: quota usage is checked
before an operation is to be performed, and the operation is *only*
denied if the user is already over quota. This makes it possible to go
over quota, but has the significant advantage of being practically
implementable within the current Forgejo architecture.
The goal of enforcement is to deny actions that can make the user go
over quota, and allow the rest. As such, deleting things should - in
almost all cases - be possible. A prime exemption is deleting files via
the web ui: that creates a new commit, which in turn increases repo
size, thus, is denied if the user is over quota.
Limitations
-----------
Because we generally work at a route decorator level, and rarely
look *into* the operation itself, `size:repos:public` and
`size:repos:private` are not enforced at this level, the engine enforces
against `size:repos:all`. This will be improved in the future.
AGit does not play very well with this system, because AGit PRs count
toward the repo they're opened against, while in the GitHub-style fork +
pull model, it counts against the fork. This too, can be improved in the
future.
There's very little done on the UI side to guard against going over
quota. What this patch implements, is enforcement, not prevention. The
UI will still let you *try* operations that *will* result in a denial.
Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
This is an implementation of a quota engine, and the API routes to
manage its settings. This does *not* contain any enforcement code: this
is just the bedrock, the engine itself.
The goal of the engine is to be flexible and future proof: to be nimble
enough to build on it further, without having to rewrite large parts of
it.
It might feel a little more complicated than necessary, because the goal
was to be able to support scenarios only very few Forgejo instances
need, scenarios the vast majority of mostly smaller instances simply do
not care about. The goal is to support both big and small, and for that,
we need a solid, flexible foundation.
There are thee big parts to the engine: counting quota use, setting
limits, and evaluating whether the usage is within the limits. Sounds
simple on paper, less so in practice!
Quota counting
==============
Quota is counted based on repo ownership, whenever possible, because
repo owners are in ultimate control over the resources they use: they
can delete repos, attachments, everything, even if they don't *own*
those themselves. They can clean up, and will always have the permission
and access required to do so. Would we count quota based on the owning
user, that could lead to situations where a user is unable to free up
space, because they uploaded a big attachment to a repo that has been
taken private since. It's both more fair, and much safer to count quota
against repo owners.
This means that if user A uploads an attachment to an issue opened
against organization O, that will count towards the quota of
organization O, rather than user A.
One's quota usage stats can be queried using the `/user/quota` API
endpoint. To figure out what's eating into it, the
`/user/repos?order_by=size`, `/user/quota/attachments`,
`/user/quota/artifacts`, and `/user/quota/packages` endpoints should be
consulted. There's also `/user/quota/check?subject=<...>` to check
whether the signed-in user is within a particular quota limit.
Quotas are counted based on sizes stored in the database.
Setting quota limits
====================
There are different "subjects" one can limit usage for. At this time,
only size-based limits are implemented, which are:
- `size:all`: As the name would imply, the total size of everything
Forgejo tracks.
- `size:repos:all`: The total size of all repositories (not including
LFS).
- `size:repos:public`: The total size of all public repositories (not
including LFS).
- `size:repos:private`: The total size of all private repositories (not
including LFS).
- `sizeall`: The total size of all git data (including all
repositories, and LFS).
- `sizelfs`: The size of all git LFS data (either in private or
public repos).
- `size:assets:all`: The size of all assets tracked by Forgejo.
- `size:assets:attachments:all`: The size of all kinds of attachments
tracked by Forgejo.
- `size:assets:attachments:issues`: Size of all attachments attached to
issues, including issue comments.
- `size:assets:attachments:releases`: Size of all attachments attached
to releases. This does *not* include automatically generated archives.
- `size:assets:artifacts`: Size of all Action artifacts.
- `size:assets:packages:all`: Size of all Packages.
- `size:wiki`: Wiki size
Wiki size is currently not tracked, and the engine will always deem it
within quota.
These subjects are built into Rules, which set a limit on *all* subjects
within a rule. Thus, we can create a rule that says: "1Gb limit on all
release assets, all packages, and git LFS, combined". For a rule to
stand, the total sum of all subjects must be below the rule's limit.
Rules are in turn collected into groups. A group is just a name, and a
list of rules. For a group to stand, all of its rules must stand. Thus,
if we have a group with two rules, one that sets a combined 1Gb limit on
release assets, all packages, and git LFS, and another rule that sets a
256Mb limit on packages, if the user has 512Mb of packages, the group
will not stand, because the second rule deems it over quota. Similarly,
if the user has only 128Mb of packages, but 900Mb of release assets, the
group will not stand, because the combined size of packages and release
assets is over the 1Gb limit of the first rule.
Groups themselves are collected into Group Lists. A group list stands
when *any* of the groups within stand. This allows an administrator to
set conservative defaults, but then place select users into additional
groups that increase some aspect of their limits.
To top it off, it is possible to set the default quota groups a user
belongs to in `app.ini`. If there's no explicit assignment, the engine
will use the default groups. This makes it possible to avoid having to
assign each and every user a list of quota groups, and only those need
to be explicitly assigned who need a different set of groups than the
defaults.
If a user has any quota groups assigned to them, the default list will
not be considered for them.
The management APIs
===================
This commit contains the engine itself, its unit tests, and the quota
management APIs. It does not contain any enforcement.
The APIs are documented in-code, and in the swagger docs, and the
integration tests can serve as an example on how to use them.
Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
Add an optional `order_by` parameter to the `user.ListMyRepos`
handler (which handles the `/api/v1/user/repos` route), allowing a user
to sort repos by name (the default), id, or size.
The latter will be useful later for figuring out which repos use most
space, which repos eat most into a user's quota.
Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
Document return type for the endpoints that fetch specific files from a
repository. This allows the swagger generated code to read the returned
data.
Co-authored-by: Giteabot <teabot@gitea.io>
(cherry picked from commit bae87dfb0958e6a2920c905e51c2a026b7b71ca6)
The PATCH if issue & pull request switched to use the service
functions instead. However, the service function changing the state is
not idempotent. Instead of doing nothing which changing from open to
open or close to close, it will fail with an error like:
Issue [2472] 0 was already closed
Regression of: 6a4bc0289d
Fixes: https://codeberg.org/forgejo/forgejo/issues/4686
The `/repos/{owner}/{repo}/issues/comments/{id}` API endpoint returns an
`assets` field, but the route handler did not load attachments, thus,
the field was never populated.
This patch fixes that, and adds a test to exercise it. The test fails
without the fix.
This addresses a bug discovered in Codeberg/Community#1607.
Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
I noticed that Forgejo does not allow HTTP range requests when downloading artifacts. All other file downloads like releases and packages support them.
So I looked at the code and found that the artifact download endpoint uses a simple io.Copy to serve the file contents instead of using the established `ServeContentByReadSeeker` function which does take range requests into account.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4218
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: ThetaDev <thetadev@magenta.de>
Co-committed-by: ThetaDev <thetadev@magenta.de>
Closes#2797
I'm aware of https://github.com/go-gitea/gitea/pull/28163 exists, but since I had it laying around on my drive and collecting dust, I might as well open a PR for it if anyone wants the feature a bit sooner than waiting for upstream to release it or to be a forgejo "native" implementation.
This PR Contains:
- Support for the `workflow_dispatch` trigger
- Inputs: boolean, string, number, choice
Things still to be done:
- [x] API Endpoint `/api/v1/<org>/<repo>/actions/workflows/<workflow id>/dispatches`
- ~~Fixing some UI bugs I had no time figuring out, like why dropdown/choice inputs's menu's behave weirdly~~ Unrelated visual bug with dropdowns inside dropdowns
- [x] Fix bug where opening the branch selection submits the form
- [x] Limit on inputs to render/process
Things not in this PR:
- Inputs: environment (First need support for environments in forgejo)
Things needed to test this:
- A patch for https://code.forgejo.org/forgejo/runner to actually consider the inputs inside the workflow.
~~One possible patch can be seen here: https://code.forgejo.org/Mai-Lapyst/runner/src/branch/support-workflow-inputs~~
[PR](https://code.forgejo.org/forgejo/runner/pulls/199)
## Testing
- Checkout PR
- Setup new development runner with [this PR](https://code.forgejo.org/forgejo/runner/pulls/199)
- Create a repo with a workflow (see below)
- Go to the actions tab, select the workflow and see the notice as in the screenshot above
- Use the button + dropdown to run the workflow
- Try also running it via the api using the `` endpoint
- ...
- Profit!
<details>
<summary>Example workflow</summary>
```yaml
on:
workflow_dispatch:
inputs:
logLevel:
description: 'Log Level'
required: true
default: 'warning'
type: choice
options:
- info
- warning
- debug
tags:
description: 'Test scenario tags'
required: false
type: boolean
boolean_default_true:
description: 'Test scenario tags'
required: true
type: boolean
default: true
boolean_default_false:
description: 'Test scenario tags'
required: false
type: boolean
default: false
number1_default:
description: 'Number w. default'
default: '100'
type: number
number2:
description: 'Number w/o. default'
type: number
string1_default:
description: 'String w. default'
default: 'Hello world'
type: string
string2:
description: 'String w/o. default'
required: true
type: string
jobs:
test:
runs-on: docker
steps:
- uses: actions/checkout@v3
- run: whoami
- run: cat /etc/issue
- run: uname -a
- run: date
- run: echo ${{ inputs.logLevel }}
- run: echo ${{ inputs.tags }}
- env:
GITHUB_CONTEXT: ${{ toJson(github) }}
run: echo "$GITHUB_CONTEXT"
- run: echo "abc"
```
</details>
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/3334
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Mai-Lapyst <mai-lapyst@noreply.codeberg.org>
Co-committed-by: Mai-Lapyst <mai-lapyst@noreply.codeberg.org>
Add tag protection manage via rest API.
---------
Co-authored-by: Alexander Kogay <kogay.a@citilink.ru>
Co-authored-by: Giteabot <teabot@gitea.io>
(cherry picked from commit d4e4226c3cbfa62a6adf15f4466747468eb208c7)
Conflicts:
modules/structs/repo_tag.go
trivial context conflict
templates/swagger/v1_json.tmpl
fixed with make generate-swagger
have repo OrderBy definitions defined in one place and use a single type
for OrderBy database options
(cherry picked from commit bb04311b0b5b7a28f94c4bc409db1c4a04bcef17)
When using the MinIO storage driver for Actions Artifacts, we found that
the chunked artifact required significantly more memory usage to both
upload and merge than the local storage driver. This seems to be related
to hardcoding a value of `-1` for the size to the MinIO client [which
has a warning about memory usage in the respective
docs](https://pkg.go.dev/github.com/minio/minio-go/v7#Client.PutObject).
Specifying the size in both the upload and merge case reduces memory
usage of the MinIO client.
Co-authored-by: Kyle D <kdumontnu@gmail.com>
(cherry picked from commit 45dbeb5600d1f552c0134721fe49e8fd1099b5a4)
Enable [unparam](https://github.com/mvdan/unparam) linter.
Often I could not tell the intention why param is unused, so I put
`//nolint` for those cases like webhook request creation functions never
using `ctx`.
---------
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: delvh <dev.lh@web.de>
(cherry picked from commit fc2d75f86d77b022ece848acf2581c14ef21d43b)
Conflicts:
modules/setting/config_env.go
modules/storage/azureblob.go
services/webhook/dingtalk.go
services/webhook/discord.go
services/webhook/feishu.go
services/webhook/matrix.go
services/webhook/msteams.go
services/webhook/packagist.go
services/webhook/slack.go
services/webhook/telegram.go
services/webhook/wechatwork.go
run make lint-go and fix Forgejo specific warnings
Fixes issue when running `choco info pkgname` where `pkgname` is also a
substring of another package Id.
Relates to #31168
---
This might fix the issue linked, but I'd like to test it with more choco
commands before closing the issue in case I find other problems if
that's ok.
---------
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
(cherry picked from commit c888c933a930ee2ba4e7bb0bf6678aaf45a9778a)
