Commit graph

4252 commits

Author SHA1 Message Date
wxiaoguang
f2a309e6c8
Improve doctor cli behavior (#28422)
1. Do not sort the "checks" slice again and again when "Register", it
just wastes CPU when the Gitea instance runs
2. If a check doesn't exist, tell the end user
3. Add some tests
2023-12-11 15:55:10 +00:00
Lunny Xiao
537fa69962
Second part of refactor db.Find (#28194)
Continue of #27798 and move more functions to `db.Find` and `db.Count`.
2023-12-11 16:56:48 +08:00
Adam Majer
d9c19899cd
Make gogit Repository.GetBranchNames consistent (#28348) 2023-12-07 12:08:17 -05:00
Lunny Xiao
dd30d9d5c0
Remove GetByBean method because sometimes it's danger when query condition parameter is zero and also introduce new generic methods (#28220)
The function `GetByBean` has an obvious defect that when the fields are
empty values, it will be ignored. Then users will get a wrong result
which is possibly used to make a security problem.

To avoid the possibility, this PR removed function `GetByBean` and all
references.
And some new generic functions have been introduced to be used.

The recommand usage like below.

```go
// if query an object according id
obj, err := db.GetByID[Object](ctx, id)
// query with other conditions
obj, err := db.Get[Object](ctx, builder.Eq{"a": a, "b":b})
```
2023-12-07 15:27:36 +08:00
Jason Song
beb71f5ef6
Include public repos in doer's dashboard for issue search (#28304)
It will fix #28268 .

<img width="1313" alt="image"
src="https://github.com/go-gitea/gitea/assets/9418365/cb1e07d5-7a12-4691-a054-8278ba255bfc">

<img width="1318" alt="image"
src="https://github.com/go-gitea/gitea/assets/9418365/4fd60820-97f1-4c2c-a233-d3671a5039e9">

## ⚠️ BREAKING ⚠️

But need to give up some features:

<img width="1312" alt="image"
src="https://github.com/go-gitea/gitea/assets/9418365/281c0d51-0e7d-473f-bbed-216e2f645610">

However, such abandonment may fix #28055 .

## Backgroud

When the user switches the dashboard context to an org, it means they
want to search issues in the repos that belong to the org. However, when
they switch to themselves, it means all repos they can access because
they may have created an issue in a public repo that they don't own.

<img width="286" alt="image"
src="https://github.com/go-gitea/gitea/assets/9418365/182dcd5b-1c20-4725-93af-96e8dfae5b97">

It's a confusing design. Think about this: What does "In your
repositories" mean when the user switches to an org? Repos belong to the
user or the org?

Whatever, it has been broken by #26012 and its following PRs. After the
PR, it searches for issues in repos that the dashboard context user owns
or has been explicitly granted access to, so it causes #28268.

## How to fix it

It's not really difficult to fix it. Just extend the repo scope to
search issues when the dashboard context user is the doer. Since the
user may create issues or be mentioned in any public repo, we can just
set `AllPublic` to true, which is already supported by indexers. The DB
condition will also support it in this PR.

But the real difficulty is how to count the search results grouped by
repos. It's something like "search issues with this keyword and those
filters, and return the total number and the top results. **Then, group
all of them by repo and return the counts of each group.**"

<img width="314" alt="image"
src="https://github.com/go-gitea/gitea/assets/9418365/5206eb20-f8f5-49b9-b45a-1be2fcf679f4">

Before #26012, it was being done in the DB, but it caused the results to
be incomplete (see the description of #26012).

And to keep this, #26012 implement it in an inefficient way, just count
the issues by repo one by one, so it cannot work when `AllPublic` is
true because it's almost impossible to do this for all public repos.


1bfcdeef4c/modules/indexer/issues/indexer.go (L318-L338)

## Give up unnecessary features

We may can resovle `TODO: use "group by" of the indexer engines to
implement it`, I'm sure it can be done with Elasticsearch, but IIRC,
Bleve and Meilisearch don't support "group by".

And the real question is, does it worth it? Why should we need to know
the counts grouped by repos?

Let me show you my search dashboard on gitea.com.

<img width="1304" alt="image"
src="https://github.com/go-gitea/gitea/assets/9418365/2bca2d46-6c71-4de1-94cb-0c9af27c62ff">

I never think the long repo list helps anything.

And if we agree to abandon it, things will be much easier. That is this
PR.

## TODO

I know it's important to filter by repos when searching issues. However,
it shouldn't be the way we have it now. It could be implemented like
this.

<img width="1316" alt="image"
src="https://github.com/go-gitea/gitea/assets/9418365/99ee5f21-cbb5-4dfe-914d-cb796cb79fbe">

The indexers support it well now, but it requires some frontend work,
which I'm not good at. So, I think someone could help do that in another
PR and merge this one to fix the bug first.

Or please block this PR and help to complete it.

Finally, "Switch dashboard context" is also a design that needs
improvement. In my opinion, it can be accomplished by adding filtering
conditions instead of "switching".
2023-12-07 13:26:18 +08:00
wxiaoguang
1e512b800c
Use filepath instead of path to create SQLite3 database file (#28374) 2023-12-06 16:57:52 +00:00
capvor
c1b86ecdab
Fix incorrect default value of [attachment].MAX_SIZE (#28373) 2023-12-06 10:59:56 -05:00
Earl Warren
876a0cb3d6
Render PyPi long description as document (#28272)
Co-authored-by: Gusted <postmaster@gusted.xyz>
2023-12-05 15:02:01 +00:00
darrinsmart
38a93a0665
Convert git commit summary to valid UTF8. (#28356)
The summary string ends up in the database, and (at least) MySQL &
PostgreSQL require valid UTF8 strings.

Fixes #28178

Co-authored-by: Darrin Smart <darrin@filmlight.ltd.uk>
2023-12-05 14:34:24 +08:00
KN4CK3R
14354e4f8e
Read previous info from git blame (#28306)
Fixes #28280

Reads the `previous` info from the `git blame` output instead of
calculating it afterwards.
2023-12-01 01:26:52 +00:00
Brecht Van Lommel
a7de14e493
Meilisearch: require all query terms to be matched (#28293)
Previously only the first term had to be matched. That default
Meilisearch behavior makes sense for e.g. some kind of autocomplete to
find and select a single result. But for filtering issues it means you
can't narrow down results by adding more terms.

This is also more consistent with other indexers and GitHub.

---

Reference:
https://www.meilisearch.com/docs/reference/api/search#matching-strategy
2023-11-29 23:00:59 +08:00
wxiaoguang
b348424c64
Ignore "non-existing" errors when getDirectorySize calculates the size (#28276)
The git command may operate the git directory (add/remove) files in any
time.

So when the code iterates the directory, some files may disappear during
the "walk". All "IsNotExist" errors should be ignored.

Fix #26765
2023-11-29 13:08:58 +08:00
Earl Warren
cb8298b717
Ignore temporary files for directory size (#28265)
Co-authored-by: Gusted <postmaster@gusted.xyz>
2023-11-28 21:53:21 +00:00
pitpalme
eacce14546
Fix delete-orphaned-repos (#28200)
gitea doctor failed at checking and fixing 'delete-orphaned-repos',
because table name 'user' needs quoting to be correctly recognized by at
least PostgreSQL.

fixes #28199
2023-11-24 14:46:19 +00:00
wxiaoguang
2ac2796a4c
Refactor graceful manager to use shared code (#28073)
Make "windows" and "unix" share as much code as possible. No logic
change.
2023-11-24 14:21:46 +00:00
Earl Warren
1075ff74b5
Use restricted sanitizer for repository description (#28141)
- Currently the repository description uses the same sanitizer as a
normal markdown document. This means that element such as heading and
images are allowed and can be abused.
- Create a minimal restricted sanitizer for the repository description,
which only allows what the postprocessor currently allows, which are
links and emojis.
- Added unit testing.
- Resolves https://codeberg.org/forgejo/forgejo/issues/1202
- Resolves https://codeberg.org/Codeberg/Community/issues/1122

(cherry picked from commit 631c87cc23)

Co-authored-by: Gusted <postmaster@gusted.xyz>
2023-11-23 16:34:25 +00:00
wxiaoguang
7c0ab8b974
Make CORS work for oauth2 handlers (#28184)
Fix #25473

Although there was `m.Post("/login/oauth/access_token", CorsHandler()...`,
it never really worked, because it still lacks the "OPTIONS" handler.
2023-11-23 21:19:26 +08:00
yp05327
a6a674e26a
Add guide page to actions when there's no workflows (#28145)
Before:

![image](https://github.com/go-gitea/gitea/assets/18380374/599d40c1-9b8d-4189-9286-c9c36fb780dd)

After:

![image](https://github.com/go-gitea/gitea/assets/18380374/848a73d1-aaec-478f-93a7-adcc7ee18907)
2023-11-21 08:27:33 +00:00
Denys Konovalov
816e46ee7c
add skip ci functionality (#28075)
Adds the possibility to skip workflow execution if the commit message
contains a string like [skip ci] or similar.

The default strings are the same as on GitHub, users can also set custom
ones in app.ini

Reference:
https://docs.github.com/en/actions/managing-workflow-runs/skipping-workflow-runs

Close #28020
2023-11-18 13:37:08 +02:00
Nanguan Lin
f63b116697
Change default size of attachments and repo files (#28100)
https://github.com/go-gitea/gitea/pull/27946 forgets to change them in
code. Sorry about that.
2023-11-17 11:42:00 +00:00
wxiaoguang
17d246cdcc
Fix incorrect pgsql conn builder behavior (#28085)
Fix #28083 and fix the tests
2023-11-17 02:30:57 +00:00
wxiaoguang
79394b340d
Improve graceful manager code/comment (#28063)
The graceful manager has some bugs (#27643, #28062). This is a
preparation for further fixes.
2023-11-15 22:02:46 +08:00
Yarden Shoham
1c0566f66d
Render email addresses as such if followed by punctuation (#27987)
Added the following characters to the regular expression for the email:

- ,
- ;
- ?
- !

Also added a test case.

- Fixes #27616 

# Before

![image](https://github.com/go-gitea/gitea/assets/20454870/c57eac26-f281-43ef-a51d-9c9a81b63efa)

# After

![image](https://github.com/go-gitea/gitea/assets/20454870/fc7d5c08-4350-4af0-a7f0-d1444d2d75af)

Signed-off-by: Yarden Shoham <git@yardenshoham.com>
2023-11-11 05:08:19 +01:00
KN4CK3R
481e738e7f
Remove title from elements on Org mode (#27968)
The Org mode rendering has some problems:
1.  `[[https://example.com][pre https://example.com/example.mp4 post]]`
 renders as
`<p><a href="https://example.com" title="pre <video
src="https://example.com/example.mp4"
title="https://example.com/example.mp4">https://example.com/example.mp4</video>
post">pre <video src="https://example.com/example.mp4"
title="https://example.com/example.mp4">https://example.com/example.mp4</video>
post</a></p>`
As you can see, the `title` attribute contains the inner html in
unescaped form. I removed the `title` attribute because it is of little
value.
3. The `title` attribute on `img` and `video` is of little value.
4. The inner elements of `video` are different depending on the `if`.
2023-11-10 01:45:13 +00:00
6543
16ba16dbe9
Allow to set explore page default sort (#27951)
as title


---
*Sponsored by Kithara Software GmbH*
2023-11-09 10:11:45 +00:00
Lunny Xiao
69d98f83f9
Fix format error (#27963) 2023-11-08 21:50:20 +08:00
KN4CK3R
4f4fea734c
Unify two factor check (#27915)
Fixes #27819

We have support for two factor logins with the normal web login and with
basic auth. For basic auth the two factor check was implemented at three
different places and you need to know that this check is necessary. This
PR moves the check into the basic auth itself.
2023-11-06 08:22:39 +00:00
Earl Warren
59f9ef9fee
Remove action runners on user deletion (#27902)
- On user deletion, delete action runners that the user has created.
- Add a database consistency check to remove action runners that have
nonexistent belonging owner.
- Resolves https://codeberg.org/forgejo/forgejo/issues/1720

(cherry picked from commit 009ca7223d)

Co-authored-by: Gusted <postmaster@gusted.xyz>
2023-11-05 13:48:32 +01:00
Earl Warren
da0c4b8d10
Remove SSH workaround (#27893)
Revert the workaround of #26409 and fix #26411, update github.com/gliderlabs/ssh to include 02f9d57300
2023-11-03 15:21:05 +00:00
Moritz Poldrack
9b6e77c489
refactor postgres connection string building (#27723)
This patchset changes the connection string builder to use net.URL and
the host/port parser to use the stdlib function for splitting host from
port. It also adds a footnote about a potentially required portnumber
for postgres UNIX sockets.

Fixes: #24552
2023-11-01 18:00:20 +00:00
Lunny Xiao
d519a39302
Support storage base path as prefix (#27827)
This PR adds a prefix path for all minio storage and override base path
will override the path.
The previous behavior is undefined officially, so it will be marked as
breaking.
2023-11-01 19:17:18 +08:00
wxiaoguang
a4b242ae7a
Clean up template locale usage (#27856)
After many refactoring PRs for the "locale" and "template context
function", now the ".locale" is not needed for web templates any more.

This PR does a clean up for:

1. Remove `ctx.Data["locale"]` for web context.
2. Use `ctx.Locale` in `500.tmpl`, for consistency.
3. Add a test check for `500 page` locale usage.
4. Remove the `Str2html` and `DotEscape` from mail template context
data, they are copy&paste errors introduced by #19169 and #16200 . These
functions are template functions (provided by the common renderer), but
not template data variables.
5. Make email `SendAsync` function mockable (I was planning to add more
tests but it would make this PR much too complex, so the tests could be
done in another PR)
2023-10-31 22:11:48 +08:00
Lunny Xiao
16d15ce087
Fix package webhook (#27839)
Fix #23742

---------

Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
2023-10-31 04:43:38 +00:00
KN4CK3R
ab3f6c1bac
Close all hashed buffers (#27787)
Add missing `.Close()` calls. The current code does not delete the
temporary files if the data grows over 32mb.
2023-10-25 21:24:24 +02:00
Nanguan Lin
1eae2aadae
Fix issue not showing on default board and add test (#27720)
See https://github.com/go-gitea/gitea/pull/27718#issuecomment-1773743014
. Add a test to ensure its behavior.
Why this test uses `ProjectBoardID=0`? Because in `SearchOptions`,
`ProjectBoardID=0` means what it is. But in `IssueOptions`,
`ProjectBoardID=0` means there is no condition, and
`ProjectBoardID=db.NoConditionID` means the board ID = 0.
It's really confusing. Probably it's better to separate the db search
engine and the other issue search code. It's really two different
systems. As far as I can see, `IssueOptions` is not necessary for most
of the code, which has very simple issue search conditions.
2023-10-25 11:51:49 +00:00
silverwind
ce83609ff6
Upgrade to golangci-lint@v1.55.0 (#27756)
https://github.com/golangci/golangci-lint/releases/tag/v1.55.0
2023-10-24 02:54:59 +00:00
Earl Warren
6352114a77
Fix label render containing invalid HTML (#27752)
- The label HTML contained a quote that wasn't being closed.

Refs: https://codeberg.org/forgejo/forgejo/pulls/1651

(cherry picked from commit e2bc2c9a1f)

Co-authored-by: Gusted <postmaster@gusted.xyz>
2023-10-23 23:02:00 +00:00
6543
b2f828db5e
refactor: make db iterate context aware (#27710)
the iteration will run until finished atm.

this changes it by checking if if the context got canceled before each
run of a loop sequence is executed

[View this pull with now
whitespace](https://github.com/go-gitea/gitea/pull/27710/files?diff=unified&w=1)
2023-10-21 10:05:29 +08:00
Nanguan Lin
eb1478791f
Clean some functions about project issue (#27705)
1. remove unused function `MoveIssueAcrossProjectBoards`
2. extract the project board condition into a function
3. use db.NoCondition instead of -1. (BTW, the usage of db.NoCondition
is too confusing. Is there any way to avoid that?)
4. remove the unnecessary comment since the ctx refactor is completed.
5. Change `b.ID != 0` to `b.ID > 0`. It's more intuitive but I think
they're the same since board ID can't be negative.
2023-10-20 14:01:25 +02:00
6543
e83f2cbbac
Add doctor dbconsistency fix to delete repos with no owner (#27290)
to address #27273
replace #24873
2023-10-19 15:20:52 +02:00
Jason Song
4e98224a45
Support allowed hosts for webhook to work with proxy (#27655)
When `webhook.PROXY_URL` has been set, the old code will check if the
proxy host is in `ALLOWED_HOST_LIST` or reject requests through the
proxy. It requires users to add the proxy host to `ALLOWED_HOST_LIST`.
However, it actually allows all requests to any port on the host, when
the proxy host is probably an internal address.

But things may be even worse. `ALLOWED_HOST_LIST` doesn't really work
when requests are sent to the allowed proxy, and the proxy could forward
them to any hosts.

This PR fixes it by:

- If the proxy has been set, always allow connectioins to the host and
port.
- Check `ALLOWED_HOST_LIST` before forwarding.
2023-10-18 09:44:36 +00:00
Jason Song
1be49fdda6
Improve retrying index issues (#27554)
Fix #27540
2023-10-15 18:56:57 +00:00
Lunny Xiao
cddf245c12
Replace more db.DefaultContext (#27628)
Target #27065
2023-10-15 17:46:06 +02:00
JakobDev
76a85a4ce9
Final round of db.DefaultContext refactor (#27587)
Last part of #27065
2023-10-14 08:37:24 +00:00
KN4CK3R
c6c829fe3f
Enhanced auth token / remember me (#27606)
Closes #27455

> The mechanism responsible for long-term authentication (the 'remember
me' cookie) uses a weak construction technique. It will hash the user's
hashed password and the rands value; it will then call the secure cookie
code, which will encrypt the user's name with the computed hash. If one
were able to dump the database, they could extract those two values to
rebuild that cookie and impersonate a user. That vulnerability exists
from the date the dump was obtained until a user changed their password.
> 
> To fix this security issue, the cookie could be created and verified
using a different technique such as the one explained at
https://paragonie.com/blog/2015/04/secure-authentication-php-with-long-term-persistence#secure-remember-me-cookies.

The PR removes the now obsolete setting `COOKIE_USERNAME`.
2023-10-14 00:56:41 +00:00
Nanguan Lin
dc04044716
Replace assert.Fail with assert.FailNow (#27578)
assert.Fail() will continue to execute the code while assert.FailNow()
not. I thought those uses of assert.Fail() should exit immediately.
PS: perhaps it's a good idea to use
[require](https://pkg.go.dev/github.com/stretchr/testify/require)
somewhere because the assert package's default behavior does not exit
when an error occurs, which makes it difficult to find the root error
reason.
2023-10-11 11:02:24 +00:00
JakobDev
ebe803e514
Penultimate round of db.DefaultContext refactor (#27414)
Part of #27065

---------

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-10-11 04:24:07 +00:00
Jason Song
2c7b6c378e
Increase queue length (#27555) 2023-10-10 18:47:49 +08:00
wxiaoguang
e2e0280108
Fix environment-to-ini inherited key bug (#27543)
Fix  #27541

The INI package has a quirk: by default, the keys are inherited.
When maintaining the keys, the newly added sub key should not be
affected by the parent key.
2023-10-10 01:10:37 +08:00
silverwind
5bf367f904
Restore warning commit status (#27504)
Partial revert of https://github.com/go-gitea/gitea/pull/25839. This
commit status is used by a number of external integrations, so I think
we should not remove it (See
https://github.com/go-gitea/gitea/pull/25839#issuecomment-1729002077).
This is a rare case where an existing migration needed to be alterted to
avoid data loss.

---------

Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: Giteabot <teabot@gitea.io>
2023-10-08 22:16:06 +00:00