- I made a mistake when specifying the `FOR` clause for the index hint,
I read it as being an required argument by XORM. The [MariaDB
documention](https://mariadb.com/kb/en/use-index/) tells that it
defaults to the `FOR JOIN` clause hence why I specified `JOIN` (As can
be seen in the previous PR's SQL analyze I didn't specify the `FOR`
clause). However apparently there seems to be some wizardy going on as
we need to tell MariaDB to use this index for the `ORDER BY` clause to
actually force MariaDB to use this index over the `updated_unix` index.
However because it's not actually required by XORM to specify this
value I leave this empty as mariadb is apparently smart enough to figure
out for which type we want to use this index.
- TL;DR make this index hint actually effective for MariaDB.
- Ref: #6146
This is a clean revert of the commits that led to grouping all jobs related to issue labeling into one workflow. The assumption that it would be more efficient was incorrect because it assumed the conditions for running each job would be evaluated statically Forgejo side. In reality the conditions are evaluated by the runner and multiplies the number of runs required instead of decreasing them. In turn, this clutters the status line of each pull request with numerous skipped runs. Finally it is more complex to maintain multiple jobs into a single workflow because the conditions for it to run have to be duplicated.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/6178
Reviewed-by: Michael Kriese <michael.kriese@gmx.de>
Co-authored-by: Earl Warren <contact@earl-warren.org>
Co-committed-by: Earl Warren <contact@earl-warren.org>
### Problem
Big instances can have huge authorized_keys files when using OpenSSH instead of the internal ssh server. Forgejo always re-generates the contents of that file when a user is deleted, even if he does not even have a public key uploaded. In case of codeberg.org, a 15MB file gets rewritten. If we batch delete 100 Spam users without ssh keys, we rewrite 1.5GB, this takes time and wears the SSD. In addition, there is a high chance of hitting a race contidion bug, when deleting users in parallel.
### Solution / Mitigation
This patch prevents rewriting authorized_keys files, when not necessary. It greatly speeds up deleting malicious users, saves IO bandwidth and SSD wear. It also greatly reduces the chance of hitting a race condition bug. Fixing the race condition is not the scope of this patch though.
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [x] I do not want this change to show in the release notes.
- [ ] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
Co-authored-by: Gusted <postmaster@gusted.xyz>
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/6097
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Andreas Shimokawa <shimokawa@fsfe.org>
Co-committed-by: Andreas Shimokawa <shimokawa@fsfe.org>
In the combination of events where the user is viewing an action that is completed (and therefore no interval-based loading of logs is occurring), and `loadJob` is called while `loadJob` is already running (commonly from expanding two log sections back-to-back), the second request for loading would be discarded and never occur.
To fix this, the invocation of `loadJob` keeps track of the log cursors it is attempting to load, and an aborted invocation stores its attempted load. If `loadJob` succeeds in loading and finds that an attempt was aborted while it was busy, it will reload the data.
Steps to reproduce:
- Open an Action UI that is already "Done".
- Click on two steps in rapid succession to expand their logs.
- (Race condition) As long as the second click is registered while the first log chunk is being loaded, its data won't load.
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes... (**not applicable**)
- [ ] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [x] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] I do not want this change to show in the release notes.
- [x] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/6122
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Reviewed-by: Otto <otto@codeberg.org>
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
- A softbreak was being preserved during the github legacy callout (this
is likely due to a change in Goldmark) while it should not. This caused
an `<br>` to be present between the attention title and attention content.
- Added unit test.
- For the notifications page the unread and pinned notifications are
gathered for doer those that and are ordered by the updated unix.
MariaDB makes a bad decision (sometimes, for most users it does not make
this decision) with this query, it uses the index for the `updated_unix`
column to speed up this query, however this is not the correct index to
be taking, if the doer does not have more than 20 (the
page size) unread and pinned notifications combined MariaDB will
traverse the whole notifications table before it realizes that there are
no more notifications to be gathered. It instead should use the index
for the `user_id` column (this is what MariaDB already does for most
users), so the list that has to be traversed is limited to the doer's
notifications which is significantly less than the whole notifications
table.
- This is a different approach than what Gitea has taken to solve this
problem, which is to add a index to the (status, userid, updated_unix)
tuple (Ref: https://github.com/go-gitea/gitea/pull/32395). Adding more
and more indexes is not a good way if we can use existing indexes to get
a query to a acceptable performance.
- The code cannot use `db.Find` as it's hard to add a index hint option
specifically for this query and not for the other instances that uses
`activities_model.FindNotificationOptions`.
- Only add a index hint for MySQL as I have not been able to test if
SQLite or PostgreSQL are smart enough to use the better index (as you
need a large enough dataset to test this meaningfully).
- Integration test added to ensure the SQL is run by all databases.
---
Performance numbers (from Codeberg's database - MariaDB
10.11.6-MariaDB-0+deb12u1):
Currently:
```sql
SELECT * FROM `notification` WHERE notification.user_id=26734 AND (notification.status=3 OR notification.status=1) ORDER BY notification.updated_unix DESC LIMIT 20;
(5.731 sec)
+------+-------------+--------------+-------+--------------------------------------------------+-------------------------------+---------+-------+---------+------------+----------+------------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | r_rows | filtered | r_filtered | Extra |
+------+-------------+--------------+-------+--------------------------------------------------+-------------------------------+---------+-------+---------+------------+----------+------------+-------------+
| 1 | SIMPLE | notification | index | IDX_notification_status,IDX_notification_user_id | IDX_notification_updated_unix | 8 | const | 1376836 | 1474066.00 | 50.03 | 0.00 | Using where |
+------+-------------+--------------+-------+--------------------------------------------------+-------------------------------+---------+-------+---------+------------+----------+------------+-------------+
```
Using the better index:
```sql
SELECT * FROM `notification` USE INDEX (IDX_notification_user_id) WHERE notification.user_id=26734 AND (notification.status=3 OR notification.status=1) ORDER BY notification.updated_unix DESC LIMIT 20;
(0.834 sec)
+------+-------------+--------------+--------+----------------------------------------------------------+--------------------------+---------+----------------------------------+-------+----------+----------+------------+----------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | r_rows | filtered | r_filtered | Extra |
+------+-------------+--------------+--------+----------------------------------------------------------+--------------------------+---------+----------------------------------+-------+----------+----------+------------+----------------------------------------------+
| 1 | PRIMARY | notification | ref | PRIMARY,IDX_notification_status,IDX_notification_user_id | IDX_notification_user_id | 8 | const | 22042 | 10756.00 | 50.03 | 0.02 | Using where; Using temporary; Using filesort |
| 1 | PRIMARY | notification | eq_ref | PRIMARY | PRIMARY | 8 | gitea_production.notification.id | 1 | 1.00 | 100.00 | 100.00 | |
+------+-------------+--------------+--------+----------------------------------------------------------+--------------------------+---------+----------------------------------+-------+----------+----------+------------+----------------------------------------------+
```
- If a organisation is set to be limited visible, then it will still be
visible for signed-in users. However `UnitPermission` didn't take this
into account, it does now.
- Add unit test.
- Resolves#6141
