- The current architecture is inherently insecure, because you can
construct the 'secret' cookie value with values that are available in
the database. Thus provides zero protection when a database is
dumped/leaked.
- This patch implements a new architecture that's inspired from: [Paragonie Initiative](https://paragonie.com/blog/2015/04/secure-authentication-php-with-long-term-persistence#secure-remember-me-cookies).
- Integration testing is added to ensure the new mechanism works.
- Removes a setting, because it's not used anymore.
(cherry picked from commit eff097448b1ebd2a280fcdd55d10b1f6081e9ccd)
[GITEA] rework long-term authentication (squash) add migration
Reminder: the migration is run via integration tests as explained
in the commit "[DB] run all Forgejo migrations in integration tests"
(cherry picked from commit 4accf7443c1c59b4d2e7787d6a6c602d725da403)
(cherry picked from commit 99d06e344ebc3b50bafb2ac4473dd95f057d1ddc)
(cherry picked from commit d8bc98a8f021d381bf72790ad246f923ac983ad4)
(cherry picked from commit 6404845df9a63802fff4c5bd6cfe1e390076e7f0)
(cherry picked from commit 72bdd4f3b9f6509d1ff3f10ecb12c621a932ed30)
(cherry picked from commit 4b01bb0ce812b6c59414ff53fed728563d8bc9cc)
Sends email with information on the new user (time of creation and time of last sign-in) and a link to manage the new user from the admin panel
closes: https://codeberg.org/forgejo/forgejo/issues/480
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/1371
Co-authored-by: Aravinth Manivannan <realaravinth@batsense.net>
Co-committed-by: Aravinth Manivannan <realaravinth@batsense.net>
(cherry picked from commit c721aa828ba6aec5ef95459cfc632a0a1f7463e9)
(cherry picked from commit 6487efcb9da61be1f802f1cd8007330153322770)
Conflicts:
modules/notification/base/notifier.go
modules/notification/base/null.go
modules/notification/notification.go
https://codeberg.org/forgejo/forgejo/pulls/1422
(cherry picked from commit 7ea66ee1c5dd21d9e6a43f961e8adc71ec79b806)
Conflicts:
services/notify/notifier.go
services/notify/notify.go
services/notify/null.go
https://codeberg.org/forgejo/forgejo/pulls/1469
(cherry picked from commit 7d2d9970115c94954dacb45684f9e3c16117ebfe)
(cherry picked from commit 435a54f14039408b315c99063bdce28c7ef6fe2f)
(cherry picked from commit 8ec7b3e4484383445fa2622a28bb4f5c990dd4f2)
[GITEA] notifies admins on new user registration (squash) performance bottleneck
Refs: https://codeberg.org/forgejo/forgejo/issues/1479
(cherry picked from commit 97ac9147ff3643cca0a059688c6b3c53479e28a7)
(cherry picked from commit 19f295c16bd392aa438477fa3c42038d63d1a06a)
(cherry picked from commit 3367dcb2cf5328e2afc89f7d5a008b64ede1c987)
[GITEA] notifies admins on new user registration (squash) cosmetic changes
Co-authored-by: delvh <dev.lh@web.de>
(cherry picked from commit 9f1670e040b469ed4346aa2689a75088e4e71c8b)
(cherry picked from commit de5bb2a224ab2ae9be891de1ee88a7454a07f7e9)
(cherry picked from commit 8f8e52f31a4da080465521747a2c5c0c51ed65e3)
(cherry picked from commit e0d51303129fe8763d87ed5f859eeae8f0cc6188)
(cherry picked from commit f1288d6d9bfc9150596cb2f7ddb7300cf7ab6952)
(cherry picked from commit f664f41658e4f2b7948a394822d77f2b9f1d0235)
(cherry picked from commit e44e6c7e47b33aafb97ae61268aebe33453db063)
(cherry picked from commit c0d958cc4c31242fc91a85a4bc2ca6dcc02cebb7)
- Databases are one of the most important parts of Forgejo, every
interaction with Forgejo uses the database in one way or another.
Therefore, it is important to maintain the database and recognize when
Forgejo is not doing well with the database. Forgejo already has the
option to log *every* SQL query along with its execution time, but
monitoring becomes impractical for larger instances and takes up
unnecessary storage in the logs.
- Add a QoL enhancement that allows instance administrators to specify a
threshold value beyond which query execution time is logged as a warning
in the xorm logger. The default value is a conservative five seconds to
avoid this becoming a source of spam in the logs.
- The use case for this patch is that with an instance the size of Codeberg, monitoring SQL logs is not very fruitful and most of them are uninteresting. Recently, in the context of persistent deadlock issues (https://codeberg.org/forgejo/forgejo/issues/220), I have noticed that certain queries hold locks on tables like comment and issue for several seconds. This patch helps to identify which queries these are and when they happen.
- Added unit test.
(cherry picked from commit 24bbe7886fb4cb9a38c8dab8c44f4c9cbfa25481)
(cherry picked from commit 6e29145b3c1455498531593d38e6a914941a12cb)
(cherry picked from commit 63731e30712872bd2395eb3cf36d9996e5793645)
(cherry picked from commit 3ce1a097369c132654de70df707b867e47bd1c40)
(cherry picked from commit a64426907de788cc0937a7a2b16af4d2f26f7fe6)
(cherry picked from commit 4b1921569156445c58d9889602733da5934c7b95)
(cherry picked from commit e6356744359fa947c049827d60c2ea0e277e03dc)
(cherry picked from commit 9cf501f1af4cd870221cef6af489618785b71186)
(cherry picked from commit 0d6b934eba1c0e9b27b364791113aae816b6b366)
(cherry picked from commit 4b6c2738795002887844a106f2fed2ef1673eed1)
(cherry picked from commit b50517139cc62f214c1629ef2fd9bcaa37b46202)
(cherry picked from commit 6546dd1fc946e620a02b6d1afed7d5ac50655fa8)
(cherry picked from commit 3eda6890e6b840237f675a2873c102a6fc86b8f1)
[GITEA] Add slow SQL query warning (squash) document the setting
(cherry picked from commit ce38599c5141c7fc6bc054819f5ff1c1b45bda1f)
(cherry picked from commit 794aa67c68c8e24ac7301eb7ef767c6e2499a78d)
(cherry picked from commit 8227673deb1b93015f56e446b27c52a0013eba29)
(cherry picked from commit 8854d1d4dda72304f43ea8aa61a941701a5deede)
(cherry picked from commit 9121a0e21f7c5de89ba2d8afa9054c5fbf210fb1)
(cherry picked from commit 88d1b53eeaa0d5ad0ed54c191236db928aadedf0)
(cherry picked from commit f63f71afad2aa8272772dba919485c5e858d2fae)
(cherry picked from commit 1f774145fc731e2d93cf8a6708dbf4c15722fa5e)
(cherry picked from commit 53e637693bcbb9fe9daca9a296c9d2c700fc915b)
(cherry picked from commit 4974cbf10a8b51d51591fa6f6281f4cd7a415c91)
(cherry picked from commit 554bca7fae7fa6f0235ab631598115875711d80e)
(cherry picked from commit 61b8cf83b988a2d8547b81cfb7aaeefaf2e17fba)
(cherry picked from commit d8bcc6f68c6dc972e8c99eb7b46a4f096f85cd61)
(cherry picked from commit ee04c340724cccada2c71db45ec510d19e92e099)
(cherry picked from commit 713153a6d63205984f2a85ee07f08269df3b0a85)
(cherry picked from commit 1d51dfa5107b82a7fac43d39deb39e24a85a895e)
(cherry picked from commit ee7ae29abb8380a30c1f467943bb6f6411d6c793)
(cherry picked from commit b20ebd82542113dea064767e5c439ac2ba34d099)
(cherry picked from commit 2e45c9fdc50584f9541384e7f5f4f81f76029f40)
(cherry picked from commit 57e597bf7e1e3bb3b7bcbcea66a4ea170a231f85)
(cherry picked from commit 643a2b0e81570e935779f6c509ebe4633fad74b9)
(cherry picked from commit f10faffb4febeef114d5be4e6abe57bd3cd72894)
(cherry picked from commit b440c5767eebdf406200e9a47446827778514425)
[TESTS] verify facts for the admin storage documentation (squash)
(cherry picked from commit d83d8ce57b8b39b4da849f5403198ecf706117ba)
(cherry picked from commit d8855ef27cd1b219184e95ce055bc6d84350ee26)
(cherry picked from commit 11230466ec0c1c4db1296cdd2ead74fc91a34491)
(cherry picked from commit b2cdd9d971b694fe32bab11f9ccdb41a38d7c6fe)
(cherry picked from commit a0a5e785241ac2c3a7493aa62637351021d48a39)
(cherry picked from commit 846413110fb936c386ba7fa80ff67e4394231464)
(cherry picked from commit 72b92d5a7854f5afbd949e0c6e53f3a4f5b72055)
(cherry picked from commit d51f29b1ba0f01caef966a200ff2946c1b25dd0f)
(cherry picked from commit 58b7ece62a1bc233f73f290782c2e2e7fac732d2)
(cherry picked from commit 751cf469a649b06287c849832b31e46cea3642f1)
(cherry picked from commit 7b2fd5fd17c576a349db659992508f59d686709d)
(cherry picked from commit b7fe7cf401f4bddd6455efc651f7ac054f3fe1cf)
(cherry picked from commit cf339eed4f4851b18448dbdd83df32d00bc6f45b)
(cherry picked from commit 4f3a16168bbeced519a60c32e10e2895d9367238)
(cherry picked from commit 6f5bbc53fcebd614f5ee8627c7d6e3c637ffa694)
(cherry picked from commit aca42b422e76668387769c15868ef77b073bb7db)
(cherry picked from commit 5a7f7580e525694d1f27e12329c8532bd29273f8)
(cherry picked from commit 06c383c807ad49e1e35b429a10c6a11c65aeebe5)
(cherry picked from commit fe831dcb53b81b6cc632be751bfcbfb9bc00efd3)
(cherry picked from commit cd12cd0dbce47c6117ea579ae5019c182155b3a9)
(cherry picked from commit cc79163703ce31706c86b88c38fb8a20ed745e20)
(cherry picked from commit f3ad9d345f7506dbe369567971af73d6c0288e32)
(cherry picked from commit 1a01baaa0f8d8108c64c68263c7c12485af4d296)
(cherry picked from commit 21123afcfd568883774b5e932b20885da28926de)
(cherry picked from commit fdfbda0f9b4ecfa565d9d8cacb718514c99c5d18)
Follow-up to d58c542579 for Forgejo.
By default, Gitea does not select any map service that can be used
to introduce a 'Show this place on a map' button in the location
field of a user profile. Before I tried upstreaming this change to
Gitea, this was the case in Forgejo. This patch essentially recovers
this functionality, which is nice for public-facing instances and
communities.
Links to original PRs:
- https://codeberg.org/forgejo/forgejo/pulls/1076
- https://github.com/go-gitea/gitea/pull/26214
(cherry picked from commit bb187d5f617f8efceb41810d6ff9adcaa60450bb)
(cherry picked from commit ce02ef9078a8731921caa4f7b0c1b0ac3b59a784)
(cherry picked from commit 6b75c40e2575e23810880ee8e368dc1781e2b4e4)
(cherry picked from commit 6bc8e9f5737f5721ddcd1ef5926a778a7f66a4a3)
(cherry picked from commit 063f8afdf7c2cca014c60dcc3d78d270fd236f8f)
(cherry picked from commit c5cc736b72e1ba980519e9258686f41e4eb78b42)
(cherry picked from commit 7b1bb4bedc6196fb6e8516d9a34092b16d7fab05)
(cherry picked from commit 2a022dceb4b391e2fc25b077193ae22a71380eee)
(cherry picked from commit a946c142d2448cb92daaea11851cf7f00921f062)
(cherry picked from commit 8a4ea0c7ab8960f41a67442fa2f485fe0a901a96)
(cherry picked from commit 37bfb05b34e61194afe407acac5778258315b756)
(cherry picked from commit 28fdcd95a5b6266af4e92f20a79b835b6b31417f)
(cherry picked from commit 65743be85df740f6b9275e36a25e216b82dc89e1)
(cherry picked from commit a3d81d07288eeb0f65afd99e14b76af47977ca87)
(cherry picked from commit 4068219ac6eae39be08c1434dd3856a80ccda186)
Backport #27723 by @mpldr
This patchset changes the connection string builder to use net.URL and
the host/port parser to use the stdlib function for splitting host from
port. It also adds a footnote about a potentially required portnumber
for postgres UNIX sockets.
Fixes: #24552
Co-authored-by: Moritz Poldrack <33086936+mpldr@users.noreply.github.com>
Backport #27555 by @wolfogre
It should be OK to increase the default queue length since the default
type is "level".
IMO, the old default length (100) is a little too small. See
https://github.com/go-gitea/gitea/issues/27540#issuecomment-1754269491
IIRC, a larger length could lead to more memory usage only when the type
is "channel," but it's an obscure case. Otherwise, it's just a limit
(for "level" or "redis").
Co-authored-by: Jason Song <i@wolfogre.com>
Backport #27543 by @wxiaoguang
Fix #27541
The INI package has a quirk: by default, the keys are inherited.
When maintaining the keys, the newly added sub key should not be
affected by the parent key.
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Backport #27000 by @wxiaoguang
This PR reduces the complexity of the system setting system.
It only needs one line to introduce a new option, and the option can be
used anywhere out-of-box.
It is still high-performant (and more performant) because the config
values are cached in the config system.
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Backport #27400 by @fantognazza
With this PR we added the possibility to configure the Actions timeouts
values for killing tasks/jobs.
Particularly this enhancement is closely related to the `act_runner`
configuration reported below:
```
# The timeout for a job to be finished.
# Please note that the Gitea instance also has a timeout (3h by default) for the job.
# So the job could be stopped by the Gitea instance if it's timeout is shorter than this.
timeout: 3h
```
---
Setting the corresponding key in the INI configuration file, it is
possible to let jobs run for more than 3 hours.
Signed-off-by: Francesco Antognazza <francesco.antognazza@gmail.com>
If the AppURL(ROOT_URL) is an HTTPS URL, then the COOKIE_SECURE's
default value should be true.
And, if a user visits an "http" site with "https" AppURL, they won't be
able to login, and they should have been warned. The only problem is
that the "language" can't be set either in such case, while I think it
is not a serious problem, and it could be fixed easily if needed.
This feature was removed by #22219 to avoid possible CSRF attack.
This PR takes reverseproxy auth for API back but with default disabled.
To prevent possbile CSRF attack, the responsibility will be the
reverseproxy but not Gitea itself.
For those want to enable this `ENABLE_REVERSE_PROXY_AUTHENTICATION_API`,
they should know what they are doing.
---------
Co-authored-by: Giteabot <teabot@gitea.io>
Currently, Artifact does not have an expiration and automatic cleanup
mechanism, and this feature needs to be added. It contains the following
key points:
- [x] add global artifact retention days option in config file. Default
value is 90 days.
- [x] add cron task to clean up expired artifacts. It should run once a
day.
- [x] support custom retention period from `retention-days: 5` in
`upload-artifact@v3`.
- [x] artifacts link in actions view should be non-clickable text when
expired.
This PR rewrites the function `getStorage` and make it more clear.
Include tests from #26435, thanks @earl-warren
---------
Co-authored-by: Earl Warren <contact@earl-warren.org>
When users put the secrets into a file (GITEA__sec__KEY__FILE), the
newline sometimes is different to avoid (eg: echo/vim/...)
So the last newline could be removed when reading, it makes the users
easier to maintain the secret files.
Co-authored-by: Giteabot <teabot@gitea.io>
This PR is an extended implementation of #25189 and builds upon the
proposal by @hickford in #25653, utilizing some ideas proposed
internally by @wxiaoguang.
Mainly, this PR consists of a mechanism to pre-register OAuth2
applications on startup, which can be enabled or disabled by modifying
the `[oauth2].DEFAULT_APPLICATIONS` parameter in app.ini. The OAuth2
applications registered this way are being marked as "locked" and
neither be deleted nor edited over UI to prevent confusing/unexpected
behavior. Instead, they're being removed if no longer enabled in config.
The implemented mechanism can also be used to pre-register other OAuth2
applications in the future, if wanted.
Co-authored-by: hickford <mirth.hickford@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
---------
Co-authored-by: M Hickford <mirth.hickford@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
This PR will fix#26264, caused by #23911.
The package configuration derive is totally wrong when storage type is
local in that PR.
This PR fixed the inherit logic when storage type is local with some
unit tests.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
1. Fix the wrong document (add the missing `MODE=`)
2. Add a more friendly log message to tell users to add `MODE=` in their
config
Co-authored-by: Giteabot <teabot@gitea.io>
Not too important, but I think that it'd be a pretty neat touch.
Also fixes some layout bugs introduced by a previous PR.
---------
Co-authored-by: Gusted <postmaster@gusted.xyz>
Co-authored-by: Caesar Schinas <caesar@caesarschinas.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
This PR includes #26007 's changes but have a UI to prompt administrator
about the deprecated settings as well as the log or console warning.
Then users will have enough time to notice the problem and don't have
surprise like before.
<img width="1293" alt="图片"
src="https://github.com/go-gitea/gitea/assets/81045/c33355f0-1ea7-4fb3-ad43-cd23cd15391d">
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Replace #25892
Close #21942
Close #25464
Major changes:
1. Serve "robots.txt" and ".well-known/security.txt" in the "public"
custom path
* All files in "public/.well-known" can be served, just like
"public/assets"
3. Add a test for ".well-known/security.txt"
4. Simplify the "FileHandlerFunc" logic, now the paths are consistent so
the code can be simpler
5. Add CORS header for ".well-known" endpoints
6. Add logs to tell users they should move some of their legacy custom
public files
```
2023/07/19 13:00:37 cmd/web.go:178:serveInstalled() [E] Found legacy public asset "img" in CustomPath. Please move it to /work/gitea/custom/public/assets/img
2023/07/19 13:00:37 cmd/web.go:182:serveInstalled() [E] Found legacy public asset "robots.txt" in CustomPath. Please move it to /work/gitea/custom/public/robots.txt
```
This PR is not breaking.
---------
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Giteabot <teabot@gitea.io>
Update WorkPath/WORK_PATH related documents, remove out-dated
information.
Remove "StaticRootPath" on the admin config display page, because few
end user really need it, it only causes misconfiguration.
Co-authored-by: Giteabot <teabot@gitea.io>
The "creating dir/file during load config" is a longstanding and complex
problem.
This PR only does a quick patch, it still needs more refactorings in the
future.
Fix#25938
Got the same problem as #25915 when updating an instance. The
`log.Fatal` should have been marked as breaking in #23911.
This PR adds a notice that the system is shutting down because of the
deprecated setting.
Replace #25580Fix#19453
The problem was: when users set "GITEA__XXX__YYY" , the "install page"
doesn't respect it.
So, to make the result consistent and avoid surprising end users, now
the "install page" also writes the environment variables to the config
file.
And, to make things clear, there are enough messages on the UI to tell
users what will happen.
There are some necessary/related changes to `environment-to-ini.go`:
* The "--clear" flag is removed and it was incorrectly written there.
The "clear" operation should be done if INSTALL_LOCK=true
* The "--prefix" flag is removed because it's never used, never
documented and it only causes inconsistent behavior.
This prevents the disk from overflowing with auth keys file
Fixes#17117
## ⚠️ BREAKING
This changes the default option for creating a backup of the authorized
key file when an update is made to default to false.
Resolve#24789
## ⚠️ BREAKING ⚠️
Before this, `DEFAULT_ACTIONS_URL` cound be set to any custom URLs like
`https://gitea.com` or `http://your-git-server,https://gitea.com`, and
the default value was `https://gitea.com`.
But now, `DEFAULT_ACTIONS_URL` supports only
`github`(`https://github.com`) or `self`(the root url of current Gitea
instance), and the default value is `github`.
If it has configured with a URL, an error log will be displayed and it
will fallback to `github`.
Actually, what we really want to do is always make it
`https://github.com`, however, this may not be acceptable for some
instances of internal use, so there's extra support for `self`, but no
more, even `https://gitea.com`.
Please note that `uses: https://xxx/yyy/zzz` always works and it does
exactly what it is supposed to do.
Although it's breaking, I belive it should be backported to `v1.20` due
to some security issues.
Follow-up on the runner side:
- https://gitea.com/gitea/act_runner/pulls/262
- https://gitea.com/gitea/act/pulls/70
More fix for #24981
* #24981Close#22361
* #22361
There were many patches for Gitea's sub-commands to satisfy the facts:
* Some sub-commands shouldn't output any log, otherwise the git protocol
would be broken
* Sometimes the users want to see "verbose" or "quiet" outputs
That's a longstanding problem, and very fragile. This PR is only a quick
patch for the problem.
In the future, the sub-command system should be refactored to a clear
solution.
----
Other changes:
* Use `ReplaceAllWriters` to replace
`RemoveAllWriters().AddWriters(writer)`, then it's an atomic operation.
* Remove unnecessary `syncLevelInternal` calls, because
`AddWriters/addWritersInternal` already calls it.
Co-authored-by: Giteabot <teabot@gitea.io>
Fix#25481
The `InitWorkPathAndCommonConfig` calls `LoadCommonSettings` which does
many checks like "current user is root or not".
Some commands like "environment-to-ini" shouldn't do such check, because
it might be run with "root" user at the moment (eg: the docker's setup
script)
ps: in the future, the docker's setup script should be improved to avoid
Gitea's command running with "root"
In modern days, there is no reason to make users set "charset" anymore.
Close#25378
## ⚠️ BREAKING
The key `[database].CHARSET` was removed completely as every newer
(>10years) MySQL database supports `utf8mb4` already.
There is a (deliberately) undocumented new fallback option if anyone
still needs to use it, but we don't recommend using it as it simply
causes problems.
# The problem
There were many "path tricks":
* By default, Gitea uses its program directory as its work path
* Gitea tries to use the "work path" to guess its "custom path" and
"custom conf (app.ini)"
* Users might want to use other directories as work path
* The non-default work path should be passed to Gitea by GITEA_WORK_DIR
or "--work-path"
* But some Gitea processes are started without these values
* The "serv" process started by OpenSSH server
* The CLI sub-commands started by site admin
* The paths are guessed by SetCustomPathAndConf again and again
* The default values of "work path / custom path / custom conf" can be
changed when compiling
# The solution
* Use `InitWorkPathAndCommonConfig` to handle these path tricks, and use
test code to cover its behaviors.
* When Gitea's web server runs, write the WORK_PATH to "app.ini", this
value must be the most correct one, because if this value is not right,
users would find that the web UI doesn't work and then they should be
able to fix it.
* Then all other sub-commands can use the WORK_PATH in app.ini to
initialize their paths.
* By the way, when Gitea starts for git protocol, it shouldn't output
any log, otherwise the git protocol gets broken and client blocks
forever.
The "work path" priority is: WORK_PATH in app.ini > cmd arg --work-path
> env var GITEA_WORK_DIR > builtin default
The "app.ini" searching order is: cmd arg --config > cmd arg "work path
/ custom path" > env var "work path / custom path" > builtin default
## ⚠️ BREAKING
If your instance's "work path / custom path / custom conf" doesn't meet
the requirements (eg: work path must be absolute), Gitea will report a
fatal error and exit. You need to set these values according to the
error log.
----
Close#24818Close#24222Close#21606Close#21498Close#25107Close#24981
Maybe close#24503
Replace #23301
Replace #22754
And maybe more
That's a longstanding INI package problem: the "MustXxx" calls change
the option values, and the following "Save" will save a lot of garbage
options into the user's config file.
Ideally we should refactor the INI package to a clear solution, but it's
a huge work.
A clear workaround is what this PR does: when "Save", load a clear INI
instance and save it.
Partially fix#25377, the "install" page needs more fine tunes.
