Commit graph

15 commits

Author SHA1 Message Date
oliverpool
8dfbbfef07 [REFACTOR] webhook matrix endpoints 2024-03-27 15:38:38 +01:00
oliverpool
702152bfde [REFACTOR] webhook.Handler interface 2024-03-24 10:35:06 +01:00
oliverpool
53f6f62ad4
Store webhook event in database (#29145)
Refactor the webhook logic, to have the type-dependent processing happen
only in one place.

---

1. An event happens
2. It is pre-processed (depending on the webhook type) and its body is
added to a task queue
3. When the task is processed, some more logic (depending on the webhook
type as well) is applied to make an HTTP request

This means that webhook-type dependant logic is needed in step 2 and 3.
This is cumbersome and brittle to maintain.

Updated webhook flow with this PR:
1. An event happens
2. It is stored as-is and added to a task queue
3. When the task is processed, the event is processed (depending on the
webhook type) to make an HTTP request

So the only webhook-type dependent logic happens in one place (step 3)
which should be much more robust.

- the raw event must be stored in the hooktask (until now, the
pre-processed body was stored)
- to ensure that previous hooktasks are correctly sent, a
`payload_version` is added (version 1: the body has already been
pre-process / version 2: the body is the raw event)

So future webhook additions will only have to deal with creating an
http.Request based on the raw event (no need to adjust the code in
multiple places, like currently).

Moreover since this processing happens when fetching from the task
queue, it ensures that the queuing of new events (upon a `git push` for
instance) does not get slowed down by a slow webhook.

As a concrete example, the PR #19307 for custom webhooks, should be
substantially smaller:
- no need to change `services/webhook/deliver.go`
- minimal change in `services/webhook/webhook.go` (add the new webhook
to the map)
- no need to change all the individual webhook files (since with this
refactor the `*webhook_model.Webhook` is provided as argument)

(cherry picked from commit 26653b196bd1d15c532af41f60351596dd4330bd)

Conflicts:
	services/webhook/deliver_test.go
	trivial context conflict
2024-03-11 23:36:59 +07:00
Lunny Xiao
16d15ce087
Fix package webhook (#27839)
Fix #23742

---------

Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
2023-10-31 04:43:38 +00:00
Lunny Xiao
f9cfd6ce5b
Use the type RefName for all the needed places and fix pull mirror sync bugs (#24634)
This PR replaces all string refName as a type `git.RefName` to make the
code more maintainable.

Fix #15367
Replaces #23070 
It also fixed a bug that tags are not sync because `git remote --prune
origin` will not remove local tags if remote removed.

We in fact should use `git fetch --prune --tags origin` but not `git
remote update origin` to do the sync.

Some answer from ChatGPT as ref.

> If the git fetch --prune --tags command is not working as expected,
there could be a few reasons why. Here are a few things to check:
> 
>Make sure that you have the latest version of Git installed on your
system. You can check the version by running git --version in your
terminal. If you have an outdated version, try updating Git and see if
that resolves the issue.
> 
>Check that your Git repository is properly configured to track the
remote repository's tags. You can check this by running git config
--get-all remote.origin.fetch and verifying that it includes
+refs/tags/*:refs/tags/*. If it does not, you can add it by running git
config --add remote.origin.fetch "+refs/tags/*:refs/tags/*".
> 
>Verify that the tags you are trying to prune actually exist on the
remote repository. You can do this by running git ls-remote --tags
origin to list all the tags on the remote repository.
> 
>Check if any local tags have been created that match the names of tags
on the remote repository. If so, these local tags may be preventing the
git fetch --prune --tags command from working properly. You can delete
local tags using the git tag -d command.

---------

Co-authored-by: delvh <dev.lh@web.de>
2023-05-26 01:04:48 +00:00
delvh
0f4e1b9ac6
Restructure webhook module (#22256)
Previously, there was an `import services/webhooks` inside
`modules/notification/webhook`.
This import was removed (after fighting against many import cycles).
Additionally, `modules/notification/webhook` was moved to
`modules/webhook`,
and a few structs/constants were extracted from `models/webhooks` to
`modules/webhook`.

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-01-01 23:23:15 +08:00
flynnnnnnnnnn
e81ccc406b
Implement FSFE REUSE for golang files (#21840)
Change all license headers to comply with REUSE specification.

Fix #16132

Co-authored-by: flynnnnnnnnnn <flynnnnnnnnnn@github>
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
2022-11-27 18:20:29 +00:00
Jim Kirisame
c8f3eb6acb
Fix wechatwork webhook sends empty content in PR review (#21762)
Wechatwork webhook is sending the following string for pull request reviews:

``` markdown
# 
>
```

This commit fixes this problem.
2022-11-19 15:19:14 +00:00
KN4CK3R
cad9adeff4
Display total commit count in hook message (#21400)
Fixes #21379

The commits are capped by `setting.UI.FeedMaxCommitNum` so
`len(commits)` is not the correct number. So this PR adds a new
`TotalCommits` field.

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-10-17 00:22:34 +08:00
Aaron F
3963625b6e
Webhook for Wiki changes (#20219)
Add support for triggering webhook notifications on wiki changes.

This PR contains frontend and backend for webhook notifications on wiki actions (create a new page, rename a page, edit a page and delete a page). The frontend got a new checkbox under the Custom Event -> Repository Events section. There is only one checkbox for create/edit/rename/delete actions, because it makes no sense to separate it and others like releases or packages follow the same schema.

![image](https://user-images.githubusercontent.com/121972/177018803-26851196-831f-4fde-9a4c-9e639b0e0d6b.png)

The actions itself are separated, so that different notifications will be executed (with the "action" field). All the webhook receivers implement the new interface method (Wiki) and the corresponding tests.

When implementing this, I encounter a little bug on editing a wiki page. Creating and editing a wiki page is technically the same action and will be handled by the ```updateWikiPage``` function. But the function need to know if it is a new wiki page or just a change. This distinction is done by the ```action``` parameter, but this will not be sent by the frontend (on form submit). This PR will fix this by adding the ```action``` parameter with the values ```_new``` or ```_edit```, which will be used by the ```updateWikiPage``` function.

I've done integration tests with matrix and gitea (http).

![image](https://user-images.githubusercontent.com/121972/177018795-eb5cdc01-9ba3-483e-a6b7-ed0e313a71fb.png)

Fix #16457

Signed-off-by: Aaron Fischer <mail@aaron-fischer.net>
2022-09-04 20:54:23 +01:00
6543
54e9ee37a7
format with gofumpt (#18184)
* gofumpt -w -l .

* gofumpt -w -l -extra .

* Add linter

* manual fix

* change make fmt
2022-01-20 18:46:10 +01:00
zjjhot
e34632bb53
Add issue hyperlinks in the webhook of wechatwork (#18102)
Co-authored-by: zjj <2031381130@qq.com>
2021-12-25 21:30:09 +08:00
Lunny Xiao
33fca2b537
Move webhook into models/webhook/ (#17579) 2021-11-10 13:13:16 +08:00
Lunny Xiao
9f31f3aa8a
Add an abstract json layout to make it's easier to change json library (#16528)
* Add an abstract json layout to make it's easier to change json library

* Fix import

* Fix import sequence

* Fix blank lines

* Fix blank lines
2021-07-24 18:03:58 +02:00
lengyuqu
078e2b2c39
Add support for corporate WeChat webhooks (#15910)
* 企业微信webhook

* 企业微信webhook

* 企业微信webhook

* Update templates/admin/hook_new.tmpl

Co-authored-by: a1012112796 <1012112796@qq.com>

* Update services/webhook/wechatwork.go

Co-authored-by: a1012112796 <1012112796@qq.com>

* 修善wechatwork

* 修善wechatwork

* fix

* Update locale_cs-CZ.ini

fix

* fix build

* fix

* fix build

* make webhooks.zh-cn.md

* delet unnecessary blank line

* delet unnecessary blank line

* 企业微信webhook

* 企业微信webhook

* 企业微信webhook

* Update templates/admin/hook_new.tmpl

Co-authored-by: a1012112796 <1012112796@qq.com>

* Update services/webhook/wechatwork.go

Co-authored-by: a1012112796 <1012112796@qq.com>

* 修善wechatwork

* 修善wechatwork

* fix

* fix build

* fix

* fix build

* make webhooks.zh-cn.md

* delet unnecessary blank line

* delet unnecessary blank line

* 企业微信webhook

* 企业微信webhook

* 企业微信webhook

* 企业微信webhook

* 企业微信webhook

* fix

* fix

* 企业微信webhook

* 企业微信webhook

* 企业微信webhook

* fix wechat

* fix wechat

* fix wechat

* fix wechat

* Fix invalid params and typo of email templates (#16394)

Signed-off-by: Meano <meanocat@gmail.com>

* Add LRU mem cache implementation (#16226)

The current default memory cache implementation is unbounded in size and number of
objects cached. This is hardly ideal.

This PR proposes creating a TwoQueue LRU cache as the underlying cache for Gitea.
The cache is limited by the number of objects stored in the cache (rather than size)
for simplicity. The default number of objects is 50000 - which is perhaps too small
as most of our objects cached are going to be much less than 1kB.

It may be worth considering using a different LRU implementation that actively limits
sizes or avoids GC - however, this is just a beginning implementation.

Signed-off-by: Andrew Thornton <art27@cantab.net>

* [skip ci] Updated translations via Crowdin

* Replace `plugins/docker` with `techknowlogick/drone-docker`in ci (#16407)

* plugins/docker -> techknowlogick/drone-docker

* It is multi-arch

* docs: rewrite email setup (#16404)

* Add intro for both the docs page and mailer methods
  * Fix numbering level in SMTP section
  * Recommends implicit TLS

Signed-off-by: Bagas Sanjaya <bagasdotme@gmail.com>

* Validate Issue Index before querying DB (#16406)

* Fix external renderer (#16401)

* fix external renderer

* use GBackground context as fallback

* no fallback, return error

Co-authored-by: Lauris BH <lauris@nix.lv>

* Add checkbox to delete pull branch after successful merge (#16049)

* Add checkbox to delete pull branch after successful merge

* Omit DeleteBranchAfterMerge field in json

* Log a warning instead of error when PR head branch deleted

* Add DefaultDeleteBranchAfterMerge to PullRequestConfig

* Add support for delete_branch_after_merge via API

* Fix for API: the branch should be deleted from the HEAD repo

If head and base repo are the same, reuse the already opened ctx.Repo.GitRepo

* Don't delegate to CleanupBranch, only reuse branch deletion code

CleanupBranch contains too much logic that has already been performed by the Merge

* Reuse gitrepo in MergePullRequest

Co-authored-by: Andrew Thornton <art27@cantab.net>

* [skip ci] Updated translations via Crowdin

* Detect encoding changes while parsing diff (#16330)

* Detect encoding changes while parsing diff

* Let branch/tag name be a valid ref to get CI status (#16400)

* fix #16384#

* refactor: move shared helper func to utils package

* extend Tests

* use ctx.Repo.GitRepo if not nil

* fix

* fix

* 企业微信webhook

* 企业微信webhook

* 企业微信webhook

* fix build

* fix build

* Apply suggestions from code review

Co-authored-by: a1012112796 <1012112796@qq.com>
Co-authored-by: myheavily <myheavily>
Co-authored-by: zhaoxin <gitea@fake.local>
Co-authored-by: Meano <Meano@foxmail.com>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: GiteaBot <teabot@gitea.io>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Bagas Sanjaya <bagasdotme@gmail.com>
Co-authored-by: Norwin <noerw@users.noreply.github.com>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: Jimmy Praet <jimmy.praet@telenet.be>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-07-23 12:41:27 +08:00