femsci/forgejo
femsci/forgejo
1
0
Fork 0
forked from NYANDEV/forgejo
Code Pull requests Activity
forgejo/modules
History
Jason Song 9958642502
Fix issues indexer document mapping (#25619) 
Fix regression of #5363 (so long ago).

The old code definded a document mapping for `issueIndexerDocType`, and
assigned it to `BleveIndexerData` as its type. (`BleveIndexerData` has
been renamed to `IndexerData` in #25174, but nothing more.) But the old
code never used `BleveIndexerData`, it wrote the index with an anonymous
struct type. Nonetheless, bleve would use the default auto-mapping for
struct it didn't know, so the indexer still worked. This means the
custom document mapping was always dead code.

The custom document mapping is not useless, it can reduce index storage,
this PR brings it back and disable default mapping to prevent it from
happening again. Since `IndexerData`(`BleveIndexerData`) has JSON tags,
and bleve uses them first, so we should use `repo_id` as the field name
instead of `RepoID`.

I did a test to compare the storage size before and after this, with
about 3k real comments that were migrated from some public repos.

Before:

```text
[ 160]  .
├── [  42]  index_meta.json
├── [  13]  rupture_meta.json
└── [ 128]  store
    ├── [6.9M]  00000000005d.zap
    └── [256K]  root.bolt
```

After:

```text
[ 160]  .
├── [  42]  index_meta.json
├── [  13]  rupture_meta.json
└── [ 128]  store
    ├── [3.5M]  000000000065.zap
    └── [256K]  root.bolt
```

It saves about half the storage space.

---------

Co-authored-by: Giteabot <teabot@gitea.io>
2023-07-04 09:05:28 +00:00
..
actions Support downloading raw task logs (#24451) 2023-06-29 10:58:56 +08:00
activitypub Add Chef package registry (#22554) 2023-02-06 09:49:21 +08:00
analyze Implement FSFE REUSE for golang files (#21840) 2022-11-27 18:20:29 +00:00
assetfs Use a general approach to access custom/static/builtin assets (#24022) 2023-04-12 18:16:45 +08:00
auth Use a general approach to access custom/static/builtin assets (#24022) 2023-04-12 18:16:45 +08:00
avatar Improve avatar uploading / resizing / compressing, remove Fomantic card module (#24653) 2023-05-13 20:59:11 +02:00
base Update octicons and use octicon-file-directory-symlink (#25453) 2023-06-22 22:05:52 +00:00
cache Update redis library to support redis v7 (#24114) 2023-04-13 18:41:04 -04:00
charset Update go tool dependencies, restructure lint targets (#24239) 2023-04-22 14:53:00 -04:00
container Implement FSFE REUSE for golang files (#21840) 2022-11-27 18:20:29 +00:00
context Sync branches into databases (#22743) 2023-06-29 10:03:20 +00:00
csv Refactor locale number (#24134) 2023-04-17 11:37:23 +08:00
doctor Refactor path & config system (#25330) 2023-06-21 13:50:26 +08:00
emoji Update emoji set to Unicode 15 (#25595) 2023-06-29 16:29:48 +00:00
eventsource Move convert package to services (#22264) 2022-12-29 10:57:15 +08:00
generate Implement FSFE REUSE for golang files (#21840) 2022-11-27 18:20:29 +00:00
git Fix index generation parallelly failure (#25235) 2023-06-15 02:14:43 +02:00
gitgraph Add context cache as a request level cache (#22294) 2023-02-15 21:37:34 +08:00
graceful Improve queue and logger context (#24924) 2023-05-26 07:31:55 +00:00
hcaptcha Consume hcaptcha and pwn deps (#22610) 2023-01-29 09:49:51 -06:00
highlight test_env: hardcode major go version in use (#23464) 2023-03-14 16:09:01 -04:00
hostmatcher Implement FSFE REUSE for golang files (#21840) 2022-11-27 18:20:29 +00:00
html Implement FSFE REUSE for golang files (#21840) 2022-11-27 18:20:29 +00:00
httpcache Use standard HTTP library to serve files (#24693) 2023-05-13 16:04:57 +02:00
httplib Use fetch to send requests to create issues/comments (#25258) 2023-06-16 06:32:43 +00:00
indexer Fix issues indexer document mapping (#25619) 2023-07-04 09:05:28 +00:00
issue/template Allow issue templates to not render title (#22589) 2023-01-26 22:45:49 -06:00
json Update gitea-vet to check FSFE REUSE (#22004) 2022-12-02 22:14:57 +08:00
label Make label templates have consistent behavior and priority (#23749) 2023-04-10 16:44:02 +08:00
lfs Rewrite logger system (#24726) 2023-05-21 22:35:11 +00:00
log Fix sub-command log level (#25537) 2023-06-28 08:02:06 +02:00
markup Refactor path & config system (#25330) 2023-06-21 13:50:26 +08:00
mcaptcha Implement FSFE REUSE for golang files (#21840) 2022-11-27 18:20:29 +00:00
metrics Use a separate admin page to show global stats, remove actions stat (#25062) 2023-06-03 22:03:41 +08:00
migration Scoped labels (#22585) 2023-02-18 21:17:39 +02:00
mirror Improve queue and logger context (#24924) 2023-05-26 07:31:55 +00:00
nosql Update redis library to support redis v7 (#24114) 2023-04-13 18:41:04 -04:00
notification Add Adopt repository event and handler (#25497) 2023-06-26 06:59:15 +00:00
options Use a general approach to access custom/static/builtin assets (#24022) 2023-04-12 18:16:45 +08:00
packages Add direct serving of package content (#25543) 2023-07-03 15:33:28 +02:00
paginator Use more specific test methods (#24265) 2023-04-22 17:56:27 -04:00
pprof Implement FSFE REUSE for golang files (#21840) 2022-11-27 18:20:29 +00:00
private Use the type RefName for all the needed places and fix pull mirror sync bugs (#24634) 2023-05-26 01:04:48 +00:00
process Do not output "Trace" level logs from process manager by default (#24952) 2023-05-27 10:55:24 +00:00
proxy Use proxy for pull mirror (#22771) 2023-02-11 08:39:50 +08:00
proxyprotocol Implement FSFE REUSE for golang files (#21840) 2022-11-27 18:20:29 +00:00
public Use standard HTTP library to serve files (#24693) 2023-05-13 16:04:57 +02:00
queue Help to recover from corrupted levelqueue (#24912) 2023-05-29 10:52:32 +08:00
recaptcha Implement FSFE REUSE for golang files (#21840) 2022-11-27 18:20:29 +00:00
references Use correct captured group range when parsing cross-reference (#22672) 2023-01-31 10:08:05 +01:00
regexplru Implement FSFE REUSE for golang files (#21840) 2022-11-27 18:20:29 +00:00
repository Fix branch commit message too long problem (#25588) 2023-06-30 17:03:05 +08:00
secret Improve decryption failure message (#24573) 2023-05-07 19:29:43 +08:00
session Update redis library to support redis v7 (#24114) 2023-04-13 18:41:04 -04:00
setting Set SSH_AUTHORIZED_KEYS_BACKUP to false (#25412) 2023-07-01 04:07:48 +00:00
sitemap Fix sitemap (#22272) 2022-12-30 23:31:00 +08:00
ssh Refactor path & config system (#25330) 2023-06-21 13:50:26 +08:00
storage Fix all possible setting error related storages and added some tests (#23911) 2023-06-14 11:42:38 +08:00
structs Add API for changing Avatars (#25369) 2023-06-29 23:22:55 +00:00
svg Use inline SVG for built-in OAuth providers (#25171) 2023-06-13 10:51:02 +00:00
sync Implement FSFE REUSE for golang files (#21840) 2022-11-27 18:20:29 +00:00
system Implement FSFE REUSE for golang files (#21840) 2022-11-27 18:20:29 +00:00
templates Fix missing commit message body when the message has leading newlines (#25418) 2023-06-21 09:14:34 +00:00
test Refactor web package and context package (#25298) 2023-06-18 09:59:09 +02:00
testlogger Refactor path & config system (#25330) 2023-06-21 13:50:26 +08:00
timeutil Fix incorrect webhook time and use relative-time to display it (#24477) 2023-05-03 19:53:43 -04:00
translation Refactor INI package (first step) (#25024) 2023-06-02 17:27:30 +08:00
turnstile Add new captcha: cloudflare turnstile (#22369) 2023-02-05 15:29:03 +08:00
typesniffer Do not recognize text files as audio (#23355) 2023-03-07 22:40:41 -05:00
updatechecker Implement FSFE REUSE for golang files (#21840) 2022-11-27 18:20:29 +00:00
upload Implement FSFE REUSE for golang files (#21840) 2022-11-27 18:20:29 +00:00
uri Implement FSFE REUSE for golang files (#21840) 2022-11-27 18:20:29 +00:00
user Implement FSFE REUSE for golang files (#21840) 2022-11-27 18:20:29 +00:00
util Store and use seconds for timeline time comments (#25392) 2023-06-23 12:12:39 +00:00
validation Map OIDC groups to Orgs/Teams (#21441) 2023-02-08 14:44:42 +08:00
web Refactor web package and context package (#25298) 2023-06-18 09:59:09 +02:00
webhook New webhook trigger for receiving Pull Request review requests (#24481) 2023-05-24 22:06:27 -04:00