When using an external renderer, STDOUT is expected to be HTML. But
anything written to STDERR is currently ignored. In cases where the
renderer fails, I would like to log any error messages that the external
program outputs to STDERR.
- Remove code that isn't being used.
Found this is my stash from a few weeks ago, not sure how I found this
in the first place.
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Change all license headers to comply with REUSE specification.
Fix#16132
Co-authored-by: flynnnnnnnnnn <flynnnnnnnnnn@github>
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
Previously mentioning a user would link to its profile, regardless of
whether the user existed. This change tests if the user exists and only
if it does - a link to its profile is added.
* Fixes#3444
Signed-off-by: Yarden Shoham <hrsi88@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
It distractingly shows up on unit tests
* Looks like a leftover from #20571
Signed-off-by: Yarden Shoham <hrsi88@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
There is a small bug in #20571 whereby `$a a$b b$` will not be correctly
detected as a math inline block of `a a$b b`. This PR fixes this.
Also reenable test cases as per #21340
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
The behaviour of `PreventSurroundingPre` has changed in
https://github.com/alecthomas/chroma/pull/618 so that apparently it now
causes line wrapper tags to be no longer emitted, but we need some form
of indication to split the HTML into lines, so I did what
https://github.com/yuin/goldmark-highlighting/pull/33 did and added the
`nopWrapper`.
Maybe there are more elegant solutions but for some reason, just
splitting the HTML string on `\n` did not work.
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
This PR adds mathematical rendering with KaTeX.
The first step is to add a Goldmark extension that detects the latex
(and tex) mathematics delimiters.
The second step to make this extension only run if math support is
enabled.
The second step is to then add KaTeX CSS and JS to the head which will
load after the dom is rendered.
Fix#3445
Signed-off-by: Andrew Thornton <art27@cantab.net>
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* feat: extend issue template for yaml
* feat: support yaml template
* feat: render form to markdown
* feat: support yaml template for pr
* chore: rename to Fields
* feat: template unmarshal
* feat: split template
* feat: render to markdown
* feat: use full name as template file name
* chore: remove useless file
* feat: use dropdown of fomantic ui
* feat: update input style
* docs: more comments
* fix: render text without render
* chore: fix lint error
* fix: support use description as about in markdown
* fix: add field class in form
* chore: generate swagger
* feat: validate template
* feat: support is_nummber and regex
* test: fix broken unit tests
* fix: ignore empty body of md template
* fix: make multiple easymde editors work in one page
* feat: better UI
* fix: js error in pr form
* chore: generate swagger
* feat: support regex validation
* chore: generate swagger
* fix: refresh each markdown editor
* chore: give up required validation
* fix: correct issue template candidates
* fix: correct checkboxes style
* chore: ignore .hugo_build.lock in docs
* docs: separate out a new doc for merge templates
* docs: introduce syntax of yaml template
* feat: show a alert for invalid templates
* test: add case for a valid template
* fix: correct attributes of required checkbox
* fix: add class not-under-easymde for dropzone
* fix: use more back-quotes
* chore: remove translation in zh-CN
* fix EasyMDE statusbar margin
* fix: remove repeated blocks
* fix: reuse regex for quotes
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* Support localized README
* Slightly simplify getting the readme file and add some tests. Ensure that i18n also
works for docs/ etc.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Update modules/markup/renderer.go
* Update modules/markup/renderer.go
* Update modules/markup/renderer.go
Co-authored-by: Andrew Thornton <art27@cantab.net>
Use body text color in for links in the repository files table
Issue/PR links (`.ref-issue`) will not be affected, as seen in other git services.
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Lauris BH <lauris@nix.lv>
- This code is only valid when `refNumeric` exist(otherwise we didn't find
such numeric PR and can skip that check) and give a free-pas to the "BEFORE" check when
`ref` is nil.
- Resolves#20109
* Prototyping
* Start work on creating offsets
* Modify tests
* Start prototyping with actual MPH
* Twiddle around
* Twiddle around comments
* Convert templates
* Fix external languages
* Fix latest translation
* Fix some test
* Tidy up code
* Use simple map
* go mod tidy
* Move back to data structure
- Uses less memory by creating for each language a map.
* Apply suggestions from code review
Co-authored-by: delvh <dev.lh@web.de>
* Add some comments
* Fix tests
* Try to fix tests
* Use en-US as defacto fallback
* Use correct slices
* refactor (#4)
* Remove TryTr, add log for missing translation key
* Refactor i18n
- Separate dev and production locale stores.
- Allow for live-reloading in dev mode.
Co-authored-by: zeripath <art27@cantab.net>
* Fix live-reloading & check for errors
* Make linter happy
* live-reload with periodic check (#5)
* Fix tests
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
* clean git support for ver < 2.0
* fine tune tests for markup (which requires git module)
* remove unnecessary comments
* try to fix tests
* try test again
* use const for GitVersionRequired instead of var
* try to fix integration test
* Refactor CheckAttributeReader to make a *git.Repository version
* update document for commit signing with Gitea's internal gitconfig
* update document for commit signing with Gitea's internal gitconfig
Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Allow render HTML with css/js external links
* Fix bug because of filename escape chars
* Fix lint
* Update docs about new configuration item
* Fix bug of render HTML in sub directory
* Add CSP head for displaying iframe in rendering file
* Fix test
* Apply suggestions from code review
Co-authored-by: delvh <dev.lh@web.de>
* Some improvements
* some improvement
* revert change in SanitizerDisabled of external renderer
* Add sandbox for iframe and support allow-scripts and allow-same-origin
* refactor
* fix
* fix lint
* fine tune
* use single option RENDER_CONTENT_MODE, use sandbox=allow-scripts
* fine tune CSP
* Apply suggestions from code review
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Automatically add sidebar in the wiki view containing a TOC for the wiki page.
Make the TOC collapsable
Signed-off-by: Andrew Thornton <art27@cantab.net>
When Gitea is running as PID 1 git will occassionally orphan child processes leading
to (defunct) processes. This PR simply sets Setpgid to true on these child processes
meaning that these defunct processes will also be correctly reaped.
Fix#19077
Signed-off-by: Andrew Thornton <art27@cantab.net>
Remove two unmaintained vendor packages `i18n` and `paginater`. Changes:
* Rewrite `i18n` package with a more clear fallback mechanism. Fix an unstable `Tr` behavior, add more tests.
* Refactor the legacy `Paginater` to `Paginator`, test cases are kept unchanged.
Trivial enhancement (no breaking for end users):
* Use the first locale in LANGS setting option as the default, add a log to prevent from surprising users.
The main purpose is to refactor the legacy `unknwon/com` package.
1. Remove most imports of `unknwon/com`, only `util/legacy.go` imports the legacy `unknwon/com`
2. Use golangci's depguard to process denied packages
3. Fix some incorrect values in golangci.yml, eg, the version should be quoted string `"1.18"`
4. Use correctly escaped content for `go-import` and `go-source` meta tags
5. Refactor `com.Expand` to our stable (and the same fast) `vars.Expand`, our `vars.Expand` can still return partially rendered content even if the template is not good (eg: key mistach).
* Add tests for references with dashes
This commit adds tests for full URLs referencing repos names and user
names containing a dash.
* Extend regex to match URLs to repos/users with dashes
* Don't panic & allow shorter sha1
- Don't panic when the full regex isn't matched and allow the usage of a
shorter sha1 being used.
- Resolves#18471
* Update modules/markup/html.go
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
This PR continues the work in #17125 by progressively ensuring that git
commands run within the request context.
This now means that the if there is a git repo already open in the context it will be used instead of reopening it.
Signed-off-by: Andrew Thornton <art27@cantab.net>
This PR registers requests with the process manager and manages hierarchy within the processes.
Git repos are then associated with a context, (usually the request's context) - with sub commands using this context as their base context.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Prevent double sanitize.
* Use SanitizeReaderToWriter.
At the moment `actualRender` uses `SanitizeReader` to sanitize the output. But `SanitizeReader` gets called in `markup.render` too so the output gets sanitized twice.
I moved the `SanitizeReader` call into `RenderRaw` because this method does not use `markup.render`. I would like to remove the `RenderRaw`/`RenderRawString` methods too because they are only called from tests, the fuzzer and the `/markup/raw` api endpoint. This endpoint is not in use so I think we could remove them. If we really in the future need a method to render markdown without PostProcessing we could achieve this with a more flexible `renderer.NeedPostProcess` method.
* Add copy button to markdown code blocks
Done mostly in JS because I think it's better not to try getting buttons
past the markup sanitizer.
* add svg module tests
* fix sanitizer regexp
* remove outdated comment
* vertically center button in issue comments as well
* add comment to css
* fix undefined on view file line copy
* combine animation less files
* Update modules/markup/markdown/markdown.go
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* add test for different sizes
* add cloneNode and add tests for it
* use deep clone
* remove useless optional chaining
* remove the svg node cache
* unify clipboard copy string and i18n
* remove unused var
* remove unused localization
* minor css tweaks to the button
* comment tweak
* remove useless attribute
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* Fixes#16558 CSV delimiter determiner
* Fixes#16558 - properly determine CSV delmiiter
* Moves quoteString to a new function
* Adds big test with lots of commas for tab delimited csv
* Adds comments
* Shortens the text of the test
* Removes single quotes from regexp as only double quotes need to be searched
* Fixes spelling
* Fixes check of length as it probalby will only be 1e4, not greater
* Makes sample size a const, properly removes truncated line
* Makes sample size a const, properly removes truncated line
* Fixes comment
* Fixes comment
* tests for FormatError() function
* Adds logic to find the limiter before or after a quoted value
* Simplifies regex
* Error tests
* Error tests
* Update modules/csv/csv.go
Co-authored-by: delvh <dev.lh@web.de>
* Update modules/csv/csv.go
Co-authored-by: delvh <dev.lh@web.de>
* Adds comments
* Update modules/csv/csv.go
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: delvh <dev.lh@web.de>
When rendering source in org mode there is a mistake in the highlight code that
causes a panic.
This PR fixes this.
Fix#17139
Signed-off-by: Andrew Thornton <art27@cantab.net>
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* In Render tolerate not being passed a context
It is possible for RenderString to be passed to an external renderer if markdown
is set to be rendered by an external renderer. No context is currently sent to these
meaning that this will error out.
Fix#16835
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Add Context to Repo calls for RenderString
All calls from routers can easily add the context - so add it.
Signed-off-by: Andrew Thornton <art27@cantab.net>
The fuzzer found an issue with the issue pattern processor where there is a spurious
path.Clean which does not need to be there. This PR also sets the default AppURL for
the fuzzer too.
Signed-off-by: Andrew Thornton <art27@cantab.net>
If a pre-post-processed file contains relative img tags these need to be updated
and joined correctly with the prefix. Finally, the node attributes need to be updated.
Fix#16308
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Unfortunately go doesn't always ensure that execd processes are completely
waited for. On linux this means that zombie processes can occur.
This PR ensures that these are waited for by using signal notifier in serv and
passing a context elsewhere.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Use html.Parse rather than html.ParseFragment
There have been a few issues with html.ParseFragment - just use html.Parse instead.
* Skip document node
Signed-off-by: Andrew Thornton <art27@cantab.net>
* More efficiently parse shas for shaPostProcessor
The shaPostProcessor currently repeatedly calls git rev-parse --verify on both backends
which is fine if there is only one thing that matches a sha - however if there are
multiple things then this becomes wildly inefficient.
This PR provides functions for both backends which are much faster to use.
Fix#16092
* Add ShaExistCache to RenderContext
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
There is an inefficiency in the design of our processors which means that Emoji
and other processors run in order n^2 time.
This PR forces the processors to process the entirety of text node before passing
back up. The fundamental inefficiency remains but it should be significantly
ameliorated.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Bluemonday sanitizer regexp rules are not additive, so the addition of the icons,
emojis and chroma syntax policy has led to this being stripped.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Links in markdown should be absolute to the repository not the server
Fix#15075
Signed-off-by: Andrew Thornton <art27@cantab.net>
* match github
Signed-off-by: Andrew Thornton <art27@cantab.net>
* add testcase
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
* Upgrade to bluemonday 1.0.7
Fix#15349
Signed-off-by: Andrew Thornton <art27@cantab.net>
* resolve unit test
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* _ to unused func options
* rm useless brakets
* rm trifial non used models functions
* rm dead code
* rm dead global vars
* fix routers/api/v1/repo/issue.go
* dont overload import module
Implements request #14320 The rendering of CSV files does match the diff style.
* Moved CSV logic into base package.
* Added method to create a tabular diff.
* Added CSV compare context.
* Added CSV diff template.
* Use new table style in CSV markup.
* Added file size limit for CSV rendering.
* Display CSV parser errors in diff.
* Lazy read single file.
* Lazy read rows for full diff.
* Added unit tests for various CSV changes.
* Fix an issue with panics related to attributes
* Wrap goldmark render in a recovery function
* Reduce memory use in render emoji
* Use a pipe for rendering goldmark - still needs more work and a limiter
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
* Make sure sibling images get a link too
Due a problem with the ast.Walker in the our transformer in goldmark
an image with a sibling image will not be transformed to gain a parent
link. This PR fixes this.
Fix#12925
Signed-off-by: Andrew Thornton <art27@cantab.net>
PostProcess is supposed to be parsing and handling HTML
fragments, but on fuzzing it appears that there is a weird
issue with NUL elements that could cause a memory address
error in downstream libraries.
The simplest solution is to strip out the weird NULs - they
should not be there in any case and would be stripped out
anyway.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Prevent panic on fuzzer provided string
The fuzzer has found that providing a <body> tag with an attribute to
PostProcess causes a panic. This PR removes any rendered html or body
tags from the output.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Placate lint
* placate lint again
Signed-off-by: Andrew Thornton <art27@cantab.net>
* minor cleanup
Signed-off-by: Andrew Thornton <art27@cantab.net>
