forked from NYANDEV/forgejo
parent
74433c91bf
commit
15c7738b3a
18 changed files with 1902 additions and 120 deletions
170
vendor/github.com/yuin/goldmark/README.md
generated
vendored
170
vendor/github.com/yuin/goldmark/README.md
generated
vendored
|
@ -6,48 +6,48 @@ goldmark
|
|||
[](https://coveralls.io/github/yuin/goldmark)
|
||||
[](https://goreportcard.com/report/github.com/yuin/goldmark)
|
||||
|
||||
> A Markdown parser written in Go. Easy to extend, standard compliant, well structured.
|
||||
> A Markdown parser written in Go. Easy to extend, standards-compliant, well-structured.
|
||||
|
||||
goldmark is compliant with CommonMark 0.29.
|
||||
|
||||
Motivation
|
||||
----------------------
|
||||
I need a Markdown parser for Go that meets following conditions:
|
||||
I needed a Markdown parser for Go that satisfies the following requirements:
|
||||
|
||||
- Easy to extend.
|
||||
- Markdown is poor in document expressions compared with other light markup languages like reStructuredText.
|
||||
- Markdown is poor in document expressions compared to other light markup languages such as reStructuredText.
|
||||
- We have extensions to the Markdown syntax, e.g. PHP Markdown Extra, GitHub Flavored Markdown.
|
||||
- Standard compliant.
|
||||
- Standards-compliant.
|
||||
- Markdown has many dialects.
|
||||
- GitHub Flavored Markdown is widely used and it is based on CommonMark aside from whether CommonMark is good specification or not.
|
||||
- CommonMark is too complicated and hard to implement.
|
||||
- Well structured.
|
||||
- AST based, and preserves source position of nodes.
|
||||
- GitHub-Flavored Markdown is widely used and is based upon CommonMark, effectively mooting the question of whether or not CommonMark is an ideal specification.
|
||||
- CommonMark is complicated and hard to implement.
|
||||
- Well-structured.
|
||||
- AST-based; preserves source position of nodes.
|
||||
- Written in pure Go.
|
||||
|
||||
[golang-commonmark](https://gitlab.com/golang-commonmark/markdown) may be a good choice, but it seems to be a copy of [markdown-it](https://github.com/markdown-it).
|
||||
|
||||
[blackfriday.v2](https://github.com/russross/blackfriday/tree/v2) is a fast and widely used implementation, but it is not CommonMark compliant and cannot be extended from outside of the package since its AST uses structs instead of interfaces.
|
||||
[blackfriday.v2](https://github.com/russross/blackfriday/tree/v2) is a fast and widely-used implementation, but is not CommonMark-compliant and cannot be extended from outside of the package, since its AST uses structs instead of interfaces.
|
||||
|
||||
Furthermore, its behavior differs from other implementations in some cases, especially regarding lists: ([Deep nested lists don't output correctly #329](https://github.com/russross/blackfriday/issues/329), [List block cannot have a second line #244](https://github.com/russross/blackfriday/issues/244), etc).
|
||||
Furthermore, its behavior differs from other implementations in some cases, especially regarding lists: [Deep nested lists don't output correctly #329](https://github.com/russross/blackfriday/issues/329), [List block cannot have a second line #244](https://github.com/russross/blackfriday/issues/244), etc.
|
||||
|
||||
This behavior sometimes causes problems. If you migrate your Markdown text to blackfriday-based wikis from GitHub, many lists will immediately be broken.
|
||||
This behavior sometimes causes problems. If you migrate your Markdown text from GitHub to blackfriday-based wikis, many lists will immediately be broken.
|
||||
|
||||
As mentioned above, CommonMark is too complicated and hard to implement, so Markdown parsers based on CommonMark barely exist.
|
||||
As mentioned above, CommonMark is complicated and hard to implement, so Markdown parsers based on CommonMark are few and far between.
|
||||
|
||||
Features
|
||||
----------------------
|
||||
|
||||
- **Standard compliant.** goldmark gets full compliance with the latest CommonMark spec.
|
||||
- **Standards-compliant.** goldmark is fully compliant with the latest [CommonMark](https://commonmark.org/) specification.
|
||||
- **Extensible.** Do you want to add a `@username` mention syntax to Markdown?
|
||||
You can easily do it in goldmark. You can add your AST nodes,
|
||||
parsers for block level elements, parsers for inline level elements,
|
||||
transformers for paragraphs, transformers for whole AST structure, and
|
||||
You can easily do so in goldmark. You can add your AST nodes,
|
||||
parsers for block-level elements, parsers for inline-level elements,
|
||||
transformers for paragraphs, transformers for the whole AST structure, and
|
||||
renderers.
|
||||
- **Performance.** goldmark performs pretty much equally to cmark,
|
||||
- **Performance.** goldmark's performance is on par with that of cmark,
|
||||
the CommonMark reference implementation written in C.
|
||||
- **Robust.** goldmark is tested with [go-fuzz](https://github.com/dvyukov/go-fuzz), a fuzz testing tool.
|
||||
- **Builtin extensions.** goldmark ships with common extensions like tables, strikethrough,
|
||||
- **Built-in extensions.** goldmark ships with common extensions like tables, strikethrough,
|
||||
task lists, and definition lists.
|
||||
- **Depends only on standard libraries.**
|
||||
|
||||
|
@ -62,15 +62,15 @@ Usage
|
|||
----------------------
|
||||
Import packages:
|
||||
|
||||
```
|
||||
```go
|
||||
import (
|
||||
"bytes"
|
||||
"github.com/yuin/goldmark"
|
||||
"bytes"
|
||||
"github.com/yuin/goldmark"
|
||||
)
|
||||
```
|
||||
|
||||
|
||||
Convert Markdown documents with the CommonMark compliant mode:
|
||||
Convert Markdown documents with the CommonMark-compliant mode:
|
||||
|
||||
```go
|
||||
var buf bytes.Buffer
|
||||
|
@ -105,11 +105,11 @@ Custom parser and renderer
|
|||
--------------------------
|
||||
```go
|
||||
import (
|
||||
"bytes"
|
||||
"github.com/yuin/goldmark"
|
||||
"github.com/yuin/goldmark/extension"
|
||||
"github.com/yuin/goldmark/parser"
|
||||
"github.com/yuin/goldmark/renderer/html"
|
||||
"bytes"
|
||||
"github.com/yuin/goldmark"
|
||||
"github.com/yuin/goldmark/extension"
|
||||
"github.com/yuin/goldmark/parser"
|
||||
"github.com/yuin/goldmark/renderer/html"
|
||||
)
|
||||
|
||||
md := goldmark.New(
|
||||
|
@ -128,6 +128,14 @@ if err := md.Convert(source, &buf); err != nil {
|
|||
}
|
||||
```
|
||||
|
||||
| Functional option | Type | Description |
|
||||
| ----------------- | ---- | ----------- |
|
||||
| `goldmark.WithParser` | `parser.Parser` | This option must be passed before `goldmark.WithParserOptions` and `goldmark.WithExtensions` |
|
||||
| `goldmark.WithRenderer` | `renderer.Renderer` | This option must be passed before `goldmark.WithRendererOptions` and `goldmark.WithExtensions` |
|
||||
| `goldmark.WithParserOptions` | `...parser.Option` | |
|
||||
| `goldmark.WithRendererOptions` | `...renderer.Option` | |
|
||||
| `goldmark.WithExtensions` | `...goldmark.Extender` | |
|
||||
|
||||
Parser and Renderer options
|
||||
------------------------------
|
||||
|
||||
|
@ -147,33 +155,33 @@ Parser and Renderer options
|
|||
| Functional option | Type | Description |
|
||||
| ----------------- | ---- | ----------- |
|
||||
| `html.WithWriter` | `html.Writer` | `html.Writer` for writing contents to an `io.Writer`. |
|
||||
| `html.WithHardWraps` | `-` | Render new lines as `<br>`.|
|
||||
| `html.WithHardWraps` | `-` | Render newlines as `<br>`.|
|
||||
| `html.WithXHTML` | `-` | Render as XHTML. |
|
||||
| `html.WithUnsafe` | `-` | By default, goldmark does not render raw HTML and potentially dangerous links. With this option, goldmark renders these contents as written. |
|
||||
| `html.WithUnsafe` | `-` | By default, goldmark does not render raw HTML or potentially dangerous links. With this option, goldmark renders such content as written. |
|
||||
|
||||
### Built-in extensions
|
||||
|
||||
- `extension.Table`
|
||||
- [GitHub Flavored Markdown: Tables](https://github.github.com/gfm/#tables-extension-)
|
||||
- [GitHub Flavored Markdown: Tables](https://github.github.com/gfm/#tables-extension-)
|
||||
- `extension.Strikethrough`
|
||||
- [GitHub Flavored Markdown: Strikethrough](https://github.github.com/gfm/#strikethrough-extension-)
|
||||
- [GitHub Flavored Markdown: Strikethrough](https://github.github.com/gfm/#strikethrough-extension-)
|
||||
- `extension.Linkify`
|
||||
- [GitHub Flavored Markdown: Autolinks](https://github.github.com/gfm/#autolinks-extension-)
|
||||
- [GitHub Flavored Markdown: Autolinks](https://github.github.com/gfm/#autolinks-extension-)
|
||||
- `extension.TaskList`
|
||||
- [GitHub Flavored Markdown: Task list items](https://github.github.com/gfm/#task-list-items-extension-)
|
||||
- [GitHub Flavored Markdown: Task list items](https://github.github.com/gfm/#task-list-items-extension-)
|
||||
- `extension.GFM`
|
||||
- This extension enables Table, Strikethrough, Linkify and TaskList.
|
||||
- This extension does not filter tags defined in [6.11: Disallowed Raw HTML (extension)](https://github.github.com/gfm/#disallowed-raw-html-extension-).
|
||||
If you need to filter HTML tags, see [Security](#security)
|
||||
- This extension enables Table, Strikethrough, Linkify and TaskList.
|
||||
- This extension does not filter tags defined in [6.11: Disallowed Raw HTML (extension)](https://github.github.com/gfm/#disallowed-raw-html-extension-).
|
||||
If you need to filter HTML tags, see [Security](#security).
|
||||
- `extension.DefinitionList`
|
||||
- [PHP Markdown Extra: Definition lists](https://michelf.ca/projects/php-markdown/extra/#def-list)
|
||||
- [PHP Markdown Extra: Definition lists](https://michelf.ca/projects/php-markdown/extra/#def-list)
|
||||
- `extension.Footnote`
|
||||
- [PHP Markdown Extra: Footnotes](https://michelf.ca/projects/php-markdown/extra/#footnotes)
|
||||
- [PHP Markdown Extra: Footnotes](https://michelf.ca/projects/php-markdown/extra/#footnotes)
|
||||
- `extension.Typographer`
|
||||
- This extension substitutes punctuations with typographic entities like [smartypants](https://daringfireball.net/projects/smartypants/).
|
||||
- This extension substitutes punctuations with typographic entities like [smartypants](https://daringfireball.net/projects/smartypants/).
|
||||
|
||||
### Attributes
|
||||
`parser.WithAttribute` option allows you to define attributes on some elements.
|
||||
The `parser.WithAttribute` option allows you to define attributes on some elements.
|
||||
|
||||
Currently only headings support attributes.
|
||||
|
||||
|
@ -197,7 +205,7 @@ heading {#id .className attrName=attrValue}
|
|||
|
||||
### Typographer extension
|
||||
|
||||
Typographer extension translates plain ASCII punctuation characters into typographic punctuation HTML entities.
|
||||
The Typographer extension translates plain ASCII punctuation characters into typographic-punctuation HTML entities.
|
||||
|
||||
Default substitutions are:
|
||||
|
||||
|
@ -211,25 +219,65 @@ Default substitutions are:
|
|||
| `<<` | `«` |
|
||||
| `>>` | `»` |
|
||||
|
||||
You can overwrite the substitutions by `extensions.WithTypographicSubstitutions`.
|
||||
You can override the defualt substitutions via `extensions.WithTypographicSubstitutions`:
|
||||
|
||||
```go
|
||||
markdown := goldmark.New(
|
||||
goldmark.WithExtensions(
|
||||
extension.NewTypographer(
|
||||
extension.WithTypographicSubstitutions(extension.TypographicSubstitutions{
|
||||
extension.LeftSingleQuote: []byte("‚"),
|
||||
extension.RightSingleQuote: nil, // nil disables a substitution
|
||||
}),
|
||||
),
|
||||
),
|
||||
goldmark.WithExtensions(
|
||||
extension.NewTypographer(
|
||||
extension.WithTypographicSubstitutions(extension.TypographicSubstitutions{
|
||||
extension.LeftSingleQuote: []byte("‚"),
|
||||
extension.RightSingleQuote: nil, // nil disables a substitution
|
||||
}),
|
||||
),
|
||||
),
|
||||
)
|
||||
```
|
||||
|
||||
### Linkify extension
|
||||
|
||||
The Linkify extension implements [Autolinks(extension)](https://github.github.com/gfm/#autolinks-extension-), as
|
||||
defined in [GitHub Flavored Markdown Spec](https://github.github.com/gfm/).
|
||||
|
||||
Since the spec does not define details about URLs, there are numerous ambiguous cases.
|
||||
|
||||
You can override autolinking patterns via options.
|
||||
|
||||
| Functional option | Type | Description |
|
||||
| ----------------- | ---- | ----------- |
|
||||
| `extension.WithLinkifyAllowedProtocols` | `[][]byte` | List of allowed protocols such as `[][]byte{ []byte("http:") }` |
|
||||
| `extension.WithLinkifyURLRegexp` | `*regexp.Regexp` | Regexp that defines URLs, including protocols |
|
||||
| `extension.WithLinkifyWWWRegexp` | `*regexp.Regexp` | Regexp that defines URL starting with `www.`. This pattern corresponds to [the extended www autolink](https://github.github.com/gfm/#extended-www-autolink) |
|
||||
| `extension.WithLinkifyEmailRegexp` | `*regexp.Regexp` | Regexp that defines email addresses` |
|
||||
|
||||
Example, using [xurls](https://github.com/mvdan/xurls):
|
||||
|
||||
```go
|
||||
import "mvdan.cc/xurls/v2"
|
||||
|
||||
markdown := goldmark.New(
|
||||
goldmark.WithRendererOptions(
|
||||
html.WithXHTML(),
|
||||
html.WithUnsafe(),
|
||||
),
|
||||
goldmark.WithExtensions(
|
||||
extension.NewLinkify(
|
||||
extension.WithLinkifyAllowedProtocols([][]byte{
|
||||
[]byte("http:"),
|
||||
[]byte("https:"),
|
||||
}),
|
||||
extension.WithLinkifyURLRegexp(
|
||||
xurls.Strict(),
|
||||
),
|
||||
),
|
||||
),
|
||||
)
|
||||
```
|
||||
|
||||
Security
|
||||
--------------------
|
||||
By default, goldmark does not render raw HTML and potentially dangerous URLs.
|
||||
If you need to gain more control over untrusted contents, it is recommended to
|
||||
By default, goldmark does not render raw HTML or potentially-dangerous URLs.
|
||||
If you need to gain more control over untrusted contents, it is recommended that you
|
||||
use an HTML sanitizer such as [bluemonday](https://github.com/microcosm-cc/bluemonday).
|
||||
|
||||
Benchmark
|
||||
|
@ -238,11 +286,10 @@ You can run this benchmark in the `_benchmark` directory.
|
|||
|
||||
### against other golang libraries
|
||||
|
||||
blackfriday v2 seems to be fastest, but it is not CommonMark compliant, so the performance of
|
||||
blackfriday v2 cannot simply be compared with that of the other CommonMark compliant libraries.
|
||||
blackfriday v2 seems to be the fastest, but as it is not CommonMark compliant, its performance cannot be directly compared to that of the CommonMark-compliant libraries.
|
||||
|
||||
Though goldmark builds clean extensible AST structure and get full compliance with
|
||||
CommonMark, it is reasonably fast and has lower memory consumption.
|
||||
goldmark, meanwhile, builds a clean, extensible AST structure, achieves full compliance with
|
||||
CommonMark, and consumes less memory, all while being reasonably fast.
|
||||
|
||||
```
|
||||
goos: darwin
|
||||
|
@ -268,21 +315,21 @@ iteration: 50
|
|||
average: 0.0040964230 sec
|
||||
```
|
||||
|
||||
As you can see, goldmark performs pretty much equally to cmark.
|
||||
As you can see, goldmark's performance is on par with cmark's.
|
||||
|
||||
Extensions
|
||||
--------------------
|
||||
|
||||
- [goldmark-meta](https://github.com/yuin/goldmark-meta): A YAML metadata
|
||||
extension for the goldmark Markdown parser.
|
||||
- [goldmark-highlighting](https://github.com/yuin/goldmark-highlighting): A Syntax highlighting extension
|
||||
- [goldmark-highlighting](https://github.com/yuin/goldmark-highlighting): A syntax-highlighting extension
|
||||
for the goldmark markdown parser.
|
||||
- [goldmark-mathjax](https://github.com/litao91/goldmark-mathjax): Mathjax support for goldmark markdown parser
|
||||
- [goldmark-mathjax](https://github.com/litao91/goldmark-mathjax): Mathjax support for the goldmark markdown parser
|
||||
|
||||
goldmark internal(for extension developers)
|
||||
----------------------------------------------
|
||||
### Overview
|
||||
goldmark's Markdown processing is outlined as a bellow diagram.
|
||||
goldmark's Markdown processing is outlined in the diagram below.
|
||||
|
||||
```
|
||||
<Markdown in []byte, parser.Context>
|
||||
|
@ -313,10 +360,11 @@ goldmark's Markdown processing is outlined as a bellow diagram.
|
|||
### Parsing
|
||||
Markdown documents are read through `text.Reader` interface.
|
||||
|
||||
AST nodes do not have concrete text. AST nodes have segment information of the documents. It is represented by `text.Segment` .
|
||||
AST nodes do not have concrete text. AST nodes have segment information of the documents, represented by `text.Segment` .
|
||||
|
||||
`text.Segment` has 3 attributes: `Start`, `End`, `Padding` .
|
||||
|
||||
(TBC)
|
||||
|
||||
**TODO**
|
||||
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue