Markdown is a widely used lightweight marking language that allows people to write documents in a simple, easy-to-read text format and is also the main article format used in [xLog] (https://xlog.app), which is presented in [xLog Flavored Markdown] (https://github.com/Crossbell-Box/xLog/blob/dev/src/markdown/index.ts) as an example of how a Markdown document can be deciphered with elegance
Structure
The process can be expressed in such a structure:
♪ Protected ♪
Key concepts:
- [unified] (https://github.com/unifiedjs): The library of deciphering, checking, converting and sequenced content through grammar trees and plugins
- [remark] (https://github.com/remarkjs): One of the ecological projects of unified, the Markdown processing library driven by plugins
- One of the ecological projects of [rehype] (https://github.com/rehypejs):unified, a plugin-driven HTML processing library
- [mdast] (https://github.com/syntax-tree/mdast): abstract syntax norms used by remark to denote Markdown
- [hast] (https://github.com/syntax-tree/hast): the abstract syntax norm used by rehype to denote HTML
In short, send the Markdown document to an eco-resolver that is identified as a unified grammatical tree, then convert it to the required content through a series of unified eco-plugins, and export it into the required format through a series of unified eco-tool libraries, which are described below in terms of the three steps:
Parse
♪ Protected ♪
Whether the input is Markdown, HTML or plain text, it needs to be interpreted into an actionable format. This format is called the Grammar Tree. Norms (e.g. mdast) define the appearance of such a syntax tree. Processors (e.g. remark in mdast) are responsible for their creation.
The simplest step we need to resolve is Marktown, so we should use [remark-parse] (https://github.com/remarkjs/remark/tree/main/packages/remark-parse) to compile the Markdown document into a syntax tree in mdast format
Corresponds to [xLog Flavored Markdown] (https://github.com/Crossbell-Box/xLog/blob/dev/src/markdown/index.ts)
♪ Protected ♪
Convert Transform
♪ Protected ♪
This is where magic happens. The user combination plugin and the order in which it runs. Plugin inserts and converts and checks the format they obtain at this stage.
This step is most critical, not only for the transformation from Markdown to HTML, but also for the personal goods that we want to bring in during the compilation process, such as the addition of non-standard grammatical sugar, the cleanup of HTML, the prevention of XSS, the enhancement of grammar, the embedding of custom components, etc.
The number of unified plugins is very large and up-to-date, basic needs are almost met, and it is easy to write conversion scripts for specific unmet needs
A special plugin is remark-rehype, which converts the mdast syntax tree to the Hast syntax tree, so it must be preceded by a remark for Markdown and then by a rehype for HTML.
[xLog Flavored Markdown] (https://github.com/Crossbell-Box/xLog/blob/dev/src/markdown/index.ts)
♪ Protected ♪
Below are some of the plugins used.
- [remark GithubAlerts] (https://github.com/hyoban/remak-github-alerts): Add the GitHub style Alerts syntax, [demonstrate] (https://xlog.xlog.app/xfm#alerts)
- [remarkBreaks] (https://github.com/remarkjs/remark-breaks): No longer need an empty line to be recognized as a new natural segment
- [remark Frontmatter] (https://github.com/remarkjs/remark-frontmatter): Support for pre-loading (YAML, TOML, etc.)
- [remarkGfm] (https://github.com/remarkjs/remark-gfm): Supporting non-standard GitHub expands a series of [grammatics] on the original Markdown syntax (https://github.github.com/gfm/) (although this series of syntaxes has been widely used as a de facto standard)
- [remarkDirective] (https://github.com/remarkkjs/remark-directive?tab=readme-ov-file] [remarkDirectiveRehyp] (https://github.com/IGassmann/remark-directive-rehype): Markdown [proposal for generic directives] (https://talk.commonmark.org/t/generic-directives-plugins-syntax/444).
- [remarkMath rehypeKatex] (https://github.com/remarkjs/remark-math): Support complex mathematical formulae, [demonstration] (https://xlog.xlog.app/xfm#supports-martical-expressions)
- [rehypeRaw] (https://github.com/rehypejs/rehype-raw): Supports the complex HTML in Markdown
- rehypeIpfs: custom plugins for photo, audio, video support address <<PROTECTED_BLONK_7> protocol
- [rehypeSlug] (https://github.com/rehypejs/rehype-slug): Add id to title
- [rehype Autolink Heatings] (https://github.com/rehypejs/rehype-autolink-headings): Add rel = “noopener noreferrer” to the title
- [rehype Sanitize] (https://github.com/rehypejs/rehype-sanitize): Cleaning HTML to ensure HTML safety from XSS attacks
- rehypeExternalLink: Custom plugin to add external links <<PROTECTED_BLONK_8> and <<PROTECTED_BLONK_9> -RehypeMermaid: Custom plugins, rendering drawings and tabulation tools [Mermaid] (https://mermaid.js.org/) The structure of this paper is republished through Mermaid
- [rehypeInferDescriptionMeta] (https://github.com/rehypejs/rehype-infer-description-meta): Description used for automatic document generation
- rehypeEmbed: custom plugins for auto-embedded cards like YouTube, Twitter, GitHub based on links -rehypeRemoveH1: Custom plugin to convert h1 to h2
- [rehypePrism] (https://github.com/timlrx/rehype-prism-plus): Highlighting in support of grammar -RehypeMention: Custom Plugin Support @DIYgod
Output
♪ Protected ♪
Last step is to convert (adjusted) format to Markdown, HTML or plain text (possibly different from input format!)
There’s a lot of unified tools to export all the formats we need.
xLog, for example, needs to display automatically generated directories on the right side of articles, needs to output pure text to calculate the estimated reading time and generate AI digests, needs to generate HTML for RSS use, needs to produce a React Element to render the page, needs to extract pictures and descriptions of articles to display the post card, using mdast-util-toc, Hast-util-to-html, Hast-util-to-jsx-runtime, unist-util-visit, respectively
Corresponds to [xLog Flavored Markdown] (https://github.com/Crossbell-Box/xLog/blob/dev/src/markdown/index.ts)
♪ Protected ♪
So we begin with the original Markdown document with grace, and get the output of the various formats we need.
In addition to this, we can use the deconstructed unified syntax tree to create a Markdown editor that can scroll and preview in both right and right motion in real time, with reference to xLog ‘s double-column Markdown editor ([code] (https://github.com/Crossbell-Box/xLog/blob/dev/src/components/dashboard/DualColumnEditor.tsx)) and have the opportunity to talk about it next time.