Add AI documentation skills
This commit is contained in:
@@ -0,0 +1,37 @@
|
||||
---
|
||||
title: Adding a badge to your project
|
||||
date: 2023-11-30
|
||||
authors: [squidfunk]
|
||||
slug: adding-a-badge-to-your-project
|
||||
description: >
|
||||
Share the love – you can now add a badge to your README, showing that your
|
||||
project is built with Material for MkDocs
|
||||
categories:
|
||||
- General
|
||||
---
|
||||
|
||||
# Adding a :simple-materialformkdocs: badge to your project
|
||||
|
||||
__You enjoy working with Material for MkDocs? Share the love! You can now add
|
||||
a badge to your project's README, showing that your project is built with
|
||||
Material for MkDocs.__
|
||||
|
||||
Material for MkDocs' logo was just added to [Simple Icons], which is used by
|
||||
[Shields.io] to include logos in badges. We generated a badge for you, which
|
||||
you can add to your project's README:
|
||||
|
||||
[![Material for MkDocs][badge]](#usage)
|
||||
|
||||
<!-- more -->
|
||||
|
||||
## Usage
|
||||
|
||||
Just copy the following snippet and paste it into your project's `README.md`:
|
||||
|
||||
``` markdown
|
||||
[](https://squidfunk.github.io/mkdocs-material/)
|
||||
```
|
||||
|
||||
[Simple Icons]: https://simpleicons.org/
|
||||
[Shields.io]: https://shields.io/
|
||||
[badge]: https://img.shields.io/badge/Material_for_MkDocs-526CFE?style=for-the-badge&logo=MaterialForMkDocs&logoColor=white
|
||||
@@ -0,0 +1,228 @@
|
||||
---
|
||||
date: 2022-09-12
|
||||
authors: [squidfunk]
|
||||
description: >
|
||||
Our new blog is built with the brand new built-in blog plugin. You can build
|
||||
a blog alongside your documentation or standalone
|
||||
categories:
|
||||
- Blog
|
||||
links:
|
||||
- setup/setting-up-a-blog.md
|
||||
- plugins/blog.md
|
||||
---
|
||||
|
||||
# Blog support just landed
|
||||
|
||||
__Hey there! You're looking at our new blog, built with the brand new
|
||||
[built-in blog plugin]. With this plugin, you can easily build a blog alongside
|
||||
your documentation or standalone.__
|
||||
|
||||
Proper support for blogging, as requested by many users over the past few years,
|
||||
was something that was desperately missing from Material for MkDocs' feature set.
|
||||
While everybody agreed that blogging support was a blind spot, it was not
|
||||
obvious whether MkDocs could be extended in a way to allow for blogging as we
|
||||
know it from [Jekyll] and friends. The [built-in blog plugin] proves that it is,
|
||||
after all, possible to build a blogging engine on top of MkDocs, in order to
|
||||
create a technical blog alongside your documentation, or as the main thing.
|
||||
|
||||
<!-- more -->
|
||||
|
||||
_This article explains how to build a standalone blog with Material for MkDocs.
|
||||
If you want to build a blog alongside your documentation, please refer to
|
||||
[the plugin's documentation][built-in blog plugin]._
|
||||
|
||||
[built-in blog plugin]: ../../plugins/blog.md
|
||||
[Jekyll]: https://jekyllrb.com/
|
||||
|
||||
## Quick start
|
||||
|
||||
### Creating a standalone blog
|
||||
|
||||
You can bootstrap a new project using the `mkdocs` executable:
|
||||
|
||||
```
|
||||
mkdocs new .
|
||||
```
|
||||
|
||||
This will create the following structure:
|
||||
|
||||
``` { .sh .no-copy }
|
||||
.
|
||||
├─ docs/
|
||||
│ └─ index.md
|
||||
└─ mkdocs.yml
|
||||
```
|
||||
|
||||
#### Configuration
|
||||
|
||||
In this article, we're going to build a standalone blog, which means that the
|
||||
blog lives at the root of your project. For this reason, open `mkdocs.yml`,
|
||||
and replace its contents with:
|
||||
|
||||
``` yaml
|
||||
site_name: My Blog
|
||||
theme:
|
||||
name: material
|
||||
features:
|
||||
- navigation.sections
|
||||
plugins:
|
||||
- blog:
|
||||
blog_dir: . # (1)!
|
||||
- search
|
||||
- tags
|
||||
nav:
|
||||
- index.md
|
||||
```
|
||||
|
||||
1. This is the important part – we're hosting the blog at the root of the
|
||||
project, and not in a subdirectory. For more information, see the
|
||||
[`blog_dir`][blog_dir] configuration option.
|
||||
|
||||
[blog_dir]: ../../plugins/blog.md#config.blog_dir
|
||||
|
||||
#### Blog setup
|
||||
|
||||
The blog index page lives in `docs/index.md`. This page was pre-filled by
|
||||
MkDocs with some content, so we're going to replace it with what we need to
|
||||
bootstrap the blog:
|
||||
|
||||
``` markdown
|
||||
# Blog
|
||||
```
|
||||
|
||||
That's it.
|
||||
|
||||
### Writing your first post
|
||||
|
||||
Now that we have set up the [built-in blog plugin], we can start writing our
|
||||
first post. All blog posts are written with the [exact same Markdown flavor] as
|
||||
already included with Material for MkDocs. First, create a folder called `posts`
|
||||
with a file called `hello-world.md`:
|
||||
|
||||
``` { .sh .no-copy }
|
||||
.
|
||||
├─ docs/
|
||||
│ ├─ posts/
|
||||
│ │ └─ hello-world.md # (1)!
|
||||
│ └─ index.md
|
||||
└─ mkdocs.yml
|
||||
```
|
||||
|
||||
1. If you'd like to arrange posts differently, you're free to do so. The URLs
|
||||
are built from the format specified in [`post_url_format`][post slugs] and
|
||||
the titles and dates of posts, no matter how they are organized
|
||||
inside the `posts` directory.
|
||||
|
||||
Then, open up `hello-world.md`, and add the following lines:
|
||||
|
||||
``` yaml
|
||||
---
|
||||
draft: true # (1)!
|
||||
date: 2022-01-31
|
||||
categories:
|
||||
- Hello
|
||||
- World
|
||||
---
|
||||
|
||||
# Hello world!
|
||||
|
||||
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque nec
|
||||
maximus ex. Sed consequat, nulla quis malesuada dapibus, elit metus vehicula
|
||||
erat, ut egestas tellus eros at risus. In hac habitasse platea dictumst.
|
||||
Phasellus id lacus pulvinar erat consequat pretium. Morbi malesuada arcu mauris
|
||||
Nam vel justo sem. Nam placerat purus non varius luctus. Integer pretium leo in
|
||||
sem rhoncus, quis gravida orci mollis. Proin id aliquam est. Vivamus in nunc ac
|
||||
metus tristique pellentesque. Suspendisse viverra urna in accumsan aliquet.
|
||||
|
||||
<!-- more -->
|
||||
|
||||
Donec volutpat, elit ac volutpat laoreet, turpis dolor semper nibh, et dictum
|
||||
massa ex pulvinar elit. Curabitur commodo sit amet dolor sed mattis. Etiam
|
||||
tempor odio eu nisi gravida cursus. Maecenas ante enim, fermentum sit amet
|
||||
molestie nec, mollis ac libero. Vivamus sagittis suscipit eros ut luctus.
|
||||
|
||||
Nunc vehicula sagittis condimentum. Cras facilisis bibendum lorem et feugiat.
|
||||
In auctor accumsan ligula, at consectetur erat commodo quis. Morbi ac nunc
|
||||
pharetra, pellentesque risus in, consectetur urna. Nulla id enim facilisis
|
||||
arcu tincidunt pulvinar. Vestibulum laoreet risus scelerisque porta congue.
|
||||
In velit purus, dictum quis neque nec, molestie viverra risus. Nam pellentesque
|
||||
tellus id elit ultricies, vel finibus erat cursus.
|
||||
```
|
||||
|
||||
1. If you mark a post as a [draft], a red marker appears next to the post date
|
||||
on index pages. When the site is built, drafts are not included in the
|
||||
output. [This behavior can be changed], e.g. for rendering drafts when
|
||||
building deploy previews.
|
||||
|
||||
When you spin up the [live preview server], you should be greeted by your first
|
||||
post! You'll also realize, that [archive] and [category] indexes have been
|
||||
automatically generated for you:
|
||||
|
||||
![Blog]
|
||||
|
||||
However, this is just the start. The [built-in blog plugin] packs a lot of
|
||||
functionality needed in day-to-day blogging. Visit the documentation of the
|
||||
plugin to learn about the following topics:
|
||||
|
||||
<div class="mdx-columns" markdown>
|
||||
|
||||
- [Adding an excerpt]
|
||||
- [Adding authors]
|
||||
- [Adding categories]
|
||||
- [Adding tags]
|
||||
- [Adding related links]
|
||||
- [Linking from and to posts]
|
||||
- [Setting the reading time]
|
||||
- [Setting defaults]
|
||||
|
||||
</div>
|
||||
|
||||
Additionally, the [built-in blog plugin] has dozens of [configuration options]
|
||||
which allow for fine-tuning the output. You can configure post slugs, general
|
||||
behavior and much more.
|
||||
|
||||
[exact same Markdown flavor]: ../../reference/index.md
|
||||
[post slugs]: ../../plugins/blog.md#config.post_url_format
|
||||
[draft]: ../../plugins/blog.md#meta.draft
|
||||
[This behavior can be changed]: ../../plugins/blog.md#config.draft
|
||||
[live preview server]: ../../creating-your-site.md#previewing-as-you-write
|
||||
[archive]: ../../plugins/blog.md#config.archive
|
||||
[category]: ../../plugins/blog.md#config.categories
|
||||
[Blog]: blog-support-just-landed/blog.png
|
||||
[Blog post]: blog-support-just-landed/blog-post.png
|
||||
[Adding an excerpt]: ../../setup/setting-up-a-blog.md#adding-an-excerpt
|
||||
[Adding authors]: ../../setup/setting-up-a-blog.md#adding-authors
|
||||
[Adding categories]: ../../setup/setting-up-a-blog.md#adding-categories
|
||||
[Adding tags]: ../../setup/setting-up-a-blog.md#adding-tags
|
||||
[Adding related links]: ../../setup/setting-up-a-blog.md#adding-related-links
|
||||
[Linking from and to posts]: ../../setup/setting-up-a-blog.md#linking-from-and-to-posts
|
||||
[Setting the reading time]: ../../setup/setting-up-a-blog.md#setting-the-reading-time
|
||||
[Setting defaults]: ../../setup/setting-up-a-blog.md#setting-defaults
|
||||
[configuration options]: ../../setup/setting-up-a-blog.md#configuration
|
||||
|
||||
## What's next?
|
||||
|
||||
Getting basic blogging support out the door was quite a challenge – the
|
||||
[built-in blog plugin] is probably the biggest release this year and already
|
||||
packs a lot of functionality. However, Material for MkDocs is used in many
|
||||
different contexts, which is why we'd expect to iterate, as always.
|
||||
|
||||
Some ideas already proposed by users:
|
||||
|
||||
- __Blog series__: Authors should be able to create so called blog series and
|
||||
assign posts to a blog series using simple identifiers. For each post that is
|
||||
part of a series, a list with links to all other posts should be included in
|
||||
the post's content.
|
||||
|
||||
- __Author indexes__: Besides [archive] and [category] indexes, authors should
|
||||
be able to create per-author indexes, which list all posts linked to an
|
||||
author. Additionally, a profile should be created for each author and linked
|
||||
from posts.
|
||||
|
||||
- __Social share buttons__: It should be easy to share blog posts via social
|
||||
media or other ways. For this reason, it should be possible to automatically
|
||||
include social sharing buttons with each post.
|
||||
|
||||
What's still missing from the brand new [built-in blog plugin]? Feel free to
|
||||
share your ideas in the comments. Together, we can build one of the best modern
|
||||
engines for technical blogging!
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 325 KiB |
@@ -0,0 +1,82 @@
|
||||
---
|
||||
date: 2022-05-05
|
||||
authors: [squidfunk]
|
||||
title: Chinese search support
|
||||
description: >
|
||||
Insiders adds Chinese language support for the built-in search plugin – a
|
||||
feature that has been requested many times
|
||||
categories:
|
||||
- Search
|
||||
links:
|
||||
- blog/posts/search-better-faster-smaller.md
|
||||
- plugins/search.md#segmentation
|
||||
---
|
||||
|
||||
# Chinese search support – 中文搜索支持
|
||||
|
||||
__Insiders adds experimental Chinese language support for the [built-in search
|
||||
plugin] – a feature that has been requested for a long time given the large
|
||||
number of Chinese users.__
|
||||
|
||||
After the United States and Germany, the third-largest country of origin of
|
||||
Material for MkDocs users is China. For a long time, the [built-in search plugin]
|
||||
didn't allow for proper segmentation of Chinese characters, mainly due to
|
||||
missing support in [lunr-languages] which is used for search tokenization and
|
||||
stemming. The latest Insiders release adds long-awaited Chinese language support
|
||||
for the built-in search plugin, something that has been requested by many users.
|
||||
|
||||
<!-- more -->
|
||||
|
||||
_Material for MkDocs終於支持中文了！文本被正確分割並且更容易找到。_
|
||||
{ style="display: inline" }
|
||||
|
||||
_This article explains how to set up Chinese language support for the built-in
|
||||
search plugin in a few minutes._
|
||||
{ style="display: inline" }
|
||||
|
||||
[built-in search plugin]: ../../plugins/search.md
|
||||
[lunr-languages]: https://github.com/MihaiValentin/lunr-languages
|
||||
|
||||
## Configuration
|
||||
|
||||
Chinese language support for Material for MkDocs is provided by [jieba], an
|
||||
excellent Chinese text segmentation library. If [jieba] is installed, the
|
||||
built-in search plugin automatically detects Chinese characters and runs them
|
||||
through the segmenter. You can install [jieba] with:
|
||||
|
||||
```
|
||||
pip install jieba
|
||||
```
|
||||
|
||||
The next step is only required if you specified the [`separator`][separator]
|
||||
configuration in `mkdocs.yml`. Text is segmented with [zero-width whitespace]
|
||||
characters, so it renders exactly the same in the search modal. Adjust
|
||||
`mkdocs.yml` so that the [`separator`][separator] includes the `\u200b`
|
||||
character:
|
||||
|
||||
``` yaml
|
||||
plugins:
|
||||
- search:
|
||||
separator: '[\s\u200b\-]'
|
||||
```
|
||||
|
||||
That's all that is necessary.
|
||||
|
||||
## Usage
|
||||
|
||||
If you followed the instructions in the configuration guide, Chinese words will
|
||||
now be tokenized using [jieba]. Try searching for
|
||||
[:octicons-search-24: 支持][q=支持] to see how it integrates with the
|
||||
built-in search plugin.
|
||||
|
||||
---
|
||||
|
||||
Note that this is an experimental feature, and I, @squidfunk, am not
|
||||
proficient in Chinese (yet?). If you find a bug or think something can be
|
||||
improved, please [open an issue].
|
||||
|
||||
[jieba]: https://pypi.org/project/jieba/
|
||||
[zero-width whitespace]: https://en.wikipedia.org/wiki/Zero-width_space
|
||||
[separator]: ../../plugins/search.md#config.separator
|
||||
[q=支持]: ?q=支持
|
||||
[open an issue]: https://github.com/squidfunk/mkdocs-material/issues/new/choose
|
||||
@@ -0,0 +1,206 @@
|
||||
---
|
||||
date: 2021-09-26
|
||||
authors: [squidfunk]
|
||||
description: >
|
||||
Three new simple ways to exclude dedicated parts of a document from the search
|
||||
index, allowing for more fine-grained control
|
||||
categories:
|
||||
- Search
|
||||
links:
|
||||
- blog/posts/search-better-faster-smaller.md
|
||||
- setup/setting-up-site-search.md#search-exclusion
|
||||
---
|
||||
|
||||
# Excluding content from search
|
||||
|
||||
__The latest Insiders release brings three new simple ways to exclude
|
||||
dedicated parts of a document from the search index, allowing for more
|
||||
fine-grained control.__
|
||||
|
||||
Two weeks ago, Material for MkDocs Insiders shipped a [brand new search
|
||||
plugin], yielding [massive improvements in usability], but also in [speed
|
||||
and size] of the search index. Interestingly, as discussed in the previous
|
||||
blog article, we only scratched the surface of what's now possible. This
|
||||
release brings some useful features that enhance the writing experience,
|
||||
allowing for more fine-grained control of what pages, sections and blocks of a
|
||||
Markdown file should be indexed by the built-in search functionality.
|
||||
|
||||
<!-- more -->
|
||||
|
||||
_The following section discusses existing solutions for excluding pages and
|
||||
sections from the search index. If you immediately want to learn what's new,
|
||||
skip to the [section just after that][what's new]._
|
||||
|
||||
[brand new search plugin]: search-better-faster-smaller.md
|
||||
[massive improvements in usability]: search-better-faster-smaller.md#whats-new
|
||||
[speed and size]: search-better-faster-smaller.md#benchmarks
|
||||
[what's new]: #whats-new
|
||||
|
||||
## Prior art
|
||||
|
||||
MkDocs has a rich and thriving ecosystem of [plugins], and it comes as no
|
||||
surprise that there's already a fantastic plugin by @chrieke to exclude specific
|
||||
sections of a Markdown file – the [mkdocs-exclude-search] plugin. It can be
|
||||
installed with:
|
||||
|
||||
```
|
||||
pip install mkdocs-exclude-search
|
||||
```
|
||||
|
||||
__How it works__: the plugin post-processes the `search_index.json` file that
|
||||
is generated by the built-in search plugin, giving the author the ability to
|
||||
exclude certain pages and sections by adding a few lines of configuration to
|
||||
`mkdocs.yml`. An example:
|
||||
|
||||
``` yaml
|
||||
plugins:
|
||||
- search
|
||||
- exclude-search:
|
||||
exclude:
|
||||
- page.md
|
||||
- page.md#section
|
||||
- directory/*
|
||||
- /*/page.md
|
||||
```
|
||||
|
||||
It's easy to see that the plugin follows a configuration-centric approach, which
|
||||
adds support for advanced filtering techniques like infix- and suffix-filtering
|
||||
using wildcards. While this is a very powerful idea, it comes with some
|
||||
downsides:
|
||||
|
||||
1. __Exclusion patterns and content are not co-located__: exclusion patterns
|
||||
need to be defined in `mkdocs.yml`, and not as part of the respective
|
||||
document or section to be excluded. This might result in stale exclusion
|
||||
patterns, leading to unintended behavior:
|
||||
|
||||
- When a headline is changed, its slug (permalink) also changes, which might
|
||||
suddenly match (or unmatch) a pattern, e.g., when an author fixes a typo
|
||||
in a headline.
|
||||
|
||||
- As exclusion patterns support the use of wildcards, different authors
|
||||
might overwrite each other's patterns without any immediate feedback since
|
||||
the plugin does only report the number of excluded documents – not _what_
|
||||
has been excluded.[^1]
|
||||
|
||||
[^1]:
|
||||
When the log level is set to `DEBUG`, the plugin will report exactly which
|
||||
pages and sections have been excluded from the search index, but MkDocs will
|
||||
now flood the terminal with debug output from its core and other plugins.
|
||||
|
||||
2. __Exclusion control might be too coarse__: The [mkdocs-exclude-search]
|
||||
plugin only allows for the exclusion of pages and sections. It's not
|
||||
possible to exclude parts of a section, e.g., content that is irrelevant
|
||||
to search but must be included as part of the documentation.
|
||||
|
||||
[plugins]: https://github.com/mkdocs/mkdocs/wiki/MkDocs-Plugins
|
||||
[mkdocs-exclude-search]: https://github.com/chrieke/mkdocs-exclude-search
|
||||
|
||||
## What's new?
|
||||
|
||||
The latest Insiders release brings fine-grained control for [__excluding pages,
|
||||
sections, and blocks__][search exclusion] from the search index, implemented
|
||||
through front matter, as well as the [Attribute Lists]. Note that it doesn't
|
||||
replace the [mkdocs-exclude-search] plugin but __complements__ it.
|
||||
|
||||
[search exclusion]: ../../setup/setting-up-site-search.md#search-exclusion
|
||||
[Attribute Lists]: ../../setup/extensions/python-markdown.md#attribute-lists
|
||||
|
||||
### Excluding pages
|
||||
|
||||
An entire page can be excluded from the search index by adding a simple
|
||||
directive to the front matter of the respective Markdown file. The good thing
|
||||
is that the author now only has to check the top of the document to learn
|
||||
whether it is excluded or not:
|
||||
|
||||
``` yaml
|
||||
---
|
||||
search:
|
||||
exclude: true
|
||||
---
|
||||
|
||||
# Page title
|
||||
...
|
||||
```
|
||||
|
||||
### Excluding sections
|
||||
|
||||
If a section should be excluded, the author can use the [Attribute Lists]
|
||||
extension to add a __pragma__ called `data-search-exclude` at the end of a
|
||||
heading. The pragma is not included in the final HTML, as search pragmas are
|
||||
filtered by the search plugin before the page is rendered:
|
||||
|
||||
=== ":octicons-file-code-16: `docs/page.md`"
|
||||
|
||||
``` markdown
|
||||
# Page title
|
||||
|
||||
## Section 1
|
||||
|
||||
The content of this section is included
|
||||
|
||||
## Section 2 { data-search-exclude }
|
||||
|
||||
The content of this section is excluded
|
||||
```
|
||||
|
||||
=== ":octicons-codescan-16: `search_index.json`"
|
||||
|
||||
``` json
|
||||
{
|
||||
...
|
||||
"docs": [
|
||||
{
|
||||
"location":"page/",
|
||||
"text":"",
|
||||
"title":"Document title"
|
||||
},
|
||||
{
|
||||
"location":"page/#section-1",
|
||||
"text":"<p>The content of this section is included</p>",
|
||||
"title":"Section 1"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Excluding blocks
|
||||
|
||||
If even more fine-grained control is desired, the __pragma__ can be added to
|
||||
any [block-level element] or [inline-level element] that is officially
|
||||
supported by the [Attribute Lists] extension:
|
||||
|
||||
=== ":octicons-file-code-16: `docs/page.md`"
|
||||
|
||||
``` markdown
|
||||
# Page title
|
||||
|
||||
The content of this block is included
|
||||
|
||||
The content of this block is excluded
|
||||
{ data-search-exclude }
|
||||
```
|
||||
|
||||
=== ":octicons-codescan-16: `search_index.json`"
|
||||
|
||||
``` json
|
||||
{
|
||||
...
|
||||
"docs": [
|
||||
{
|
||||
"location":"page/",
|
||||
"text":"<p>The content of this block is included</p>",
|
||||
"title":"Document title"
|
||||
},
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
[block-level element]: https://python-markdown.github.io/extensions/attr_list/#block-level
|
||||
[inline-level element]: https://python-markdown.github.io/extensions/attr_list/#inline
|
||||
|
||||
## Conclusion
|
||||
|
||||
The latest release brings three simple ways to control more precisely what goes
|
||||
into the search index and what doesn't. It complements the already very powerful
|
||||
[mkdocs-exclude-search] plugin, allowing for new methods of shaping the
|
||||
structure, size and content of the search index.
|
||||
@@ -0,0 +1,100 @@
|
||||
---
|
||||
date: 2023-09-22
|
||||
authors: [squidfunk]
|
||||
categories:
|
||||
- Build
|
||||
- Performance
|
||||
links:
|
||||
- publishing-your-site.md#with-github-actions
|
||||
- creating-your-site.md#building-your-site
|
||||
---
|
||||
|
||||
# Using `git sparse-checkout` for faster documentation builds
|
||||
|
||||
__Leveraging `git sparse-checkout` in GitHub Actions enabled us to speed up
|
||||
documentation builds in our repository, cutting checkout times from 20 to 30
|
||||
seconds to just 2 seconds.__
|
||||
|
||||
Developing an efficient approach to build documentation in CI workflows is
|
||||
essential, especially when working in large repositories with thousands of
|
||||
commits, like ours. Of course, we want to build documentation quickly and
|
||||
efficiently, ensuring fast and productive workflows. When using both the
|
||||
wonderful [`git-committers`][git-committers] and [`git-revision-date-localized`]
|
||||
[git-revision-date-localized] plugins to display [document contributors] and
|
||||
[dates] at the bottom of each page, we are required to set `fetch-depth: 0`,
|
||||
which resulted in checkout times of 20 to 30 seconds on our repository. By
|
||||
leveraging [`git sparse-checkout`][git sparse-checkout] within [GitHub Actions],
|
||||
check out time was brought down to 2 seconds.
|
||||
|
||||
[git sparse-checkout]: https://git-scm.com/docs/git-sparse-checkout
|
||||
[GitHub Actions]: ../../publishing-your-site.md#with-github-actions
|
||||
[git-revision-date-localized]: https://github.com/timvink/mkdocs-git-revision-date-localized-plugin
|
||||
[git-committers]: https://github.com/ojacques/mkdocs-git-committers-plugin-2
|
||||
[document contributors]: ../../setup/adding-a-git-repository.md#document-contributors
|
||||
[dates]: ../../setup/adding-a-git-repository.md#document-dates
|
||||
|
||||
<!-- more -->
|
||||
|
||||
## A Primer
|
||||
|
||||
[`git sparse-checkout`][git sparse-checkout] allows you to check out only a
|
||||
subset of the files in a repository, making it incredibly useful for large
|
||||
repositories where a full checkout takes long and includes many files that are
|
||||
not relevant when building documentation.
|
||||
|
||||
## GitHub Actions
|
||||
|
||||
To enable [`git sparse-checkout`][git sparse-checkout] within [GitHub Actions]
|
||||
and ensure that you are only building the documentation that you need, add the
|
||||
following lines to your workflow file:
|
||||
|
||||
``` yaml
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
sparse-checkout: |
|
||||
docs
|
||||
includes
|
||||
```
|
||||
|
||||
[`git sparse-checkout`][git sparse-checkout] always checks out all files
|
||||
residing in the repository’s root. This means that regardless of the specified
|
||||
paths or directories for sparse checkout, the files located in the root of the
|
||||
repository will always be included in the checkout process.
|
||||
|
||||
Thus, you only need to specify the directories that are necessary for building
|
||||
documentation. In our case, we only need the `docs` and `includes` folders,
|
||||
but if you need additional directories, you can just add them to the end of the
|
||||
list. A complete example workflow for [GitHub Actions]:
|
||||
|
||||
``` yaml hl_lines="13-18"
|
||||
name: documentation
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- master
|
||||
- main
|
||||
permissions:
|
||||
contents: write
|
||||
jobs:
|
||||
deploy:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
sparse-checkout: |
|
||||
docs
|
||||
includes
|
||||
- uses: actions/setup-python@v4
|
||||
with:
|
||||
python-version: 3.x
|
||||
- run: pip install mkdocs-material
|
||||
- run: mkdocs gh-deploy --force
|
||||
```
|
||||
|
||||
## Conclusion
|
||||
|
||||
That's all there is! We're super happy with the results and hope that this will
|
||||
help you to speed up your documentation builds in [GitHub Actions] as well. As
|
||||
always, feel free to share your thoughts and experiences in the comments below.
|
||||
@@ -0,0 +1,164 @@
|
||||
---
|
||||
date: 2025-11-18
|
||||
authors:
|
||||
- squidfunk
|
||||
- alexvoss
|
||||
- katharinalisalin
|
||||
- pawamoy
|
||||
categories:
|
||||
- General
|
||||
icon: material/comment-off-outline
|
||||
description: >
|
||||
As Material for MkDocs entered maintenance mode and we're focusing our efforts on Zensical, our discussion board is now read-only
|
||||
---
|
||||
|
||||
# Goodbye, GitHub Discussions
|
||||
|
||||
__With the launch of [Zensical], we're evolving how we work together with the community to ensure long-term sustainability and professional support, fostering meaningful collaboration.__
|
||||
|
||||
The launch of Zensical [two weeks ago] marked a significant milestone, as it establishes a new foundation for our mission to build scalable Open Source systems for technical writing. Additionally, we're adopting a structured, transparent, and community-driven approach to guide the evolution of our projects – prioritizing what to build, and shaping Zensical's future together.
|
||||
|
||||
[Our new approach] distills a decade of experience maintaining Material for MkDocs, allowing us to professionalize and scale our efforts from a small team to a fully organized operation, while keeping the software free and open to everyone.
|
||||
|
||||
As Material for MkDocs has entered [maintenance mode], __our discussion board is now read-only__.
|
||||
|
||||
In this article, we reflect on the challenges we've navigated, the lessons we've learned, as these experiences have shaped our approach to community management and support that we're building with Zensical.
|
||||
|
||||
_You can subscribe to [our newsletter] to stay in the loop_.
|
||||
|
||||
<!-- more -->
|
||||
|
||||
[Our new approach]: https://zensical.org/docs/community/how-we-work/
|
||||
[maintenance mode]: https://github.com/squidfunk/mkdocs-material/issues/8523
|
||||
[Zensical]: https://zensical.org
|
||||
[two weeks ago]: zensical.md
|
||||
[our newsletter]: https://zensical.org/about/newsletter/
|
||||
|
||||
---
|
||||
|
||||
__This is the fourth and final article in a four-part series:__
|
||||
|
||||
1. [Transforming Material for MkDocs]
|
||||
2. [Zensical – A modern static site generator built by the creators of Material for MkDocs]
|
||||
3. [Material for MkDocs Insiders – Now free for everyone]
|
||||
4. [Goodbye, GitHub Discussions]
|
||||
|
||||
[Transforming Material for MkDocs]: transforming-material-for-mkdocs.md
|
||||
[Zensical – A modern static site generator built by the creators of Material for MkDocs]: zensical.md
|
||||
[Material for MkDocs Insiders – Now free for everyone]: insiders-now-free-for-everyone.md
|
||||
[Goodbye, GitHub Discussions]: goodbye-github-discussions.md
|
||||
|
||||
## GitHub Issues
|
||||
|
||||
Engagement with an Open Source project can take many forms – reporting bugs, submitting feature requests, seeking help, sharing ideas, participating in discussions, and contributing code. Each type of interaction requires a distinct approach to management.
|
||||
|
||||
Historically, we relied on our GitHub issue tracker to handle bug reports and feature requests. However, a significant portion of user inquiries – questions, support requests, and guidance – also found their way into the issue tracker. Over time, this blurred the lines between development work and support, making it increasingly difficult to maintain focus on core development tasks.
|
||||
|
||||
This is not unique to our project; many Open Source maintainers face similar challenges due to diverse skill levels and varying expectations within their communities. Clearly, Open Source projects needed a better solution to separate user support from development work.
|
||||
|
||||
## GitHub Discussions
|
||||
|
||||
In 2020, GitHub launched [GitHub Discussions] as a dedicated space for community interaction – a new channel to move support requests away from issue trackers. We immediately enabled discussions for Material for MkDocs, and redirected users to post questions and seek help.[^1]
|
||||
|
||||
[^1]:
|
||||
Before adopting GitHub Discussions, we maintained a Gitter channel, but ultimately chose to [consolidate our support efforts into a single platform]. While GitHub Discussions appeared promising, its mechanics proved ineffective.
|
||||
|
||||
[consolidate our support efforts into a single platform]: sunsetting-gitter.md
|
||||
|
||||
Then, as our community continued to expand – and as the vast majority of participants were seeking help rather than offering it – we launched the [Community Experts Program], giving those community members free access to Insiders for consistently providing high-quality assistance.
|
||||
|
||||
### Structural challenges
|
||||
|
||||
Despite these measures, we faced serveral structural challenges:
|
||||
|
||||
1. __Material for MkDocs? MkDocs? Same, same__ – All too often, we were perceived as the entry point for the entire ecosystem. As a result, many issues and discussions that actually belonged upstream – such as problems with Python Markdown, MkDocs plugins, or MkDocs itself – landed on our desk. This created additional triage work for redirecting issues.
|
||||
|
||||
2. __Yet another inbox__ – Although the discussion board was separate from the issue tracker, we ended up handling the majority of questions, effectively turning the discussion board into yet another inbox for us. Because the board is tied to the repository, unanswered discussions could be perceived as the project being unmaintained, putting pressure on us to respond.
|
||||
|
||||
3. __A notification for everything__ – GitHub's notification system contributed to this problem. Any interaction within an issue or a discussion subscribes you to the entire thread, resulting in notifications for every comment, including: __+1__ – taking 2 seconds to type and submit, delivering a notification to the inbox of all other participants.
|
||||
|
||||
4. __GitHub Discussions is a chat__ – Many users began treating GitHub Discussions more like a chat platform than a structured knowledge base. This created notification fatigue, reinforcing a pattern in which the community implicitly relied on maintainers to handle support requests. Everybody got a message, so the assumption was that things were taken care of – right?
|
||||
|
||||
5. __Bug reports on our discussion board__ – From time to time, our [contribution guidelines] were ignored, leading to spurious bug reports on our discussion board instead of our issue tracker – "search doesn't work" is not an actionable bug report. However, when users followed our [bug reporting guidelines], we were able to fix most bugs within 24-48 hours.
|
||||
|
||||
[Community Experts Program]: https://github.com/squidfunk/mkdocs-material/discussions/6363
|
||||
[GitHub Discussions]: https://github.com/features/discussions
|
||||
[contribution guidelines]: https://squidfunk.github.io/mkdocs-material/contributing/
|
||||
[bug reporting guidelines]: https://squidfunk.github.io/mkdocs-material/contributing/reporting-a-bug/
|
||||
|
||||
### Addressing these challenges
|
||||
|
||||
We actively addressed these challenges in various ways over time: by introducing carefully crafted issue templates on our issue tracker and creating detailed step-by-step guides on how to report bugs, request changes, and create pull requests effectively.
|
||||
|
||||
Despite these efforts, success was limited. The underlying problem was that GitHub Discussions, while a step in the right direction, was not designed to handle the scale and diversity of our community. It lacked the structure and features necessary to support a large, active user base seeking help, making it impossible to fully resolve the challenges we faced. We believe that this is not unique to our project, but rather a systemic issue with GitHub Discussions, and that many Open Source maintainers encounter similar difficulties.
|
||||
|
||||
The [Community Experts Program] helped reduce some of the support burden, but it could not fully address the structural limitations of using GitHub Discussions for user support.
|
||||
|
||||
## Discussions now read-only
|
||||
|
||||
With Material for MkDocs in [maintenance mode], __our GitHub Discussion board is now read-only__ – it remains accessible but will no longer accept new discussions or comments. All existing answers stay in place, ensuring that users can still find solutions to previously solved problems.
|
||||
|
||||
!!! warning "Material for MkDocs is now in maintenance mode"
|
||||
|
||||
We want to be transparent about the risks of staying on Material for MkDocs. With [MkDocs unmaintained] and facing fundamental supply chain concerns, we cannot guarantee Material for MkDocs will continue working reliably in the future. We're aware that transitioning takes time, which is why we commit to support it at least for the next 12 months, fixing critical bugs and security vulnerabilities as needed, but the path forward is with Zensical.
|
||||
|
||||
If documentation plays a critical role in your organization, and you're worried how this might affect your business, consider joining [Zensical Spark](https://zensical.org/spark/), or feel free to schedule a call by reaching out at hello@zensical.org.
|
||||
|
||||
[MkDocs unmaintained]: https://github.com/squidfunk/mkdocs-material/discussions/8461
|
||||
|
||||
### What this means for you
|
||||
|
||||
For issues and questions related to the broader MkDocs ecosystem, we encourage users to ask for assistance on the issue trackers and discussion boards of the respective upstream projects:
|
||||
|
||||
__MkDocs & Python Markdown__
|
||||
|
||||
- [MkDocs] – [Issues][MkDocs issues], [Discussions][MkDocs discussions]
|
||||
- [Python Markdown] – [Issues][Python Markdown issues]
|
||||
- [Pymdown Extensions] – [Issues][Pymdown Extensions issues], [Discussions][Pymdown Extensions discussions]
|
||||
|
||||
__Popular MkDocs plugins__
|
||||
|
||||
- [Mike] – [Issues][Mike issues], [Discussions][Mike discussions]
|
||||
- [Macros] – [Issues][Macros issues], [Discussions][Macros discussions]
|
||||
- [Git Revision Date Localized] – [Issues][Git Revision Date Localized issues]
|
||||
- [Awesome Nav] – [Issues][Awesome Nav issues]
|
||||
- [Static i18n] – [Issues][Static i18n issues]
|
||||
- [MkDocs Literate Nav] – [Issues][MkDocs Literate Nav issues]
|
||||
|
||||
_For all other MkDocs plugins and Markdown extensions, please consult the [MkDocs catalog], and reach out to the maintainers of the respective projects._
|
||||
|
||||
_Join our [Discord] to become part of Zensical's growing community!_
|
||||
|
||||
## Looking ahead
|
||||
|
||||
This article concludes our four-part series reflecting on the journey of maintaining Material for MkDocs and how it has paved the way for Zensical – our next chapter.
|
||||
|
||||
[Our new approach] to community management and support is built directly on the lessons we've learned over the years. We are excited to begin this next chapter together with you and look forward to cultivating a vibrant, engaged, and sustainable community together.
|
||||
|
||||
_You can subscribe to [our newsletter] to stay in the loop_.
|
||||
|
||||
[MkDocs]: https://github.com/mkdocs/mkdocs
|
||||
[MkDocs issues]: https://github.com/mkdocs/mkdocs/issues
|
||||
[MkDocs discussions]: https://github.com/mkdocs/mkdocs/discussions
|
||||
[Pymdown Extensions]: https://github.com/facelessuser/pymdown-extensions
|
||||
[Pymdown Extensions discussions]: https://github.com/facelessuser/pymdown-extensions/discussions
|
||||
[Pymdown Extensions issues]: https://github.com/facelessuser/pymdown-extensions/issues
|
||||
[Python Markdown]: https://github.com/Python-Markdown/markdown
|
||||
[Python Markdown issues]: https://github.com/Python-Markdown/markdown/issues
|
||||
[Mike]: https://github.com/jimporter/mike
|
||||
[Mike issues]: https://github.com/jimporter/mike/issues
|
||||
[Mike discussions]: https://github.com/jimporter/mike/discussions
|
||||
[Macros]: https://github.com/fralau/mkdocs-macros-plugin
|
||||
[Macros issues]: https://github.com/fralau/mkdocs-macros-plugin/issues
|
||||
[Macros discussions]: https://github.com/fralau/mkdocs-macros-plugin/discussions
|
||||
[Git Revision Date Localized]: https://github.com/timvink/mkdocs-git-revision-date-localized-plugin
|
||||
[Git Revision Date Localized issues]: https://github.com/timvink/mkdocs-git-revision-date-localized-plugin/issues
|
||||
[Awesome Nav]: https://github.com/lukasgeiter/mkdocs-awesome-nav
|
||||
[Awesome Nav issues]: https://github.com/lukasgeiter/mkdocs-awesome-nav/issues
|
||||
[Static i18n]: https://github.com/ultrabug/mkdocs-static-i18n
|
||||
[Static i18n issues]: https://github.com/ultrabug/mkdocs-static-i18n/issues
|
||||
[Static i18n discussions]: https://github.com/ultrabug/mkdocs-static-i18n/discussions
|
||||
[MkDocs Literate Nav]: https://github.com/oprypin/mkdocs-literate-nav
|
||||
[MkDocs Literate Nav issues]: https://github.com/oprypin/mkdocs-literate-nav/issues
|
||||
[MkDocs catalog]: https://github.com/mkdocs/catalog
|
||||
[Discord]: https://discord.gg/hqXRNq9CjT
|
||||
@@ -0,0 +1,205 @@
|
||||
---
|
||||
date: 2025-11-11
|
||||
authors:
|
||||
- squidfunk
|
||||
- alexvoss
|
||||
- katharinalisalin
|
||||
- pawamoy
|
||||
categories:
|
||||
- General
|
||||
icon: material/lock-open
|
||||
description: >
|
||||
We just released 9.7.0 – the final version of Material for MkDocs, which includes all features that were previously exclusive to sponsors
|
||||
title: Insiders – Now free for everyone
|
||||
slug: insiders-now-free-for-everyone
|
||||
---
|
||||
|
||||
# Material for MkDocs Insiders – Now free for everyone
|
||||
|
||||
__[9.7.0], the final version of Material for MkDocs, includes all features that were previously exclusive to sponsors, making Material for MkDocs Insiders available to everyone!__
|
||||
|
||||
As we're shifting our efforts to [Zensical], Material for MkDocs is entering [maintenance mode]. This means that while we'll continue to fix critical bugs and security issues for 12 month at least, no new features will be added to Material for MkDocs.
|
||||
|
||||
We're also discontinuing our sponsorware model, saying [goodbye to GitHub Sponsors]. If you were a sponsor of our work, you already received an email mentioning that your sponsorship was cancelled. As one of the numerous individuals and organizations sponsoring Material for MkDocs over the past years – thank you! Your continued support has been invaluable.
|
||||
|
||||
Now, we want everyone to benefit from all features we have developed for Material for MkDocs, which is why we're making all Insiders features available to everyone!
|
||||
|
||||
This is the logical next step in our journey as we focus on Zensical – our next-generation static site generator built from the ground up to overcome MkDocs' technical limitations. Zensical is fully [Open Source, licensed under MIT], maintains [compatibility with Material for MkDocs], and can build your existing projects with minimal changes.
|
||||
|
||||
In the coming months, we'll close the [feature parity] gap, bringing the expressiveness of Material for MkDocs to Zensical.
|
||||
|
||||
_You can subscribe to [our newsletter] to stay in the loop_.
|
||||
|
||||
<!-- more -->
|
||||
|
||||
[9.7.0]: ../../changelog/index.md#9.7.0
|
||||
[Zensical]: https://zensical.org
|
||||
[maintenance mode]: https://github.com/squidfunk/mkdocs-material/issues/8523
|
||||
[goodbye to GitHub Sponsors]: zensical.md#goodbye-github-sponsors
|
||||
[compatibility with Material for MkDocs]: zensical.md#maximum-compatibility
|
||||
[Open Source, licensed under MIT]: https://zensical.org/about/license/
|
||||
[feature parity]: https://zensical.org/compatibility/features/
|
||||
[our newsletter]: https://zensical.org/about/newsletter/
|
||||
|
||||
---
|
||||
|
||||
__This is the third article in a four-part series:__
|
||||
|
||||
1. [Transforming Material for MkDocs]
|
||||
2. [Zensical – A modern static site generator built by the creators of Material for MkDocs]
|
||||
3. [Material for MkDocs Insiders – Now free for everyone]
|
||||
4. [Goodbye, GitHub Discussions]
|
||||
|
||||
[Transforming Material for MkDocs]: transforming-material-for-mkdocs.md
|
||||
[Zensical – A modern static site generator built by the creators of Material for MkDocs]: zensical.md
|
||||
[Material for MkDocs Insiders – Now free for everyone]: insiders-now-free-for-everyone.md
|
||||
[Goodbye, GitHub Discussions]: goodbye-github-discussions.md
|
||||
|
||||
## Available features
|
||||
|
||||
Our sponsors have enjoyed exclusive access to the following premium features
|
||||
for quite some time. With the release of [9.7.0], all these features are now available to everyone:
|
||||
|
||||
<div class="mdx-columns" markdown>
|
||||
|
||||
- [x] [Blog plugin: pinned posts]
|
||||
- [x] [Instant previews]
|
||||
- [x] [Footnote tooltips]
|
||||
- [x] [Tags plugin: advanced settings]
|
||||
- [x] [Tags plugin: nested tags]
|
||||
- [x] [Tags plugin: shadow tags]
|
||||
- [x] [Stay on page when switching languages]
|
||||
- [x] [Blog plugin: author profiles]
|
||||
- [x] [Blog plugin: advanced settings]
|
||||
- [x] [Projects plugin]
|
||||
- [x] [Instant prefetching]
|
||||
- [x] [Social plugin: custom layouts]
|
||||
- [x] [Social plugin: background images]
|
||||
- [x] [Code range selection]
|
||||
- [x] [Code annotations: custom selectors]
|
||||
- [x] [Privacy plugin: advanced settings]
|
||||
- [x] [Optimize plugin]
|
||||
- [x] [Navigation path] (Breadcrumbs)
|
||||
- [x] [Typeset plugin]
|
||||
- [x] [Privacy plugin: external links]
|
||||
|
||||
</div>
|
||||
|
||||
[Optimize plugin]: ../../plugins/optimize.md
|
||||
[Navigation path]: ../../setup/setting-up-navigation.md#navigation-path
|
||||
[Blog plugin: advanced settings]: ../../setup/setting-up-a-blog.md#advanced-settings
|
||||
[Blog plugin: author profiles]: ../../setup/setting-up-a-blog.md#adding-author-profiles
|
||||
[Blog plugin: pinned posts]: ../../setup/setting-up-a-blog.md#pinning-a-post
|
||||
[Instant prefetching]: ../../setup/setting-up-navigation.md#instant-prefetching
|
||||
[Typeset plugin]: ../../plugins/typeset.md
|
||||
[Footnote tooltips]: ../../reference/footnotes.md#footnote-tooltips
|
||||
[Privacy plugin: external links]: ../../plugins/privacy.md#external-links
|
||||
[Privacy plugin: advanced settings]: ../../setup/ensuring-data-privacy.md#advanced-settings
|
||||
[Instant previews]: ../../setup/setting-up-navigation.md#instant-previews
|
||||
[Social plugin: custom layouts]: ../../setup/setting-up-social-cards.md#customization
|
||||
[Social plugin: background images]: ../../plugins/social.md#option.background_image
|
||||
[Code range selection]: ../../reference/code-blocks.md#code-selection-button
|
||||
[Code annotations: custom selectors]: ../../reference/code-blocks.md#custom-selectors
|
||||
[Stay on page when switching languages]: ../../setup/changing-the-language.md#stay-on-page
|
||||
[Projects plugin]: ../../plugins/projects.md
|
||||
[Tags plugin: nested tags]: ../../setup/setting-up-tags.md#nested-tags
|
||||
[Tags plugin: shadow tags]: ../../setup/setting-up-tags.md#shadow-tags
|
||||
[Tags plugin: advanced settings]: ../../setup/setting-up-tags.md#advanced-settings
|
||||
|
||||
!!! tip "[mkdocstrings Insiders is now free] as well"
|
||||
|
||||
With [Timothée joining the Zensical team], he announced that all features previously reserved to his sponsors as part of [mkdocstrings] Insiders are now free for everyone as well!
|
||||
|
||||
[mkdocstrings Insiders is now free]: https://pawamoy.github.io/posts/sunsetting-the-sponsorware-strategy/
|
||||
[Timothée joining the Zensical team]: zensical.md#were-growing-our-team
|
||||
[mkdocstrings]: https://mkdocstrings.github.io/
|
||||
|
||||
## How to upgrade
|
||||
|
||||
You can upgrade with the following command:
|
||||
|
||||
```
|
||||
pip install --upgrade --force-reinstall mkdocs-material
|
||||
```
|
||||
|
||||
## Switching from Insiders
|
||||
|
||||
If you've been a user of Insiders, we recommend to switch to the community edition as soon as possible, as it includes all Insiders features. This will make it much easier to handle third-party contributions, since no personal access tokens are necessary.
|
||||
|
||||
__From now on, bug fixes that we make to Material for MkDocs will only be released to the community edition. Security vulnerabilities will be fixed in both editions.__
|
||||
|
||||
Thus, please adjust your `requirements.txt` and GitHub Actions workflows:
|
||||
|
||||
```diff
|
||||
- pip install git+https://${GH_TOKEN}@github.com/squidfunk/mkdocs-material-insiders. git
|
||||
+ pip install mkdocs-material
|
||||
|
||||
```
|
||||
|
||||
The Insiders repository itself will remain available for the next 6 months. When you build your project with Insiders, it will now show an informational message pointing to this blog post.
|
||||
|
||||
- __On February 1, 2026, this message will be turned into a warning__.
|
||||
- __On May 1, 2026, the Insiders repository will be deleted__.
|
||||
|
||||
## Sunsetting preparation
|
||||
|
||||
Entering [maintenance mode], we're preparing Material for MkDocs for sunsetting.
|
||||
|
||||
!!! warning "Material for MkDocs is now in maintenance mode"
|
||||
|
||||
We want to be transparent about the risks of staying on Material for MkDocs. With [MkDocs unmaintained] and facing fundamental supply chain concerns, we cannot guarantee Material for MkDocs will continue working reliably in the future. We're aware that transitioning takes time, which is why we commit to support it at least for the next 12 months, fixing critical bugs and security vulnerabilities as needed, but the path forward is with Zensical.
|
||||
|
||||
If documentation plays a critical role in your organization, and you're worried how this might affect your business, consider joining [Zensical Spark](https://zensical.org/spark/), or feel free to schedule a call by reaching out at hello@zensical.org.
|
||||
|
||||
[MkDocs unmaintained]: https://github.com/squidfunk/mkdocs-material/discussions/8461
|
||||
|
||||
### Deprecations
|
||||
|
||||
While we release all features to the general public, at the same time, we're deprecating the [Projects plugin] and the [Typeset plugin] due to maintainability issues. This means that these plugins will not receive any further updates, including no more bug fixes.
|
||||
|
||||
The reason for this decision is that both plugins rely on too many workarounds to make them work with MkDocs, and subsequently have been key motivators to create [Zensical]. If you rely on these plugins, and they work for your use case, you can of course continue to use them.
|
||||
|
||||
__With Zensical, we'll be shipping proper [sub-project support], including [internationalization] and [versioning], designing these features together with our professional users in [Zensical Spark].__
|
||||
|
||||
[Zensical Spark]: https://zensical.org/spark/
|
||||
[sub-project support]: https://zensical.org/about/roadmap/#subprojects
|
||||
[internationalization]: https://zensical.org/about/roadmap/#internationalization
|
||||
[versioning]: https://zensical.org/about/roadmap/#versioning
|
||||
|
||||
### Version ranges
|
||||
|
||||
Material for MkDocs has used semver version ranges for dependencies to ensure compatibility. With the advent of [9.7.0], we're switching from semver to minimal version ranges. This provides more flexibility in dependency resolution, specifically to allow users to use newer versions of dependencies that include important bug fixes or security patches.
|
||||
|
||||
### Security
|
||||
|
||||
We will not transfer ownership of the Material for MkDocs repository to another individual or organization. The repository and PyPI package will remain under the ownership of @squidfunk, which preserves the trusted supply chain our users depend on.
|
||||
|
||||
Thus, if you wish to take on maintenance of Material for MkDocs, please create a fork.
|
||||
|
||||
## Looking ahead
|
||||
|
||||
### Achieving sustainability
|
||||
|
||||
Where Material for MkDocs relied on sponsorware, Zensical takes a new approach,
|
||||
to ensure it evolves to meet the needs of organizations building complex, enterprise-scale documentation.
|
||||
|
||||
[Zensical Spark] is a collaborative space where professional users have a direct voice in shaping Zensical's future. Through a [structured design process] and together with our Zensical Spark members, we identify opportunities, validate proposals, and define priorities – turning their real-world documentation challenges into features that benefit the entire community.
|
||||
|
||||
Reach out at hello@zensical.org to schedule a call to learn more about Zensical Spark, discuss your organization's needs, and how it helps us to make Zensical sustainable.
|
||||
|
||||
[Zensical Spark]: https://zensical.org/spark/
|
||||
[structured design process]: https://zensical.org/docs/community/how-we-work/
|
||||
|
||||
### Our commitment to you
|
||||
|
||||
If you're currently using Material for MkDocs, there's no need to rush. We're committed to keeping it secure and functional for the next 12 months while we focus our efforts on [Zensical].
|
||||
|
||||
The [9.7.0] release marks a significant shift – every Insiders feature is now available to everyone, with no sponsorship required. As we build [Zensical], each of these features will be rearchitected and improved. Zensical is entirely free and Open Source, ensuring the entire community benefits from our work without barriers.
|
||||
|
||||
When you're ready to explore what's next, [Zensical is compatible with Material for MkDocs] and designed to be a natural evolution of the ideas and principles you already know.
|
||||
|
||||
_If you loved Material for MkDocs and are excited about Zensical, we'll be providing new methods to support our work in the coming months, with the possibility of getting exclusive goodies._
|
||||
|
||||
_Subscribe to [our newsletter] to stay in the loop._
|
||||
|
||||
[Zensical is compatible with Material for MkDocs]: zensical.md#maximum-compatibility
|
||||
@@ -0,0 +1,629 @@
|
||||
---
|
||||
date: 2021-09-13
|
||||
authors: [squidfunk]
|
||||
readtime: 15
|
||||
description: >
|
||||
How we rebuilt client-side search, delivering a better user experience while
|
||||
making it faster and smaller at the same time
|
||||
categories:
|
||||
- Search
|
||||
- Performance
|
||||
links:
|
||||
- plugins/search.md
|
||||
---
|
||||
|
||||
# Search: better, faster, smaller
|
||||
|
||||
__This is the story of how we managed to completely rebuild client-side search,
|
||||
delivering a significantly better user experience while making it faster and
|
||||
smaller at the same time.__
|
||||
|
||||
The [search] of Material for MkDocs is by far one of its best and most-loved
|
||||
assets: [multilingual], [offline-capable], and most importantly: _all
|
||||
client-side_. It provides a solution to empower the users of your documentation
|
||||
to find what they're searching for instantly without the headache of managing
|
||||
additional servers. However, even though several iterations have been made,
|
||||
there's still some room for improvement, which is why we rebuilt the search
|
||||
plugin and integration from the ground up. This article shines some light on the
|
||||
internals of the new search, why it's much more powerful than the previous
|
||||
version, and what's about to come.
|
||||
|
||||
<!-- more -->
|
||||
|
||||
_The next section discusses the architecture and issues of the current search
|
||||
implementation. If you immediately want to learn what's new, skip to the
|
||||
[section just after that][what's new]._
|
||||
|
||||
[search]: ../../setup/setting-up-site-search.md
|
||||
[multilingual]: ../../plugins/search.md#config.lang
|
||||
[offline-capable]: ../../setup/building-for-offline-usage.md
|
||||
[what's new]: #whats-new
|
||||
|
||||
## Architecture
|
||||
|
||||
Material for MkDocs uses [lunr] together with [lunr-languages] to implement
|
||||
its client-side search capabilities. When a documentation page is loaded and
|
||||
JavaScript is available, the search index as generated by the
|
||||
[built-in search plugin] during the build process is requested from the
|
||||
server:
|
||||
|
||||
``` ts
|
||||
const index$ = document.forms.namedItem("search")
|
||||
? __search?.index || requestJSON<SearchIndex>(
|
||||
new URL("search/search_index.json", config.base)
|
||||
)
|
||||
: NEVER
|
||||
```
|
||||
|
||||
[lunr]: https://lunrjs.com
|
||||
[lunr-languages]: https://github.com/MihaiValentin/lunr-languages
|
||||
[built-in search plugin]: ../../plugins/search.md
|
||||
|
||||
### Search index
|
||||
|
||||
The search index includes a stripped-down version of all pages. Let's take a
|
||||
look at an example to understand precisely what the search index contains from
|
||||
the original Markdown file:
|
||||
|
||||
??? example "Expand to inspect example"
|
||||
|
||||
=== ":octicons-file-code-16: `docs/page.md`"
|
||||
|
||||
```` markdown
|
||||
# Example
|
||||
|
||||
## Text
|
||||
|
||||
It's very easy to make some words **bold** and other words *italic*
|
||||
with Markdown. You can even add [links](#), or even `code`:
|
||||
|
||||
```
|
||||
if (isAwesome) {
|
||||
return true
|
||||
}
|
||||
```
|
||||
|
||||
## Lists
|
||||
|
||||
Sometimes you want numbered lists:
|
||||
|
||||
1. One
|
||||
2. Two
|
||||
3. Three
|
||||
|
||||
Sometimes you want bullet points:
|
||||
|
||||
* Start a line with a star
|
||||
* Profit!
|
||||
````
|
||||
|
||||
=== ":octicons-codescan-16: `search_index.json`"
|
||||
|
||||
``` json
|
||||
{
|
||||
"config": {
|
||||
"indexing": "full",
|
||||
"lang": [
|
||||
"en"
|
||||
],
|
||||
"min_search_length": 3,
|
||||
"prebuild_index": false,
|
||||
"separator": "[\\s\\-]+"
|
||||
},
|
||||
"docs": [
|
||||
{
|
||||
"location": "page/",
|
||||
"title": "Example",
|
||||
"text": "Example Text It's very easy to make some words bold and other words italic with Markdown. You can even add links , or even code : if (isAwesome) { return true } Lists Sometimes you want numbered lists: One Two Three Sometimes you want bullet points: Start a line with a star Profit!"
|
||||
},
|
||||
{
|
||||
"location": "page/#example",
|
||||
"title": "Example",
|
||||
"text": ""
|
||||
},
|
||||
{
|
||||
"location": "page/#text",
|
||||
"title": "Text",
|
||||
"text": "It's very easy to make some words bold and other words italic with Markdown. You can even add links , or even code : if (isAwesome) { return true }"
|
||||
},
|
||||
{
|
||||
"location": "page/#lists",
|
||||
"title": "Lists",
|
||||
"text": "Sometimes you want numbered lists: One Two Three Sometimes you want bullet points: Start a line with a star Profit!"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
If we inspect the search index, we immediately see several problems:
|
||||
|
||||
1. __All content is included twice__: the search index contains one entry
|
||||
with the entire contents of the page, and one entry for each section of
|
||||
the page, i.e., each block preceded by a headline or subheadline. This
|
||||
significantly contributes to the size of the search index.
|
||||
|
||||
2. __All structure is lost__: when the search index is built, all structural
|
||||
information like HTML tags and attributes are stripped from the content.
|
||||
While this approach works well for paragraphs and inline formatting, it
|
||||
might be problematic for lists and code blocks. An excerpt:
|
||||
|
||||
```
|
||||
… links , or even code : if (isAwesome) { … } Lists Sometimes you want …
|
||||
```
|
||||
|
||||
- __Context__: for an untrained eye, the result can look like gibberish, as
|
||||
it's not immediately apparent what classifies as text and what as code.
|
||||
Furthermore, it's not clear that `Lists` is a headline as it's merged
|
||||
with the code block before and the paragraph after it.
|
||||
|
||||
- __Punctuation__: inline elements like links that are immediately followed
|
||||
by punctuation are separated by whitespace (see `,` and `:` in the
|
||||
excerpt). This is because all extracted text is joined with a whitespace
|
||||
character during the construction of the search index.
|
||||
|
||||
It's not difficult to see that it can be quite challenging to implement a good
|
||||
search experience for theme authors, which is why Material for MkDocs (up to
|
||||
now) did some [monkey patching] to be able to render slightly more
|
||||
meaningful search previews.
|
||||
|
||||
[monkey patching]: https://github.com/squidfunk/mkdocs-material/blob/ec7ccd2b2d15dd033740f388912f7be7738feec2/src/assets/javascripts/integrations/search/document/index.ts#L68-L71
|
||||
|
||||
### Search worker
|
||||
|
||||
The actual search functionality is implemented as part of a web worker[^1],
|
||||
which creates and manages the [lunr] search index. When search is initialized,
|
||||
the following steps are taken:
|
||||
|
||||
[^1]:
|
||||
Prior to <!-- md:version 5.0.0 -->, search was carried out in the main
|
||||
thread which locked up the browser, rendering it unusable. This problem was
|
||||
first reported in #904 and, after some back and forth, fixed and released in
|
||||
<!-- md:version 5.0.0 -->.
|
||||
|
||||
1. __Linking sections with pages__: The search index is parsed, and each
|
||||
section is linked to its parent page. The parent page itself is _not
|
||||
indexed_, as it would lead to duplicate results, so only the sections
|
||||
remain. Linking is necessary, as search results are grouped by page.
|
||||
|
||||
2. __Tokenization__: The `title` and `text` values of each section are split
|
||||
into tokens by using the [`separator`][separator] as configured in
|
||||
`mkdocs.yml`. Tokenization itself is carried out by
|
||||
[lunr's default tokenizer][default tokenizer], which doesn't allow for
|
||||
lookahead or separators spanning multiple characters.
|
||||
|
||||
> Why is this important and a big deal? We will see later how much more we
|
||||
> can achieve with a tokenizer that is capable of separating strings with
|
||||
> lookahead.
|
||||
|
||||
3. __Indexing__: As a final step, each section is indexed. When querying the
|
||||
index, if a search query includes one of the tokens as returned by step 2.,
|
||||
the section is considered to be part of the search result and passed to the
|
||||
main thread.
|
||||
|
||||
Now, that's basically how the search worker operates. Sure, there's a little
|
||||
more magic involved, e.g., search results are [post-processed] and [rescored] to
|
||||
account for some shortcomings of [lunr], but in general, this is how data gets
|
||||
into and out of the index.
|
||||
|
||||
[separator]: ../../plugins/search.md#config.separator
|
||||
[default tokenizer]: https://github.com/olivernn/lunr.js/blob/aa5a878f62a6bba1e8e5b95714899e17e8150b38/lunr.js#L413-L456
|
||||
[post-processed]: https://github.com/squidfunk/mkdocs-material/blob/ec7ccd2b2d15dd033740f388912f7be7738feec2/src/assets/javascripts/integrations/search/_/index.ts#L249-L272
|
||||
[rescored]: https://github.com/squidfunk/mkdocs-material/blob/ec7ccd2b2d15dd033740f388912f7be7738feec2/src/assets/javascripts/integrations/search/_/index.ts#L274-L275
|
||||
|
||||
### Search previews
|
||||
|
||||
Users should be able to quickly scan and evaluate the relevance of a search
|
||||
result in the given context, which is why a concise summary with highlighted
|
||||
occurrences of the search terms found is an essential part of a great search
|
||||
experience.
|
||||
|
||||
This is where the current search preview generation falls short, as some of the
|
||||
search previews appear not to include any occurrence of any of the search
|
||||
terms. This was due to the fact that search previews were [truncated after a
|
||||
maximum of 320 characters][truncated], as can be seen here:
|
||||
|
||||
<figure markdown>
|
||||
|
||||
![search preview]
|
||||
|
||||
<figcaption markdown>
|
||||
|
||||
The first two results look like they're not relevant, as they don't seem to
|
||||
include the query string the user just searched for. Yet, they are.
|
||||
|
||||
</figcaption>
|
||||
</figure>
|
||||
|
||||
A better solution to this problem has been on the roadmap for a very, very long
|
||||
time, but in order to solve this once and for all, several factors need to be
|
||||
carefully considered:
|
||||
|
||||
1. __Word boundaries__: some themes[^2] for static site generators generate
|
||||
search previews by expanding the text left and right next to an occurrence,
|
||||
stopping at a whitespace character when enough words have been consumed. A
|
||||
preview might look like this:
|
||||
|
||||
```
|
||||
… channels, e.g., or which can be configured via mkdocs.yml …
|
||||
```
|
||||
|
||||
While this may work for languages that use whitespace as a separator
|
||||
between words, it breaks down for languages like Japanese or Chinese[^3],
|
||||
as they have non-whitespace word boundaries and use dedicated segmenters to
|
||||
split strings into tokens.
|
||||
|
||||
[^2]:
|
||||
At the time of writing, [Just the Docs] and [Docusaurus] use this method
|
||||
for generating search previews. Note that the latter also integrates with
|
||||
Algolia, which is a fully managed server-based solution.
|
||||
|
||||
[^3]:
|
||||
China and Japan are both within the top 5 countries of origin of users of
|
||||
Material for MkDocs.
|
||||
|
||||
[truncated]: https://github.com/squidfunk/mkdocs-material/blob/master/src/templates/assets/javascripts/templates/search/index.tsx#L90
|
||||
[search preview]: search-better-faster-smaller/search-preview.png
|
||||
[Just the Docs]: https://just-the-docs.com
|
||||
[Docusaurus]: https://github.com/lelouch77/docusaurus-lunr-search
|
||||
|
||||
1. __Context-awareness__: Although whitespace doesn't work for all languages,
|
||||
one could argue that it could be a good enough solution. Unfortunately, this
|
||||
is not necessarily true for code blocks, as the removal of whitespace might
|
||||
change meaning in some languages.
|
||||
|
||||
3. __Structure__: Preserving structural information is not a must, but
|
||||
apparently beneficial to build more meaningful search previews which allow
|
||||
for a quick evaluation of relevance. If a word occurrence is part of a code
|
||||
block, it should be rendered as a code block.
|
||||
|
||||
## What's new?
|
||||
|
||||
After we built a solid understanding of the problem space and before we dive
|
||||
into the internals of our new search implementation to see which of the
|
||||
problems it already solves, a quick overview of what features and improvements
|
||||
it brings:
|
||||
|
||||
- __Better__: support for [rich search previews], preserving the structural
|
||||
information of code blocks, inline code, and lists, so they are rendered
|
||||
as-is, as well as [lookahead tokenization], [more accurate highlighting], and
|
||||
improved stability of typeahead. Also, a [slightly better UX].
|
||||
- __Faster__ and __smaller__: significant decrease in search index size of up
|
||||
to 48% due to improved extraction and construction techniques, resulting in a
|
||||
search experience that is up to 95% faster, which is particularly helpful for
|
||||
large documentation projects.
|
||||
|
||||
[rich search previews]: #rich-search-previews
|
||||
[lookahead tokenization]: #tokenizer-lookahead
|
||||
[more accurate highlighting]: #accurate-highlighting
|
||||
[slightly better UX]: #user-interface
|
||||
|
||||
### Rich search previews
|
||||
|
||||
As we rebuilt the search plugin from scratch, we reworked the construction of
|
||||
the search index to preserve the structural information of code blocks, inline
|
||||
code, as well as unordered and ordered lists. Using the example from the
|
||||
[search index] section, here's how it looks:
|
||||
|
||||
=== "Now"
|
||||
|
||||
![search preview now]
|
||||
|
||||
=== "Before"
|
||||
|
||||
![search preview before]
|
||||
|
||||
Now, __code blocks are first-class citizens of search previews__, and even
|
||||
inline code formatting is preserved. Let's take a look at the new structure of
|
||||
the search index to understand why:
|
||||
|
||||
??? example "Expand to inspect search index"
|
||||
|
||||
=== "Now"
|
||||
|
||||
``` json
|
||||
{
|
||||
...
|
||||
"docs": [
|
||||
{
|
||||
"location": "page/",
|
||||
"title": "Example",
|
||||
"text": ""
|
||||
},
|
||||
{
|
||||
"location": "page/#text",
|
||||
"title": "Text",
|
||||
"text": "<p>It's very easy to make some words bold and other words italic with Markdown. You can even add links, or even <code>code</code>:</p> <pre><code>if (isAwesome){\n return true\n}\n</code></pre>"
|
||||
},
|
||||
{
|
||||
"location": "page/#lists",
|
||||
"title": "Lists",
|
||||
"text": "<p>Sometimes you want numbered lists:</p> <ol> <li>One</li> <li>Two</li> <li>Three</li> </ol> <p>Sometimes you want bullet points:</p> <ul> <li>Start a line with a star</li> <li>Profit!</li> </ul>"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
=== "Before"
|
||||
|
||||
``` json
|
||||
{
|
||||
...
|
||||
"docs": [
|
||||
{
|
||||
"location": "page/",
|
||||
"title": "Example",
|
||||
"text": "Example Text It's very easy to make some words bold and other words italic with Markdown. You can even add links , or even code : if (isAwesome) { return true } Lists Sometimes you want numbered lists: One Two Three Sometimes you want bullet points: Start a line with a star Profit!"
|
||||
},
|
||||
{
|
||||
"location": "page/#example",
|
||||
"title": "Example",
|
||||
"text": ""
|
||||
},
|
||||
{
|
||||
"location": "page/#text",
|
||||
"title": "Text",
|
||||
"text": "It's very easy to make some words bold and other words italic with Markdown. You can even add links , or even code : if (isAwesome) { return true }"
|
||||
},
|
||||
{
|
||||
"location": "page/#lists",
|
||||
"title": "Lists",
|
||||
"text": "Sometimes you want numbered lists: One Two Three Sometimes you want bullet points: Start a line with a star Profit!"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
If we inspect the search index again, we can see how the situation improved:
|
||||
|
||||
1. __Content is included only once__: the search index does not include the
|
||||
content of the page twice, as only the sections of a page are part of the
|
||||
search index. This leads to a significant reduction in size, fewer bytes to
|
||||
transfer, and a smaller search index.
|
||||
|
||||
2. __Some structure is preserved__: each section of the search index includes
|
||||
a small subset of HTML to provide the necessary structure to allow for more
|
||||
sophisticated search previews. Revisiting our example from before, let's
|
||||
look at an excerpt:
|
||||
|
||||
=== "Now"
|
||||
|
||||
``` html
|
||||
… links, or even <code>code</code>:</p> <pre><code>if (isAwesome){ … }\n</code></pre>
|
||||
```
|
||||
|
||||
=== "Before"
|
||||
|
||||
```
|
||||
… links , or even code : if (isAwesome) { … }
|
||||
```
|
||||
|
||||
The punctuation issue is gone, as no additional whitespace is inserted, and
|
||||
the preserved markup yields additional context to make scanning search
|
||||
results more effective.
|
||||
|
||||
On to the next step in the process: __tokenization__.
|
||||
|
||||
[search index]: #search-index
|
||||
[search preview now]: search-better-faster-smaller/search-preview-now.png
|
||||
[search preview before]: search-better-faster-smaller/search-preview-before.png
|
||||
|
||||
### Tokenizer lookahead
|
||||
|
||||
The [default tokenizer] of [lunr] uses a regular expression to split a given
|
||||
string by matching each character against the [`separator`][separator] as
|
||||
defined in `mkdocs.yml`. This doesn't allow for more complex separators based
|
||||
on lookahead or multiple characters.
|
||||
|
||||
Fortunately, __our new search implementation provides an advanced tokenizer__
|
||||
that doesn't have these shortcomings and supports more complex regular
|
||||
expressions. As a result, Material for MkDocs just changed its own separator
|
||||
configuration to the following value:
|
||||
|
||||
```
|
||||
[\s\-,:!=\[\]()"/]+|(?!\b)(?=[A-Z][a-z])|\.(?!\d)|&[lg]t;
|
||||
```
|
||||
|
||||
While the first part up to the first `|` contains a list of single control
|
||||
characters at which the string should be split, the following three sections
|
||||
explain the remainder of the regular expression.[^4]
|
||||
|
||||
[^4]:
|
||||
As a fun fact: the [`separator`][separator] [default value] of the search
|
||||
plugin being `[\s\-]+` always has been kind of irritating, as it suggests
|
||||
that multiple characters can be considered being a separator. However, the
|
||||
`+` is completely irrelevant, as regular expression groups involving
|
||||
multiple characters were never supported by
|
||||
[lunr's default tokenizer][default tokenizer].
|
||||
|
||||
[default value]: https://www.mkdocs.org/user-guide/configuration/#separator
|
||||
|
||||
#### Case changes
|
||||
|
||||
Many programming languages use `PascalCase` or `camelCase` naming conventions.
|
||||
When a user searches for the term `case`, it's quite natural to expect for
|
||||
`PascalCase` and `camelCase` to show up. By adding the following match group to
|
||||
the separator, this can now be achieved with ease:
|
||||
|
||||
```
|
||||
(?!\b)(?=[A-Z][a-z])
|
||||
```
|
||||
|
||||
This regular expression is a combination of a negative lookahead (`\b`, i.e.,
|
||||
not a word boundary) and a positive lookahead (`[A-Z][a-z]`, i.e., an uppercase
|
||||
character followed by a lowercase character), and has the following behavior:
|
||||
|
||||
- `PascalCase` :octicons-arrow-right-24: `Pascal`, `Case`
|
||||
- `camelCase` :octicons-arrow-right-24: `camel`, `Case`
|
||||
- `UPPERCASE` :octicons-arrow-right-24: `UPPERCASE`
|
||||
|
||||
Searching for [:octicons-search-24: searchHighlight][q=searchHighlight]
|
||||
now brings up the section discussing the `search.highlight` feature flag, which
|
||||
also demonstrates that this now even works properly for search queries.[^5]
|
||||
|
||||
[^5]:
|
||||
Previously, the search query was not correctly tokenized due to the way
|
||||
[lunr] treats wildcards, as it disables the pipeline for search terms that
|
||||
contain wildcards. In order to provide a good typeahead experience,
|
||||
Material for MkDocs adds wildcards to the end of each search term not
|
||||
explicitly preceded with `+` or `-`, effectively disabling tokenization.
|
||||
|
||||
[q=searchHighlight]: ?q=searchHighlight
|
||||
|
||||
#### Version numbers
|
||||
|
||||
Indexing version numbers is another problem that can be solved with a small
|
||||
lookahead. Usually, `.` should be considered a separator to split words like
|
||||
`search.highlight`. However, splitting version numbers at `.` will make them
|
||||
undiscoverable. Thus, the following expression:
|
||||
|
||||
```
|
||||
\.(?!\d)
|
||||
```
|
||||
|
||||
This regular expression matches a `.` only if not immediately followed by a
|
||||
digit `\d`, which leaves version numbers discoverable. Searching for
|
||||
[:octicons-search-24: 7.2.6][q=7.2.6] brings up the [7.2.6] release notes.
|
||||
|
||||
[q=7.2.6]: ?q=7.2.6
|
||||
[7.2.6]: ../../changelog/index.md#7.2.6
|
||||
|
||||
#### HTML/XML tags
|
||||
|
||||
If your documentation includes HTML/XML code examples, you may want to allow
|
||||
users to find specific tag names. Unfortunately, the `<` and `>` control
|
||||
characters are encoded in code blocks as `<` and `>`. Now, adding the
|
||||
following expression to the separator allows for just that:
|
||||
|
||||
```
|
||||
&[lg]t;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
_We've only just begun to scratch the surface of the new possibilities
|
||||
tokenizer lookahead brings. If you found other useful expressions, you're
|
||||
invited to share them in the comment section._
|
||||
|
||||
### Accurate highlighting
|
||||
|
||||
Highlighting is the last step in the process of search and involves the
|
||||
highlighting of all search term occurrences in a given search result. For a
|
||||
long time, highlighting was implemented through dynamically generated
|
||||
[regular expressions].[^6]
|
||||
|
||||
This approach has some problems with non-whitespace languages like Japanese or
|
||||
Chinese[^3] since it only works if the highlighted term is at a word boundary.
|
||||
However, Asian languages are tokenized using a [dedicated segmenter], which
|
||||
cannot be modeled with regular expressions.
|
||||
|
||||
[^6]:
|
||||
Using the separator as defined in `mkdocs.yml`, a regular expression was
|
||||
constructed that was trying to mimic the tokenizer. As an example, the
|
||||
search query `search highlight` was transformed into the rather cumbersome
|
||||
regular expression `(^|<separator>)(search|highlight)`, which only matches
|
||||
at word boundaries.
|
||||
|
||||
Now, as a direct result of the [new tokenization approach], __our new search
|
||||
implementation uses token positions for highlighting__, making it exactly as
|
||||
powerful as tokenization:
|
||||
|
||||
1. __Word boundaries__: as the new highlighter uses token positions, word
|
||||
boundaries are equal to token boundaries. This means that more complex cases
|
||||
of tokenization (e.g., [case changes], [version numbers], [HTML/XML tags]),
|
||||
are now all highlighted accurately.
|
||||
|
||||
2. __Context-awareness__: as the new search index preserves some of the
|
||||
structural information of the original document, the content of a section
|
||||
is now divided into separate content blocks – paragraphs, code blocks, and
|
||||
lists.
|
||||
|
||||
Now, only the content blocks that actually contain occurrences of one of
|
||||
the search terms are considered for inclusion into the search preview. If a
|
||||
term only occurs in a code block, it's the code block that gets rendered,
|
||||
see, for example, the results of
|
||||
[:octicons-search-24: twitter][q=twitter].
|
||||
|
||||
[regular expressions]: https://github.com/squidfunk/mkdocs-material/blob/ec7ccd2b2d15dd033740f388912f7be7738feec2/src/assets/javascripts/integrations/search/highlighter/index.ts#L61-L91
|
||||
[dedicated segmenter]: http://chasen.org/~taku/software/TinySegmenter/
|
||||
[new tokenization approach]: #tokenizer-lookahead
|
||||
[case changes]: #case-changes
|
||||
[version numbers]: #version-numbers
|
||||
[HTML/XML tags]: #htmlxml-tags
|
||||
[q=twitter]: ?q=twitter
|
||||
|
||||
### Benchmarks
|
||||
|
||||
We conducted two benchmarks – one with the documentation of Material for MkDocs
|
||||
itself, and one with a very massive corpus of Markdown files with more than
|
||||
800,000 words – a size most documentation projects will likely never
|
||||
reach:
|
||||
|
||||
<figure markdown>
|
||||
|
||||
| | Before | Now | Relative |
|
||||
| ----------------------- | -------: | -------------: | -----------: |
|
||||
| __Material for MkDocs__ | | | |
|
||||
| Index size | 573 kB | __335 kB__ | __–42%__ |
|
||||
| Index size (`gzip`) | 105 kB | __78 kB__ | __–27%__ |
|
||||
| Indexing time[^7] | 265 ms | __177 ms__ | __–34%__ |
|
||||
| __KJV Markdown[^8]__ | | | |
|
||||
| Index size | 8.2 MB | __4.4 MB__ | __–47%__ |
|
||||
| Index size (`gzip`) | 2.3 MB | __1.2 MB__ | __–48%__ |
|
||||
| Indexing time | 2,700 ms | __1,390 ms__ | __–48%__ |
|
||||
|
||||
<figcaption>
|
||||
<p>Benchmark results</p>
|
||||
</figcaption>
|
||||
|
||||
</figure>
|
||||
|
||||
[^7]:
|
||||
Smallest value of ten distinct runs.
|
||||
|
||||
[^8]:
|
||||
We agnostically use [KJV Markdown] as a tool for testing to learn how
|
||||
Material for MkDocs behaves on large corpora, as it's a very large set of
|
||||
Markdown files with over 800k words.
|
||||
|
||||
The results show that indexing time, which is the time that it takes to set up
|
||||
the search when the page is loaded, has dropped by up to 48%, which means __the
|
||||
new search is up to 95% faster__. This is a significant improvement,
|
||||
particularly relevant for large documentation projects.
|
||||
|
||||
While 1,3s still may sound like a long time, using the new client-side search
|
||||
together with [instant loading] only creates the search index on the initial
|
||||
page load. When navigating, the search index is preserved across pages, so the
|
||||
cost does only have to be paid once.
|
||||
|
||||
[KJV Markdown]: https://github.com/arleym/kjv-markdown
|
||||
[instant loading]: ../../setup/setting-up-navigation.md#instant-loading
|
||||
|
||||
### User interface
|
||||
|
||||
Additionally, some small improvements have been made, most prominently the
|
||||
__more results on this page__ button, which now sticks to the top of the search
|
||||
result list when open. This enables the user to jump out of the list more
|
||||
quickly.
|
||||
|
||||
## What's next?
|
||||
|
||||
Our new search implementation is a big improvement to Material for MkDocs. It
|
||||
solves some long-standing issues which needed to be tackled for years. Yet,
|
||||
it's only the start of a search experience that is going to get better and
|
||||
better. Next up:
|
||||
|
||||
- __Context-aware search summarization__: currently, the first two matching
|
||||
content blocks are rendered as a search preview. With the new tokenization
|
||||
technique, we laid the groundwork for more sophisticated shortening and
|
||||
summarization methods, which we're tackling next.
|
||||
|
||||
- __User interface improvements__: as we now gained full control over the
|
||||
search plugin, we can now add meaningful metadata to provide more context and
|
||||
a better experience. We'll explore some of those paths in the future.
|
||||
|
||||
If you've made it this far, thank you for your time and interest in Material
|
||||
for MkDocs! This is the first blog article that I decided to write after a
|
||||
short [Twitter survey] made me to. You're invited to leave a comment
|
||||
to share your experiences with the new search implementation.
|
||||
|
||||
[X survey]: https://x.com/squidfunk/status/1434477478823743488
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 225 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 174 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 55 KiB |
@@ -0,0 +1,96 @@
|
||||
---
|
||||
date: 2023-10-02
|
||||
authors: [squidfunk, alexvoss]
|
||||
categories:
|
||||
- General
|
||||
---
|
||||
|
||||
# Sunsetting Gitter: Towards Efficient Community Engagement
|
||||
|
||||
__As we're starting to build a team around Material for MkDocs, we've decided to
|
||||
sunset and archive our [Gitter] channel on October 13, 2023 in favor of
|
||||
[GitHub Discussions].__
|
||||
|
||||
As part of our efforts to improve the processes for maintaining Material for
|
||||
MkDocs and for supporting the community, we have reviewed the use of different
|
||||
communication channels. At the moment, both [Gitter] and [GitHub Discussions]
|
||||
allow to ask the community for support and to discuss ideas and issues. In the
|
||||
past weeks, we have begun to question whether this duplication is in the best
|
||||
interest of our project. This post explains the rationale behind our decision.
|
||||
|
||||
[Gitter]: https://gitter.im/squidfunk/mkdocs-material
|
||||
[GitHub discussions]: https://github.com/squidfunk/mkdocs-material/discussions
|
||||
|
||||
<!-- more -->
|
||||
|
||||
## The Constraints of Gitter
|
||||
|
||||
In the first years of Material for MkDocs, [Gitter] served as a helpful platform
|
||||
for quick community interactions and questions. However, there are some inherent
|
||||
constraints that necessitated a shift. Here's a breakdown of the challenges we
|
||||
encountered:
|
||||
|
||||
1. __Thread utilization__ –
|
||||
Even with the introduction of threads, many Gitter users were reticent to
|
||||
use them, leading to cluttered and disconnected conversations, making it
|
||||
difficult to follow.
|
||||
|
||||
2. __Search and discovery__ –
|
||||
Gitter's structure and interface makes it cumbersome to search for and
|
||||
retrieve past messages, resulting in lost insights and repeated conversations.
|
||||
|
||||
1. __User interface__ –
|
||||
After the acquisition by [Element], the user interface of Gitter has
|
||||
experienced a severe decline, becoming more complicated, less intuitive and
|
||||
user-friendly.
|
||||
|
||||
1. __Access mandate__ –
|
||||
The necessity for users to grant access to their GitHub accounts to use
|
||||
Gitter posed an unnecessary access barrier and potential security concerns.
|
||||
|
||||
[Element]: https://element.io/blog/gitter-is-joining-element/
|
||||
|
||||
## GitHub Discussions
|
||||
|
||||
Thankfully, [GitHub Discussions] emerged and continues to evolve to be a much
|
||||
better alternative, eliminating many of Gitter's constraints and offering
|
||||
enhanced functionality:
|
||||
|
||||
1. __Threaded conversations__ –
|
||||
GitHub Discussions supports structured, threaded conversations, ensuring
|
||||
discussions are coherent, contextual, and easy to follow for everyone.
|
||||
|
||||
2. __Integration and linking__ –
|
||||
the platform facilitates seamless linking to other discussions, issues, code,
|
||||
and much more, enriching conversations and providing comprehensive context.
|
||||
|
||||
3. __A unified platform__ –
|
||||
GitHub acts as a one-stop shop for support, allowing users to access varied
|
||||
resources and support channels in one place, greatly improving user
|
||||
experience.
|
||||
|
||||
4. __Discoverability__ –
|
||||
GitHub Discussions are easily searchable and allow for categorization,
|
||||
preventing loss of valuable insights and fostering knowledge sharing.
|
||||
|
||||
## Conclusion
|
||||
|
||||
Sunsetting [Gitter] is a leap towards creating more efficient, user-friendly,
|
||||
and enriched community interactions. By consolidating communication to
|
||||
[GitHub Discussions], we seek to mitigate the fragmentation of information,
|
||||
ease the user journey, and fortify the bonds within our community.
|
||||
|
||||
This concentrated approach enables us to provide consistent, fast and quality
|
||||
support to our users. In the past, the nature of queries raised on Gitter
|
||||
typically necessitated supplementary information or reproductions, prompting us
|
||||
to advise users to initiate a discussion subsequently. The shift to a
|
||||
centralized discussion forum is designed to simplify and accelerate this
|
||||
procedure, ensuring swift and efficient resolutions for our community. It is a
|
||||
testament to our ongoing commitment to support the community that helps Material
|
||||
for MkDocs flourish.
|
||||
|
||||
We are confident that this streamlined approach will spur more
|
||||
fruitful discussions, innovative ideas, and collective growth, and we are
|
||||
excited to continue our journey with the unwavering support and active
|
||||
participation of our community.
|
||||
|
||||
@@ -0,0 +1,263 @@
|
||||
---
|
||||
date: 2021-12-27
|
||||
authors: [squidfunk]
|
||||
description: >
|
||||
2021 was a fantastic year for this project as we shipped many new awesome
|
||||
features and made this project sustainable
|
||||
categories:
|
||||
- General
|
||||
---
|
||||
|
||||
# The past, present and future
|
||||
|
||||
__2021 was a fantastic year for this project as we shipped many new awesome
|
||||
features, saw significant user growth and leveraged GitHub Sponsors to make the
|
||||
project sustainable.__
|
||||
|
||||
Today, together, [MkDocs] and Material for MkDocs are among the most popular
|
||||
options when it comes to choosing a static site generator and theme for your
|
||||
technical documentation project. Material for MkDocs ensures that your
|
||||
content is always perfectly presented to your audience, regardless of screen
|
||||
resolution or device capabilities. It has evolved to a framework for technical
|
||||
writing, offering many features, some of which are yet to be found in other
|
||||
static site generators. However, we're far from the end, as 2022 is going to
|
||||
bring some interesting new capabilities.
|
||||
|
||||
<!-- more -->
|
||||
|
||||
_This article showcases all features that were added in 2021 and gives an
|
||||
outlook on what's going to drop in 2022. Additionally, it provides some context
|
||||
on the history of the project._
|
||||
|
||||
[MkDocs]: https://www.mkdocs.org
|
||||
|
||||
## A little history
|
||||
|
||||
In 2015, albeit 10 years in the industry, I was still quite new in Open Source.
|
||||
I wanted to release my latest Open Source project [protobluff], a zero-copy
|
||||
Protocol Buffers implementation for C, which I've created as part of my former
|
||||
startup. As the project has a significant degree of complexity, I quickly
|
||||
realized that I was in need of good user documentation.
|
||||
|
||||
After evaluating static site generators in general and Hugo, Sphinx and MkDocs
|
||||
in particular, I quickly decided that MkDocs seemed a good choice, as it was
|
||||
specifically aimed at technical project documentation and easy to use.
|
||||
Unfortunately, all of the available themes looked dated and given that I'm a
|
||||
very visual person, I just couldn't convince myself to call it a day.
|
||||
|
||||
I _had_ to build a theme.
|
||||
|
||||
Months later, in February 2016, I released [the first version] of Material for
|
||||
MkDocs (and with it, [protobluff], the project I wanted to release in the first
|
||||
place), and it looked like this:
|
||||
|
||||
![Material for MkDocs 0.1.0][Material for MkDocs 0.1.0]
|
||||
|
||||
It was already fully responsive and overall, well, quite okayish, but barely
|
||||
customizable, as only the logo could be changed. Beyond that, it had no options:
|
||||
No color or navigation options, no instant loading, etc. The search was very
|
||||
rudimentary and only supported section titles:
|
||||
|
||||
![Material for MkDocs 0.1.0 Search][Material for MkDocs 0.1.0 Search]
|
||||
|
||||
It's important to know that at this point in time I've built Material for
|
||||
MkDocs for [protobluff], the project I was really caring about. Almost 6 years
|
||||
later, nobody knows protobluff, but this little side project has taken off. If
|
||||
back in those days, you would've told me big organizations like AWS,
|
||||
Microsoft and CERN, as well as extremely popular Open Source projects like
|
||||
FastAPI and Kubernetes will be using this project in the future – I would've
|
||||
declared you insane.
|
||||
|
||||
I still find the success of this project quite surprising, as I thought that
|
||||
themes exist in abundance and are very much a solved problem. There's no glory
|
||||
in themes, no stars to earn (remember that I was new in Open Source, so this was
|
||||
the metric I was optimizing for), as there are thousands and thousands of
|
||||
options. However, as the years progressed, I learned that _execution matters_:
|
||||
although Material for MkDocs solves a problem for which thousands of solutions
|
||||
exist, it excels in a specific niche, and that niche is to be known as
|
||||
_technical project documentation_.
|
||||
|
||||
Today, this project is not only popular but funded by almost 300 individuals
|
||||
and organizations, resulting in a yearly budget of more than $50,000. This
|
||||
allows me to set aside enough time for the development of new features,
|
||||
bug fixing, stability improvement, issue triage and general support and still
|
||||
feels like a dream to me, as there are many failed stories of Open Source
|
||||
funding and people telling you: _don't do Open Source, you'll be working for
|
||||
free._
|
||||
|
||||
Making Open Source sustainable is, after all, possible in 2021.
|
||||
|
||||
[the first version]: https://github.com/squidfunk/mkdocs-material/releases/tag/0.1.0
|
||||
[Material for MkDocs 0.1.0]: the-past-present-and-future/mkdocs-material-0.1.0.png
|
||||
[Material for MkDocs 0.1.0 Search]: the-past-present-and-future/mkdocs-material-0.1.0-search.png
|
||||
[protobluff]: https://github.com/squidfunk/protobluff
|
||||
|
||||
## 2021 in numbers
|
||||
|
||||
2021 was an exciting year, as the project has seen significant growth.
|
||||
|
||||
__166k people__ visited the official documentation in 2021, totalling in __1,6m
|
||||
page views__ which is an increase of 83% when compared to 2020. The average
|
||||
visitor spends __1,5min__ on the site. While mobile phones make up 12% of
|
||||
visits, tablets only account for 0.6%. Visitors come from as many as __213
|
||||
countries__, which covers almost the whole world.
|
||||
|
||||
### Features
|
||||
|
||||
It's absolutely mind-blowing that __38 new features__ were added to Material
|
||||
for MkDocs throughout 2021 – __that's a new feature every 9,6 days__ –
|
||||
which was only possible because of the constantly improving funding situation.
|
||||
Following is a list of all features shipped in alphabetical order, some of which
|
||||
are still exclusively available to sponsors as part of [Insiders]:
|
||||
|
||||
<div class="mdx-columns" markdown>
|
||||
|
||||
- [Admonition inline blocks]
|
||||
- [Advanced search highlighting]
|
||||
- [Anchor tracking]
|
||||
- [Back-to-top button]
|
||||
- [Boosting pages in search]
|
||||
- [Brand new search plugin]
|
||||
- [Code annotations]
|
||||
- Code annotations: anchor links
|
||||
- [Code annotations: strip comments]
|
||||
- [Code block titles]
|
||||
- [Code block line anchors]
|
||||
- [Color palette toggle]
|
||||
- [Content tabs: improved support]
|
||||
- [Content tabs: auto-linking]
|
||||
- Content tabs: animated indicator
|
||||
- [Cookie consent]
|
||||
- [Custom admonition icons]
|
||||
- [Dark mode support for images]
|
||||
- [Dismissable announcement bar]
|
||||
- [Excluding content from search]
|
||||
- Latest release tag
|
||||
- [Mermaid.js integration]
|
||||
- [Navigation icons]
|
||||
- [Remove generator notice]
|
||||
- [Rich search previews]
|
||||
- Stay on page when switching versions
|
||||
- [Search highlighting]
|
||||
- [Search sharing]
|
||||
- [Search suggestions]
|
||||
- [Section index pages]
|
||||
- [Site language selection]
|
||||
- [Social cards]
|
||||
- [Sticky navigation tabs]
|
||||
- [Tags with search integration]
|
||||
- [Tokenizer with lookahead]
|
||||
- [Versioning]
|
||||
- [Version warning]
|
||||
- [Was this page helpful?]
|
||||
|
||||
</div>
|
||||
|
||||
Additionally, a lot of bugs were fixed in the __1,000 commits__ that were pushed
|
||||
to the repository this year. The [changelog] includes a list of all fixes.
|
||||
Furthermore, a large amount of time was invested into refactoring the code base
|
||||
to keep it in good shape. While the `mkdocs-material` package was released
|
||||
__55__ times, `mkdocs-material-insiders` was shipped __72__ times.
|
||||
|
||||
[Insiders]: ../../insiders/index.md
|
||||
[Admonition inline blocks]: ../../reference/admonitions.md#inline-blocks
|
||||
[Advanced search highlighting]: search-better-faster-smaller.md#accurate-highlighting
|
||||
[Anchor tracking]: ../../setup/setting-up-navigation.md#anchor-tracking
|
||||
[Back-to-top button]: ../../setup/setting-up-navigation.md#back-to-top-button
|
||||
[Boosting pages in search]: ../../setup/setting-up-site-search.md#search-boosting
|
||||
[Brand new search plugin]: search-better-faster-smaller.md
|
||||
[Code annotations]: ../../reference/code-blocks.md#adding-annotations
|
||||
[Code annotations: strip comments]: ../../reference/code-blocks.md#stripping-comments
|
||||
[Code block titles]: ../../reference/code-blocks.md#adding-a-title
|
||||
[Code block line anchors]: ../../setup/extensions/python-markdown-extensions.md#+pymdownx.highlight.anchor_linenums
|
||||
[Color palette toggle]: ../../setup/changing-the-colors.md#color-palette-toggle
|
||||
[Content tabs: improved support]: ../../reference/content-tabs.md
|
||||
[Content tabs: auto-linking]: ../../reference/content-tabs.md#linked-content-tabs
|
||||
[Cookie consent]: ../../setup/ensuring-data-privacy.md#cookie-consent
|
||||
[Custom admonition icons]: ../../reference/admonitions.md#admonition-icons
|
||||
[Dark mode support for images]: ../../reference/images.md#light-and-dark-mode
|
||||
[Dismissable announcement bar]: ../../setup/setting-up-the-header.md#mark-as-read
|
||||
[Excluding content from search]: ../../setup/setting-up-site-search.md#search-exclusion
|
||||
[Mermaid.js integration]: ../../reference/diagrams.md
|
||||
[Navigation icons]: ../../reference/index.md#setting-the-page-icon
|
||||
[Remove generator notice]: ../../setup/setting-up-the-footer.md#generator-notice
|
||||
[Rich search previews]: search-better-faster-smaller.md#rich-search-previews
|
||||
[Search highlighting]: ../../setup/setting-up-site-search.md#search-highlighting
|
||||
[Search sharing]: ../../setup/setting-up-site-search.md#search-sharing
|
||||
[Search suggestions]: ../../setup/setting-up-site-search.md#search-suggestions
|
||||
[Section index pages]: ../../setup/setting-up-navigation.md#section-index-pages
|
||||
[Site language selection]: ../../setup/changing-the-language.md#site-language-selector
|
||||
[Social cards]: ../../setup/setting-up-social-cards.md
|
||||
[Sticky navigation tabs]: ../../setup/setting-up-navigation.md#sticky-navigation-tabs
|
||||
[Tags with search integration]: ../../setup/setting-up-tags.md
|
||||
[Tokenizer with lookahead]: search-better-faster-smaller.md#tokenizer-lookahead
|
||||
[Versioning]: ../../setup/setting-up-versioning.md#versioning
|
||||
[Version warning]: ../../setup/setting-up-versioning.md#version-warning
|
||||
[Was this page helpful?]: ../../setup/setting-up-site-analytics.md#was-this-page-helpful
|
||||
[changelog]: ../../changelog/index.md
|
||||
|
||||
### Funding
|
||||
|
||||
In 2021, monthly funding increased from $1,050 in the beginning of January to
|
||||
more than $4,300 (Dec 27, 2021), totaling in a yearly budget of more than
|
||||
$50,000. Compared to last year, __revenue from funding has increased by 617%__
|
||||
– which is absolutely unbelievable:
|
||||
|
||||
![Funding]
|
||||
|
||||
[Funding]: the-past-present-and-future/funding.png
|
||||
|
||||
I'm solely providing these numbers to fulfill the transparency pledge I'm giving
|
||||
to my awesome sponsors, and to show that it's possible to make existing Open
|
||||
Source projects sustainable by following a well-designed release strategy.
|
||||
|
||||
You can learn about the strategy in the [Insiders] guide.
|
||||
|
||||
## 2022
|
||||
|
||||
Standing at the verge of the next year, it's safe to say that the project will
|
||||
continue to prosper and evolve, yielding many awesome features that will make
|
||||
technical writing more comfortable and flexible. Here's an excerpt of the
|
||||
features that will see the light of day in 2022:
|
||||
|
||||
- __Instant previews__: [instant previews] will render a specific page section
|
||||
inside a tooltip when hovering an internal link, which will allow to implement
|
||||
things like glossaries. Further support for improving glossary functionality
|
||||
will also be investigated.
|
||||
|
||||
- __Text annotations__: as a logical progression of [code annotations] which
|
||||
were added in 2021, authors will be able to add annotations to plain text,
|
||||
yielding excellent opportunities for side content. Of course, text annotations
|
||||
will be as easy to use as code annotations.
|
||||
|
||||
- __Navigation pruning__: to optimize large documentation projects, Material
|
||||
for MkDocs will introduce a new feature flag called `navigation.prune` that
|
||||
will lead to significantly smaller HTML files for documentation projects with
|
||||
huge navigation hierarchies.
|
||||
|
||||
- __Navigation status badge__: as an addition to the recently added
|
||||
[navigation icon][Navigation icons] support, a status will be attributable to
|
||||
each page, allowing to mark a page in the navigation tree with an icon as
|
||||
:material-alert-decagram: __new__ or :material-trash-can: __deprecated__.
|
||||
Custom status types will also be supported.
|
||||
|
||||
- __Card grids__: as a further component in the toolkit of technical writing,
|
||||
[card grids] will allow arranging content in grids, which is especially
|
||||
useful for overview pages. They will allow to arrange arbitrary content,
|
||||
including code blocks, admonitions, etc.
|
||||
|
||||
- __Blog support__: blogging support is still [under investigation] and expected
|
||||
to be one of the major additions in 2022. Blogging will perfectly integrate
|
||||
with writing documentation, allowing to use all components available in
|
||||
Material for MkDocs.
|
||||
|
||||
This list is incomplete. Additionally, many new smaller features will be added
|
||||
next year, just as in 2021. You can follow [@squidfunk on X] to stay
|
||||
updated.
|
||||
|
||||
__Happy new year!__ :tada:
|
||||
|
||||
[Instant previews]: https://x.com/squidfunk/status/1466794654213492743
|
||||
[card grids]: https://github.com/squidfunk/mkdocs-material/issues/3018
|
||||
[under investigation]: https://github.com/squidfunk/mkdocs-material/issues/3353
|
||||
[@squidfunk on X]: https://x.com/squidfunk
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 94 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 74 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 93 KiB |
@@ -0,0 +1,225 @@
|
||||
---
|
||||
date: 2024-08-19
|
||||
authors:
|
||||
- squidfunk
|
||||
- alexvoss
|
||||
- katharinalisalin
|
||||
categories:
|
||||
- General
|
||||
description: >
|
||||
We are transforming Material for MkDocs to ensure its community continues to thrive and doubling down on our commitment to Open Source.
|
||||
title: How we're transforming Material for MkDocs
|
||||
---
|
||||
|
||||
# Transforming Material for MkDocs
|
||||
|
||||
__We are working on an amazing set of features which has required some behind-the-scenes work we are now ready to share in a series of posts. Here, we give an overview of our goals, features in the making, things that kept us up at night, and our commitment to Open Source.__
|
||||
|
||||
We know it's been quite a while since our last update, which is why we're eager to share what's happening in and around Material for MkDocs with you. For the largest part of 2024, we've been focused on transforming the core of Material for MkDocs preparing the ground for new, interconnected features that are amongst the most frequently requested.
|
||||
|
||||
This article is part of a series where we'll explore how we aim to support our users through improved information retrieval, provide better support for multi-lingual sites and versioning, as well as improve the overall authoring experience. We outline our vision for the projects' evolution and describe what we have been working on. In this and the coming blog posts, we will share our progress with you, and we're excited to hear your thoughts.
|
||||
|
||||
<!-- more -->
|
||||
|
||||
_Please note that this post includes several technical details in the footnotes, specifically on challenges. Feel free to skip them if you're not interested in the specifics._
|
||||
|
||||
---
|
||||
|
||||
__This is the first article in a four-part series:__
|
||||
|
||||
1. [Transforming Material for MkDocs]
|
||||
2. [Zensical – A modern static site generator built by the creators of Material for MkDocs]
|
||||
3. [Material for MkDocs Insiders – Now free for everyone]
|
||||
4. [Goodbye, GitHub Discussions]
|
||||
|
||||
[Transforming Material for MkDocs]: transforming-material-for-mkdocs.md
|
||||
[Zensical – A modern static site generator built by the creators of Material for MkDocs]: zensical.md
|
||||
[Material for MkDocs Insiders – Now free for everyone]: insiders-now-free-for-everyone.md
|
||||
[Goodbye, GitHub Discussions]: goodbye-github-discussions.md
|
||||
|
||||
## A success story
|
||||
|
||||
In 2024, Material for MkDocs has firmly established itself as a leading tool in the documentation framework landscape, with more than 5 million downloads each month as of this writing. What began as @squidfunk's personal project has evolved into a versatile resource for creating comprehensive documentation sites, personal websites, blogs, and notably, for managing knowledge both within and outside of organizations.
|
||||
|
||||
With almost [50,000] public GitHub projects depending on Material for MkDocs, it's clear that the framework has made a substantial impact. Tens of thousands of authors rely on us to deliver documentation to millions of users each month.[^1] Beyond its adoption by [many popular Open Source projects], Material for MkDocs is trusted and financially supported by major corporations such as [AWS], [Microsoft], and [Siemens], among many other companies and individuals. We're very grateful for the continued support we receive, which allows us to devote our time to this project, making writing documentation enjoyable.
|
||||
|
||||
[^1]:
|
||||
We've collected invaluable feedback from enterprises and other Open Source maintainers, which revealed that the actual numbers are even higher. Many organizations leverage the framework within private infrastructures, such as self-hosted platforms like GitLab, for internal knowledge management. This suggests that the true reach of Material for MkDocs extends far beyond what is publicly visible.
|
||||
|
||||
[50,000]: https://github.com/squidfunk/mkdocs-material/network/dependents
|
||||
[many popular Open Source projects]: https://github.com/squidfunk/mkdocs-material?tab=readme-ov-file#trusted-by-
|
||||
[AWS]: https://x.com/squidfunk/status/1740389441284579767
|
||||
[Microsoft]: https://x.com/squidfunk/status/1801909506391105840
|
||||
[Siemens]: https://x.com/squidfunk/status/1699799988069646761
|
||||
|
||||
Our users particularly appreciate Material for MkDocs for its ease of use and straightforward setup. It simplifies the process by handling the complexities of web technologies for you, so you can build a production-ready static site in minutes without needing to invest years into mastering frontend development or JavaScript. This makes it accessible to a wide range of users, regardless of their technical background. Additionally, Material for MkDocs is MIT-licensed and free to use, which has contributed to its widespread adoption, and allows everybody to build sophisticated documentation sites at no cost.
|
||||
|
||||
Our vision is to provide you with the tools that allow you to __own your docs__, enable you to develop your own processes for producing amazing documentation and to avoid being locked into costly subscription services even for doing simple things. We believe it is important that these tools should be easy to set up and configure to suit your needs but should also give you the freedom to adapt them further, if needed. This is why we are always striving to ensure our architectures are modular and future-proof.[^2]
|
||||
|
||||
[^2]:
|
||||
The [built-in plugins] that Material for MkDocs ships perfectly outline this principle, as they are complementary to each other and can be used in combination to build sophisticated pipelines. This modular design allows users to pick and choose the features they need, ensuring that the framework remains lightweight and flexible.
|
||||
|
||||
For instance, the [privacy plugin] can work together with the [optimize plugin] so that external assets can be passed through the same optimization pipeline as the rest of your documentation. This means you can store and edit unoptimized files outside of your repository, and let both plugins automatically build an optimized site for you.
|
||||
|
||||
[built-in plugins]: ../../plugins/index.md
|
||||
[privacy plugin]: ../../plugins/privacy.md
|
||||
[optimize plugin]: ../../plugins/optimize.md
|
||||
|
||||
## Challenges
|
||||
|
||||
Now, let's talk about our journey and the particular challenges we're addressing. With "we", we're referring to the incredible team that @squidfunk was fortunate to build around his original work, thanks to the financial support he receives from his [sponsors]. This remarkable team includes @alexvoss and @katharinalisalin, whose invaluable contributions in research, development, and community support have been essential to the project's ongoing success.
|
||||
|
||||
Together, we've started exploring new technologies, incorporating the feedback we received from our users, and rethinking critical components from first principles, serving our growing community one of the best frameworks to create documentation.
|
||||
|
||||
[sponsors]: https://github.com/sponsors/squidfunk
|
||||
|
||||
This section highlights the key areas we've been focusing on.
|
||||
|
||||
### Search and discovery
|
||||
|
||||
From our perspective, search is a cornerstone of any effective documentation site, enabling users to swiftly locate the information they need. From the start, we've relied on [Lunr.js], a popular client-side search library that has streamlined the deployment of documentation sites without needing an external service. Over the years, Lunr.js has served us well by answering millions of queries on sites built with Material for MkDocs. Yet, as our users' needs evolved, we encountered limitations that proved difficult to overcome.
|
||||
|
||||
[Lunr.js]: https://github.com/olivernn/lunr.js
|
||||
|
||||
We learned that the [BM25 ranking algorithm] at the core of Lunr.js (as in many other search implementations) isn't well-suited for effective and stable typeahead. Adding new documents can influence existing rankings, requiring manual adjustments[^3] that are cumbersome and hard to scale. Furthermore, Lunr.js has performance issues that stem from its design not fully leveraging modern JavaScript engines and optimizations, making it slower and memory-intensive.[^4]
|
||||
|
||||
[^3]:
|
||||
Boosting documents in [BM25][BM25 ranking algorithm] can lead to challenges, particularly with a changing corpus of documents. Relevance is calculated based on term frequency and the importance of terms across the entire corpus. Boosting a document involves adjusting its weight to make it more prominent in search results.
|
||||
|
||||
As new documents are added or existing ones are modified, the overall distribution of term frequencies and their importance can shift. This recalibration can diminish the effectiveness of the boost, making it harder to maintain consistent ranking across a changing dataset. Essentially, the boosted documents may not stay as prominent or relevant as intended, leading to imbalances and scalability issues in search results.
|
||||
|
||||
[^4]:
|
||||
Lunr.js uses JavaScript objects to index and manage search data, which introduces inefficiencies due to how JavaScript engines handle object operations. JavaScript engines optimize performance using techniques like inline caching and object shape optimization. However, these optimizations rely on predictable and consistent object structures.
|
||||
|
||||
The dynamic nature of Lunr.js' indexing — where documents can have varying structures — prevents the engine from applying these optimizations effectively. This means that while JavaScript engines can optimize for fixed, predictable object structures, they struggle with the polymorphic and fluid nature of Lunr.js's indexing, leading to performance issues as the amount of data grows.
|
||||
|
||||
[BM25 ranking algorithm]: https://en.wikipedia.org/wiki/Okapi_BM25
|
||||
|
||||
Over the last years, we've invested substantially into [improving the search experience]. For instance, expanding Lunr.js' functionality to add support for [rich search previews] and [tokenizer lookahead] demanded substantial engineering effort. Lunr.js allows to customize tasks such as stemming, stopword filtering, and trimming with [pipeline functions], but it makes it particularly hard to add extensions like term highlighting or alternative ranking methods. Compounded by the fact that Lunr.js has been [unmaintained since 2020], it became clear that we needed to find a more powerful solution. We've looked at alternative JavaScript-based libraries to keep our client-side search, but none met our requirements or lived up to our expectations.
|
||||
|
||||
[improving the search experience]: search-better-faster-smaller.md
|
||||
[rich search previews]: search-better-faster-smaller.md#rich-search-previews
|
||||
[tokenizer lookahead]: search-better-faster-smaller.md#tokenizer-lookahead
|
||||
[pipeline functions]: https://lunrjs.com/guides/customising.html#pipeline-functions
|
||||
[unmaintained since 2020]: https://github.com/olivernn/lunr.js/tag/
|
||||
|
||||
To address these challenges, we've embarked on developing a new search system from first principles that not only matches but already exceeds the capabilities of Lunr.js. Built from the ground up, this system is faster, more compact, and most importantly: modular. It is based on a growing core evolving around two core concepts we isolated to be essential – engines and plugins – allowing for flexible configuration and assembly of components like text indexing, vector embeddings, filtering, ranking, highlighting, and more. Every part of it can be replaced or extended, enabling users to tailor the search system to their specific needs.
|
||||
|
||||
Our new search system, which we will release as a separate MIT-licensed Open Source project, is designed to handle a wide range of scenarios — from small blogs to large documentation projects. Of course, it supports offline use and integrates seamlessly with both client and server environments. The advanced ranking system provides fine-tuned control, and integration with third-party services is now more straightforward.
|
||||
|
||||
__:octicons-goal-16: Goal – Empower users to quickly find the information they need while freeing authors from managing external services, by providing a search system that adapts to diverse content types.__
|
||||
|
||||
You might wonder why it's not yet available. The process of developing this system from scratch and uncovering its potential has led us to re-evaluate core concepts of Material for MkDocs. As a result, we've decided to postpone the release of the new search system to integrate it into the broader update that we've started to work on. If you keep on reading, you'll learn about further reasons why we've decided to follow this approach.
|
||||
|
||||
We're excited to share more details about this update in one of the next posts in this series.
|
||||
|
||||
### Translations and versioning
|
||||
|
||||
Supporting multi-language sites in MkDocs is the [most requested feature on our discussion board] and in conversations with users, yet it presents significant challenges, as MkDocs does not natively support it. The same applies to versioning, which also involves synchronisation of multi-project builds. While the MkDocs ecosystem has developed [various plugins and tools] to address these issues, there is still substantial untapped potential. We began exploring these areas but quickly encountered problems that hindered our progress.
|
||||
|
||||
[most requested feature on our discussion board]: https://github.com/squidfunk/mkdocs-material/discussions/2346
|
||||
[various plugins and tools]: https://github.com/mkdocs/catalog?tab=readme-ov-file#-site-building-site-management
|
||||
|
||||
As you may know, our initial effort involved the [projects plugin] that aims to extend MkDocs to add support for multi-project environments as a solid foundation to support multi-language sites and versioning. Unfortunately, we encountered significant challenges due to MkDocs' internal architecture and design constraints, which we're working actively on resolving.[^5]
|
||||
|
||||
[^5]:
|
||||
When developing the [projects plugin], we initially made [good progress], including adding support for nested projects and allowing for a tree structure where sub-projects can have further sub-projects. However, we quickly encountered difficulties, particularly with cross-project navigation. To illustrate, imagine that each project can link to any other project, which makes handling these interconnections complex, especially when dealing with circular dependencies — such as Project A linking to Project B and vice versa.
|
||||
|
||||
Implementing multi-project support in MkDocs is particularly challenging due to the lack of an official programmatic API, which complicates efforts to extend its functionality. Moreover, resolving navigation issues before building projects is crucial for ensuring proper interconnectivity. These challenges combined made the development of the projects plugin a complex endeavor.
|
||||
|
||||
[projects plugin]: ../../plugins/projects.md
|
||||
[good progress]: https://github.com/squidfunk/mkdocs-material/discussions/5800
|
||||
|
||||
__:octicons-goal-16: Goal – Enable scaling documentation to any size or team structure by offering seamless methods for integrating multiple documentation projects, whether they involve different languages, versions, or distinct sections of an overall body of work.__
|
||||
|
||||
As a result, we are developing a new approach to offer a more comprehensive and robust solution for both multi-language support and versioning. This new approach also intersects with adjacent functionalities like search, as many of our users are interested in [federated search] capabilities that combine results from multiple documentation sites into a unified search interface. Overcoming this challenge is one of the major hurdles we need to address before releasing the new search system.
|
||||
|
||||
[federated search]: https://github.com/squidfunk/mkdocs-material/issues/5230
|
||||
|
||||
### Editing and collaboration
|
||||
|
||||
We had considered developing a live editor in response to MkDocs' [performance issues with large projects], which in most cases stem from compute-intensive plugins that don't employ caching. A [proof of concept] based on [Pyodide] (= running Python in the browser) [generated significant interest] among users and prompted many organizations and individuals to share their collaborative workflows for feedback. Sadly, implementing this live editor proved to be very challenging, as it would require rebuilding substantial parts of MkDocs.[^6] After discontinuing work on this approach, our progress with multi-project support has renewed our belief that we can finally solve the sluggish editing experience that was reported several times over the last years.[^7]
|
||||
|
||||
[performance issues with large projects]: https://github.com/mkdocs/mkdocs/issues/3695#issuecomment-2093299518
|
||||
[proof of concept]: https://x.com/squidfunk/status/1338252230265360391
|
||||
[Pyodide]: https://pyodide.org/
|
||||
[generated significant interest]: https://github.com/squidfunk/mkdocs-material/issues/2110
|
||||
|
||||
[^6]:
|
||||
Our [proof of concept] supported some features of Material for MkDocs but didn't cover them all. For instance, integrating support for icons or linking between documents would have necessitated reimplementing parts of MkDocs to bypass a full rebuild — something we obviously wanted to avoid. Additionally, certain links, such as those to blog posts generated from schemas, are not merely translated but computed dynamically, which means they cannot be inferred by replacing the `.md` extension with `.html`.
|
||||
|
||||
[^7]:
|
||||
After [we raised this issue] with the maintainers of MkDocs again[^8], and [maintainership changed] mid 2024, work on a [ground-up rewrite] that aims to address slow reloads by [rendering only the page currently being built] has started. It's still unclear how this rewrite will integrate with the requirements of existing plugins. Complex plugins such as [mkdocstrings], or our [built-in blog] and [tags] plugins require a coordinated build of all pages to accurately resolve links between pages and to computed resources, which cannot be determined without building the entire project.
|
||||
|
||||
__Update__{ title="August 21, 2024" }: The new maintainer now publicly stated that he's working towards a new version of MkDocs that [does not require or support plugins], and mentions it will be equally capable through offering customization via templating, themes, and styling, which he also mentioned to us and several other users in [a team call on August 1]. In this call, [we raised objections multiple times] in regards to how this will impact the ecosystem, to no avail. No intention or roadmap for plugin support was provided. This development is orthogonal to our goal empowering users and organizations to adapt the core framework to their requirements by the means of modularity. We're working on resolving this situation, and will provide a way forward for our community.
|
||||
|
||||
[we raised this issue]: https://github.com/mkdocs/mkdocs/issues/3695
|
||||
[maintainership changed]: https://github.com/mkdocs/mkdocs/discussions/3677
|
||||
[ground-up rewrite]: https://github.com/mkdocs/sketch
|
||||
[rendering only the page currently being built]: https://github.com/mkdocs/mkdocs/issues/3695#issuecomment-2117939743
|
||||
[mkdocstrings]: https://mkdocstrings.github.io/
|
||||
[built-in blog]: ../../plugins/blog.md
|
||||
[tags]: ../../plugins/tags.md
|
||||
[does not require or support plugins]: https://github.com/mkdocs/mkdocs/discussions/3815#discussioncomment-10398312
|
||||
[a team call on August 1]: https://github.com/mkdocs/mkdocs/discussions/3671#discussioncomment-10164237
|
||||
[we raised objections multiple times]: https://github.com/mkdocs/mkdocs/discussions/3671#discussioncomment-10215445
|
||||
|
||||
[^8]:
|
||||
Previously raised issues include [#2418], [#2384], and [#1900].
|
||||
|
||||
[#1900]: https://github.com/mkdocs/mkdocs/issues/1900
|
||||
[#2384]: https://github.com/mkdocs/mkdocs/issues/2384
|
||||
[#2418]: https://github.com/mkdocs/mkdocs/issues/2418
|
||||
|
||||
This brings us to collaboration, which wasn't initially on our list of priorities. However, throughout 2024, conversations with various organizations and maintainers of popular Open Source projects highlighted a frequent request for enhanced collaboration features. Many users expressed a need for capabilities that would allow non-technical team members to suggest and make changes to the documentation. We're genuinely grateful for this feedback, as it has come at a pivotal time. We recognize the need to streamline tracking and discussing changes, as well as to facilitate drive-by contributions.
|
||||
|
||||
__:octicons-goal-16: Goal – Everyone, regardless of their technical skill level, can easily work on and improve the documentation and contribute to a growing corpus of knowledge.__[^9]
|
||||
|
||||
[^9]:
|
||||
We are actively aligning our future development efforts to address this, recognizing it as a key area for improvement. While it's not something we can deliver immediately, we are committed to making this vision a reality in our work.
|
||||
|
||||
This focus on collaboration aligns with how knowledge is managed in enterprises. In large organizations, documentation often exists in [information silos] — decentralized yet essential repositories of information. We understand the need to be able to unify these disparate sources into a coherent body of knowledge while preserving decentralized ownership. This also nicely aligns with our previously outlined work on multi-project support, as well as the new search system to implement [federated search] among multiple projects.
|
||||
|
||||
[information silos]: https://en.wikipedia.org/wiki/Information_silo
|
||||
|
||||
### Large Language Models (LLMs)
|
||||
|
||||
Almost a year ago, we introduced an [experimental chatbot] on our documentation site. It quickly became one of the most anticipated features, with users requesting the ability to deploy similar functionality on their own sites, underscoring the growing demand for interactive documentation tools. However, we found that adding to the myriad of existing chat solutions and simply building another thin wrapper on top of ChatGPT is nonsense.
|
||||
|
||||
__:octicons-goal-16: Goal – We're envisioning creating a unified interface that seamlessly integrates advanced search, chat, and summarization features, providing an interactive documentation experience.__
|
||||
|
||||
[experimental chatbot]: https://github.com/squidfunk/mkdocs-material/discussions/6240
|
||||
|
||||
As we delved into this ambitious project, we gained valuable insights from user feedback. Users began interacting with the chatbot in their native languages, an outcome we hadn't anticipated given that our documentation is in English. Remarkably (or obviously to those that work on LLMs year round), the chatbot responded in the same language. This ability of LLMs is one of the genuinely exciting features of these machine learning models as it has the potential to improve the accessibility of the documentation. However, while we employed state-of-the-art RAG methodologies, the results were mixed, and occasional hallucinations surfaced.
|
||||
|
||||
These experiences led us to prioritize enhancing our search capabilities before integrating LLM-based features. Building a search engine from scratch was already a substantial effort, and adding more complexity without a solid search foundation would be premature. By rearchitecting our search functionalities, we aim to create a robust platform that will seamlessly support advanced information retrieval and deliver a cohesive interactive documentation experience.
|
||||
|
||||
## Team, transparency, and growth
|
||||
|
||||
While we navigate the challenges and explore the opportunities of this project, we consider it essential to demonstrate how we're building a solid foundation for its continued growth and success. Please consider this an overview rather than a formal roadmap — our detailed plans are in the works. The goals we've highlighted represent the most impactful areas we aim to address.
|
||||
|
||||
Thanks to the generous support from our sponsors, we're fortunate to be assembling a team capable of dedicating significant time and expertise to this endeavor. This newfound capacity allows us to delve deeper into core development while also engaging more comprehensively with our user community. A special mention goes to @kamilkrzyskow, one of our invaluable community experts, who has been essential in supporting users and fostering discussions on our platform.
|
||||
|
||||
With the team's support, @squidfunk can concentrate on the heart of development, while we have begun investing in user research. This effort is helping us understand how organizations and individuals interact with our tools, guiding the project's future direction based on real feedback from numerous conversations with users and companies.
|
||||
|
||||
Looking to expand our team further, we are committed to improving transparency and communication. Our previous work often happened behind the scenes due to time constraints, but we're now focused on making our processes more open and inviting for new contributors. By embracing this collaborative approach, we aim to enhance our tools and ensure they meet the evolving needs of our community.
|
||||
|
||||
## What's ahead of us
|
||||
|
||||
As we look into the future, some of the groundwork we're laying now is crucial for the exciting developments on the horizon. Many of the initiatives we've discussed represent foundational work that will set the stage for much more ambitious and innovative features. Once these core elements are in place, we'll deliver a range of exciting new functionalities that align with our vision and goals.
|
||||
|
||||
In the coming months, we'll be sharing more details about our plans and how they will contribute to our overarching objectives. While growth and innovation are at the forefront of our plans, we want to assure you that our core values remain unchanged. We are committed to maintaining the principles that have guided us thus far, ensuring that our growth is both healthy and consistent:
|
||||
|
||||
- Against recent industry trends with companies moving away from core ideas of Open Source, we are doubling down on our commitment to Open Source because we believe it's at the heart of the value proposition of our work and the [docs as code] approach.
|
||||
|
||||
[docs as code]: https://www.writethedocs.org/guide/docs-as-code/
|
||||
|
||||
- Our [organic approach to growth] is part of this strategy as it keeps us independent of individual funding sources and pressures to provide a return on investment, which is what causes many other projects to turn away from the principles of Open Source.
|
||||
|
||||
[organic approach to growth]: https://star-history.com/#squidfunk/mkdocs-material
|
||||
|
||||
- Likewise, we are driven by the needs of the community for a rich ecosystem of adaptations of the core framework. Extensibility and modularity are key for this and we are working hard to improve the developer experience by providing clear interfaces.
|
||||
|
||||
Stay tuned for updates as we continue to build on our progress and explore new possibilities. We're excited about the future and look forward to sharing more with you as we advance.
|
||||
|
||||
_Martin, Alex and Kathi_ :octicons-heart-fill-24:{ .mdx-heart .mdx-insiders }
|
||||
@@ -0,0 +1,236 @@
|
||||
---
|
||||
date: 2025-11-05
|
||||
authors:
|
||||
- squidfunk
|
||||
- alexvoss
|
||||
- katharinalisalin
|
||||
- pawamoy
|
||||
categories:
|
||||
- General
|
||||
description: >
|
||||
We are thrilled to announce Zensical, our next-gen static site generator
|
||||
that addresses and overcomes the technical limitations of MkDocs
|
||||
social:
|
||||
cards_layout: default/only/image
|
||||
cards_layout_options:
|
||||
background_image: docs/assets/images/zensical-social.png
|
||||
title: Zensical - A modern static site generator
|
||||
slug: zensical
|
||||
---
|
||||
|
||||
# Zensical – A modern static site generator built by the Material for MkDocs team
|
||||
|
||||
__We are thrilled to announce [Zensical], our next-gen static site generator designed to simplify the process of building documentation sites. Distilled from a decade of experience, Zensical is our effort to overcome the technical limitations of MkDocs, reaching far beyond its capabilities.__
|
||||
|
||||
Zensical is the result of thousands of hours of work – built from the ground up for a modern and comfortable authoring experience, while making it easy for developers to extend and customize Zensical through its upcoming module system. Our goal is to support docs-as-code workflows with tens of thousands of pages, without compromising performance or usability.
|
||||
|
||||
To make the transition seamless, [compatibility] comes first. We're putting significant effort into ensuring a smooth migration from Material for MkDocs for all users. Zensical can natively read `mkdocs.yml`, allowing you to build your existing project with minimal changes. As of now, a subset of plugins is supported, and we're working on feature parity in the coming months.
|
||||
|
||||
Zensical is fully Open Source, licensed under MIT, and can be used for any purpose, including for commercial use. We're also saying goodbye to our sponsorware model, replacing it with our new offering for professional users: [Zensical Spark]. This allows us to stay independent, maximizing user value, as we shape the future of Zensical together with you.
|
||||
|
||||
_You can subscribe to [our newsletter] to stay in the loop_.
|
||||
|
||||
<!-- more -->
|
||||
|
||||
[Zensical]: https://zensical.org/
|
||||
[compatibility]: #maximum-compatibility
|
||||
[Zensical Spark]: #zensical-spark
|
||||
[our newsletter]: https://zensical.org/about/newsletter/
|
||||
|
||||
---
|
||||
|
||||
__This is the second article in a four-part series:__
|
||||
|
||||
1. [Transforming Material for MkDocs]
|
||||
2. [Zensical – A modern static site generator built by the creators of Material for MkDocs]
|
||||
3. [Material for MkDocs Insiders – Now free for everyone]
|
||||
4. [Goodbye, GitHub Discussions]
|
||||
|
||||
[Transforming Material for MkDocs]: transforming-material-for-mkdocs.md
|
||||
[Zensical – A modern static site generator built by the creators of Material for MkDocs]: zensical.md
|
||||
[Material for MkDocs Insiders – Now free for everyone]: insiders-now-free-for-everyone.md
|
||||
[Goodbye, GitHub Discussions]: goodbye-github-discussions.md
|
||||
|
||||
## Why Zensical?
|
||||
|
||||
Since its initial release in 2016, Material for MkDocs has helped tens of thousands of teams to publish and maintain reliable documentation. However, in recent years, it has become apparent that we were running up against limitations of our core dependency, MkDocs. These limitations proved impossible to overcome as they are deeply rooted in its architecture.
|
||||
|
||||
We also mentioned in our [update on our foundational work] that MkDocs must be considered a supply chain risk, since it's unmaintained since August 2024. It has seen no releases in over a year and is accumulating unresolved issues and pull requests. These developments have forced us to cut our ties to MkDocs as a dependency.
|
||||
|
||||
In order to map out a path forward, we went back to the drawing board, talked to dozens of our professional users and thoroughly analyzed the MkDocs ecosystem. We didn't just want to create a fork or port of MkDocs, but decided to rethink static site generation from first principles.
|
||||
|
||||
With Zensical, we are creating a modern static site generator, which is compatible with your content and customizations, and addresses MkDocs' limitations. While Material for MkDocs is built on top of MkDocs, __Zensical consolidates both projects into one coherent stack__, covering static site generation, theming, and customization. What you can expect today:
|
||||
|
||||
- [5x faster rebuilds](#authoring-experience)
|
||||
- [Modern design](#modern-design)
|
||||
- [Blazing-fast search](#blazing-fast-search)
|
||||
|
||||
Although we haven't reached full feature parity yet, you can already use Zensical to build your existing Material for MkDocs projects with minimal changes.
|
||||
|
||||
_You can jump to the [compatibility] section to learn what is already supported._
|
||||
|
||||
[update on our foundational work]: https://github.com/squidfunk/mkdocs-material/discussions/8461
|
||||
|
||||
## What you can expect
|
||||
|
||||
### Solid foundation
|
||||
|
||||
Our goal with Zensical is to create a coherent and modern stack, vertically integrating all parts of the authoring experience (AX), developer experience (DX), and user experience (UX). This gives us a significant competitive advantage over solutions that overly rely on third-party frameworks and dependencies, helping us to create much more robust Open Source software.
|
||||
|
||||
[ZRX], our new differential build engine, creates a solid foundation for Zensical, and is an Open Source project of its own. It's a fresh take on making differential data flows easy to build and a joy to work with. Most engineering effort has gone into ZRX, as it forms the backbone of Zensical, and will allow us to ship features faster.
|
||||
|
||||
Following the principle of architectural hoisting, we moved essential, reusable functionality into ZRX, which allows us to keep Zensical's core simple and focused on static site generation. ZRX handles the heavy lifting – differential builds, caching, and data flow orchestration.
|
||||
|
||||
With the upcoming [module system] and [component system], both of which are on our public [roadmap], Zensical will gain more degrees of freedom in the coming months, allowing you to extend and customize Zensical in ways that were previously impossible with MkDocs.
|
||||
|
||||
[ZRX]: https://github.com/zensical/zrx/
|
||||
[module system]: https://zensical.org/about/roadmap/#module-system
|
||||
[component system]: https://zensical.org/about/roadmap/#component-system
|
||||
|
||||
### Modern design
|
||||
|
||||
Zensical brings a fresh, modern design that breaks out of the Materal Design aesthetic, creating a visual foundation that is more easily brandable and adaptable to different use cases. The new design prioritizes clarity, simplicity, and usability, while having a more professional finish:
|
||||
|
||||
<figure markdown>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<figcaption markdown>Our public [roadmap], built with Zensical</figcaption>
|
||||
</figure>
|
||||
|
||||
Right now, the layout and site structure of Zensical match Material for MkDocs closely, as we're focusing on ensuring maximum compatibility. Once we finish work on our upcoming [component system], we'll provide an alternative that is much more flexible and adaptable, and can be tailored to different use cases and branding requirements more easily.
|
||||
|
||||
_You can also keep the Material for MkDocs look and feel with a single line of configuration._
|
||||
|
||||
[roadmap]: https://zensical.org/about/roadmap/
|
||||
|
||||
### Blazing-fast search
|
||||
|
||||
Client-side search isn't a compromise – for the vast majority of static sites, it's the best solution, since it's faster, involves zero maintenance, and doesn't require you to pay for a service.
|
||||
|
||||
As covered in depth in [the first part of this series], the current search implementation in Material for MkDocs has severe limitations, and is based on a now unmaintained library, which is why we decided to build a new search engine from scratch. It's based on the same goals as Zensical itself: performance, flexibility, and extensibility.
|
||||
|
||||
Disco, our modular and blazing-fast client-side search engine, is exclusively available in Zensical. When you build your site with Zensical, your users will immediately benefit from Disco's improved ranking algorithm, as well as its filtering and aggregation capabilities:
|
||||
|
||||
[the first part of this series]: transforming-material-for-mkdocs.md#search-and-discovery
|
||||
|
||||
<figure markdown>
|
||||
|
||||
|
||||
|
||||
|
||||
<figcaption markdown>Disco on [zensical.org]</figcaption>
|
||||
</figure>
|
||||
|
||||
In early 2026, we'll be releasing Disco as a standalone Open Source project. With the feedback of our professional users in [Zensical Spark], we're going to evolve the search experience, turning Disco into a highly configurable and customizable search engine that adapts to your needs.
|
||||
|
||||
_You can subscribe to [our newsletter] to receive news about Disco_.
|
||||
|
||||
[zensical.org]: https://zensical.org/
|
||||
|
||||
### Authoring experience
|
||||
|
||||
Slow feedback loops can be a major pain point when writing documentation. Almost all of us know the feeling of waiting for the static site generator to finish building the site, just to see a small change reflected in the output. With Zensical, we're finally addressing this issue.
|
||||
|
||||
It's important to understand that we're not yet utilizing the differential capabilities of [ZRX] to the fullest extent, as we're forced to make several compromises to ensure maximum [compatibility] with Material for MkDocs at the moment. Markdown rendering needs to go through Python Markdown, which forces us to pay for extra marshalling costs.
|
||||
|
||||
While the initial build can sometimes be slower than with MkDocs, repeated builds – especially when serving the site – are already 4 to 5x faster, as only changed files need to be rebuilt.
|
||||
|
||||
We're also working on a new Markdown toolchain based on a CommonMark-compliant parser written in Rust, which will make Markdown processing significantly faster. We'll be tackling this as part of the upcoming [component system], which we'll start working on in early 2026. Once our new Markdown toolchain is ready, we'll provide automated tools to translate between Python Markdown and CommonMark, so you don't need to manually migrate your content.
|
||||
|
||||
### Maximum compatibility
|
||||
|
||||
[Compatibility with Material for MkDocs] is our top priority. We understand that switching to a new static site generator can be challenging, especially for large projects with many customizations. Therefore, we've put significant effort into ensuring that Zensical understands `mkdocs.yml` configuration files, so that you can build your projects with minimal changes.
|
||||
|
||||
This means your existing Markdown files, template overrides, CSS and JavaScript extensions don't need to be touched, primarily because we did not change the generated HTML, and rely on Python Markdown for processing your content.
|
||||
|
||||
However, plugins are a different story. In MkDocs, practically all plugins have side effects, making it impossible to parallelize builds. We started from first principles and asked: what should extensibility look like in a modern static site generator?
|
||||
Our answer is the upcoming [module system], which takes a fundamentally different approach based on four core principles:
|
||||
|
||||
- Modules can inject, extend, and re-define functionality
|
||||
- Modules are deterministic through topological ordering
|
||||
- Modules foster reusability, with the possibility to remix them
|
||||
- Modules can cooperate through well-defined contracts
|
||||
|
||||
We're working on shipping essential functionality as provided by MkDocs plugins as built-in modules. In early 2026, we will open the module system to third-party developers, so they can start building their own modules, as we see Zensical as the heart of a thriving ecosystem.
|
||||
|
||||
[Compatibility with Material for MkDocs]: https://zensical.org/compatibility/
|
||||
[feature parity]: https://zensical.org/compatibility/features/
|
||||
[search]: ../../plugins/search.md
|
||||
[offline]: ../../plugins/offline.md
|
||||
|
||||
## Zensical Spark
|
||||
|
||||
Zensical Spark, [our offering for professionals], is the result of countless calls with professional users of Material for MkDocs. From startups to large enterprises, we enable organizations to realize complex projects in diverse environments. For this, we've created Zensical Spark as a collaborative space. If you're a professional user, Zensical Spark is for you, since:
|
||||
|
||||
- You can be confident that Zensical will continue to be developed and maintained in the long term as a set of interconnected and sustainable OSI-compliant Open Source projects.
|
||||
|
||||
- You can receive the support you need to successfully use, configure and customize Zensical in your organization, receiving first-class support from the Zensical team.
|
||||
|
||||
- You can influence the future development of Zensical by participating in [our new approach] to Open Source software development, helping us to build exactly what you need.
|
||||
|
||||
_Let's talk! If you're working in a professional context, reach out to hello@zensical.org to schedule a call and learn how Zensical Spark enables your team to transition to Zensical smoothly and have a voice in its continued development._
|
||||
|
||||
_You should also consider joining the [waiting list], since seats are limited._
|
||||
|
||||
[our offering for professionals]: https://zensical.org/spark/
|
||||
[our new approach]: https://zensical.org/docs/community/how-we-work/
|
||||
[waiting list]: https://zensical.org/spark/join/
|
||||
|
||||
## We're growing our team
|
||||
|
||||
We're also excited to announce that we're growing [our team]:
|
||||
|
||||
__Timothée Mazzucotelli, also known as @pawamoy, is joining Zensical!__
|
||||
|
||||
At Zensical, Tim is focusing on providing the same seamless experience for generating API reference documentation from source code (via docstrings) as he has done with [mkdocstrings], the second biggest project in the MkDocs ecosystem. With his expertise, and Zensical's new stack, we'll be pushing the boundaries of what's possible with API reference documentation.
|
||||
|
||||
[mkdocstrings]: https://mkdocstrings.github.io/
|
||||
[our team]: https://zensical.org/about/team/
|
||||
|
||||
## Goodbye, GitHub Sponsors
|
||||
|
||||
Thank you! To all of you who have supported us over the years through GitHub Sponsors – we are incredibly grateful for your support. It has been invaluable in helping us to build, maintain and evolve Material for MkDocs, and we couldn't have done it without you. __Seriously, thank you!__
|
||||
|
||||
Material for MkDocs gave us something invaluable: experience building for tens of thousands of users, and the opportunity to build a team around Open Source software. It showed us that making a living from Open Source isn't just possible – we grew it into one of the largest sponsorware projects on GitHub and inspired others to pursue similar paths.
|
||||
|
||||
Now we're breaking new ground. Zensical is our next chapter, and we're professionalizing how we approach Open Source development. Our vision is to make Zensical free for everyone to use while building a sustainable business around it through [our new approach].
|
||||
|
||||
This transition means saying goodbye to GitHub Sponsors. It has served us exceptionally well, but as we professionalize and scale, we're making the leap from personal project to company – building a business and team that can meet the growing demands of professional users while staying true to our values.
|
||||
|
||||
We're doubling down on Open Source, developing software for everyone.
|
||||
|
||||
_If you want to continue supporting our work, please subscribe to [our newsletter]. We'll be providing new methods to support us in the coming months, with the possibility of getting exclusive goodies._
|
||||
|
||||
## Looking Ahead
|
||||
|
||||
Material for MkDocs grew organically in a pot that eventually became too small. With Zensical, we're building on solid foundations designed to grow with us – and with you.
|
||||
|
||||
!!! warning "Material for MkDocs is now in maintenance mode"
|
||||
|
||||
We want to be transparent about the risks of staying on Material for MkDocs. With MkDocs unmaintained and facing fundamental supply chain concerns, we cannot guarantee Material for MkDocs will continue working reliably in the future. We're aware that transitioning takes time, which is why we commit to support it at least for the next 12 months, fixing critical bugs and security vulnerabilities as needed, but the path forward is with Zensical.
|
||||
|
||||
If documentation plays a critical role in your organization, and you're worried how this might affect your business, consider joining [Zensical Spark](https://zensical.org/spark/), or feel free to schedule a call by reaching out at hello@zensical.org.
|
||||
|
||||
### Where we'll be in 12 months
|
||||
|
||||
Over the next 12 months, following our [phased transition strategy], we'll reach Phase 2 and 3 – introducing our [module system] and [component system], as well as CommonMark support. By replacing Python Markdown with a Rust-based Markdown parser, we'll unlock performance improvements and the modularity needed for flexible templating. This is where Zensical truly starts to unfold its capabilities.
|
||||
|
||||
Zensical is already powering real projects due to extensive [compatibility with Material for MkDocs][Compatibility with Material for MkDocs]. We're actively working on closing the gap to reach full [feature parity].
|
||||
|
||||
You can [install Zensical now], and build your existing Material for MkDocs projects with it. If you run into a bug, please don't hesitate to [open an issue] – we're here to help.
|
||||
|
||||
### Connect with us
|
||||
|
||||
If you have questions we haven't addressed, please reach out to us at hello@zensical.org. We're currently collecting questions from the community about Zensical, and will address them in an FAQ section as part of our documentation in the coming weeks.
|
||||
|
||||
We're incredibly thankful that you have been part of our journey so far. With Zensical, we're embarking on a new chapter, and we couldn't be more excited to have you with us.
|
||||
|
||||
_You can subscribe to [our newsletter] to stay in the loop_.
|
||||
|
||||
[phased transition strategy]: https://zensical.org/compatibility/#phased-transition-strategy
|
||||
[install Zensical now]: https://zensical.org/docs/get-started/
|
||||
[GitHub repository]: https://github.com/zensical/zensical
|
||||
[open an issue]: https://zensical.org/docs/community/get-involved/
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 549 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 648 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 581 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 501 KiB |
Reference in New Issue
Block a user