tildes-issue-log/Development.md

Development Guide

Welcome to the Tildes Issue Log development guide, this will be very concise.

Setup

1. Clone the repository.
2. Create a config.json file from the sample config.sample.json file. The gitlabToken should be a Personal Access Token with the api scope available.

Project Overview

The Tildes Issue Log is set up with 7 scripts, 6 of which can be run on their own.

If you want to run any of these on their own, use yarn ts-node 'source/scripts/<script>.ts'.

These 7 are the following:

assets.ts

Extracts the fonts and favicons .tar files located in source/pages/assets/ and outputs them in public/.

download.ts

Downloads all the commit, issue and merge request data with the GitLab API and saves it to temp/data/. If any of these already exist for today (meaning you have downloaded them previously) it won't try to redownload them and will use the existing ones instead.

feeds.ts

This is the script that can't be run on its own.

Generates the Atom, JSON and RSS feeds from the post list that is created by html.ts. The feeds are output to public/feed.{atom,json,rss}.

html.ts

Takes in all the generated data by download.ts, official-topics.ts and statistics.ts, and the Markdown posts in source/pages/posts/ and generates all the HTML for the site.

The HTML is generated using Nunjucks templates, all of which can be found in source/pages/templates/.

The HTML generation also linkifies Tildes-style @user mentions and ~group mentions.

This script also writes a Markdown file to temp/email.md to be used for the mailing list. If you plan on sending the email, don't forget to change the header so instead of it being something like this:

<h2 id="may-2020">
May 2020
<span id="authors">by @Bauke</span>
</h2>

It looks something like this:

# Tildes Issue Log

## May 2020 by @Bauke

official-topics.ts

Scrapes ~tildes.official and extracts all the official topics that were posted after the latest one we have saved in source/pages/data/official-topics.json.

redirects.ts

Creates HTML files for the posts before the 2.0.0 redesign that contain a <meta http-equiv="refresh"> tag that redirects to the new wanted location after 5 seconds.

The old URL scheme was https://til.bauke.xyz/posts/<month>-<year>.html.

The current URL scheme is https://til.bauke.xyz/<year>/<month>.html.

An example: https://til.bauke.xyz/posts/january-2020.html

statistics.ts

Generates various statistics from the downloaded data download.ts creates.

Writing Posts

To start writing a new post, create the Markdown file as source/pages/posts/<year>/<month>.md. Every post should contain the same header similar to this one:

<h2 id="may-2020">
  May 2020
  <span id="authors">by @Bauke</span>
</h2>

The @user mentions are automatically linkified, so it's simple to add multiple authors.

After that it's all up to you. As mentioned in the html.ts section, @user mentions and ~group mentions are automatically linkified.

If you write and intend to publish a completely new post, don't forget to update the latest post link in the ReadMe.

Building

To test and build the entire site, you can run this command:

yarn test && yarn download && yarn statistics && yarn official-topics && yarn build

This will run all the scripts in the desired sequence. It's not necessary to run it like this, once you have successfully ran yarn download && yarn statistics && yarn official-topics once you can run yarn build by itself after, but this way everything is generated and ready for the post.

yarn build runs all the build:... commands inside package.json. Some of these commands are do not run scripts, so if you want to build the entire site use yarn build to include everything.

Testing/Linting

The JavaScript and TypeScript is linted by XO. If XO ever complains, you can usually fix it quickly with xo --fix. If not it will usually tell you what exactly to fix.

The SCSS is linted by Stylelint with an XO-like config.

If you only want to lint JS/TS, use yarn xo.

If you only want to lint SCSS, use yarn stylelint 'source/pages/scss/**'. Stylelint might try to lint the compiled CSS in public/ without the specified path and blow up with errors, so don't forget to include it.

You can run both of these linters at the same time with yarn test.

Committing

If you have written a new post, there's a few things you should do before you commit.

• Proofread the new post.
• Build the site, copy the entire contents of the post from the site. (CTRL+A and then CTRL+C). Then paste this into LibreOffice Writer or equivalent program and do a spellcheck. Ignore any false positives it might mark like @user mentions or unknown but correct words.
• Make sure there are new statistics for this month in source/pages/data/statistics.json. Even if they are all 0, they should be included.
• If any official topics were posted, make they are added correctly in source/pages/data/official-topics.json.

Deploying

Deployment is completely and automatically handled via the GitLab CI/CD and hosted with GitLab Pages.

The CI/CD will first run yarn test to make sure everything is good. And then it will run yarn build. If successful it will save the generated public/ directory as artifacts and upload it to be used on the live site. If either of these steps failed nothing will be uploaded.