Docusaurus does not have prebuilt options for creating drafts. By drafts, I mean documents that should be viewable during development but not in production. I wanted my drafts to be fully deleted and not just hidden from the production build.

I came up with this simple method to selectively deploy drafts to selectively release drafts while being able to write, save, and preview drafts that are not yet ready.

The solution I came up with is to prepend draft folders and files with DRAFT, and remove these files within the deployment pipeline. I hope the following post is helpful!

Indicating draft folders and files​

Since my pipeline systematically deletes folders and files marked as drafts, I needed a quick way to distinguish draft documents. I toyed with the idea of creating JSON documents to store draft status but ultimately decided that special folder/file names would be easy and sufficient for my purposes.

Start by prepending draft folders and files with DRAFT like so:

• 2023-05-21-create-and-deploy-docusaurus becomes DRAFT_2023-05-21-create-and-deploy-docusaurus
• create-and-deploy-docusaurus.md becomes DRAFT_create-and-deploy-docusaurus.md

Docusaurus documents and blog posts can either be stored as single markdown files, or as folders with multiple markdown files, a media folder, etc. We will delete all files as well as recursively delete directories marked as DRAFT. If only part of a directory is not ready for release, simply don't mark the parent folder as a draft.

Removing draft folders and files​

This blog is built and deployed using an Azure DevOps Pipeline written in YAML. Some configure Azure to deploy to Static Web App upon pushing a build to a remote repo. Here's what my pipeline looked like before implementing the drafting abilities:

build-and-deploy.yaml (before draft support)

trigger: none

pool:
  vmImage: ubuntu-latest

variables:
  sourceDirectory: $(System.DefaultWorkingDirectory)
  docusaurDirectory: docusaur

steps:
  - task: AzureStaticWebApp@0
    inputs:
      app_location: $(docusaurDirectory)
      app_build_command: npm run build
      output_location: build
    env:
      azure_static_web_apps_api_token: $(swa_deployment_token)

The AzureStaticWebApp@0 task runs relative to the virtual machine's default working directory. System.DefaultWorkingDirectory is a variable predefined by Azure for the path where source code is automatically downloaded. You can read more about predefined variables at Azure DevOps System Variables.

System.DefaultWorkingDirectory is the path of the source code copy. Deleting folders and files within this directory will prevent them from being published in later steps.

To remove folders and files starting with DRAFT, we can add the following script to the pipeline. Please read on to understand the parts before implementing the script as rm -rf is dangerous and may result in file loss:

build-and-deploy.yaml (excerpt)

- script: |
    find $(sourceDirectory)/$(docusaurDirectory) -type f,d -name 'DRAFT*' -exec rm -rf {} +
  displayName: 'Remove draft files and directories.'

Let's break down the script.

• find $(System.DefaultWorkingDirectory)/$(docusaurDirectory): find is a bash command for recursively searching for files. The path $(System.DefaultWorkingDirectory) is the root of the locally-downloaded source code, while /$(docusaurDirectory) is the root of my Docusaurus project. This path may vary based on how your project is set up.

• -type f,d: Search for files (f) and directories (d).

• -name 'DRAFT*': Find files and directories that start with DRAFT. You could also use DRAFT_* if you want the underscore to be significant.

• exec <command> {} +: As files or directories are found, execute <command> on the current file, represented by the placeholder {}. + signifies the end of the -exec command.

• rm -rf: Remove files and directories recursively (forced). This deletes everything within a file tree. While rm -r does the same thing, it asks the user at each file and directory whether they want to proceed. rm -rf is forced, so it will indiscriminately delete everything in that file tree. This portion of the script should be considered dangerous! We are running it on a copy of source-controlled code, so deleting the wrong files can easily be undone. If you run rm -rf in the wrong place, however, you could cause permanent file loss.

Full pipeline​

Here's the full pipeline. Be sure to make changes to fit this solution to your project structure.

trigger: none

pool:
  vmImage: ubuntu-latest

variables:
  sourceDirectory: $(System.DefaultWorkingDirectory)
  docusaurDirectory: docusaur

steps:
  - script: |
      find $(sourceDirectory)/$(docusaurDirectory) -type f,d -name 'DRAFT*' -exec rm -rf {} +
    displayName: 'Remove draft files and directories.'

  - task: AzureStaticWebApp@0
    inputs:
      app_location: $(docusaurDirectory)
      app_build_command: npm run build
      output_location: build
    env:
      azure_static_web_apps_api_token: $(swa_deployment_token)

Publishing drafts​

To publish drafts, simply remove the DRAFT prefix and run the pipeline.

Limitations​

This method is quick and easy, but it has a few limitations.

1. Docusaurus internal linking depends on the file names. If a document contains a 'Go to Final Section' link, the link will break after the draft prefix is removed.

2. Git source control has no concept of moving or renaming files. Removing the 'DRAFT' prefix will appear to Git as a deletion of the original file and creation of a new file. No file history is lost, but it may make reverting to previous versions more difficult.

3. It requires a YAML-style deployment. I imagine it could be implemented with other methods with some changes.

Thanks for reading!​

I hope this method is helpful or helps you think of a better way to enable drafting in Docusaurus! Thanks for stopping by.

Fixed route transit agencies typically have timetables defining the daily schedule of their buses. To make route information available of Google Maps, transit agencies must conform their data to the General Transit Feed Specification (GTFS).

I created a GTFS schedule for University of Wyoming's bus service using:

1. Python for data engineering and
2. React for visualization

This post is a brief overview of the project, and more detailed information will be soon available in the Projects tab of this blog, or at Introduction to the General Transit Feed Specification.

Motivation​

With this new blog, I needed some projects to write about. Being a bus driver for University of Wyoming, publishing their data to Google Maps is something I've been wanting to do for a while. In fact, last year I spent months developing an awesome full-stack web app for creating and publishing route timetables called SmallTown Transit.

The app ended up being a never-ending project and was a huge lesson in Agile development (Check out the Agile Manifesto if you are unfamiliar with Agile).

I aimed in this project to have a shippable product in two weeks, and that's what I did.

Outline​

1. Clean the existing timetables. UWYO's timetables are Excel documents that are made to be human-readable. I created standardized CSV documents where every cell is plain text because this would be easier to manage with Python.

2. Write a Python script for creating the feed. Some of the txt files in the feed are manually created, while others like trips.txt, stop_times.txt, and shapes.txt lend themselves to being programmatically generated.

3. Create a quick React app to view and check the data for accuracy. Using the Google Maps API, I am able to view the stop locations and route shapes.

Cleaning the Timetables​

I started the project with timetables from the transit department. These show a leave time for every bus stop along a route.

The original tables​

In the original timetables columns are stops ('Downtown' and 'East Lot'), rows are trips (one lap around a route), and cells are stop times ('6:30 AM at Downtown'). See the picture below.

These provided timetables are great for human readability as well as recalculating times. The cells are variously colored, merged, and contain plain text, numbers, and calculations.

I needed to:

1. Flatten these tables into standard, 100% plain text CSV files.
2. Replace stop names ('Walmart') with stop IDs ('WM') so that I could reference a stops.txt table.
3. Add information about stop types (ie. can passengers board here? Do passengers need to notify the driver to stop here?). This required insight into how the department and routes work, as this information is not included in their timetables.

The formatted table​

The exclamation mark designate a stop request: A stop where passengers must get the driver's attention (typically by pulling the stop request cord).

I likewise used the question mark and pipe to designate an interpolated timepoint and a do-not-board stop respectively.

More symbols could be added, but these few allowed for the layers of information I needed.

Writing a Python Script​

I chose to use Python to create the GTFS feed. I used a Jupyter Notebook for quick development and integrated documentation.

The main purpose of the Python script is to convert formatted timetables to the CSV files required by the GTFS. The below tables demonstrates the transition from timetable to stop_times and trips. The tables are simplified versions of the files required by GTFS. Notice how each timetable row gets a row in trips.txt and each cell gets a row in stop_times.txt

Python script input (simplified)​

link_timetable.csv

Python script output (simplified)​

Python script other tasks​

Beyond creating stop_times.txt and trips.txt, the Python script also handles:

1. Importing all necessary data files (stops.xlsx, routes.xlsx, agency.xlsx, etc.), processing data if necessary, and exporting them as .txt files.

2. Generate shapes.txt using Google Maps Directions API. shapes.txt is used to display a colored line for a given route on Google Maps. Directions API returns an encoded polyline which must be decoded. Luckily, there is a Python package for this.

Creating a React App for Viewing and Verifying GTFS​

The React App I built can be found on this site at the GTFS Viewer. If you don't have a GTFS feed, look at the screenshots below from the GTFS Viewer. For the app, I used MDX, the combination of markdown and JSX that allows for directly using React components in markdown files.

Viewing route shapes and stop locations​

Viewing stop times by stop​

This app ended up being helpful not only for viewing and verifying data, but also showing the transit agency their data in a way that is easy to visualize.

Final Thoughts​

I enjoyed building this project. My department is still in the process of applying to be a Google Transit Partner, but before long, this data should be visible on Google Maps.

Thanks for stopping by, and watch out for more posts soon!

Documentation is an necessary element of most projects. Facebook's open source project, Docusaurus, makes creating beautiful and clean documentation easy. We will deploy the Docusaurus app onto Azure Static Web Apps using Azure DevOps and a pipeline written in YAML.