Automating changelogs

We've been publishing ~weekly changelogs at Membrane as we approach our open beta launch. The process is pretty repetitive, so we wrote a Membrane program to handle the menial bits: membrane/changelog

The process

Each week on Friday, I go through the same process of looking through git commits since the last changelog, writing up a summary, and publishing.

Now, the changelog program runs on a cron every Friday, using the membrane/github, membrane/openai, and membrane/slack drivers to prep and send a changelog draft. It uses:

1. github to fetch commits since the last changelog
2. openai to draft the post, including past posts as context
3. slack to send us the draft
4. state to store version numbers and samples
5. Timers to run the program every Friday

The code

// changelog/index.ts

import { nodes, state } from "membrane";

export interface State {
  version: number;
  samples: Array<{ name: string; content?: string }>;
}

export async function draft() {
  const commits = await nodes.repo.commits.page.items.$query(
    "{ message author }"
  );

  const changes = commits.slice(
    0,
    commits.findIndex(({ message }) => {
      return message?.toLowerCase().includes(`changelog 0.${state.version}`);
    })
  );

  const prompt = `
    You will be drafting a weekly changelog for Membrane, a developer tools startup.
    Membrane is a fast, powerful way to write internal tools in TypeScript, connecting the apps you already use at work.
    The Membrane team writes a weekly changelog blog post to broadcast new features and bug fixes to users.
    Here are the GitHub commits that the team has shipped since the last changelog:
    ${JSON.stringify(changes)}
    And here are five sample changelogs written by the Membrane team, so that you can emulate their writing style:
    ${JSON.stringify(state.samples)}
    Please draft a changelog blog post for Changelog 0.${state.version + 1}.
    DO NOT INCLUDE ANY CONTENT FROM PREVIOUS CHANGELOGS IN YOUR DRAFT!
    The draft you create should only include content from the GitHub commits provided.
    The samples are provided purely to help you with formatting and writing style.
  `;

  const gpt = nodes.openai.models.one({ id: "gpt-4-turbo" });
  const messages = [{ role: "user", content: prompt }];
  const completion: any = await gpt.completeChat({ messages });
  const draft = completion.content;

  await nodes.slack.sendMessage({
    text: `*Reminder: publish changelog 0.${
      state.version + 1
    }*\nHere's a draft:\n\`\`\`\n${draft}\n\`\`\``,
  });
}

Make it your own

Truth be told, I much prefer writing the changelog myself. Writing is the enjoyable part, and I generally dislike LLM-generated blog posts. So I may end up removing the openai connection of this program and just keep the time-saving core logic. I'd also consider:

1. Using membrane/email, membrane/sms, or membrane/discord instead of slack to send the draft
2. Using the membrane/height driver to create a task for each changelog
3. Creating an action to mark each changelog as done, e.g. by responding to the email or subscribing to a GitHub commit event
4. Opening a PR on GitHub rather than just sending a draft on Slack

If you'd like a hand reworking this program to your needs, send us a note on Discord, via email, or book a live session.

- Pete Millspaugh (pete@membrane.io)