Progress of your scripts

A script that imports data, a nightly job, a long migration: the town can show how far it got, with no SDK and no dependency. The script writes one line of JSON on its standard output each time it has news, and Myrtille turns it into the progress, the message or the state of its citizen. Without Myrtille, it is an ordinary log line that nobody minds.

This is level 2 of the integration of Myrtille: one log line in a script. Nothing else changes on your side.

The convention

One JSON object per line, on standard output, with the key myrtille:

{"myrtille":"progress","done":42,"total":120,"unit":"poi","message":"Import Lisbon"}
{"myrtille":"state","state":"error","message":"Quota reached"}

A progress line:

Key	Required	Value
myrtille	yes	"progress"
done	yes	a whole number, 0 or more
total	no	a whole number, 1 or more
unit	no	a short word, such as poi or city (40 characters are kept)
message	no	what the script is doing, shown in the panel of the citizen (500 kept)

A state line:

Key	Required	Value
myrtille	yes	"state"
state	yes	one of idle, running, waiting, error, done, offline, unknown
message	no	why, shown in the bubble of an error and in the panel of the citizen (500 kept)

The rules:

• The line is the JSON object alone, at most 4,096 characters. Spaces around it are fine; a log prefix before it is not.
• Any other key is ignored and never leaves the machine. A line that does not follow the convention is an ordinary log line.
• Standard output only: standard error stays a log, whatever it holds.
• At most one progress and one state per citizen and per second reach the town: write as often as you like, the latest one wins. A state said again unchanged is sent again only after 30 seconds. A run of myrtille run reads at most 1,200 structured lines a minute; past that, the lines in between are skipped, but its latest progress and its latest state still reach the town, so the town always ends on the last state the script wrote, and myrtille run says so once. All the scripts of a workstation send at most 600 events a minute. A busy script can neither push the permission requests of Claude Code out of the queue of the agent nor spend what the town hall accepts from it.
• The message and the unit are masked like the logs of the town (tokens, passwords, keys, addresses with credentials) before they leave the machine. The masking is a safety net, not a guarantee: never write a secret in them.

Where the lines are read

• Scripts of a workstation, through myrtille run and the local agent of the workstation (below).
• Containers of a server, through the Docker agent that Coolify deploys beside them: the lines of a container update the citizen of that container. Nothing to add but the lines.

Run a script with myrtille run

On a workstation where the local agent runs (Claude Code explains how to connect it with myrtille connect and myrtille agent):

myrtille run -- python3 import_pois.py
myrtille run --name nightly-report -- bash report.sh --since yesterday

• Everything after -- is your command, as you would type it. Its output reaches your terminal unchanged, and myrtille run ends with its exit code.
• The script becomes the citizen local:<agent>:script.<name>, a citizen of kind automation. Its name is the file it runs (import_pois.py gives import-pois), or --name (lower-case letters, digits and dashes).
• The town places it in the district of the repository whose folder you run it from (repo.path of the town plan), else in a district of kind local, never in production.
• It is running while the script runs, done when it exits with 0, and error with its exit code otherwise, which also puts an incident in the mail. A state line of the script changes the state in between.
• Only the lines that follow the convention go to the agent, over its private channel on the workstation. Every other line stays in your terminal.
• The agent says every minute that the script still runs, so a quiet script of several hours keeps its colour. If the command is killed without a word, the citizen turns grey after ten minutes: the town never shows a state it does not know.
• Without an agent, the script runs all the same; myrtille run says so once.

What the town shows

While the citizen is running, its bubble shows the progress: 42/120 poi, 42/120, 7 city or 7. The panel of the citizen shows the progress and the last message. A state line moves the citizen within a second: error makes it shake with its message in the bubble, done makes it jump.

Examples

The three examples do the same work: five cities, one progress line after each, and a state line at the end. Run them as they are to see the lines, then with myrtille run to see them in the town. They are also in the repository, under packages/agent/examples/structured-logs/, and the test suite of Myrtille runs all three through the local agent and the town hall.

bash

printf writes the line; for text that may hold quotes or backslashes, build the line with jq -nc instead.

#!/usr/bin/env bash
# The structured log convention of Myrtille (level 2): one JSON line on standard output per update.
# Without Myrtille these are ordinary log lines. With it: myrtille run -- bash progress.sh
set -euo pipefail

cities=(lisbon porto faro braga coimbra)
total=${#cities[@]}
count=0
for city in "${cities[@]}"; do
  echo "importing ${city}" # an ordinary line: it stays in the terminal
  sleep 0.1
  count=$((count + 1))
  printf '{"myrtille":"progress","done":%d,"total":%d,"unit":"city","message":"Imported %s"}\n' \
    "$count" "$total" "$city"
done
echo '{"myrtille":"state","state":"done","message":"5 cities imported"}'

Node

// The structured log convention of Myrtille (level 2): one JSON line on standard output per update.
// Without Myrtille these are ordinary log lines. With it: myrtille run -- node progress.mjs
const myrtille = (fields) => {
  process.stdout.write(`${JSON.stringify(fields)}\n`);
};
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

const cities = ['lisbon', 'porto', 'faro', 'braga', 'coimbra'];
for (const [index, city] of cities.entries()) {
  console.log(`importing ${city}`); // an ordinary line: it stays in the terminal
  await sleep(100);
  myrtille({
    myrtille: 'progress',
    done: index + 1,
    total: cities.length,
    unit: 'city',
    message: `Imported ${city}`,
  });
}
myrtille({ myrtille: 'state', state: 'done', message: `${cities.length} cities imported` });

Python

flush=True matters: when the output is a pipe, Python keeps it in a buffer, and the town would only see the progress at the end. python3 -u does the same for a whole script.

#!/usr/bin/env python3
# The structured log convention of Myrtille (level 2): one JSON line on standard output per update.
# Without Myrtille these are ordinary log lines. With it: myrtille run -- python3 progress.py
import json
import time


def myrtille(**fields):
    # flush=True: a pipe is buffered, and the town would see the progress only at the end.
    print(json.dumps(fields, separators=(",", ":")), flush=True)


cities = ["lisbon", "porto", "faro", "braga", "coimbra"]
for index, city in enumerate(cities, start=1):
    print(f"importing {city}", flush=True)  # an ordinary line: it stays in the terminal
    time.sleep(0.1)
    myrtille(myrtille="progress", done=index, total=len(cities), unit="city", message=f"Imported {city}")
myrtille(myrtille="state", state="done", message=f"{len(cities)} cities imported")