Skip to content

About

A durable croak for ramabana and vishalakshi

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

27 Commits

Folders and files

Repository files navigation

pobblebonk

the clock and the notebook for an agent

pobblebonk adds callbacks, durable lists, and per-reader notes to honker. Schedules, queued work, list items, retries, and notes share one SQLite file.

There is no scheduler daemon. Call Pob.tick() on a heartbeat, and each tick asks honker for due fires and runs their callbacks. pobblebonk.heartbeat installs that recurring call through cron, launchd, or Task Scheduler.

Install

uv add pobblebonk

Python 3.12 or later is required.

Schedule a callback

Register a callback, add its schedule, then call tick. The callback return value becomes a note.

import time

pob = Pob()
beats = []

@pob.on('heartbeat')
def beat(fire):
    beats.append(fire.fire_at)
    return f'beat {len(beats)} at {fire.fire_at}'

pob.add('heartbeat', every='1s')
for _ in range(3):
    time.sleep(1.05)
    pob.tick()

beats
[1788060622, 1788060623, 1788060624]

fire.fire_at is the scheduled boundary, not the time the callback happened. The one-second gaps show that the cadence held.

[b-a for a, b in zip(beats, beats[1:])]
[1, 1]

drain returns the notes a reader has not seen. Each reader has an independent cursor.

pob.drain('leela')
[{'title': 'heartbeat', 'body': 'beat 1 at 1788060622', 'ref': 1, 'used': 0, 'offset': 1}, {'title': 'heartbeat', 'body': 'beat 2 at 1788060623', 'ref': 2, 'used': 0, 'offset': 2}, {'title': 'heartbeat', 'body': 'beat 3 at 1788060624', 'ref': 3, 'used': 0, 'offset': 3}]
pob.drain('leela'), len(pob.drain('phone'))
([], 3)

Run ordinary Python

A callback is a Python function. It can call a library, write a file, or run a command.

import subprocess

pob2 = Pob()

@pob2.on('git roundup')
def roundup(fire):
    out = subprocess.run(['git', 'log', '--oneline', '--since=30 days ago'],
                         capture_output=True, text=True).stdout.strip().splitlines()
    return f'{len(out)} commits in the last 30 days'

pob2.add('git roundup', cron='0 17 * * *')      # 5pm daily
time.sleep(1.05)
pob2.tick(at=int(time.time()) + 86400).ran[0].result
'17 commits in the last 30 days'

Give a callback a durable list

push adds an item to a named list. A schedule with needs runs only when that list is not empty. Its callback receives the open items as fire.list.

The callback marks its items used when it returns. If it raises, the items remain open for the retry. A key makes repeated open items idempotent.

pob3 = Pob()

@pob3.on('cart')
def cart(fire):
    return 'added: ' + ', '.join(i.text for i in fire.list)

pob3.add('cart', cron='0 20 * * 3', needs='shopping')     # 8pm on Wednesdays
pob3.push('shopping', 'yoghurt, the greek one', key='img_2201.heic')
pob3.push('shopping', 'yoghurt, the greek one', key='img_2201.heic')   # the same photo twice
pob3.push('shopping', 'oat milk')
pob3.items('shopping').attrgot('text')
['yoghurt, the greek one', 'oat milk']
time.sleep(1.05)
got = pob3.tick(at=int(time.time()) + 7*86400).ran[0]
got.status, got.result, got.used
('ok', 'added: yoghurt, the greek one, oat milk', 2)

When the list is empty, the next fire is skipped. It is not a callback failure and produces no note.

time.sleep(1.05)
nxt = pob3.tick(at=int(time.time()) + 14*86400).ran[0]
nxt.status, nxt.why
('skipped', 'the shopping list is empty')

Use a model callback

A model is another callable dependency. The documentation build does not run this example because it needs a LiteRT model.

import rishi

pob4 = Pob()

@pob4.on('news')
def digest(fire):
    chat = rishi.Chat('gpt-4.1')
    topics = ', '.join(i.text for i in fire.list)
    return rishi.resp_text(chat(f'Name one thing worth reading about each of: {topics}. One line each.'))

pob4.add('news', every='1s', needs='interests')
pob4.push('interests', 'sanskrit grammar')
pob4.push('interests', 'sqlite internals')
time.sleep(1.05)
pob4.tick().ran[0].result.split('\n')
['- **Sanskrit grammar:** The concept of *Sandhi*—how sounds combine at word boundaries—reveals the language’s precision and flexibility.',
 '- **SQLite internals:** The B-tree structure in SQLite’s storage engine is central to how it efficiently manages and retrieves data on disk.']

Operate schedules

Use pause, resume, update, and drop to maintain schedules. update changes only the fields you pass. drop also unregisters the callback in the current process.

pob4.pause('news')
pob4.update('news', cron='30 7 * * *', retries=5)
pob4.resume('news')
pob4.drop('news')
True
# Queue due fires without running callbacks.
queued = pob.tick(run=False)

# Run queued fires separately, with a bounded batch.
results = pob.work(worker='scheduler', limit=100)

Failures and missed fires

A fire retries with exponential backoff when its callback raises. The default attempt budget is three. Set retries on Pob or on one schedule. After the final attempt, the fire is dead-lettered and a note records the error.

catchup='once' keeps the latest fire missed while the machine was off. This is the default. catchup='all' keeps every missed fire, and a positive integer keeps that many of the most recent. start= sets the first fire; without it the cadence picks the next boundary from now.

Use tick(run=False) when scheduling and callback execution belong in separate processes. It queues due fires without running them. work claims and runs queued fires.

pob = Pob('~/.pobblebonk/pob.db', retries=3)
pob.add('roundup', cron='0 17 * * *', catchup='once', retries=5)
RuntimeError: Database error: unable to open database file: ~/.pobblebonk/pob.db
�[31m---------------------------------------------------------------------------�[39m
�[31mRuntimeError�[39m                              Traceback (most recent call last)
�[36mCell�[39m�[36m �[39m�[32mIn[13]�[39m�[32m, line 2�[39m
�[32m      1�[39m �[38;5;66;03m#| eval: false�[39;00m
�[32m----> �[39m�[32m2�[39m pob = Pob(�[33m'~/.pobblebonk/pob.db'�[39m, retries=�[32m3�[39m)
�[32m      3�[39m pob.add(�[33m'roundup'�[39m, cron=�[33m'0 17 * * *'�[39m, catchup=�[33m'once'�[39m, retries=�[32m5�[39m)

�[36mFile �[39m�[32m~/code/pobblebonk/pobblebonk/core.py:65�[39m, in �[36mPob.__init__�[39m�[34m(self, path, db, retries, base)�[39m
�[32m     58�[39m �[38;5;28;01mdef�[39;00m�[38;5;250m �[39m�[34m__init__�[39m(�[38;5;28mself�[39m,
�[32m     59�[39m              path=�[38;5;28;01mNone�[39;00m,          �[38;5;66;03m# the file; None makes a temporary one�[39;00m
�[32m     60�[39m              db=�[38;5;28;01mNone�[39;00m,            �[38;5;66;03m# or a honker database something else already opened�[39;00m
�[32m     61�[39m              retries:�[38;5;28mint�[39m=RETRIES,�[38;5;66;03m# attempts a failing fire gets before it is dead-lettered�[39;00m
�[32m     62�[39m              base:�[38;5;28mfloat�[39m=BASE):   �[38;5;66;03m# seconds before the first retry; doubles per attempt�[39;00m
�[32m     63�[39m     �[38;5;66;03m# honker watches `PRAGMA data_version` on a file, so `:memory:` is not an option�[39;00m
�[32m     64�[39m     �[38;5;28;01mif�[39;00m db �[38;5;129;01mis�[39;00m �[38;5;28;01mNone�[39;00m �[38;5;129;01mand�[39;00m path �[38;5;129;01mis�[39;00m �[38;5;28;01mNone�[39;00m: path = Path(mkdtemp())/�[33m'�[39m�[33mpob.db�[39m�[33m'�[39m
�[32m---> �[39m�[32m65�[39m     �[38;5;28mself�[39m.db = db �[38;5;28;01mif�[39;00m db �[38;5;129;01mis�[39;00m �[38;5;129;01mnot�[39;00m �[38;5;28;01mNone�[39;00m �[38;5;28;01melse�[39;00m �[30;43mhonker�[39;49m�[30;43m.�[39;49m�[30;43mopen�[39;49m�[30;43m(�[39;49m�[30;43mstr�[39;49m�[30;43m(�[39;49m�[30;43mpath�[39;49m�[30;43m)�[39;49m�[30;43m)�[39;49m
�[32m     66�[39m     �[38;5;28mself�[39m.retries, �[38;5;28mself�[39m.base = �[38;5;28mmax�[39m(�[32m1�[39m, �[38;5;28mint�[39m(retries)), �[38;5;28mfloat�[39m(base)
�[32m     67�[39m     �[38;5;28mself�[39m.q, �[38;5;28mself�[39m.stream = �[38;5;28mself�[39m.db.queue(FIRES), �[38;5;28mself�[39m.db.stream(NOTES)

�[36mFile �[39m�[32m~/code/pobblebonk/.venv/lib/python3.13/site-packages/honker/_honker.py:1433�[39m, in �[36mopen�[39m�[34m(path, max_readers, watcher_backend, watcher_poll_interval_ms)�[39m
�[32m   1412�[39m �[38;5;28;01mdef�[39;00m�[38;5;250m �[39m�[34mopen�[39m(
�[32m   1413�[39m     path: �[38;5;28mstr�[39m,
�[32m   1414�[39m     max_readers: �[38;5;28mint�[39m = �[32m8�[39m,
�[32m   1415�[39m     watcher_backend: Optional[�[38;5;28mstr�[39m] = �[38;5;28;01mNone�[39;00m,
�[32m   1416�[39m     watcher_poll_interval_ms: Optional[�[38;5;28mint�[39m] = �[38;5;28;01mNone�[39;00m,
�[32m   1417�[39m ) -> Database:
�[32m   1418�[39m �[38;5;250m    �[39m�[33;03m"""Open a Honker database at `path`.�[39;00m
�[32m   1419�[39m 
�[32m   1420�[39m �[33;03m    `watcher_backend` selects the update-detection strategy:�[39;00m
�[32m   (...)�[39m�[32m   1431�[39m �[33;03m    when lower idle CPU matters more than lowest-latency wakeups.�[39;00m
�[32m   1432�[39m �[33;03m    """�[39;00m
�[32m-> �[39m�[32m1433�[39m     �[38;5;28;01mreturn�[39;00m Database(�[30;43m_core_open�[39;49m�[30;43m(�[39;49m
�[32m   1434�[39m �[30;43m        �[39;49m�[30;43mpath�[39;49m�[30;43m,�[39;49m
�[32m   1435�[39m �[30;43m        �[39;49m�[30;43mmax_readers�[39;49m�[30;43m=�[39;49m�[30;43mmax_readers�[39;49m�[30;43m,�[39;49m
�[32m   1436�[39m �[30;43m        �[39;49m�[30;43mwatcher_backend�[39;49m�[30;43m=�[39;49m�[30;43mwatcher_backend�[39;49m�[30;43m,�[39;49m
�[32m   1437�[39m �[30;43m        �[39;49m�[30;43mwatcher_poll_interval_ms�[39;49m�[30;43m=�[39;49m�[30;43mwatcher_poll_interval_ms�[39;49m�[30;43m,�[39;49m
�[32m   1438�[39m �[30;43m    �[39;49m�[30;43m)�[39;49m)

�[36mFile �[39m�[32m~/code/pobblebonk/.venv/lib/python3.13/site-packages/honker/_honker.py:23�[39m, in �[36m_core_open�[39m�[34m(path, max_readers, watcher_backend, watcher_poll_interval_ms)�[39m
�[32m     21�[39m �[38;5;28;01mdef�[39;00m�[38;5;250m �[39m�[34m_core_open�[39m(path, max_readers, watcher_backend=�[38;5;28;01mNone�[39;00m, watcher_poll_interval_ms=�[38;5;28;01mNone�[39;00m):
�[32m     22�[39m     �[38;5;28;01mfrom�[39;00m�[38;5;250m �[39m�[34;01mhonker�[39;00m�[34;01m.�[39;00m�[34;01m_honker_native�[39;00m�[38;5;250m �[39m�[38;5;28;01mimport�[39;00m �[38;5;28mopen�[39m �[38;5;28;01mas�[39;00m _open
�[32m---> �[39m�[32m23�[39m     �[38;5;28;01mreturn�[39;00m �[30;43m_open�[39;49m�[30;43m(�[39;49m
�[32m     24�[39m �[30;43m        �[39;49m�[30;43mpath�[39;49m�[30;43m,�[39;49m
�[32m     25�[39m �[30;43m        �[39;49m�[30;43mmax_readers�[39;49m�[30;43m=�[39;49m�[30;43mmax_readers�[39;49m�[30;43m,�[39;49m
�[32m     26�[39m �[30;43m        �[39;49m�[30;43mwatcher_backend�[39;49m�[30;43m=�[39;49m�[30;43mwatcher_backend�[39;49m�[30;43m,�[39;49m
�[32m     27�[39m �[30;43m        �[39;49m�[30;43mwatcher_poll_interval_ms�[39;49m�[30;43m=�[39;49m�[30;43mwatcher_poll_interval_ms�[39;49m�[30;43m,�[39;49m
�[32m     28�[39m �[30;43m    �[39;49m�[30;43m)�[39;49m

�[31mRuntimeError�[39m: Database error: unable to open database file: ~/.pobblebonk/pob.db

Long jobs

A fire’s claim lasts lease seconds, five minutes by default. While a callback runs, pobblebonk renews the claim every third of the lease. A job that takes a day keeps its claim, and no other work runs it twice. If the worker dies, the renewing stops and the fire is claimable again after the lease.

batch caps how many list items one fire takes. With batch=1 each fire gets the oldest open item and marks only that one used. The rest wait for later fires. Inside a callback, fire.note(body) leaves a progress note whose ref is the fire’s job ID, and fire.beat() renews the claim by hand.

pob5 = Pob(lease=600)

@pob5.on('render')
def render(fire):
    fire.note(f'starting {fire.list[0].text}')
    return f'rendered {fire.list[0].text}'

pob5.add('render', every='1s', needs='renders', batch=1)
for t in ('Rudram', 'Chamakam'): pob5.push('renders', t)
time.sleep(1.05)
pob5.tick().ran.attrgot('result'), pob5.items('renders').attrgot('text')
(['rendered Rudram'], ['Chamakam'])

drop_item(id) deletes an open item, for a job cancelled before it starts.

Run the tick on a heartbeat

The application owns one Pob database and one tick script. The script registers every callback before it calls tick.

# tick.py
from pobblebonk.core import Pob

pob = Pob('/absolute/path/to/pob.db')

@pob.on('cart')
def cart(fire):
    return 'added: ' + ', '.join(item.text for item in fire.list)

if __name__ == '__main__': print(pob.tick())

Run the script once before scheduling it. This catches import and path errors directly.

/absolute/path/to/uv run --directory /absolute/path/to/app python tick.py

Install one machine heartbeat after the script works. The command must use absolute paths because schedulers have a small environment.

from pobblebonk.heartbeat import install, installed, uninstall

install('/absolute/path/to/uv run --directory /absolute/path/to/app python tick.py', every=300)
installed()

install creates ~/.pobblebonk/pobblebonk.sh and schedules it every five minutes. every is seconds and defaults to 60. Portable values are whole minutes that divide an hour. Output is appended to ~/.pobblebonk/pobblebonk.log. On Linux it installs a cron line. On macOS it installs a LaunchAgent. On Windows it installs a Task Scheduler job.

The heartbeat only calls tick. Schedule times, retries, pauses, and catch-up policy stay in SQLite.

Inspect the log when a callback does not run. Remove the machine job when the application no longer needs it.

from pathlib import Path

print(Path.home().joinpath('.pobblebonk/pobblebonk.log').read_text())
uninstall()
assert installed() is None

The Linux backend needs uv add 'pobblebonk[cron]'.

Share an existing database

Pass an open honker database to keep pobblebonk data beside another application.

import honker

db = honker.open('app.db')
pob = Pob(db=db)

Develop

uv sync --group dev
uv run nbdev-export
uv run --extra cron nbdev-test
uv run nbdev-clean

nbs/00_core.ipynb contains schedules, lists, notes, and tick execution. nbs/01_heartbeat.ipynb contains the machine heartbeat. nbs/index.ipynb generates this README.

About

A durable croak for ramabana and vishalakshi

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages