Repository navigation
Read tasks the way Logseq does in every command (#92, part 1) - #117
Merged
Merged
Conversation
The package kept six marker lists, none complete, and counted tasks two ways: get-todos from the database marker, analyze-* from a pattern over the page text. logseq_cli/tasks.py is now the one place for the eleven markers Logseq's parser reads (measured with mldoc 1.5.7, Logseq 0.10.15), their three states as Logseq draws them (open, done, cancelled), and how a marker is read from and swapped in a block's text. get-todos takes the task text from tasks.task_text; the code moved over unchanged, including how it reads an unclosed :LOGBOOK: and one inside a code fence. No command behaves differently, and --help of every command is byte-identical. tasks.STATES is the three states in the order of the markers, derived once from STATE. blocktext now owns the rule for planning lines (PLANNING_LINE_RE moved from dates) and attached_line_mask, the one definition of which lines belong to the block above. Nothing reads the mask yet. tests/test_marker_lists.py scans the package for a second marker list. It lists the twelve places where one still sits; later changes remove them. tasks joins the modules that need no API (ADR 0003, package layering test), and CONTEXT.md defines Task, Marker and Task State. Mutations tried, each caught by: - attached_line_mask ignoring code blocks: planning line in a code block - property lines in attached_line_mask: the disjointness test - unclosed :LOGBOOK: read as a drawer: opener without closer is text - planning regex without "<": no angle bracket - STARTED dropped from ORDER: test_order - CANCELLED as open: test_states - no heading prefix, tab as separator, list character before the marker: the marker_of cases - lower-case marker before a newline: marker_before_newline cases - trailing space after a swap at the end of the text: with_marker cases - box check on every line, or behind a list character or heading: the starts_with_box cases - task_text built on attached_line_mask, marker removed on every line, properties kept: the task_text cases - a count in the exception list raised, a new marker list added: the scan - an api parameter in tasks.py: the layering test Existing tests changed: only PURE and the docstring in test_package_layering.py. Part of #92.
get-todos built a task's text with a reading of :LOGBOOK: of its own: any :LOGBOOK: line opened a drawer, and everything up to a :END: was dropped, whether or not one ever came. mldoc reads it differently (measured with mldoc 1.5.7): a :LOGBOOK: that no :END: closes is text, and a line inside a code fence is code. For a block with such a line the two disagreed, and content, --match and --tag lost everything after it. task_text now takes the lines that belong to the block above from the one rule in blocktext. An unclosed :LOGBOOK: stays in the text, so does a :LOGBOOK: or a SCHEDULED:/DEADLINE: line inside a code fence and a stray :END:. Closed drawers, planning lines and property lines outside code read as before. Each of the four cases was red before the change; --match and --tag now find text behind the opener, which the old reading hid. Mutations tried, each caught by: - task_text back on the old line-by-line reading: the four task_text cases and the --match/--tag cases through get-todos - a pattern of its own instead of attached_line_mask: the source check (test_the_source_reads_the_shared_rule) and the unclosed-opener case Existing assertions changed: the four cases that pinned the old reading of task_text (TestTaskTextKeepsTheOldBehaviour, now TestTaskTextKeepsWhatItAlwaysDid for the rest), pinned to the shared rule in TestTaskTextReadsTheSharedRule. Part of #92.
The test doubles worked out a block's marker from the first word of its text, knew only eight of the eleven markers, and clocked a task in or out by the marker it moved to. Logseq stores :block/marker when it parses the text, so a block such as "TODO\nnotes" has none, and it starts or closes the clock by the pair of the old and the new marker (time tracking, on by default). A mock that derives the marker from text would confirm the rule a test is meant to check. The marker is now stored data on each node, given by the fixture. The HTTP double sets it again in one place only, after a write, with the rule the package uses and a test compares with mldoc. Its time tracking follows tasks.CLOCK_IN_STEPS and CLOCK_OUT_STEPS: for example WAIT -> DOING starts no clock, which the old target sets did. A source test keeps marker lists and marker patterns out of the test helpers. No command behaves differently. One assertion changed: the step DONE -> NOW of the time tracking test expected a new CLOCK line. With time tracking on, 0.10.15 starts the clock only for the pairs in CLOCK_IN_STEPS (with-marker-time in handler/editor.cljs, read in the source, not measured: tracking was off in the measured graph), and DONE -> NOW is not one of them. Mutations tried, each caught by: - a tuple of its own in the double instead of tasks.CLOCK_IN_STEPS: the source test for marker lists in the test helpers - target sets instead of pairs: the WAIT -> DOING case of TestTimeTrackingFollowsThePair - no second reading of the marker after updateBlock: TestMarkerIsStoredState - DOING -> DOING starting a clock with a logbook present: the DOING -> DOING and NOW -> NOW cases with a closed drawer - PageGraph leaving the marker out of getBlock, with a temporary check in set-todo-status that the fixture's marker agrees with its text: 19 tests Part of #92.
set-todo-status took the first word of the text for the marker. A heading task (## TODO ship it) became DONE ## TODO ship it, which Logseq no longer reads as a task, a block without a marker (todo x, no marker) became a task, and TODO followed by a space and a line break lost the space, which also made it text (measured with Logseq's parser, mldoc 1.5.7). The command now decides by the marker Logseq keeps in its database and swaps it where Logseq reads it, behind a heading prefix. A block with no marker, one whose marker is not at the start of the text, and one that only points to another block are refused before the preview, also under --dry-run. --content that matches blocks but no task is refused too, except with --follow-refs, where a match that only points to a task is followed as before. A block that already carries the status answers "No change (block already has this status)." Not found, ambiguous and a missing selector answer under --json with a reason. get-todos reports the content of a heading task without its marker. Mutations tried, and the test that caught each: - marker not taken from the database: the "TODO x" refusal case - no wording for a pointer, none for a line break: their refusal cases - no refusal for matches without a task, or one that also holds with --follow-refs: TestSetTodoStatusNeedsATask - no "no change" answer: the unchanged cases - the space before a line break lost: the "space before the line break" case - only the database marker checked: the "##<TAB>TODO x" case - old_marker read from the first word: the heading dry-run case - the page named as given, not as the alias resolves: the alias case - matches listing only ten uuids, a candidate line cut at 80: the ambiguity case - --id also searching --content, the selector checked after a request: TestSetTodoStatusReasons - heading prefix of spaces only, of any blank, or dropped: the heading prefix cases of task_text and get-todos - the marker of the pull moved before the uuid, another default, one of the four queries left out: test_find_blocks_task_fields Existing assertions changed, with the reason: - test_set_todo_status_on_a_block_that_starts_with_a_fence: a block that starts with a fence has no marker, so it is refused as no task before a marker could be put in front of the fence - test_a_labelled_ref_is_a_link and test_what_logseq_reads_as_no_embed: Logseq reads neither as an embed, and no marker in them; they were rewritten into a task, and are refused now - test_only_the_marker_changes: the bare marker, code under a bare marker and no marker cases are refusals now, the heading and line break cases are new - TestMarkerIsStrippedForEveryMarker takes its markers from tasks.ORDER Seen red before the change: the refusal, unchanged, reason, heading and alias cases against the code as it was. Part of #92.
add-note-content --content $'TODO x\nSCHEDULED: <2026-09-25 Fri>' made two blocks: the SCHEDULED line became a block of its own and had no effect, and the dry run said "2 block(s)". A task with a date could not be written in one call. In Logseq's files these lines, and a :LOGBOOK: drawer, belong to the block above, as a key:: value line does. The outline parser now keeps them there, through the one rule in blocktext that says which lines belong to the block above, in every command that reads its text as an outline. A :LOGBOOK: that no :END: closes is text, not a drawer (measured with Logseq's parser, mldoc 1.5.7), so it hides nothing and swallows no bullet after it. Confirmed live on a probe page, 2026-10-06, Logseq 0.10.15: one block per task, Logseq sets the scheduled and deadline dates, the drawer stays in its block. Mutations tried, and the test that caught each: - a SCHEDULED line not attached: the scheduled case of the outline tests - an unclosed :LOGBOOK: read as a drawer: the mask case for an opener without a closer - a literal of its own in the outline module: the source check that the shared rule is the only one - the mask over the whole line list instead of the lines the loop reads: the bullet fence with a drawer inside - no block to join for a leading line: the leading planning line case Existing assertions changed, with the reason: none. A "- ```" line inside a code block that starts without a bullet no longer opens another code block, so the real closer is not cut out and a drawer behind it stays with its block. A bullet line ends the reach of a drawer opener: "- a", ":LOGBOOK:", "- b", ":END:" are four blocks and nothing is swallowed. A drawer that closes before the next bullet stays with its block, a blank line inside it still goes. Part of #92.
get-todos took --status with a default of TODO, DOING, NOW and LATER, so a task marked WAIT, WAITING, IN-PROGRESS or STARTED never showed up unless the caller named it. Logseq draws the first three as open tasks, and its parser reads STARTED as a marker too, although its own interface does not offer it. --status also took any text: --status foo ran and found nothing. --state open|done|cancelled (repeatable, default open) now selects by state. --status takes the eleven markers Logseq reads, in any case, has no default and wins over --state, with a note when both are given. --include-done stays, hidden from --help, and adds DONE to either. Tasks sort by marker. The default changes under the same name and key; only open tasks are added, each carrying its marker, so nothing reads wrongly. The changelog names the old default and how to keep it. The two tests that compare the command reference and the skill with the registry now leave hidden options out. Mutations tried, and the test that caught each: - markers_in without WAIT: the open-markers case without a filter - the old sort table: the marker order case - --state winning over --status: the status-wins case - --status with its old default: the source scan for marker lists (no behaviour test can see it, since "given" is read from the source) - "given" read from the value, with the default back: the case of an explicit --state equal to the default, and the case without a filter - --state as free text or case sensitive, --status case sensitive: the case and usage error cases - the note on stdout: the case of the note under --json Existing assertions changed: none. The reference and skill tests only leave hidden options out, green before and after. Cases seen failing before the change: the thirteen cases of the state selection class, against the code as it was. The markers go into the query through tasks.marker_clause, which checks each of them against the eleven; get-todos and its reference query built that set by hand before. Part of #92.
--status took a fixed list of six markers, told upper case from lower case, and so refused WAIT, WAITING, IN-PROGRESS and CANCELLED, which Logseq sets itself, and --status done, although get-todos took --status todo. The choice is now tasks.FRONTEND_MARKERS: every marker Logseq's parser reads but STARTED, which its interface does not offer, so the CLI does not write a marker Logseq never writes. Case does not matter, and the help says why the option takes a marker and not a state: only the marker DOING tells DOING from TODO. The help lists are made from the constant, and the epilog of update-block no longer lists markers. Mutations tried, and the test that caught each: - the choice case sensitive: the lower-case WAITING case - the old list of six: the WAITING case - STARTED among the markers: the STARTED case - the marker hint left out of the help: the help case Existing assertions changed: none. Part of #92.
get-todos worked out next_due from the first repeater in a task's text, whichever timestamp it belonged to. A task with SCHEDULED <... .+1d> and DEADLINE <... +1w> reports the deadline, but next_due followed the daily repeater, so --due-from and --due-to could place it up to six days off. next_due now comes from the timestamp whose date is reported (DEADLINE before SCHEDULED, as before), with that timestamp's own repeater. A timestamp without a repeater does not move, as Logseq moves only timestamps that have one, so next_due is its date. dates.timestamps reads every timestamp of a line with its kind, date, time and repeater; dates.parse_timestamp reads the text between the angle brackets and does not check the weekday. Timestamps are read only from the SCHEDULED and DEADLINE lines of a block, outside code fences, as Logseq reads them; parse_repeater also found one in running text. parse_repeater is gone: nothing calls it any more, and keeping it would leave a second rule, "the first repeater in the text", next to the right one. Its seven cases are timestamps cases now, with the same inputs. Mutations tried, and the test that caught each: - timestamps reading only the first timestamp of a line: the case of both on one line - the kind taken from the wrong group, the bare + left out of the repeaters, .+ read as +: the timestamps table cases - no line rule and no code fence rule: the running text case - the time ignored: the case with 18:00 - the weekday checked, the hour read with two digits only: the parse_timestamp cases - next_due from the first repeater in the text, from the other kind, without a date for a timestamp without repeater, or with the date from the text: the cases of next_due following the reported timestamp Existing assertions changed: none; the cases of TestParseRepeater moved to the timestamps cases with the same inputs. Cases seen failing before the fix: the two-repeater case, the deadline without a repeater, and the repeater of the other timestamp not borrowed. timestamps returns early for a text without a SCHEDULED or DEADLINE word, so the code-block scan runs only for blocks that can hold a timestamp. Part of #92.
analyze-graph counted tasks with a pattern over the page text, while get-todos counts blocks that carry a marker. On one graph the two disagreed: the pattern read TODO: , a lowercase todo, a block that starts with [ ] and a marker on a later line as tasks, none of which Logseq does. analyze-graph now asks for every block with one of the eleven markers Logseq reads and sorts them into open, done and cancelled by one classification, the same one get-todos uses. total_todos is replaced by tasks (open, done, cancelled, open_by_marker). A note on stderr says how many blocks start with [ ], which Logseq shows as text. --days limits only the Recently updated list, and its help says so. smart-query --request tasks asks for the same open markers, so it now also finds WAIT, WAITING, IN-PROGRESS and STARTED. The pattern is matched by every keyword it lists, so all of them are affected. The tree walker of meta.py is replaced by outlinetext.preorder_blocks, which the note uses per block too. Live check 2026-10-06, Logseq 0.10.15: 8 of 8 quantities equal on a probe page (and a deliberately shifted expectation was reported as unequal); the page was removed. Mutations tried and caught: - the query asking for the open markers only, the count taken per page: the test that the query names exactly the eleven markers, once - a checkbox looked for on every line, or behind a list character or a heading: the note case - the parenthesis kept at zero open tasks, the open markers sorted by the alphabet: the text line and the order cases - a missing tree not skipped, a marker outside the states not skipped, a query answer of none not read as empty: the three cases of that name - the empty-tree rule copied into a second place: the page text cases - the old list in the smart-query pattern, done also asking for CANCELED: the pattern cases, one per keyword - the tree walker in post-order, not descending, reading the children key unguarded: the walker cases Changed existing tests: the five total_todos cases of analyze-graph are replaced; the cases they held, prose, a logbook line, a marker on a later line, a bracket pair in prose, are marker and checkbox cases of the task rules now, and the marker cases are covered by the mock that answers the query. Two test gaps are closed with it: a checkbox block nested below the top level is counted, and --days limiting only the Recently updated list has a behaviour test. Part of #92.
largest_cluster was the size of the first cluster the search found, not of the largest one. The clusters come in the order of the pages, so a graph whose first linked group was small under-reported it: with a group of two pages before a group of three, the report said two. largest_cluster is now the largest of the sizes, 0 without a cluster. Mutations tried, and the test that caught each: - the first cluster again: the case of two groups of 2 and 3 pages and the text line case - the largest taken over the number of clusters, not their sizes: all four cases - no default for the largest of nothing: the case without a cluster The case with two groups was red against the code as it was (2 instead of 3). Part of #92.
analyze-journal-patterns counted tasks with two patterns over the text of each journal page. They counted Markdown checkboxes, which Logseq does not read as tasks, counted CANCELED and CANCELLED as done, and could be fooled by text that only looks like a task (TODO: x, TODO and a line break). The count now comes from one query for the blocks that carry one of the eleven markers, on the journal days of the time frame, sorted into open, done and cancelled by the same classification as get-todos. tasks holds open, done, cancelled and done_rate; the tasks of each entry hold open, done and cancelled. The rate is named done_rate and not completion_rate on purpose: its value changes (cancelled tasks do not count, checkbox lines drop out), and a key that keeps its name while it returns another number breaks quietly, whereas a missing one breaks loudly. tasks.total_complete, tasks.total_incomplete, tasks.completion_rate and the per-entry tasks_complete and tasks_incomplete are gone. Tasks are dated by the page their block is on, as before; get-todos --from/--to also counts the days a block ref carried a task to. Habit tracking still counts the text pattern "- [ ]" / "- [x]". Both are said in --help and in the text output now, not in the JSON. Mood, topics, projects and habit tracking are unchanged; habit tracking, which had no test, is pinned by one written before the change. Mutations tried, and the test that caught each: - the text counted as well, cancelled as done, rows counted instead of blocks: the cases of those names - the rate under its old name: the empty journal case (the key set) - the query asking for the open markers only, or for other days than the pages: the case of the query text - no guard for a row of fewer than three fields, for a marker outside the states, for an answer of none: one case each - the sentences left out of the help or the text output, or put into the JSON: the cases of the output Existing tests changed: the task cases of analyze-journal-patterns now give the database rows and the new keys; the checkbox case says that checkbox lines are no tasks; the doubled bullet case is gone (Logseq reads no marker behind a list character; a marker rule case holds it). The class that tested copies of the old patterns, not the code, is removed. Capitalised words opening a sentence are cases of the marker rule. The marker scan loses its last exception with this commit, so the list of exceptions goes away: a marker list outside the tasks module fails the test directly; only docstrings and the first argument of output.fail stay out of it, found by their position in the syntax tree. Checked against Logseq 0.10.15 on 2026-10-06: the counts of analyze-graph and get-todos --no-follow-refs equal Logseq's own task query for open, done and cancelled (6 of 6 pairs equal, nothing written to the graph). Mutations of the scan, each seen red and reverted: - a marker set added under commands/, the old marker list back in the smart-query pattern, an enumeration of markers in a help text: test_no_module_but_tasks_keeps_a_marker_list Part of #92.
get-todos stopped with a traceback and exit 1 on a block whose repeater holds a number no date can reach. Two sizes, two ways to fail: - Eleven digits or so (SCHEDULED: <2026-01-01 Thu +99999999999d>) made _add_interval raise OverflowError; 99999 months or years raised ValueError for a year past 9999. get_todos catches only (ValueError, TypeError) around next_occurrence, so the OverflowError ended the command. - More than 4300 digits made int() raise ValueError inside parse_timestamp, outside that try, so no caller could catch it. One such block, typed or pasted, made get-todos unusable for the whole graph. Logseq does not write such a number; it was already on main (parse_repeater, same two failures). parse_timestamp now reads a number of more than twelve digits as 10**12, a value no date lies away. _add_interval returns None for a result no date can hold, which is what next_occurrence already answers "cannot be derived" with. The task lists like one whose interval cannot be read: no next_due, left out of a due range, counted in repeating_excluded and named on stderr. Mutations tried, each caught: - int(digits) without the cap: the 5000-digit cases, in dates and through get-todos (13 tests) - _add_interval without the except: all 44 cases of the new class - only OverflowError caught: the 99999 months and years cases (6) - the cap from three digits on: the case that .+100y is not cut The cases were red before the change (44 failed), with the two exceptions above.
next_occurrence stepped once past a stored date that was today or still ahead for "+" and ".+", so get-todos reported next_due one repeat too late and --due-from/--due-to left the task out. Only "++" kept such a date.
next_occurrence compared with "> today" in the single step and in the loop, so an overdue repeating task whose next date is today was reported for the occurrence after it, and --due-from today --due-to today missed it.
…read update_block reads the block again, past the cache, directly before a write that says what it replaces, and raises BlockChanged when the text is another. set-todo-status, replace-text, set-property and remove-property sent the whole text back and wrote over a change made in between, and the read-back passed because it compares with what was just sent. An id:: line is left out of the comparison: a ref written earlier in the same call stores the id of a block that call then replaces.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part 1 of #92. Every command that reads or changes tasks now uses one definition of a task (new
tasks.py): the marker Logseq stores for a block, mldoc's eleven markers, and the lines that stay with a block (drawers, code fences,SCHEDULED,DEADLINE,LOGBOOK).What changes for a user
get-todostakes--state open|done|cancelled. Without it, it lists every open marker, not onlyTODO.next_duecomes from the repeater of the timestamp it reports.set-todo-statusrefuses a block Logseq does not read as a task, offers every marker the frontend knows, and keeps theSCHEDULED,DEADLINEandLOGBOOKlines with the block.analyze-graphandanalyze-journal-patternscount tasks from the database, likeget-todos.tasksreplacestotal_todos, andtasks.total_completeis gone (see "Removed" in the CHANGELOG).smart-query --request tasksalso findsWAIT,WAITINGandIN-PROGRESS.Fixed along the way (all were already on
main)get-todos.next_duewas one repeat too late for a date from today onward, and skipped an occurrence that falls on today.block_changed. The window shrinks, it does not close.analyze-graphreported the first cluster's size aslargest_cluster.Not in this PR: renaming to
get-tasks/set-task-status, and the repeater choice. Both are part 2.Checks: 3777 tests,
ruff checkclean. Each fix has a test that fails without it. The write path was run against a live Logseq 0.10.15 on a throwaway page,replace-textand the id-line case included.