Skip to content

editorial-notes.md

Latest commit

 

History

History
315 lines (251 loc) · 12 KB

editorial-notes.md

File metadata and controls

315 lines (251 loc) · 12 KB
Raw

AI Editorial Notes

Summary

The AI Editorial Notes experiment adds a block-by-block AI editorial review to the WordPress post editor. Clicking "Generate Editorial Notes" in the post sidebar triggers the AI to examine each reviewable block and create WordPress Notes directly on the relevant blocks with concise, actionable suggestions across four categories: Accessibility, Readability, Grammar, and SEO.

At a Glance

  • Type: Experiment (may change or be removed in any release; try it on a staging site first)
  • Runs without a user action: No, only when you ask
  • When it runs: When you click Generate Editorial Notes
  • Sent to your AI provider: Each block's text with surrounding content
  • Requests and data: One request per block
  • Good to know: Notes are saved as comments on the post as soon as they're generated.

Overview

For End Users

When enabled, a "Generate Editorial Notes" button appears in the post status info panel (the sidebar area below the post status). Clicking it triggers a review pass:

  1. The button label updates to show review progress (Reviewing blocks… (2 of 8))
  2. Each content block is sent individually to the AI for analysis
  3. Notes with suggestions appear directly on the blocks inside the Notes panel
  4. After completion, a count of new suggestions is shown beneath the button

Key Features:

  • Block-level Notes with suggestions scoped to each block's content and type
  • Four review categories: Accessibility, Readability, Grammar, SEO
  • Accumulating history: subsequent review runs reply to the block's existing AI Note thread rather than creating duplicate threads
  • Works alongside human Notes: the AI keeps its own thread on each block, and Notes added through the Notes panel stay attached to the block
  • Prior suggestions are sent back to the AI as context so it avoids repeating itself
  • Blocks whose AI Note thread has been resolved (marked as approved) are skipped on re-run
  • Works with common block types: paragraphs, headings, images, lists, tables, quotes, and more

For Developers

Architecture & Implementation

Block Types Reviewed

const REVIEWABLE_BLOCK_TYPES = [
  'core/paragraph',
  'core/heading',
  'core/list',
  'core/list-item',
  'core/quote',
  'core/verse',
  'core/image',
  'core/table',
  'core/preformatted',
  'core/pullquote',
];

Blocks with fewer than 20 characters of text content are skipped. The review is capped at 25 blocks per run to control cost.

Input Schema

array(
    'block_type'     => array(
        'type'              => 'string',
        'sanitize_callback' => 'sanitize_text_field',
        'description'       => 'The block type, e.g. core/paragraph, core/heading.',
    ),
    'block_content'  => array(
        'type'              => 'string',
        'sanitize_callback' => 'sanitize_text_field',
        'description'       => 'The plain-text content of the block to review.',
    ),
    'context'        => array(
        'type'              => 'string',
        'sanitize_callback' => 'sanitize_text_field',
        'description'       => 'Surrounding content to improve review relevance.',
    ),
    'post_id'        => array(
        'type'              => 'integer',
        'sanitize_callback' => 'absint',
        'description' => 'ID of the post being reviewed.',
    ),
    'existing_notes' => array(
        'type'        => 'array',
        'items'       => array( 'type' => 'string' ),
        'description' => 'Existing Note texts for this block from prior review runs, used to avoid repeating suggestions.',
    ),
    'review_types'   => array(
        'type'        => 'array',
        'items'       => array( 'type' => 'string', 'enum' => array( 'accessibility', 'readability', 'grammar', 'seo' ) ),
        'description' => 'Review types to perform.',
    ),
)

Output Schema

array(
    'type'       => 'object',
    'properties' => array(
        'suggestions' => array(
            'type'  => 'array',
            'items' => array(
                'type'       => 'object',
                'properties' => array(
                    'review_type' => array( 'type' => 'string' ),
                    'text'        => array( 'type' => 'string' ),
                ),
            ),
        ),
    ),
)

Permissions

The ability's permission_callback has two paths:

  • With a numeric post_id (post ID): Validates that the post exists, the current user has edit_post capability for that specific post, and the post type is registered with show_in_rest => true. Returns false if the post type is not REST-accessible.
  • Without a post ID: Requires current_user_can( 'edit_posts' ).

In both cases, users without the required capability receive an insufficient_capabilities WP_Error.

Note Association

Notes are WP_Comment objects with comment_type = 'note' and status = 'hold'. Block association is maintained via block metadata:

  • noteId format: metadata.noteId is expected to be either a number (12) or an array of numbers ([12, 34]).
  • New thread: POST /wp/v2/comments with parent: 0 → response id is added to block.attributes.metadata.noteId via updateBlockAttributes. Existing IDs are kept, and the block's current format is preserved: arrays are appended to, and a number is only set when the block has no Note yet.
  • Reply: POST /wp/v2/comments with parent: <AI thread ID> → block metadata unchanged. The AI thread is the block's root Note with meta.wpai_note = true. When noteId is a number, the AI replies to that thread.
  • AI author: All Notes created by this experiment include meta: { wpai_note: true }. The rest_pre_insert_comment filter intercepts this and sets the author to "WordPress AI" with no email, URL, or user ID, so Notes are not attributed to the authenticated user's account.
  • Resolved Notes: A resolved (status = 'approve') AI thread causes its block to be skipped entirely on the next review run. Resolved human threads do not, except when noteId is a number, in which case the block is skipped once that thread is no longer pending.

Using the Ability via REST API

Endpoint

POST /wp-json/wp-abilities/v1/abilities/ai/editorial-notes/run

Authentication

See TESTING_REST_API.md for authentication details (application passwords or cookie + nonce).

Request Examples

Review a Paragraph Block

curl -X POST "https://yoursite.com/wp-json/wp-abilities/v1/abilities/ai/editorial-notes/run" \
  -u "username:application-password" \
  -H "Content-Type: application/json" \
  -d '{
    "input": {
      "block_type": "core/paragraph",
      "block_content": "The committee was formed by the director in order to study the problem and make recommendations.",
      "review_types": ["readability", "grammar"],
      "existing_notes": [],
      "post_id": 42
    }
  }'

Response:

{
  "suggestions": [
    {
      "review_type": "readability",
      "text": "Rewrite in active voice: \"The director formed a committee to study the problem and make recommendations.\""
    }
  ]
}

Review an Image Block (Accessibility Only)

curl -X POST "https://yoursite.com/wp-json/wp-abilities/v1/abilities/ai/editorial-notes/run" \
  -u "username:application-password" \
  -H "Content-Type: application/json" \
  -d '{
    "input": {
      "block_type": "core/image",
      "block_content": "",
      "review_types": ["accessibility"],
      "existing_notes": []
    }
  }'

Response (missing alt text):

{
  "suggestions": [
    {
      "review_type": "accessibility",
      "text": "Add descriptive alt text to this image so screen reader users understand its content."
    }
  ]
}

Using WordPress API Fetch (in Gutenberg)

import apiFetch from '@wordpress/api-fetch';

async function reviewBlock( blockType, blockContent, existingNotes = [] ) {
  const result = await apiFetch( {
    path: '/wp-abilities/v1/abilities/ai/editorial-notes/run',
    method: 'POST',
    data: {
      input: {
        block_type: blockType,
        block_content: blockContent,
        review_types: [ 'accessibility', 'readability', 'grammar', 'seo' ],
        existing_notes: existingNotes,
        context: String( postId ), // numeric post ID as string
      },
    },
  } );

  return result.suggestions; // Array of { review_type, text }
}

Error Responses

Code Meaning
block_content_required block_content was empty
post_not_found The post ID passed does not exist
insufficient_capabilities User lacks edit_posts (or edit_post for the specific post)

Testing

Manual Testing Steps

  1. Enable the experiment:

    • Go to Settings → AI
    • Enable AI Editorial Notes
    • Ensure valid AI credentials are configured

  2. Run a review:

    • Create or open a post with a mix of block types (headings, paragraphs, an image without alt text, a list)
    • Open the post sidebar (click the Settings button in the toolbar)
    • Click Generate Editorial Notes in the post info panel
    • Watch the progress counter advance (Reviewing blocks… 2 of 8)
    • After completion, open the Notes panel (via the block toolbar or the comments icon)
    • Verify Notes appear on relevant blocks, formatted as [REVIEW_TYPE] Suggestion text.
    • Verify Notes show "WordPress AI" as the author rather than your account name

  3. Re-run accumulation:

    • Click Generate Editorial Notes a second time
    • Verify existing Note threads gain replies rather than new top-level Notes
    • Verify prior suggestions are not repeated

  4. Resolved Notes:

    • Mark a Note as resolved in the Notes panel
    • Run the review again
    • Verify the resolved block is skipped entirely

  5. Multiple Note threads:

    • Add a Note to a paragraph block through the Notes panel
    • Run the review
    • Verify the AI's suggestions appear as a separate thread and the human Note is still attached
    • In the Code Editor, verify the block's metadata.noteId contains both IDs (e.g. [123, 124])
    • Run the review again and verify the AI replies in its own thread
    • Resolve the human Note, run again, and verify the block is still reviewed

  6. Note deletion cleanup:

    • Delete a Note from the Notes panel
    • Save the post
    • Verify the deleted block no longer has a noteId in its block metadata (inspect via the Code Editor)

  7. Edge cases:

    • Post with only very short blocks → button completes instantly with "No new suggestions found."
    • All blocks already have Notes → second run skips repeats
    • Disable experiment → button disappears from sidebar

Notes & Considerations

Requirements

  • Valid AI credentials configured in Settings → Connectors
  • User must have edit_posts capability (or edit_post for the specific post when a post ID is provided)
  • The block editor must be active (classic editor is not supported)

Performance

  • Each block generates one API call; blocks are processed in parallel batches of 4
  • The review is capped at 25 blocks per run to control cost
  • Blocks with fewer than 20 characters of text are skipped

Note Storage

  • Notes are stored as WordPress comments with comment_type = 'note' and comment_author = 'WordPress AI'
  • Block association is stored in block.attributes.metadata.noteId, as either a number or an array of numbers
  • Block metadata is saved as part of the post content when the editor saves
  • Note threads accumulate across review runs by design
  • Deleting or trashing a root Note automatically removes its ID from the associated block's metadata.noteId

Limitations

  • Image block review is limited to alt text presence; it does not analyze the image itself
  • Block metadata (noteId) is only persisted after the post is saved
  • The 25-block cap means very long posts will have only the first 25 reviewable blocks analyzed per run
  • Blocks with a resolved AI thread are skipped in full; they will not receive new suggestions until the thread is un-resolved or deleted
  • When noteId is a number, the AI replies to that existing thread even if a human started it