Implement Refine from Notes batch processing and progress indicator

[zeus2611]

What?

Closes #250
See #251

This PR introduces the new "Refine from Notes" feature, enabling users to automatically apply pending editorial feedback/notes to their content using AI.

Why?

Feature request #250 proposed allowing users to generate automated refinement updates based heavily on unresolved editorial notes attached to blocks. This enables a smoother editing workflow where suggestions can be adopted contextually instead of applied manually one-by-one.

We matched the batch UX already provided by "Review Notes" and supply users with a safe, native diff interface via WordPress Revisions to examine and accept/rollback any generated AI changes before publishing.

How?

• Created useRefineNotes hook to loop comprehensively through the document blocks, matching pending and threaded ExistingNote collections to associated parent clientIds.
• Utilized an asynchronous batch progression of 4 block items per network request.
• Published surface-level progress data to a new RefineNotesPlugin.tsx component in the sidebar to dynamically step the button text (e.g. Refining block (1 of 7)...).
• Injected an autosave() trigger to establish a clean DB checkpoint after processing.
• Exposed a "Review in Revisions" action to the success snackbar notification linking directly to revision.php?revision=XXX.

Use of AI Tools

Implementation was planned and executed with assistance from Gemini (Antigravity). All code was reviewed, tested, and validated manually in a local WordPress environment.

Testing Instructions

1. Run and build the plugin locally on WP. Ensure AI experiments are enabled.
2. Create a Post and generate some blocks.
3. Attach multiple Editorial Notes/Comments across various blocks suggesting changes (e.g. "Fix capitalization", "Make this sound professional").
4. Click the "Refine from Notes" button from the main Plugin Sidebar.
5. Verify it updates the button text to show progression through the blocks.
6. Check that the snackbar notification appears with a "Review in Revisions" action upon completion.
7. Click "Review in Revisions" and verify the WP Core diff viewer accurately shows the before & after changes.

Testing Instructions for Keyboard

1. Navigate via Tab to the Plugin Sidebar -> Refine from Notes button.
2. Hit Enter to start the action.
3. Once completed, Tab over to the auto-focused snackbar notification to click "Review in Revisions".

Screenshots or screencast

[jeffpaul]

@zeus2611 I see this is still in draft, but PR description seems like it might be ready for review/testing; mind confirming if this is ready or still in progress?

[codecov]

Codecov Report

❌ Patch coverage is 70.66667% with 44 lines in your changes missing coverage. Please review.
✅ Project coverage is 68.44%. Comparing base (8322852) to head (a504d39).
⚠️ Report is 65 commits behind head on develop.

Files with missing lines	Patch %	Lines
includes/Experiments/Refine_Notes/Refine_Notes.php	30.30%	23 Missing ⚠️
includes/Abilities/Refine_Notes/Refine_Notes.php	82.60%	20 Missing ⚠️
...udes/Abilities/Refine_Notes/system-instruction.php	50.00%	1 Missing ⚠️

Additional details and impacted files

@@              Coverage Diff              @@
##             develop     #289      +/-   ##
=============================================
+ Coverage      68.36%   68.44%   +0.08%     
- Complexity       813      846      +33     
=============================================
  Files             53       56       +3     
  Lines           3945     4095     +150     
=============================================
+ Hits            2697     2803     +106     
- Misses          1248     1292      +44     

Flag	Coverage Δ
unit	68.44% <70.66%> (+0.08%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[zeus2611]

Hey @jeffpaul — the core feature is complete and working as expected. I'm currently working on improving the e2e test coverage before marking this ready for review. Should have that wrapped up shortly and will convert from draft then!

[github-actions]

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: zeus2611 <n1schay@git.wordpress.org>
Co-authored-by: dkotter <dkotter@git.wordpress.org>
Co-authored-by: jeffpaul <jeffpaul@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

[zeus2611]

Hey @dkotter — the implementation is complete and ready for review. Marking this as ready and removing the Draft status.

[jeffpaul]

@zeus2611 one thing that I think would be a great extension of this in a future, follow-up, PR is leveraging Real Time Collab and having a new WordPress AI (or just AI) user on the site and showing it within the editor interface making these changes. I'll open an issue to document that, but will otherwise review/test this issue without that in mind.

[zeus2611]

Sounds great, looking forward to the scope of the feature.

One thing I've been thinking about while building this: the individual features we're shipping (Review Notes, Refine from Notes, etc.) are really just discrete steps in a larger editorial workflow. There's an opportunity to tie these together as an agentic flow — e.g., a single "Review & Refine" pipeline where the AI first surfaces annotations as editorial notes and then automatically applies accepted suggestions in one pass, rather than requiring two separate manual actions.

I think this connects well to #282 — if the Chat Workspace is going to be the home for multi-step AI interactions, these editor-native workflows could be surfaced there as named agent actions ("Refine my draft", "Run editorial review"), giving users a coherent experience whether they're acting from the sidebar or the chat interface. Worth keeping in mind as the #282 design takes shape.

[jeffpaul]

There's an opportunity to tie these together as an agentic flow — e.g., a single "Review & Refine" pipeline where the AI first surfaces annotations as editorial notes and then automatically applies accepted suggestions in one pass, rather than requiring two separate manual actions.

I'm not certain I want to design a YOLO-mode into this by default, I'd prefer a human step in between the Review and Refine steps where an author or editor could perhaps resolve certain Review Notes that they don't want actioned and then have the AI continue on with the Refine steps.

[dkotter]

Haven't fully tested things yet, overall looking good but have a handful of things I think we need addressed, though let me know if there's any questions on these.

[jeffpaul]

I'm not certain that the flow through the "Review in Revisions" toast message and then historical diff view for Revisions is the most optimal way to get folks through this feature. I'd argue that immediately applying these updates to the post and allowing a user to leverage revisions to roll back any changes they don't like would be better. Alternatively presenting each change for editor/author review where accepting applies the change and resolves the respective Note and rejecting the change skips the change and leaves the respective Note?

[zeus2611]

The current implementation already applies changes directly to the post — the toast is just a convenience link. We could improve this by redirecting users straight to the revisions view after refinement completes, which would pair nicely with 7.0's block-aware visual diffing.

True per-block rollback isn't feasible yet since revisions are still whole-post snapshots — reverting one block's refinement would undo all others from that pass.

The accept/reject per-change approach would give the most granular control but requires a custom inline diff review UI. For this PR, would keeping "apply all + redirect to revisions" as the MVP and opening a follow-up issue for per-block accept/reject work?

[dkotter]

Code looks good to me, nice work here!

In testing this out, if I click on the link to view the revision in the notice that shows after processing Notes, I'm seeing two issues:

1. I get the "Are you sure, you'll lose unsaved changes" message from my browser. If I click ok in that, I do end up in the old revisions view but if I then go back to the post view, the changes are lost. Do we need to trigger a save after the Notes are processed? Otherwise changes are lost if they click on that revisions link
2. Looking at Jeff's feedback, seems he asked to link to the new revisions view. I'm still seeing the old one being linked to though

[zeus2611]

Thanks for testing!

1. "Unsaved changes" warning/changes lost:
   Good catch. We originally used autosave() intentionally to avoid AI directly saving changes, giving users a chance to review in revisions first. But without a way to selectively accept/reject individual changes, this creates the UX issue you described. We'll switch to savePost() for now — the granular accept/reject flow will come with Evolve Refine from Notes into collaborative and agentic editorial workflows #324.

2. Visual revisions view:
   We looked into this after Jeff's feedback. The block-based visual diff is triggered via a private editor API (setStylesPath('/revisions')) locked behind __dangerousOptInToUnstableAPIsOnlyForCoreModules — inaccessible to external plugins. Waiting for WP 7.0 stable to see how this gets exposed publicly. Also tracked in Evolve Refine from Notes into collaborative and agentic editorial workflows #324.

[zeus2611]

@dkotter All CI tests have passed successfully. I believe this is ready to be merged.

[dkotter]

@zeus2611 I've pushed a few commits to bring things inline with recent changes we've made elsewhere. Overall things work but few more pieces of feedback:

1. If an error happens during refinement, we don't surface the specific error making it harder to figure out what went wrong. As an example, if I have no AI Connectors in place and I try and refine, I'll get the following error: Refinement failed for all blocks.. Ideally I should be getting the error message that comes from the Ability, in this case Refinement generation failed. Please ensure you have a connected provider that supports text generation.
2. We recently made some changes to system instructions to try and ensure the content the LLM returns matches the language of the content it receives. I think we'll want a few changes here to support that, namely adding something similar to these new system instructions and also removing the extra context we add (the What follows is surrounding article content, where the block being refined has been replaced... bit). As detailed here, the LLM will often prioritize this instead of matching the content language.

[zeus2611]

@dkotter Ability errors now surface directly in the notice (e.g. "Please ensure you have a connected provider...") instead of the generic "Refinement failed for all blocks." Added language-matching rule to the system instruction and removed the English context wrapper per the #357 pattern. Also fixed the PHPUnit test broken by the develop merge (Loader::init()). All CI checks pass — for now, at least. Develop is merging faster than CI can run, so I might be stuck in an infinite rebase loop soon. 😄

[dkotter]

Thanks for all the work here @zeus2611!