fix(mito2): introduce PartitionExprChange in staging flow and keep memtables on metadata-only updates

[WenyXu]

I hereby agree to the terms of the GreptimeDB CLA.

Refer to a related PR or issue link (optional)

#6558
#7678

What's changed and what's your intention?

This PR refines staging manifest action semantics for repartition flow and ensures staging exit does not rebuild/drop memtables when only partition expression metadata changes.

1. Introduce dedicated partition-expression action

• Add RegionMetaAction::PartitionExprChange with payload RegionPartitionExprChange { partition_expr }.

2. Apply PartitionExprChange during manifest replay

• RegionManifestBuilder now supports apply_partition_expr_change.
• RegionManifestManager replay paths (recover/install/update) apply this action by updating metadata partition expression only.

3. Strengthen merge/split and conflict validation

• split_region_change_and_edit now returns partition-expr change + change + edit components.
• In exit_staging_on_success, PartitionExprChange and Change together are treated as invalid and fail fast.

4. Enter-staging now emits the right action type

• handle_enter_staging now writes PartitionExprChange instead of using RegionChange for partition-only updates.

5. Change handling in staging exit is now metadata-safe

• For PartitionExprChange: apply metadata-only update path.
• For Change: require change.metadata.column_metadatas == current.column_metadatas; if equal, apply metadata-only update, otherwise fail fast.
• Add VersionControl::alter_metadata to update metadata without rebuilding memtables.

6. Test coverage additions

• Engine integration tests:
  • verify enter-staging writes PartitionExprChange action (and not Change);
  • verify PartitionExprChange + Change conflict is rejected on staging exit.
• Apply-staging integration tests:
  • Change + Edit with same columns succeeds;
  • Change + Edit with different columns fails.

PR Checklist

Please convert it to a draft if some of the following conditions are not met.

• I have written the necessary rustdoc comments.
• I have added the necessary unit tests and integration tests.
• This PR requires documentation updates.
• API changes are backward compatible.
• Schema or data changes are backward compatible.

[gemini-code-assist]

Summary of Changes

Hello @WenyXu, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly refines the handling of region repartitioning within the Mito2 engine. By introducing a dedicated manifest action for partition expression changes and optimizing the staging exit process, it ensures that metadata-only updates are applied efficiently without unnecessary memtable operations. This change enhances the robustness of the staging flow, improves performance for certain metadata alterations, and prevents potential conflicts during region state transitions.

Highlights

• New Manifest Action Type: Introduced RegionMetaAction::PartitionExprChange to specifically handle updates to a region's partition expression metadata, distinguishing it from broader RegionChange actions.
• Optimized Staging Exit for Metadata-Only Updates: Refined the staging exit process to apply metadata-only partition expression changes without rebuilding or dropping memtables, improving efficiency for repartitioning flows.
• Enhanced Conflict Validation: Implemented stricter validation during staging exit, preventing simultaneous PartitionExprChange and RegionChange actions to avoid conflicts and ensure data integrity.
• Dedicated Metadata Alteration Method: Added VersionControl::alter_metadata to allow updating region metadata without affecting memtables, supporting the optimized handling of partition expression changes.
• Improved EnterStaging Behavior: Modified handle_enter_staging to correctly emit PartitionExprChange for partition-only updates, ensuring the appropriate action type is recorded in the manifest.
• Comprehensive Test Coverage: Added new integration tests to verify the correct behavior of PartitionExprChange actions, conflict detection, and metadata-only updates during staging entry and exit.

Changelog

• src/mito2/src/engine/apply_staging_manifest_test.rs
  • Added Arc import.
  • Imported new manifest action types and FormatType.
  • Added test_apply_staging_manifest_change_edit_same_columns_success to verify successful manifest application when column metadata is consistent.
  • Added test_apply_staging_manifest_change_edit_different_columns_fails to verify manifest application failure when column metadata differs.
• src/mito2/src/engine/staging_test.rs
  • Imported SetRegionRoleStateResponse.
  • Imported new manifest action types and FormatType.
  • Added test_enter_staging_writes_partition_expr_change_action to confirm EnterStaging uses the new partition expression change action.
  • Added test_staging_exit_conflict_partition_expr_change_and_change to verify conflict detection during staging exit.
• src/mito2/src/manifest/action.rs
  • Introduced RegionMetaAction::PartitionExprChange enum variant.
  • Defined RegionPartitionExprChange struct for partition expression metadata.
  • Added is_partition_expr_change method to RegionMetaAction.
  • Added apply_partition_expr_change method to RegionManifestBuilder.
  • Modified split_region_change_and_edit to also return Option<RegionPartitionExprChange>.
  • Updated split_region_change_and_edit logic to process PartitionExprChange actions.
  • Added a test for RegionPartitionExprChange deserialization.
• src/mito2/src/manifest/manager.rs
  • Updated replay_actions, install_actions, and update methods to apply RegionMetaAction::PartitionExprChange.
• src/mito2/src/region.rs
  • Modified exit_staging_on_success to ensure Change and PartitionExprChange actions do not coexist.
  • Updated exit_staging_on_success to apply PartitionExprChange via alter_metadata and validate Change actions against current column metadata.
  • Added helper functions build_test_region and empty_edit for testing.
  • Added test_exit_staging_partition_expr_change_and_edit_success for successful staging exit with partition expression changes.
  • Added test_exit_staging_change_with_same_columns_success for successful staging exit with consistent column metadata changes.
  • Added test_exit_staging_change_with_different_columns_fails for failed staging exit due to differing column metadata.
  • Added test_exit_staging_partition_expr_change_and_change_conflict_fails for failed staging exit due to conflicting actions.
• src/mito2/src/region/version.rs
  • Added alter_metadata method to VersionControl for metadata updates without memtable rebuilds.
• src/mito2/src/worker/handle_enter_staging.rs
  • Removed unused Arc import.
  • Updated imports to use RegionPartitionExprChange.
  • Modified handle_enter_staging to generate RegionMetaAction::PartitionExprChange for partition expression updates.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a new PartitionExprChange manifest action to handle partition expression updates without rebuilding memtables. The changes are well-structured and include necessary modifications to manifest handling, version control, and staging flows. The addition of VersionControl::alter_metadata is a key improvement for performance. The new logic is also well-covered by integration tests. I have one suggestion to improve test code maintainability by reducing duplication.

[WenyXu]

@codex review @cursor review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 110b14f916

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[fengjiachun]

LGTM