Is there an existing issue for the same bug?
Environment
Actual behavior
MergeDeep preserves every base field that is absent from the operator override and appends every base fileservice entry whose name is not matched by an operator-generated entry. It has no schema-aware deletion or migration semantics.
This creates several unsafe cases.
1. Stale backend-specific fields survive provider migration
If the user configuration contains an operator-managed entry such as:
[[fileservice]]
name = "S3"
backend = "DISK"
data-dir = "/old/path"
and shared storage is later changed from FileSystem to S3, the operator overrides backend with S3 and adds the s3 table but does not remove data-dir. The rendered entry contains backend = S3, s3.*, and the stale data-dir.
The reverse S3 to FileSystem transition can similarly retain a stale s3 table.
Current MatrixOne code accepts data-dir as a known field and does not use it in the S3 constructor, so this particular mixed configuration is currently ignored rather than rejected. It is nevertheless stale configuration with future compatibility and operational-debugging risk.
2. Unmanaged entries are forwarded to MatrixOne
A user entry such as name = "CUSTOM" or name = "SHARED" does not match the operator-managed LOCAL, S3, or ETL names and is appended to the generated list. MatrixOne iterates and initializes every fileservice entry; it does not ignore additional entries.
A malformed backend, inaccessible local data directory, failed object-storage initialization, or duplicate service name can therefore prevent the MO process from starting.
The existing TestMergeDeep case named user adds extra fileservice entry explicitly codifies this append behavior without testing the MatrixOne startup consequences.
3. Operator and MatrixOne use different name matching semantics
mergeNamedMapSlice matches names case-sensitively. MatrixOne registers fileservices using lowercase names and rejects duplicates case-insensitively.
For example, a user entry named s3 is not merged with the operator entry named S3, so both entries are rendered. MatrixOne then normalizes both to s3 and returns a duplicate service name error during startup.
Expected behavior
Fileservice merging should be schema-aware:
- Operator-managed names should be matched case-insensitively and rendered with canonical names.
- Backend-specific fields that are invalid or irrelevant after a provider transition should be removed.
- The policy for unmanaged fileservice entries should be explicit. They should either be rejected by operator validation or supported only after validating that MatrixOne can initialize them safely.
- User tuning fields that remain valid for the selected backend should still be preserved.
Steps to reproduce
Stale data-dir
- Configure a user fileservice named S3 with backend DISK and data-dir.
- Configure the LogSet shared storage provider as S3.
- Build a CN or DN ConfigMap.
- Observe that the rendered S3 entry contains both the generated S3 configuration and the old data-dir.
Case-insensitive duplicate
- Configure a user fileservice named s3.
- Let the operator generate its normal S3 entry.
- Observe both s3 and S3 in the rendered TOML.
- Start MatrixOne and observe duplicate service-name validation after names are normalized to lowercase.
Suggested test coverage
- FileSystem to S3 removes data-dir from the managed S3 entry.
- S3 to FileSystem removes the stale s3 table.
- Managed names are matched case-insensitively.
- Unmanaged CUSTOM or SHARED entries follow the chosen validation policy.
- Invalid extra entries cannot reach MatrixOne startup.
- Valid user tuning fields under s3 and cache remain preserved.
Acceptance criteria
- Provider transitions do not leave incompatible backend-specific fields in generated TOML.
- s3 and S3 cannot be emitted as separate entries.
- Unmanaged fileservice behavior is documented and validated.
- Regression tests cover final controller-generated TOML, not only the merge helper.
References
Is there an existing issue for the same bug?
Environment
Actual behavior
MergeDeep preserves every base field that is absent from the operator override and appends every base fileservice entry whose name is not matched by an operator-generated entry. It has no schema-aware deletion or migration semantics.
This creates several unsafe cases.
1. Stale backend-specific fields survive provider migration
If the user configuration contains an operator-managed entry such as:
and shared storage is later changed from FileSystem to S3, the operator overrides backend with S3 and adds the s3 table but does not remove data-dir. The rendered entry contains backend = S3, s3.*, and the stale data-dir.
The reverse S3 to FileSystem transition can similarly retain a stale s3 table.
Current MatrixOne code accepts data-dir as a known field and does not use it in the S3 constructor, so this particular mixed configuration is currently ignored rather than rejected. It is nevertheless stale configuration with future compatibility and operational-debugging risk.
2. Unmanaged entries are forwarded to MatrixOne
A user entry such as name = "CUSTOM" or name = "SHARED" does not match the operator-managed LOCAL, S3, or ETL names and is appended to the generated list. MatrixOne iterates and initializes every fileservice entry; it does not ignore additional entries.
A malformed backend, inaccessible local data directory, failed object-storage initialization, or duplicate service name can therefore prevent the MO process from starting.
The existing TestMergeDeep case named user adds extra fileservice entry explicitly codifies this append behavior without testing the MatrixOne startup consequences.
3. Operator and MatrixOne use different name matching semantics
mergeNamedMapSlice matches names case-sensitively. MatrixOne registers fileservices using lowercase names and rejects duplicates case-insensitively.
For example, a user entry named s3 is not merged with the operator entry named S3, so both entries are rendered. MatrixOne then normalizes both to s3 and returns a duplicate service name error during startup.
Expected behavior
Fileservice merging should be schema-aware:
Steps to reproduce
Stale data-dir
Case-insensitive duplicate
Suggested test coverage
Acceptance criteria
References