Watchflow Rules Validation Issues

[dimeloper]

Problem Description

There are two critical issues with the current Watchflow rules validation system that impact developer experience and rule testing workflows:

1. Misleading Success Status When Rules File is Missing

Current Behavior:

• When .watchflow/rules.yaml is missing, the check shows an error message but still reports "All checks have passed" and "All rules passed"
• This creates confusion as the status indicates success despite the missing configuration

Example Output:

❌ **Watchflow rules file not found**
The file `.watchflow/rules.yaml` is missing from your repository.

[... error message continues ...]

Merge info
All checks have passed ✅
2 successful checks
successful checks
* CI - Lint and Test / lint-and-test (pull_request) ✅
* Watchflow Rules — All rules passed ✅

Expected Behavior:

• When rules file is missing, the check should fail with appropriate status
• Status should clearly indicate that Watchflow validation failed
• Should prevent merge if configured as a required check

2. Rules Only Read from Main Branch

Current Behavior:

• Watchflow always reads rules from the main branch, regardless of which branch is being validated
• This prevents testing rule changes in feature branches before merging
• Makes it impossible to validate new rules alongside the code changes they're meant to protect

Impact:

• Developers cannot iterate on rules in feature branches
• Rule changes must be merged blindly without validation
• No way to test rule modifications against the current PR changes

Expected Behavior:

• Watchflow should provide a parameter/configuration option to read rules from the current branch being validated
• By default, rules should continue to be read from main branch (maintaining backward compatibility)
• When enabled, this allows rule development and testing in feature branches
• Enables validation of both code and rule changes together in the same PR

Proposed Solutions

For Issue #1: Fix Missing Rules Status

• Change check status to failed when .watchflow/rules.yaml is missing
• Update status message to clearly indicate validation failure
• Ensure GitHub branch protection rules can properly block merges

For Issue #2: Enable Branch-Based Rules

• Add a configuration parameter to control rules source branch (default: main)
• When parameter is set, read rules from the current branch being validated
• Maintain default behavior of reading from main branch for backward compatibility
• Update documentation with branch-based rule development workflow and new parameter usage

Acceptance Criteria

• Missing rules file causes check to fail (not pass)
• GitHub status clearly indicates Watchflow validation failure
• Configuration parameter available to read rules from current branch
• Default behavior remains reading rules from main branch
• When enabled, rules can be read from feature branches
• Rule changes can be tested before merging to main when parameter is configured
• Backward compatibility maintained for existing setups
• Documentation updated with new workflow patterns and parameter usage

Additional Context

These issues significantly impact the developer experience when working with Watchflow rules. The first issue can lead to false confidence in merge safety, while the second prevents proper rule development workflows when teams need to iterate on rules alongside their code changes.

The proposed parameter-based approach for branch rules maintains backward compatibility while enabling advanced workflows for teams that need to test rule modifications in feature branches.

Both changes would align Watchflow behavior with standard expectations for CI/CD tools and GitHub status checks.

[dkargatzis]

Thanks @dimeloper for the detailed feedback!

Issue 1: Missing rules status

Nice catch! When .watchflow/rules.yaml is missing, the check should have the appropriate status, not pass.

Issue 2: Branch-based rules

For testing rules on branch PRs, I suggest a simple approach with a separate file.

Test rules file

• This allows teams to test rule changes in feature branches without any configuration
• No breaking changes to existing workflows
• Perfect for the "test rules alongside code changes" use case

This approach is much simpler than branch parameters and solves the core problem. Teams can create rules-test.yaml in their feature branches to test rule changes before merging.

What do you think?

[dimeloper]

@dkargatzis sounds good, thanks for the quick response and the thoughtful solution!

The proposed solution with a separate file actually feels more natural than a parameter-based approach. Having a dedicated test file makes the workflow explicit and discoverable for other team members.

[dkargatzis]

Status Update

Point 1: Fixed - PR #11 has been merged and addresses the misleading success status when rules file is missing. The check now shows "neutral" status with clear setup instructions instead of misleading "success".

Point 2: Under Discussion - The branch-based rule testing workflow is being explored in Discussion #12. This is a more complex feature that requires careful consideration of security implications and user workflow preferences.

Next Steps:

• Point 1 changes are now live in main
• Point 2 will be addressed in a future release after community discussion

Thanks @dimeloper for raising these issues! The first point is now resolved, and we're actively working on the second.