[ENG-1404] Endpoint updates

[JadeCara]

Ticket ENG-1404

Description Of Changes

🎯 Fides must provide a way for privacy admins to quickly identify and surface in the request manager UI privacy requests which are likely duplicates submitted by the same user over a period of time.

This PR adds the check_for_duplicates function to duplicate_detection.py. This function allows us to check for duplicates in a number of scenarios, inluding creation, receiving identity validation and the cases where everything is auto approved and a request goes straight to the runner.

Code Changes

• added check_for_duplicates function
• checking for duplicates in more places
• Updated the actions in PrivacyRequestFilter to accept a list (request from @lucanovera )
• Added IdentityValue schema for better openapi functionality (request from @lucanovera )

Steps to Confirm

1. Running Fides or Fideplus pointed at this branch
2. enable duplicate_privacy_request_detection by hitting PATCH /api/v1/config with the following

{
  {
  "execution": {
    "subject_identity_verification_required": false, (try both true and false)
    "require_manual_request_approval": false (try both true and false)
  },
  "privacy_request_duplicate_detection": {
    "enabled": true
  }
}

3. Create a couple of Privacy requests - one that is a duplicate of the other.
4. Try with both identity verified and not verified.
5. Try creating a new privacy request when the previous has completed (should not be a duplicate)
6. Try creating a new privacy request when the previous is still running.

Pre-Merge Checklist

• Issue requirements met
• All CI pipelines succeeded
• CHANGELOG.md updated
  • Add a db-migration This indicates that a change includes a database migration label to the entry if your change includes a DB migration
  • Add a high-risk This issue suggests changes that have a high-probability of breaking existing code label to the entry if your change includes a high-risk change (i.e. potential for performance impact or unexpected regression) that should be flagged
  • Updates unreleased work already in Changelog, no new entry necessary
• UX feedback:
  • All UX related changes have been reviewed by a designer
  • No UX review needed
• Followup issues:
  • Followup issues created
  • No followup issues
• Database migrations:
  • Ensure that your downrev is up to date with the latest revision on main
  • Ensure that your downgrade() migration is correct and works
    • If a downgrade migration is not possible for this change, please call this out in the PR description!
  • No migrations
• Documentation:
  • Documentation complete, PR opened in fidesdocs
  • Documentation issue created in fidesdocs
  • If there are any new client scopes created as part of the pull request, remember to update public-facing documentation that references our scope registry
  • No documentation updates required

[JadeCara]

@greptile please review this pr

[greptile-apps]

Greptile Overview

Greptile Summary

This PR refactors duplicate detection to use a centralized check_for_duplicates function that consolidates duplicate checking logic across multiple scenarios (request creation, identity verification, and auto-approval flows).

Key changes:

• Added check_for_duplicates helper function in duplication_detection.py that encapsulates duplicate detection service instantiation and checking
• Integrated duplicate checking into handle_approval function to catch duplicates when requests are manually approved
• Updated PrivacyRequestFilter.action_type to accept list of action types for better API flexibility
• Added IdentityValue schema for improved OpenAPI documentation
• Added frontend support for displaying duplicate status across admin UI and privacy center
• Comprehensive test coverage including runner service integration tests

Review findings:

• Clean refactoring that improves code reusability and maintainability
• Proper separation of concerns with duplicate checking happening at appropriate lifecycle points
• Good test coverage for various duplicate detection scenarios

Confidence Score: 5/5

• This PR is safe to merge with minimal risk
• The changes are well-structured refactoring that improves code organization without introducing new logic. The duplicate detection logic itself was already implemented and tested in the base branch - this PR simply consolidates it into a reusable function and adds integration points. Comprehensive test coverage validates the refactoring.
• No files require special attention

Important Files Changed

File Analysis

Filename	Score	Overview
src/fides/api/service/privacy_request/duplication_detection.py	5/5	Added check_for_duplicates function to consolidate duplicate detection logic across different scenarios (creation, identity validation, auto-approval). Clean refactoring that extracts reusable logic.
src/fides/service/privacy_request/privacy_request_service.py	5/5	Refactored to use new check_for_duplicates function instead of directly instantiating DuplicateDetectionService. Simplified logic and added duplicate check to handle_approval function.
src/fides/api/service/privacy_request/request_runner_service.py	5/5	Updated to use check_for_duplicates function, simplified duplicate detection logic in the runner service. Consistent with other service changes.
src/fides/api/schemas/privacy_request.py	5/5	Added IdentityValue schema for better OpenAPI documentation. Updated PrivacyRequestFilter.action_type to accept list of action types, improving API flexibility.
tests/ops/service/privacy_request/test_duplication_detection.py	5/5	Comprehensive test coverage for duplicate detection scenarios including verified identities, time windows, actioned requests, and runner service integration.

_{15 files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}

[lucanovera]

Tested the feature with real requests from the privacy center and can confirm it works as expected. 💯