Skip to content

Conversation

@grantdfoster
Copy link
Collaborator

@grantdfoster grantdfoster commented Oct 28, 2025

PR summary (vs main)

  • Scope: 5 files changed, 18 insertions, 30 deletions
  • Commits:
    • feat: add residential proxies to tiktok query and web
    • chore: add residential proxy to tiktok query

High-level changes

  • Residential proxy support added: Introduces a typed ApifyProxy with residential group and wires it into both web scraper and TikTok Apify requests.
  • Typed Apify inputs: Replaces map[string]any proxy payloads with a strongly-typed struct and passes typed requests directly to the Apify client (removing JSON marshal/unmarshal round-trips).
  • Schema alignment: Web scraper request now uses proxyConfiguration in the JSON payload.

File-by-file highlights

  • api/types/apify.go (new)
    • Adds ApifyProxy { UseApifyProxy bool, ApifyProxyGroups []string }.
  • api/types/web.go
    • WebScraperRequest gains Proxy ApifyProxy with JSON proxyConfiguration.
  • api/args/web/scraper/scraper.go
    • ToScraperRequest() sets Proxy to UseApifyProxy: true with ApifyProxyGroups: ["RESIDENTIAL"].
  • internal/jobs/tiktokapify/client.go
    • TikTokSearchByQueryRequest.Proxy becomes types.ApifyProxy.
    • Request construction sets residential proxy group.
    • Removes marshal/unmarshal to map[string]any; calls RunActorAndGetResponse with the typed request for both search and trending flows.
  • internal/jobs/tiktok_test.go
    • Removes assertion relying on legacy "proxy": {"use_apify_proxy": true} map shape.

Impact/Risks

  • Type/API surface change: Any callers expecting map[string]any for proxy/request inputs must use types.ApifyProxy.
  • JSON shape changes: Web scraper payload uses proxyConfiguration with useApifyProxy/apifyProxyGroups (camelCase), replacing any prior snake_case usage. Ensure external actor inputs match this schema.

Copilot AI review requested due to automatic review settings October 28, 2025 01:46
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR fixes residential actor proxy configuration across TikTok and web scraper clients by introducing a typed ApifyProxy struct to replace unstructured proxy configurations.

Key changes:

  • Introduced types.ApifyProxy struct to standardize proxy configuration with residential proxy groups
  • Simplified TikTok client by removing unnecessary JSON marshaling/unmarshaling steps
  • Updated web scraper to use the new typed proxy configuration

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file
File Description
api/types/apify.go Introduces new ApifyProxy struct with typed fields for proxy configuration
internal/jobs/tiktokapify/client.go Updates TikTok client to use typed proxy struct and removes redundant marshaling logic
internal/jobs/tiktok_test.go Removes proxy parameter from test case as it's now handled internally
api/types/web.go Adds Proxy field using ApifyProxy type to web scraper request
api/args/web/scraper/scraper.go Configures default residential proxy settings in web scraper arguments

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@grantdfoster grantdfoster requested a review from Copilot October 28, 2025 01:47
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@grantdfoster
Copy link
Collaborator Author

Fixes #184

Copy link
Collaborator

@rapidfix rapidfix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good!

@grantdfoster grantdfoster merged commit ea74a79 into main Oct 28, 2025
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants