# Datafold

## Docs

- [Get Audit Logs](https://docs.datafold.com/api-reference/audit-logs/get-audit-logs.md)
- [Create a DBT BI integration](https://docs.datafold.com/api-reference/bi/create-a-dbt-bi-integration.md)
- [Create a Hightouch integration](https://docs.datafold.com/api-reference/bi/create-a-hightouch-integration.md)
- [Create a Looker integration](https://docs.datafold.com/api-reference/bi/create-a-looker-integration.md)
- [Create a Mode Analytics integration](https://docs.datafold.com/api-reference/bi/create-a-mode-analytics-integration.md)
- [Create a Power BI integration](https://docs.datafold.com/api-reference/bi/create-a-power-bi-integration.md)
- [Create a Tableau integration](https://docs.datafold.com/api-reference/bi/create-a-tableau-integration.md)
- [Get an integration](https://docs.datafold.com/api-reference/bi/get-an-integration.md): Returns the integration for Mode/Tableau/Looker/HighTouch by its id.
- [List all integrations](https://docs.datafold.com/api-reference/bi/list-all-integrations.md): Return all integrations for Mode/Tableau/Looker
- [Remove an integration](https://docs.datafold.com/api-reference/bi/remove-an-integration.md)
- [Rename a Power BI integration](https://docs.datafold.com/api-reference/bi/rename-a-power-bi-integration.md): It can only update the name. Returns the integration with changed fields.
- [Sync a BI integration](https://docs.datafold.com/api-reference/bi/sync-a-bi-integration.md): Start an unscheduled synchronization of the integration.
- [Update a DBT BI integration](https://docs.datafold.com/api-reference/bi/update-a-dbt-bi-integration.md): Returns the integration with changed fields.
- [Update a Hightouch integration](https://docs.datafold.com/api-reference/bi/update-a-hightouch-integration.md): It can only update the schedule. Returns the integration with changed fields.
- [Update a Looker integration](https://docs.datafold.com/api-reference/bi/update-a-looker-integration.md): It can only update the schedule. Returns the integration with changed fields.
- [Update a Mode Analytics integration](https://docs.datafold.com/api-reference/bi/update-a-mode-analytics-integration.md): It can only update the schedule. Returns the integration with changed fields.
- [Update a Tableau integration](https://docs.datafold.com/api-reference/bi/update-a-tableau-integration.md): It can only update the schedule. Returns the integration with changed fields.
- [List CI runs](https://docs.datafold.com/api-reference/ci/list-ci-runs.md)
- [Trigger a PR/MR run](https://docs.datafold.com/api-reference/ci/trigger-a-prmr-run.md)
- [Upload PR/MR changes](https://docs.datafold.com/api-reference/ci/upload-prmr-changes.md)
- [Create a data diff](https://docs.datafold.com/api-reference/data-diffs/create-a-data-diff.md): Launches a new data diff to compare two datasets (tables or queries).

A data diff identifies differences between two datasets by comparing:
- Row-level changes (added, removed, modified rows)
- Schema differences
- Column-level statistics

The diff runs asynchronously. Use the returned diff ID to poll for status and retrieve results.
- [Get a data diff](https://docs.datafold.com/api-reference/data-diffs/get-a-data-diff.md)
- [Get a data diff summary](https://docs.datafold.com/api-reference/data-diffs/get-a-data-diff-summary.md)
- [Get a human-readable summary of a DataDiff comparison](https://docs.datafold.com/api-reference/data-diffs/get-a-human-readable-summary-of-a-datadiff-comparison.md): Retrieves a comprehensive, human-readable summary of a completed data diff.

This endpoint provides the most useful information for understanding diff results:
- Overall status and result (success/failure)
- Human-readable feedback explaining the differences found
- Key statistics (row counts, differences, match rates)
- Configuration details (tables compared, primary keys used)
- Error messages if the diff failed

Use this after a diff completes to get actionable insights. For diffs still running,
check status with get_datadiff first.
- [List data diffs](https://docs.datafold.com/api-reference/data-diffs/list-data-diffs.md): All fields support multiple items, using just comma delimiter
Date fields also support ranges using the following syntax:

- ``<DATETIME`` = before DATETIME
- ``>DATETIME`` = after DATETIME
- ``DATETIME`` = between DATETIME and DATETIME + 1 MINUTE
- ``DATE`` = start of that DATE until DATE + 1 DAY
- ``DATETIME1<<DATETIME2`` = between DATETIME1 and DATETIME2
- ``DATE1<<DATE2`` = between DATE1 and DATE2
- [Update a data diff](https://docs.datafold.com/api-reference/data-diffs/update-a-data-diff.md)
- [Create a data source](https://docs.datafold.com/api-reference/data-sources/create-a-data-source.md)
- [Execute a SQL query against a data source](https://docs.datafold.com/api-reference/data-sources/execute-a-sql-query-against-a-data-source.md): Executes a SQL query against the specified data source and returns the results.

This endpoint allows you to run ad-hoc SQL queries for data exploration, validation, or analysis.
The query is executed using the data source's native query runner with the appropriate credentials.

Returns:
- Query results as rows with column metadata (name, type, description)
- Limited to a reasonable number of rows for performance
- [Get a data source](https://docs.datafold.com/api-reference/data-sources/get-a-data-source.md)
- [Get a data source summary](https://docs.datafold.com/api-reference/data-sources/get-a-data-source-summary.md)
- [Get data source testing results](https://docs.datafold.com/api-reference/data-sources/get-data-source-testing-results.md)
- [List data source types](https://docs.datafold.com/api-reference/data-sources/list-data-source-types.md)
- [List data sources](https://docs.datafold.com/api-reference/data-sources/list-data-sources.md): Retrieves all data sources accessible to the authenticated user.

Returns active data sources (not deleted, hidden, or draft) that the user has permission to access.
For non-admin users, only data sources belonging to their assigned groups are returned.
- [Test a data source connection](https://docs.datafold.com/api-reference/data-sources/test-a-data-source-connection.md)
- [Datafold SDK](https://docs.datafold.com/api-reference/datafold-sdk.md)
- [Get column downstreams](https://docs.datafold.com/api-reference/explore/get-column-downstreams.md): Retrieve a list of columns or tables which depend on the given column.
- [Get column upstreams](https://docs.datafold.com/api-reference/explore/get-column-upstreams.md): Retrieve a list of columns or tables which the given column depends on.
- [Get table downstreams](https://docs.datafold.com/api-reference/explore/get-table-downstreams.md): Retrieve a list of tables which depend on the given table.
- [Get table upstreams](https://docs.datafold.com/api-reference/explore/get-table-upstreams.md): Retrieve a list of tables which the given table depends on.
- [Introduction](https://docs.datafold.com/api-reference/introduction.md)
- [Create a Data Diff Monitor](https://docs.datafold.com/api-reference/monitors/create-a-data-diff-monitor.md)
- [Create a Data Test Monitor](https://docs.datafold.com/api-reference/monitors/create-a-data-test-monitor.md)
- [Create a Metric Monitor](https://docs.datafold.com/api-reference/monitors/create-a-metric-monitor.md)
- [Create a Schema Change Monitor](https://docs.datafold.com/api-reference/monitors/create-a-schema-change-monitor.md)
- [Delete a Monitor](https://docs.datafold.com/api-reference/monitors/delete-a-monitor.md)
- [Get Monitor](https://docs.datafold.com/api-reference/monitors/get-monitor.md)
- [Get Monitor Run](https://docs.datafold.com/api-reference/monitors/get-monitor-run.md)
- [List Monitor Runs](https://docs.datafold.com/api-reference/monitors/list-monitor-runs.md)
- [List Monitors](https://docs.datafold.com/api-reference/monitors/list-monitors.md)
- [Toggle a Monitor](https://docs.datafold.com/api-reference/monitors/toggle-a-monitor.md)
- [Trigger a run](https://docs.datafold.com/api-reference/monitors/trigger-a-run.md)
- [Update a Monitor](https://docs.datafold.com/api-reference/monitors/update-a-monitor.md)
- [Connection Budgets](https://docs.datafold.com/data-diff/connection-budgets.md): How connection budgets are enforced across data diffs in Datafold
- [Best Practices](https://docs.datafold.com/data-diff/cross-database-diffing/best-practices.md): When dealing with large datasets, it's crucial to approach diffing with specific optimization strategies in mind. We share best practices that will help you get the most accurate and efficient results from your data diffs.
- [Creating a New Data Diff](https://docs.datafold.com/data-diff/cross-database-diffing/creating-a-new-data-diff.md): Datafold's Data Diff can compare data across databases (e.g., PostgreSQL <> Snowflake, or between two SQL Server instances) to validate migrations, meet regulatory and compliance requirements, or ensure data is flowing successfully from source to target.
- [Results](https://docs.datafold.com/data-diff/cross-database-diffing/results.md): Once your data diff is complete, Datafold provides a concise, high-level summary of the detected changes in the Overview tab.
- [File Diffing](https://docs.datafold.com/data-diff/file-diffing.md): Datafold allows you to diff files (e.g. CSV, Excel, Parquet, etc.) in a similar way to how you diff tables.
- [How Datafold Diffs Data](https://docs.datafold.com/data-diff/how-datafold-diffs-data.md): Data diffs allow you to perform value-level comparisons between any two datasets within the same database, across different databases, or even between files.
- [Best Practices](https://docs.datafold.com/data-diff/in-database-diffing/best-practices.md): We share best practices that will help you get the most accurate and efficient results from your data diffs.
- [Creating a New Data Diff](https://docs.datafold.com/data-diff/in-database-diffing/creating-a-new-data-diff.md): Setting up a new data diff in Datafold is straightforward.
- [Results](https://docs.datafold.com/data-diff/in-database-diffing/results.md): Once your data diff is complete, Datafold provides a concise, high-level summary of the detected changes in the Overview tab
- [What's a Data Diff?](https://docs.datafold.com/data-diff/what-is-data-diff.md): A data diff is the value-level comparison between two tables, used to identify critical changes to your data and guarantee data quality.
- [dbt Metadata Sync](https://docs.datafold.com/data-explorer/best-practices/dbt-metadata-sync.md): Datafold can automatically ingest dbt metadata from your production environment and display it in Data Explorer.
- [How It Works](https://docs.datafold.com/data-explorer/how-it-works.md): The UI visually maps workflows and tracks column-level or tabular lineages, helping users understand the impact of upstream changes.
- [Lineage](https://docs.datafold.com/data-explorer/lineage.md): Datafold offers a column-level and tabular lineage view.
- [Profile](https://docs.datafold.com/data-explorer/profile.md): View a data profile that summarizes key table and column-level statistics, and any upstream dependencies.
- [Cross-Database Diffing for Migrations](https://docs.datafold.com/data-migration-automation/cross-database-diffing-migrations.md): Validate migration parity with Datafold's cross-database diffing solution.
- [Datafold Migration Agent](https://docs.datafold.com/data-migration-automation/datafold-migration-agent.md): Automatically migrate data environments of any scale and complexity with Datafold's Migration Agent.
- [Datafold for Migration Automation](https://docs.datafold.com/data-migration-automation/datafold-migration-automation.md): Datafold provides full-cycle migration automation with SQL code translation and cross-database validation for data warehouse, transformation framework, and hybrid migrations.
- [Monitor Types](https://docs.datafold.com/data-monitoring/monitor-types.md): Monitoring your data for unexpected changes is one of the cornerstones of data observability.
- [Monitors as Code](https://docs.datafold.com/data-monitoring/monitors-as-code.md): Manage Datafold monitors via version-controlled YAML for greater scalability, governance, and flexibility in code-based workflows.
- [Data Diff Monitors](https://docs.datafold.com/data-monitoring/monitors/data-diff-monitors.md): Data Diff monitors compare datasets across or within databases, identifying row and column discrepancies with customizable scheduling and notifications.
- [Data Test Monitors](https://docs.datafold.com/data-monitoring/monitors/data-test-monitors.md): Data Tests validate your data against off-the-shelf checks or custom business rules.
- [Metric Monitors](https://docs.datafold.com/data-monitoring/monitors/metric-monitors.md): Metric monitors detect anomalies in your data using ML-based algorithms or manual thresholds, supporting standard and custom metrics for tables or columns.
- [Schema Change Monitors](https://docs.datafold.com/data-monitoring/monitors/schema-change-monitors.md): Schema Change monitors notify you when a table’s schema changes, such as when columns are added, removed, or data types are modified.
- [Deployment Options](https://docs.datafold.com/datafold-deployment/datafold-deployment-options.md): Datafold is a web-based application with multiple deployment options, including multi-tenant SaaS and dedicated cloud (either customer- or Datafold-hosted).
- [Datafold VPC Deployment on AWS](https://docs.datafold.com/datafold-deployment/dedicated-cloud/aws.md): Learn how to deploy Datafold in a Virtual Private Cloud (VPC) on AWS.
- [Datafold VPC Deployment on Azure](https://docs.datafold.com/datafold-deployment/dedicated-cloud/azure.md): Learn how to deploy Datafold in a Virtual Private Cloud (VPC) on Azure.
- [Datafold VPC Deployment on GCP](https://docs.datafold.com/datafold-deployment/dedicated-cloud/gcp.md): Learn how to deploy Datafold in a Virtual Private Cloud (VPC) on GCP.
- [Best Practices](https://docs.datafold.com/deployment-testing/best-practices.md): Explore best practices for CI/CD testing in Datafold.
- [Handling Data Drift](https://docs.datafold.com/deployment-testing/best-practices/handling-data-drift.md): Ensuring Datafold in CI executes apples-to-apples comparison between staging and production environments.
- [Slim Diff](https://docs.datafold.com/deployment-testing/best-practices/slim-diff.md): Choose which downstream tables to diff to optimize time, cost, and performance.
- [Configuration](https://docs.datafold.com/deployment-testing/configuration.md): Explore configuration options for CI/CD testing in Datafold.
- [Column Remapping](https://docs.datafold.com/deployment-testing/configuration/column-remapping.md): Specify column renaming in your git commit message so Datafold can map renamed columns to their original counterparts in production for accurate comparison.
- [Running Data Diff for Specific PRs/MRs](https://docs.datafold.com/deployment-testing/configuration/datafold-ci/on-demand.md): By default, Datafold CI runs on every new pull/merge request and commits to existing ones.
- [Running Data Diff on Specific Branches](https://docs.datafold.com/deployment-testing/configuration/datafold-ci/specifc.md): By default, Datafold CI runs on every new pull/merge request and commits to existing ones.
- [Diff Timeline](https://docs.datafold.com/deployment-testing/configuration/model-specific-ci/diff-timeline.md): Specify a `time_column` to visualize match rates between tables for each column over time.
- [Excluding Models](https://docs.datafold.com/deployment-testing/configuration/model-specific-ci/excluding-models.md): Use `never_diff` to exclude a model or subdirectory of models from data diffs.
- [Including/Excluding Columns](https://docs.datafold.com/deployment-testing/configuration/model-specific-ci/including-excluding-columns.md): Specify columns to include or exclude from the data diff using `include_columns` and `exclude_columns`.
- [SQL Filters](https://docs.datafold.com/deployment-testing/configuration/model-specific-ci/sql-filters.md): Use dbt YAML configuration to set model-specific filters for Datafold CI.
- [Time Travel](https://docs.datafold.com/deployment-testing/configuration/model-specific-ci/time-travel.md): Use `prod_time_travel` and `pr_time_travel` to diff tables from specific points in time.
- [Primary Key Inference](https://docs.datafold.com/deployment-testing/configuration/primary-key.md): Datafold requires a primary key to perform data diffs. Using dbt metadata, Datafold identifies the column to use as the primary key for accurate data diffs.
- [Getting Started with CI/CD Testing](https://docs.datafold.com/deployment-testing/getting-started.md): Learn how to set up CI/CD testing with Datafold by integrating your data connections, code repositories, and CI pipeline for automated testing.
- [API](https://docs.datafold.com/deployment-testing/getting-started/universal/api.md): Learn how to set up and configure Datafold's API for CI/CD testing.
- [Fully-Automated](https://docs.datafold.com/deployment-testing/getting-started/universal/fully-automated.md): Automatically diff tables modified in a pull request with Datafold's Fully-Automated CI integration.
- [No-Code](https://docs.datafold.com/deployment-testing/getting-started/universal/no-code.md): Set up Datafold's No-Code CI integration to create and manage Data Diffs without writing code.
- [How Datafold in CI Works](https://docs.datafold.com/deployment-testing/how-it-works.md): Learn how Datafold integrates with your Continuous Integration (CI) process to create Data Diffs for all SQL code changes, catching issues before they make it into production.
- [CI/CD Testing](https://docs.datafold.com/faq/ci-cd-testing.md)
- [Data Diffing](https://docs.datafold.com/faq/data-diffing.md)
- [Data Migration Automation](https://docs.datafold.com/faq/data-migration-automation.md)
- [Data Monitoring and Observability](https://docs.datafold.com/faq/data-monitoring-observability.md)
- [Data Reconciliation](https://docs.datafold.com/faq/data-reconciliation.md)
- [Data Storage and Security](https://docs.datafold.com/faq/data-storage-and-security.md)
- [Integrating Datafold with dbt](https://docs.datafold.com/faq/datafold-with-dbt.md)
- [Overview](https://docs.datafold.com/faq/overview.md): Get answers to the most common questions regarding our product.
- [Performance and Scalability](https://docs.datafold.com/faq/performance-and-scalability.md)
- [Resource Management](https://docs.datafold.com/faq/resource-management.md)
- [dbt Exposures](https://docs.datafold.com/integrations/bi-data-apps/dbt.md): Incorporate dbt Exposures into your Datafold lineage.
- [Hightouch](https://docs.datafold.com/integrations/bi-data-apps/hightouch.md): Navigate to Settings > Integrations > Data Apps and add a Hightouch Integration.
- [Looker](https://docs.datafold.com/integrations/bi-data-apps/looker.md)
- [Mode](https://docs.datafold.com/integrations/bi-data-apps/mode.md)
- [Power BI](https://docs.datafold.com/integrations/bi-data-apps/power-bi.md): Include Power BI entities in Data Explorer and column-level lineage.
- [Tableau](https://docs.datafold.com/integrations/bi-data-apps/tableau.md): Visualize downstream Tableau dependencies and understand how warehouse changes impact your BI layer.
- [Tracking Jobs](https://docs.datafold.com/integrations/bi-data-apps/tracking-jobs.md): Track the completion and success of your data app integration syncs.
- [Integrate with Code Repositories](https://docs.datafold.com/integrations/code-repositories.md): Connect your code repositories with Datafold.
- [Azure DevOps](https://docs.datafold.com/integrations/code-repositories/azure-devops.md)
- [Bitbucket](https://docs.datafold.com/integrations/code-repositories/bitbucket.md)
- [GitHub](https://docs.datafold.com/integrations/code-repositories/github.md)
- [GitLab](https://docs.datafold.com/integrations/code-repositories/gitlab.md)
- [Set Up Your Data Connection](https://docs.datafold.com/integrations/databases.md): Set up your Data Connection with Datafold.
- [Azure Data Lake Storage (ADLS)](https://docs.datafold.com/integrations/databases/adls.md)
- [Amazon S3](https://docs.datafold.com/integrations/databases/amazon-s3.md)
- [Athena](https://docs.datafold.com/integrations/databases/athena.md)
- [BigQuery](https://docs.datafold.com/integrations/databases/bigquery.md)
- [Databricks](https://docs.datafold.com/integrations/databases/databricks.md)
- [Dremio](https://docs.datafold.com/integrations/databases/dremio.md)
- [Google Cloud Storage (GCS)](https://docs.datafold.com/integrations/databases/google-cloud-storage.md)
- [MariaDB](https://docs.datafold.com/integrations/databases/mariadb.md)
- [MongoDB](https://docs.datafold.com/integrations/databases/mongodb.md): Our MongoDB integration allows you to diff data within MongoDB, or between MongoDB and a relational database (or even a file!).
- [MySQL](https://docs.datafold.com/integrations/databases/mysql.md)
- [Netezza](https://docs.datafold.com/integrations/databases/netezza.md)
- [Oracle](https://docs.datafold.com/integrations/databases/oracle.md)
- [PostgreSQL](https://docs.datafold.com/integrations/databases/postgresql.md)
- [Redshift](https://docs.datafold.com/integrations/databases/redshift.md)
- [SAP HANA](https://docs.datafold.com/integrations/databases/sap-hana.md)
- [Snowflake](https://docs.datafold.com/integrations/databases/snowflake.md)
- [Microsoft SQL Server](https://docs.datafold.com/integrations/databases/sql-server.md)
- [Starburst](https://docs.datafold.com/integrations/databases/starburst.md)
- [Teradata](https://docs.datafold.com/integrations/databases/teradata.md)
- [Trino](https://docs.datafold.com/integrations/databases/trino.md)
- [OpenText Analytics Database (Vertica)](https://docs.datafold.com/integrations/databases/vertica.md)
- [Microsoft Teams](https://docs.datafold.com/integrations/notifications/microsoft-teams.md): Receive notifications for monitors in Microsoft Teams.
- [PagerDuty](https://docs.datafold.com/integrations/notifications/pagerduty.md): Receive notifications for monitors in PagerDuty.
- [Slack](https://docs.datafold.com/integrations/notifications/slack.md): Receive notifications for monitors in Slack.
- [OAuth Support](https://docs.datafold.com/integrations/oauth.md): Set up OAuth App Connections in your supported data warehouses to securely execute data diffs on behalf of your users.
- [Integrate with Orchestrators](https://docs.datafold.com/integrations/orchestrators.md): Integrate Datafold with dbt Core, dbt Cloud, Airflow, or custom orchestrators to streamline your data workflows with automated monitoring, testing, and seamless CI integration.
- [Custom Integrations](https://docs.datafold.com/integrations/orchestrators/custom-integrations.md): Integrate Datafold with your custom orchestration using the Datafold SDK and REST API.
- [dbt Cloud](https://docs.datafold.com/integrations/orchestrators/dbt-cloud.md): Integrate Datafold with dbt Cloud to automate Data Diffs in your CI pipeline, leveraging dbt jobs to detect changes and ensure data quality before merging.
- [dbt Core](https://docs.datafold.com/integrations/orchestrators/dbt-core.md): Set up Datafold’s integration with dbt Core to automate Data Diffs in your CI pipeline.
- [Compliance & Trust Center](https://docs.datafold.com/security/compilance-trust-center.md)
- [Database OAuth](https://docs.datafold.com/security/database-oauth.md): Datafold enables secure workflows like data diffs through OAuth, ensuring compliance with user-specific database permissions.
- [Securing Connections](https://docs.datafold.com/security/securing-connections.md): Datafold supports multiple options to secure connections between your resources (e.g., databases and BI tools) and Datafold.
- [Single Sign-On](https://docs.datafold.com/security/single-sign-on.md): Set up Single Sign-On with one of the following options.
- [Google OAuth](https://docs.datafold.com/security/single-sign-on/google-oauth.md)
- [Okta (OIDC)](https://docs.datafold.com/security/single-sign-on/okta.md)
- [SAML](https://docs.datafold.com/security/single-sign-on/saml.md): SAML (Security Assertion Markup Language) is a protocol that enables secure user authentication by integrating Identity Providers (IdPs) with Service Providers (SPs).
- [Google](https://docs.datafold.com/security/single-sign-on/saml/examples/google.md)
- [Microsoft Entra ID](https://docs.datafold.com/security/single-sign-on/saml/examples/microsoft-entra-id-configuration.md)
- [Okta](https://docs.datafold.com/security/single-sign-on/saml/examples/okta.md)
- [null](https://docs.datafold.com/security/single-sign-on/saml/group-provisioning.md): Automatically sync group membership with your SAML Identity Provider (IdP).
- [User Roles and Permissions](https://docs.datafold.com/security/user-roles-and-permissions.md): Datafold uses role-based access control to manage user permissions and actions.
- [FAQ](https://docs.datafold.com/support/faq-redirect.md)
- [Support](https://docs.datafold.com/support/support.md): Datafold offers multiple support channels to assist users with troubleshooting and inquiries.
- [Welcome](https://docs.datafold.com/welcome.md): Datafold is the unified platform proactive data quality that combines automated data testing, data reconciliation, and observability to help data teams prevent data quality issues and accelerate their development velocity.


## Optional

- [About Datafold](https://www.datafold.com/)
- [Blog](https://www.datafold.com/blog?)