This plugin ports dbt functionality to Apache Doris, based on dbt-mysql.

Still under development.

This is an experimental plugin:

• We have not tested it extensively
• Storage engines other than the default of InnoDB are untested
• MariaDB compatibility is untested
• Only tested with dbt-adapter-tests with MySQL 5.6, 5.7, and 8.0
• Compatiblity with other dbt packages (like dbt_utils) is also untested

Please read these docs carefully and use at your own risk. Issues and PRs welcome!

• Installation
• Supported features
  • MySQL 8.0
  • MySQL 5.6 and 5.7
    • MySQL 5.6 configuration gotchas
    • MySQL 5.7 configuration gotchas
• Configuring your profile
• Notes
• Running Tests
• Reporting bugs and contributing code

This plugin can be installed via pip:

$ pip install dbt-mysql

Notes:

• Ephemeral materializations rely upon Common Table Expressions (CTEs), which are not supported until MySQL 8.0
• MySQL 5.6 and 5.7 have some configuration gotchas that affect snapshots (see below).

dbt snapshots might not work properly due to automatic initialization and updating for TIMESTAMP if:

• the output of SHOW VARIABLES LIKE 'sql_mode' includes NO_ZERO_DATE
• the output of SHOW GLOBAL VARIABLES LIKE 'explicit_defaults_for_timestamp' has a value of OFF

A solution is to include the following in a *.cnf file: Configuration to include in a *.cnf file:

[mysqld]
explicit_defaults_for_timestamp = true

dbt snapshots might not work properly due to automatic initialization and updating for TIMESTAMP if: dbt snapshots might not work properly if:

• the output of SHOW VARIABLES LIKE 'sql_mode' includes NO_ZERO_DATE

A solution is to include the following in a *.cnf file: Configuration to include in a *.cnf file:

[mysqld]
explicit_defaults_for_timestamp = true
sql_mode = "ALLOW_INVALID_DATES,{other_sql_modes}"

where {other_sql_modes} is the rest of the modes from the SHOW VARIABLES LIKE 'sql_mode' output.

A dbt profile can be configured to run against MySQL using the following configuration example:

Use type: mysql for MySQL 8.x and type: mysql5 for MySQL 5.x

Example entry for profiles.yml:

your_profile_name:
  target: dev
  outputs:
    dev:
      type: mysql
      server: localhost
      port: 3306
      schema: analytics
      username: your_mysql_username
      password: your_mysql_password

Conflicting terminology is used between:

• dbt
• Database management systems (DBMS) like MySQL, Postgres, and Snowflake
• metadata in the ANSI-standard information_schema

The conflicts include both:

• the same word meaning different things
• different words meaning the same thing

For example, a "database" in MySQL is not the same as a "database" in dbt, but it is equivalent to a "schema" in Postgres 🤯.

dbt-mysql uses the dbt terms. The native MySQL verbiage is restricted to SQL statements.

This cross-walk aligns the terminology:

Additionally, many DBMS have relation names with three parts whereas MySQL has only two. E.g., a fully-qualified table name in Postgres is database.schema.table versus database.table in MySQL. The missing part in MySQL is the information_schema "catalog".

See test/README.md for details on running the integration tests.

• Want to report a bug or request a feature? See the contributing guidelines, or open an issue.

dbt-mysql borrows from dbt-spark and dbt-sqlite since Spark and SQLite also use two-part relation names.