Overview

What is dataform?

Dataform makes it easy to manage complex SQL data pipelines in your data warehouse. Dataform's powerful API lets you quickly transform your data and perform other actions by writing simple statements.

What can it do?

With Dataform you can:

Dataform currently supports Google BigQuery, Amazon Redshift, and Snowflake data warehouses.

How can I use it?

You can get started quickly with https://dataform.co, or check out our command line interface and open-source packages to download and develop Dataform projects yourself.

How does it work?

  1. You write SQL files enriched with Dataform's API and templating functions.
  2. Dataform compiles, validates, and executes the generated SQL statements against your warehouse, automating boiler plate such as create table and insert statements.
  3. You have a clean, well defined datasets pushed back to your warehouse, that the rest of your team can use for anything from dashboards to machine learning.

Dataform's enriched SQL format allows you to:

  • Reference and declare dependencies between datasets
  • Re-use common SQL across many queries
  • Write tests against your data
  • Document your tables fields
  • Write custom functions in JavaScript