Adding tests to a project helps validate that your models are working correctly. These tests are run every time your project updates, sending alerts if the tests fail. Data quality tests in Dataform are called assertions.
We want to create an assertion for our order_stats
table that will fail if there is more than one row in the dataset with the same value for id
.
Navigate back to your order_stats
table:
Develop Project
.order_stats
file.In the config block add your assertion:
1config { 2 type: "table", 3 assertions: { 4 uniqueKey: ["id"] 5 } 6}
Dataform automatically creates a view in your warehouse containing the results of the compiled assertion query. This makes it easy to inspect the rows that caused the assertion to fail.
You can also choose to use assertions as dependencies. Assertions create named actions in your project that can be depended upon by using the dependencies config
parameter. If you would like another dataset, assertion, or operation to only run if a specific assertion passes, you can add the assertion to that action's dependencies.
Dataform allows you to add documentation to the datasets defined in your project. Table and field descriptions are added using the same config block where you wrote your assertion.
In the config block write your documentation:
1config { 2 type: "table", 3 description: "This table joins orders information from Shopify & payment information from Stripe", 4 columns: { 5 order_date: "The date when a customer placed their order", 6 id: "Order ID as defined by Shopify", 7 order_status: "The status of an order e.g. sent, delivered", 8 customer_id: "Unique customer ID", 9 payment_status: "The status of a payment e.g. pending, paid", 10 payment_method: "How the customer chose to pay", 11 item_count: "The number of items the customer ordered", 12 amount: "The amount the customer paid" 13 }, 14 assertions: { 15 uniqueKey: ["id"] 16 } 17}
View the Dependency tree
:
Dependency tree
tab in the hamburger menu in the top left hand corner:order_stats
table on the left.You have now created an assertion which will fail if any value in your id
field is duplicated and you have given your order_stats
table and its fields descriptions.
You can find more information on assertions and documentation in our docs.