ePrivacy and GPDR Cookie Consent by Cookie Consent Skip to main content

Processor Great Expectations

Great Expectations is a processor that runs user-defined tests on their dataset. It can generally validate the data and check for inconsistencies.

Data In/ Data Out

Data In

A CSV file that you want to test.
The file should be in folder data/in/table/

Data Out

The same file as Data In, the file can be found in folder data/out/table/
All test results can be found in the Activity Logs.
If any test fails, the component will fail and returns an error.

Learn more: about the folder structure please go to this article.

Parameters

Screenshot 2021-11-26 at 4.14.34 PM.png

Input Table (required)

Name of the file to be tested.

Delimiter (required)

Delimiter of the file, for a CSV, the delimiter is a comma ,

Test (required)

The test is to be applied.

Examples of tests that can be applied:
- expect_column_to_exist(<column name>)
- expect_column_values_to_be_in_set(<column name>, [0,1,2])

- expect_table_row_count_to_be_between(0,1000)
- expect_column_values_to_be_of_type(<column name>, 'float')

For the full lists of tests, you may check out the link here.