Processor Great Expectations

Great Expectations is a processor that runs user-defined tests on their dataset. It can generally validate the data and check for inconsistencies.

Data In/ Data Out

Data In	A CSV file that you want to test. The file should be in folder `data/in/table/`
Data Out	The same file as Data In, the file can be found in folder `data/out/table/` All test results can be found in the Activity Logs. If any test fails, the component will fail and returns an error.

Learn more: about the folder structure please go to this article.

Parameters

Input Table (required)

Name of the file to be tested.

Delimiter (required)

Delimiter of the file, for a CSV, the delimiter is a comma ,

Test (required)

The test is to be applied.

Examples of tests that can be applied:
- expect_column_to_exist(<column name>)
- expect_column_values_to_be_in_set(<column name>, [0,1,2])

- expect_table_row_count_to_be_between(0,1000)
- expect_column_values_to_be_of_type(<column name>, 'float')

For the full lists of tests, you may check out the link here.