ePrivacy and GPDR Cookie Consent by Cookie Consent Skip to main content

Load events to CDP

Learn from this article:

A component for loading data into the CDP data model
Configuration: PostgreSQL backend and common config
Actions: Loading new customer_events, Upserting customer events, Upserting attribute tags

A component for loading data into the CDP data model

This component provides a set of primitives for interacting with the CDP database - tables customer_events and attribute_tags. For using the component use this link to repo: https://github.com/meiroio-components/cdp-db-loader.git

Warning: The order of the columns in the input CSV DOES matter(!) and it's checked at runtime.


PostgreSQL backend
"action": "insert_customer_events",
"auth": {
"dbname": "{{CDP_DB_NAME}}",
"host": "{{CDP_DB_HOST}}",
"password": "{{#CDP_DB_PASSWORD}}",
"port": "{{CDP_DB_PORT}}",
"user": "{{CDP_DB_USER}}"
"backend": "postgres",
"debug": true,
"dry_run": false,
"schema": {
"cdp": "public",
"customer_events": "cdp_ce",
"profile_stitching": "cdp_ps"
dry_run(default false)

If true do not commit any changes to the database. If false changes are commited after all input files are processed.


Optional if, for some reason, the data model doesn't reside in the public schema this sets the search_path to the specified value (which means the default $user,public value doesn't apply anymore). Only use this if you know what you are doing.

    Common config

    All CVSs in in/tables/*.csv are processed according to the action parameter.

    debug defaults to true

    Available actions are:

    • insert_customer_events
    • upsert_customer_events
    • upsert_attribute_tags


    Loading new customer_events

    "action": "insert_customer_events"

    Inserts new events. If there is a conflict on id column, skip this row altogether.

    Remember: The event_id must be already defined in the events table before upload.

    The structure of the input file:

    "c",,md5(source_id || type || version),"test_source",2018-06-23T12:34:56Z,"subscribed","1-0-0","{""foo"": 126}","2018-06-23T12:34:56Z"
    "d",,md5(source_id || type || version),"test_source",2018-06-23T12:34:56Z,"purchase","1-0-0","{""foo"": 126}","2018-06-23T12:34:56Z"
    Upserting customer events

    "action": "upsert_customer_events"

    Inserts new events. If there is a conflict on id column, updates payload and leaves everything else untouched. Input file should have the same structure as for loading new customer events (customer_events.csv).

    It can be useful when you have a set of events loaded in the database and later you find out that the payloads are missing some field and you want to backfill the data.

    Warning: This must be done in a backwards-compatible way (adding new field to the json shouldn't break any follow-up customer attributes). Before using this action, make sure you udnerstand what you are doing and the implications of it.

    Upserting attribute tags

    "action": "upsert_attribute_tags"

    Upserts tag_ids for the attributes in the attribute_tags table (PII data, contact information tags, etc). Doesn't delete anything from the table.

    The structure of the input file: