Processing Diagnostics

The Diagnostics page provides an overview of how customer data is currently being processed. It allows administrators to monitor the health of the data processing pipeline, understand whether data is being processed normally, and identify potential processing issues or errors.

Every event that enters the system, such as a website visit, purchase, or form submission, goes through several processing stages before it becomes available for segmentation, analytics, and activations. The Diagnostics page visualizes these stages and shows how many items are waiting to be processed, how quickly they are being processed, and whether the system is operating normally.

You can open the Diagnostics page in Administration → Diagnostics.

Data Processing Pipeline

At the top of the page, you will see the Data Processing Pipeline section. This section displays the three main stages that your customer data passes through as it is processed by the system.

These stages are displayed as cards arranged from left to right, showing the sequential flow of data through the platform. Data first passes through Identity Stitching, then Profile Recalculation, and finally Profile Synchronization.

Each card contains several metrics that help you understand the current state of that processing stage.

The values shown in these cards are updated regularly to reflect the current state of the system.

Identity Stitching

Identity stitching is the first stage of the data processing pipeline. In this step, incoming events are matched with existing customer profiles. The system uses identifiers such as email address, cookie ID, or phone number to determine which customer profile an event belongs to.

For example, if a user visits your website and later makes a purchase using the same email address, the system will connect those events to the same customer profile.

The Identity Stitching card displays two main metrics: the number of events waiting to be processed and the current processing speed.

1. The Events in queue value shows how many incoming events are currently waiting to be stitched to customer profiles. This number increases when new events arrive faster than they can be processed. Short spikes are normal, especially during periods of high activity.
2. The Processing speed value shows how many events are being processed per minute. This number indicates the current processing throughput of the identity stitching engine.

If the queue grows significantly while the processing speed remains low, this may indicate that the system is temporarily processing a backlog of events.

Profile Recalculation

After an event has been successfully linked to a customer profile, the system may need to update that profile’s attributes. This happens in the Profile Recalculation stage.

Attributes are calculated values stored on the customer profile. Examples include the total number of purchases, the date of the last activity, or the number of times a user visited your website.

Whenever an event changes data that affects these attributes, the system schedules a recalculation for the affected profile.

1. The Profiles in queue value shows how many customer profiles are currently waiting for attribute recalculation. If many profiles are affected by incoming events, this number may temporarily increase.
2. The Processing speed value shows how many profiles are recalculated per minute.

If this queue grows continuously while the processing speed remains low, it may indicate that attribute recalculations are taking longer than expected.

Profile Synchronization

The final stage of the pipeline is Profile Synchronization. Once profile attributes have been recalculated, the updated profiles must be synchronized with the search index used by the platform.

This synchronization ensures that the most recent profile data is available for segmentation, targeting, and activation workflows.

1. The Profiles in queue value shows how many profiles are waiting to be synchronized.
2. The Processing speed value shows how many profiles are being synchronized per minute.

When synchronization is complete, updated profile data becomes available for segmentation and campaign execution.

Status Indicators

Each pipeline stage displays a status message that helps you understand whether data is being processed normally or if delays are occurring.

These statuses reflect the relationship between the number of items waiting in the queue and the current processing speed.

1. If the status shows “System is overloaded”, the queue has reached a high level, and processing is significantly delayed. This indicates that the system is currently not able to keep up with incoming data, and it may take time to process the backlog. During this period, newly collected data may not be immediately available in segmentation or activations.
2. When the status shows “Processing is running well”, the system is operating normally. The processing speed is sufficient to handle incoming data, and queues are either small or being cleared at a steady rate.
3. If the status shows “Profiles are piling up”, it means that items are entering the queue faster than they are being processed. In this situation, the queue is growing, and data updates may start to be delayed. This can happen temporarily during periods of increased activity or large data volumes.

These status indicators provide a quick overview of system health and help you identify whether further investigation may be needed.

Queue Update Information

Above the pipeline, you will see a message indicating when the queue size values will be updated.

For example, the system may display a message such as “Next queue size update in 4 minutes.” Queue size values are refreshed every 10 minutes, and this message shows how much time is left until the next refresh starts.

This update applies only to the queue size values, such as the number of events or profiles currently waiting in each processing stage. It does not mean that all values on the page are refreshed at the same time.

Other metrics, such as processing speed, are updated separately based on recent processing activity. Because of this, queue size and processing speed may change at different times and should not always be expected to refresh together.

During the queue update process, the message may temporarily change to “Queue size update is running.”

This information helps explain when the next queue size refresh is expected and clarifies that the timer applies specifically to queue statistics, not to every metric shown on the Diagnostics page.

Error Log

The lower section of the Diagnostics page contains the Error Log. This section lists processing errors that occurred while the system was handling events or recalculating profile attributes.

Errors can occur for various reasons, such as incorrect attribute definitions, unexpected data formats, or conflicts in identity resolution.

Each error entry contains a short description of the issue, the type of processing step where the error occurred, and the time when the error was recorded.

Filtering Errors

You can use the filtering controls above the error list to locate specific errors.

The search field allows you to search for errors by keyword or message content. To perform a search, you must enter at least three characters.

You can also filter errors by type, which allows you to view only errors related to identity stitching or attribute recalculation.

The time range filter allows you to limit results to specific time periods, such as the last hour, last 24 hours, or the last 7 days.

Viewing Error Details

Each error row can be expanded to view additional information about the issue.

When expanded, the system displays the detailed context of the error, including the related customer entity ID, attribute information when applicable, and the technical data used by the system during processing.

This information helps administrators understand the root cause of the issue and identify potential configuration problems.

When to Use the Diagnostics Page

The Diagnostics page is particularly useful when investigating delays in data updates or troubleshooting processing issues.

If segments are not updating as expected, campaign targeting appears outdated, or profile attributes seem incorrect, this page can help you determine whether the system is processing data normally or experiencing errors.

By monitoring queue sizes, processing speeds, and error logs, administrators can quickly identify potential issues and ensure that customer data is processed reliably.