PROFILE PIPELINE

Runs a single batch in the foreground to provide detailed resource metrics for each operation in your pipeline. The batch itself is not persisted.

Syntax

PROFILE PIPELINE pipeline_name

Remarks

• pipe_line is the configured pipeline.

To use PROFILE PIPELINE, you must gather data source offsets through TEST PIPELINE. Verify TEST PIPELINE does not return an empty set before calling PROFILE PIPELINE.

See TEST PIPELINE for more details.

Output

The output of PROFILE PIPELINE is similar to the SHOW PROFILE output described in PROFILE.

Example

TEST PIPELINE kafka LIMIT 10;
+---------+
| id      |
+---------+
| rlkj    |
| slkjrl  |
| sdlkj   |
| sk      |
| slkjrls |
| lkjrljs |
| slkjr   |
+---------+
7 rows in set (0.82 sec)

PROFILE PIPELINE kafka;
+------------------------------------------------------------------------------------------------------------------------------------------------------+
| PROFILE                                                                                                                                              |
+------------------------------------------------------------------------------------------------------------------------------------------------------+
| Gather partitions:all est_rows:264 alias:remote_0 actual_rows: 7 exec_time: 0ms                                                                      |
| InsertInto quickstart_kafka.messages [messages_src.id] local:yes est_rows:264 actual_rows: 7 exec_time: 0ms start_time: [00:00:00.259, 00:00:00.280] |
| ExternalIngest pipeline:kafka                                                                                                                        |
| |---CSVParse actual_rows: 7 exec_time: 0ms start_time: [00:00:00.005, 00:00:00.030] end_time: [00:00:00.259, 00:00:00.280]                           |
| |---Transform streamed_bytes: 0.043000 KB exec_time: 4ms start_time: [00:00:00.005, 00:00:00.030] end_time: [00:00:00.259, 00:00:00.280]             |
| +---Extract streamed_bytes: 0.092000 KB exec_time: 269ms start_time: [00:00:00.005, 00:00:00.030] end_time: [00:00:00.256, 00:00:00.278]             |
+------------------------------------------------------------------------------------------------------------------------------------------------------+
6 rows in set (0.98 sec)

Was this Article Helpful?

Thanks For Your Feedback!