ChatInference
The ChatInference table stores information about inference requests for Chat Functions made to the TensorZero Gateway.
A ChatInference row can be associated with one or more ModelInference rows, depending on the variant’s type.
For chat_completion, there will be a one-to-one relationship between rows in the two tables.
For other variant types, there might be more associated rows.
| Column | Type | Notes |
|---|---|---|
id | UUID | Must be a UUIDv7 |
function_name | String | |
variant_name | String | |
episode_id | UUID | Must be a UUIDv7 |
input | String (JSON) | input field in the /inference request body |
output | String (JSON) | Array of content blocks |
tool_params | String (JSON) | Deprecated. Only present for legacy data. Use dynamic_tools, dynamic_provider_tools, allowed_tools, tool_choice, and parallel_tool_calls instead. |
inference_params | String (JSON) | Object with any inference parameters per variant type (e.g. {"chat_completion": {"temperature": 0.5}}) |
processing_time_ms | UInt32 | |
timestamp | DateTime | Materialized from id (using UUIDv7ToDateTime function) |
tags | Map(String, String) | User-assigned tags (e.g. {"user_id": "123"}) |
extra_body | Nullable(String) | Additional request body parameters passed through to the model provider |
ttft_ms | Nullable(UInt32) | Time to first token (only available in streaming inferences) |
dynamic_tools | Array(String) | Dynamic tool definitions provided at inference time (each element is a JSON string) |
dynamic_provider_tools | Array(String) | Dynamic provider tool definitions provided at inference time (each element is a JSON string) |
allowed_tools | Nullable(String) | Allowed tools specification (JSON string) |
tool_choice | Nullable(String) | Tool choice specification (JSON string) |
parallel_tool_calls | Nullable(Bool) | Whether parallel tool calls are allowed |
snapshot_hash | Nullable(UInt256) | Configuration snapshot hash for data provenance |
JsonInference
The JsonInference table stores information about inference requests for JSON Functions made to the TensorZero Gateway.
A JsonInference row can be associated with one or more ModelInference rows, depending on the variant’s type.
For chat_completion, there will be a one-to-one relationship between rows in the two tables.
For other variant types, there might be more associated rows.
| Column | Type | Notes |
|---|---|---|
id | UUID | Must be a UUIDv7 |
function_name | String | |
variant_name | String | |
episode_id | UUID | Must be a UUIDv7 |
input | String (JSON) | input field in the /inference request body |
output | String (JSON) | Object with parsed and raw fields |
output_schema | String (JSON) | Schema that the output must conform to |
inference_params | String (JSON) | Object with any inference parameters per variant type (e.g. {"chat_completion": {"temperature": 0.5}}) |
processing_time_ms | UInt32 | |
timestamp | DateTime | Materialized from id (using UUIDv7ToDateTime function) |
tags | Map(String, String) | User-assigned tags (e.g. {"user_id": "123"}) |
extra_body | Nullable(String) | Additional request body parameters passed through to the model provider |
auxiliary_content | String | Auxiliary content produced during inference (e.g. Thought blocks in chain-of-thought) |
ttft_ms | Nullable(UInt32) | Time to first token (only available in streaming inferences) |
snapshot_hash | Nullable(UInt256) | Configuration snapshot hash for data provenance |
ModelInference
The ModelInference table stores information about each inference request to a model provider.
This is the inference request you’d make if you had called the model provider directly.
| Column | Type | Notes |
|---|---|---|
id | UUID | Must be a UUIDv7 |
inference_id | UUID | Must be a UUIDv7 |
raw_request | String | Raw request as sent to the model provider (varies) |
raw_response | String | Raw response from the model provider (varies) |
model_name | String | Name of the model used for the inference |
model_provider_name | String | Name of the model provider used for the inference |
input_tokens | Nullable(UInt32) | |
output_tokens | Nullable(UInt32) | |
response_time_ms | Nullable(UInt32) | |
ttft_ms | Nullable(UInt32) | Only available in streaming inferences |
timestamp | DateTime | Materialized from id (using UUIDv7ToDateTime function) |
system | Nullable(String) | The system input to the model |
input_messages | Array(RequestMessage) | The user and assistant messages input to the model |
output | Array(ContentBlock) | The output of the model |
finish_reason | Nullable(Enum(‘stop’, ‘length’, ‘tool_call’, ‘content_filter’, ‘unknown’, ‘stop_sequence’)) | Reason the model stopped generating |
snapshot_hash | Nullable(UInt256) | Configuration snapshot hash for data provenance |
RequestMessage is an object with shape {role: "user" | "assistant", content: List[ContentBlock]} (content blocks are defined here).
DynamicInContextLearningExample
The DynamicInContextLearningExample table stores examples for dynamic in-context learning variants.
| Column | Type | Notes |
|---|---|---|
id | UUID | Must be a UUIDv7 |
function_name | String | |
variant_name | String | |
namespace | String | |
input | String (JSON) | |
output | String | |
embedding | Array(Float32) | |
timestamp | DateTime | Materialized from id (using UUIDv7ToDateTime function) |
BooleanMetricFeedback
The BooleanMetricFeedback table stores feedback for metrics of type = "boolean".
| Column | Type | Notes |
|---|---|---|
id | UUID | Must be a UUIDv7 |
target_id | UUID | Must be a UUIDv7 that is either inference_id or episode_id depending on level in metric config |
metric_name | String | |
value | Bool | |
timestamp | DateTime | Materialized from id (using UUIDv7ToDateTime function) |
tags | Map(String, String) | User-assigned tags (e.g. {"author": "Alice"}) |
snapshot_hash | Nullable(UInt256) | Configuration snapshot hash for data provenance |
FloatMetricFeedback
The FloatMetricFeedback table stores feedback for metrics of type = "float".
| Column | Type | Notes |
|---|---|---|
id | UUID | Must be a UUIDv7 |
target_id | UUID | Must be a UUIDv7 that is either inference_id or episode_id depending on level in metric config |
metric_name | String | |
value | Float32 | |
timestamp | DateTime | Materialized from id (using UUIDv7ToDateTime function) |
tags | Map(String, String) | User-assigned tags (e.g. {"author": "Alice"}) |
snapshot_hash | Nullable(UInt256) | Configuration snapshot hash for data provenance |
CommentFeedback
The CommentFeedback table stores feedback provided with metric_name of "comment".
Comments are free-form text feedbacks.
| Column | Type | Notes |
|---|---|---|
id | UUID | Must be a UUIDv7 |
target_id | UUID | Must be a UUIDv7 that is either inference_id or episode_id depending on level in metric config |
target_type | "inference" or "episode" | |
value | String | |
timestamp | DateTime | Materialized from id (using UUIDv7ToDateTime function) |
tags | Map(String, String) | User-assigned tags (e.g. {"author": "Alice"}) |
snapshot_hash | Nullable(UInt256) | Configuration snapshot hash for data provenance |
DemonstrationFeedback
The DemonstrationFeedback table stores feedback in the form of demonstrations.
Demonstrations are examples of good behaviors.
| Column | Type | Notes |
|---|---|---|
id | UUID | Must be a UUIDv7 |
inference_id | UUID | Must be a UUIDv7 |
value | String | The demonstration or example provided as feedback (must match function output) |
timestamp | DateTime | Materialized from id (using UUIDv7ToDateTime function) |
tags | Map(String, String) | User-assigned tags (e.g. {"author": "Alice"}) |
snapshot_hash | Nullable(UInt256) | Configuration snapshot hash for data provenance |
ModelInferenceCache
The ModelInferenceCache table stores cached model inference results to avoid duplicate requests.
| Column | Type | Notes |
|---|---|---|
short_cache_key | UInt64 | First part of composite key for fast lookups |
long_cache_key | FixedString(64) | Hex-encoded 256-bit key for full cache validation |
timestamp | DateTime | When this cache entry was created, defaults to now() |
output | String | The cached model output |
raw_request | String | Raw request that was sent to the model provider |
raw_response | String | Raw response received from the model provider |
is_deleted | Bool | Soft deletion flag, defaults to false |
input_tokens | Nullable(UInt32) | Cached input token count |
output_tokens | Nullable(UInt32) | Cached output token count |
finish_reason | Nullable(Enum(‘stop’, ‘length’, ‘tool_call’, ‘content_filter’, ‘unknown’, ‘stop_sequence’)) | Cached finish reason |
ReplacingMergeTree engine with timestamp and is_deleted columns for deduplication.
It is partitioned by month and ordered by the composite cache key (short_cache_key, long_cache_key).
The short_cache_key serves as the primary key for performance, while a bloom filter index on long_cache_key
helps optimize point queries.
ChatInferenceDatapoint
The ChatInferenceDatapoint table stores chat inference examples organized into datasets.
| Column | Type | Notes |
|---|---|---|
dataset_name | LowCardinality(String) | Name of the dataset this example belongs to |
function_name | LowCardinality(String) | Name of the function this example is for |
id | UUID | Must be a UUIDv7, often the inference ID if generated from an inference |
episode_id | UUID | Must be a UUIDv7 |
input | String (JSON) | input field in the /inference request body |
output | Nullable(String) (JSON) | Array of content blocks |
tool_params | String (JSON) | Object with any tool parameters (e.g. tool_choice, tools_available) used for the inference |
tags | Map(String, String) | User-assigned tags (e.g. {"user_id": "123"}) |
auxiliary | String | Additional JSON data (unstructured) |
is_deleted | Bool | Soft deletion flag, defaults to false |
updated_at | DateTime64 | When this dataset entry was updated, defaults to now64() |
staled_at | Nullable(DateTime64) | Timestamp when the datapoint was staled (edited or deleted) |
source_inference_id | Nullable(UUID) | Source inference ID if generated from an inference |
is_custom | Bool | Whether the datapoint has been customized beyond the original output, defaults to false |
name | Nullable(String) | Optional human-readable label for the datapoint |
dynamic_tools | Array(String) | Dynamic tool definitions |
dynamic_provider_tools | Array(String) | Dynamic provider tool definitions |
allowed_tools | Nullable(String) | Allowed tools specification |
tool_choice | Nullable(String) | Tool choice specification |
parallel_tool_calls | Nullable(Bool) | Whether parallel tool calls are allowed |
snapshot_hash | Nullable(UInt256) | Configuration snapshot hash for data provenance |
ReplacingMergeTree engine with updated_at and is_deleted columns for deduplication.
It is ordered by dataset_name, function_name, and id to optimize queries filtering by dataset and function.
JsonInferenceDatapoint
The JsonInferenceDatapoint table stores JSON inference examples organized into datasets.
| Column | Type | Notes |
|---|---|---|
dataset_name | LowCardinality(String) | Name of the dataset this example belongs to |
function_name | LowCardinality(String) | Name of the function this example is for |
id | UUID | Must be a UUIDv7, often the inference ID if generated from an inference |
episode_id | UUID | Must be a UUIDv7 |
input | String (JSON) | input field in the /inference request body |
output | String (JSON) | Object with parsed and raw fields |
output_schema | String (JSON) | Schema that the output must conform to |
tags | Map(String, String) | User-assigned tags (e.g. {"user_id": "123"}) |
auxiliary | String | Additional JSON data (unstructured) |
is_deleted | Bool | Soft deletion flag, defaults to false |
updated_at | DateTime64 | When this dataset entry was updated, defaults to now64() |
staled_at | Nullable(DateTime64) | Timestamp when the datapoint was staled (edited or deleted) |
source_inference_id | Nullable(UUID) | Source inference ID if generated from an inference |
is_custom | Bool | Whether the datapoint has been customized beyond the original output, defaults to false |
name | Nullable(String) | Optional human-readable label for the datapoint |
snapshot_hash | Nullable(UInt256) | Configuration snapshot hash for data provenance |
ReplacingMergeTree engine with updated_at and is_deleted columns for deduplication.
It is ordered by dataset_name, function_name, and id to optimize queries filtering by dataset and function.
BatchRequest
The BatchRequest table stores information about batch requests made to model providers. We update it every time a particular batch_id is created or polled.
| Column | Type | Notes |
|---|---|---|
batch_id | UUID | Must be a UUIDv7 |
id | UUID | Must be a UUIDv7 |
batch_params | String | Parameters used for the batch request |
model_name | String | Name of the model used |
model_provider_name | String | Name of the model provider |
status | String | One of: ‘pending’, ‘completed’, ‘failed’ |
errors | Array(String) | Array of error messages if status is ‘failed’ |
timestamp | DateTime | Materialized from id (using UUIDv7ToDateTime function) |
raw_request | String | Raw request sent to the model provider |
raw_response | String | Raw response received from the model provider |
function_name | String | Name of the function being called |
variant_name | String | Name of the function variant |
snapshot_hash | Nullable(UInt256) | Configuration snapshot hash for data provenance |
BatchModelInference
The BatchModelInference table stores information about inferences made as part of a batch request.
Once the request succeeds, we use this information to populate the ChatInference, JsonInference, and ModelInference tables.
| Column | Type | Notes |
|---|---|---|
inference_id | UUID | Must be a UUIDv7 |
batch_id | UUID | Must be a UUIDv7 |
function_name | String | Name of the function being called |
variant_name | String | Name of the function variant |
episode_id | UUID | Must be a UUIDv7 |
input | String (JSON) | input field in the /inference request body |
system | String | The system input to the model |
input_messages | Array(RequestMessage) | The user and assistant messages input to the model |
tool_params | String (JSON) | Object with any tool parameters (e.g. tool_choice, tools_available) used for the inference |
inference_params | String (JSON) | Object with any inference parameters per variant type (e.g. {"chat_completion": {"temperature": 0.5}}) |
raw_request | String | Raw request sent to the model provider |
model_name | String | Name of the model used |
model_provider_name | String | Name of the model provider |
output_schema | String | Optional schema for JSON outputs |
tags | Map(String, String) | User-assigned tags (e.g. {"author": "Alice"}) |
timestamp | DateTime | Materialized from inference_id (using UUIDv7ToDateTime function) |
dynamic_tools | Array(String) | Dynamic tool definitions provided at inference time |
dynamic_provider_tools | Array(String) | Dynamic provider tool definitions provided at inference time |
allowed_tools | Nullable(String) | Allowed tools specification |
tool_choice | Nullable(String) | Tool choice specification |
parallel_tool_calls | Nullable(Bool) | Whether parallel tool calls are allowed |
snapshot_hash | Nullable(UInt256) | Configuration snapshot hash for data provenance |
ConfigSnapshot
The ConfigSnapshot table stores snapshots of the TensorZero configuration for data provenance.
Each time the gateway starts with a new configuration, a snapshot is stored.
| Column | Type | Notes |
|---|---|---|
config | String | Serialized configuration |
extra_templates | Map(String, String) | Additional template files |
hash | UInt256 | Hash of the configuration |
tensorzero_version | String | Version of TensorZero that wrote the snapshot |
created_at | DateTime64 | When this snapshot was first created |
last_used | DateTime64 | When this snapshot was last used |
tags | Map(String, String) | Tags associated with the snapshot |
ReplacingMergeTree engine ordered by hash.
DynamicEvaluationRun
The DynamicEvaluationRun table stores information about workflow evaluation runs.
A workflow evaluation run is a related set of episodes with a common set of variant pins and experiment tags.
Note: This table uses the “DynamicEvaluation” prefix for historical reasons. This feature is externally called “Workflow Evaluations”.
| Column | Type | Notes |
|---|---|---|
run_id_uint | UInt128 | UUID encoded as a UInt128 |
variant_pins | Map(String, String) | Variant pins for this run |
tags | Map(String, String) | User-assigned tags |
project_name | Nullable(String) | Name of the project this run belongs to |
run_display_name | Nullable(String) | Human-readable display name for the run |
is_deleted | Bool | Soft deletion flag, defaults to false |
updated_at | DateTime64 | When this entry was last updated, defaults to now() |
snapshot_hash | Nullable(UInt256) | Configuration snapshot hash for data provenance |
ReplacingMergeTree engine ordered by run_id_uint.
DynamicEvaluationRunEpisode
The DynamicEvaluationRunEpisode table stores information about individual episodes within a workflow evaluation run.
Note: This table uses the “DynamicEvaluation” prefix for historical reasons. This feature is externally called “Workflow Evaluations”.
| Column | Type | Notes |
|---|---|---|
run_id | UUID | References the parent run |
episode_id_uint | UInt128 | UUID encoded as a UInt128 |
variant_pins | Map(String, String) | Variant pins (duplicated from run for efficiency) |
datapoint_name | Nullable(String) | Name of the task/datapoint being evaluated |
tags | Map(String, String) | User-assigned tags |
is_deleted | Bool | Soft deletion flag, defaults to false |
updated_at | DateTime64 | When this entry was last updated, defaults to now() |
snapshot_hash | Nullable(UInt256) | Configuration snapshot hash for data provenance |
ReplacingMergeTree engine ordered by episode_id_uint.
StaticEvaluationHumanFeedback
The StaticEvaluationHumanFeedback table stores human feedback for inference evaluations in an easy-to-reference format.
Note: This table uses the “StaticEvaluation” prefix for historical reasons. This feature is externally called “Inference Evaluations”.
| Column | Type | Notes |
|---|---|---|
metric_name | String | Name of the metric |
datapoint_id | UUID | ID of the datapoint being evaluated |
output | String | The inference output associated with the feedback |
value | String | JSON-encoded value of the feedback |
feedback_id | UUID | ID of the feedback entry |
evaluator_inference_id | UUID | ID of the evaluator inference that generated the feedback |
timestamp | DateTime | Materialized from feedback_id (using UUIDv7ToDateTime function) |
MergeTree engine ordered by (metric_name, datapoint_id, output).
Materialized View Tables
Materialized View Tables
Materialized views in columnar databases like ClickHouse pre-compute alternative indexings of data, dramatically improving query performance compared to computing results on-the-fly.
In TensorZero’s case, we store denormalized data about inferences and feedback in the materialized views below to support efficient queries for common downstream use cases.
The
The
The
The
The
The
The table uses the
The
The
The
The
The
The table uses the
The
The table uses the
The
The table uses the
The
The table uses the
The
The table uses the
The
The table uses the
The
The table uses the
The
The table uses the
FeedbackTag
The FeedbackTag table stores tags associated with various feedback types. Tags are used to categorize and add metadata to feedback entries, allowing for user-defined filtering later on. Data is inserted into this table by materialized views reading from the BooleanMetricFeedback, CommentFeedback, DemonstrationFeedback, and FloatMetricFeedback tables.| Column | Type | Notes |
|---|---|---|
metric_name | String | Name of the metric the tag is associated with. |
key | String | Key of the tag. |
value | String | Value of the tag. |
feedback_id | UUID | UUID referencing the related feedback entry (e.g., BooleanMetricFeedback.id). |
snapshot_hash | Nullable(UInt256) | Configuration snapshot hash for data provenance |
InferenceById
The InferenceById table is a materialized view that combines data from ChatInference and JSONInference.
Notably, it indexes the table by id_uint for fast lookup by the gateway to validate feedback requests.
We store id_uint as a UInt128 so that they are sorted in the natural order by time as ClickHouse sorts UUIDs in little-endian order.| Column | Type | Notes |
|---|---|---|
id_uint | UInt128 | Integer representation of UUIDv7 for sorting order |
function_name | String | |
variant_name | String | |
episode_id | UUID | Must be a UUIDv7 |
function_type | String | Either 'chat' or 'json' |
snapshot_hash | Nullable(UInt256) | Configuration snapshot hash for data provenance |
InferenceByEpisodeId
The InferenceByEpisodeId table is a materialized view that indexes inferences by their episode ID, enabling efficient lookup of all inferences within an episode.
We store episode_id_uint as a UInt128 so that they are sorted in the natural order by time as ClickHouse sorts UUIDs in little-endian order.| Column | Type | Notes |
|---|---|---|
episode_id_uint | UInt128 | Integer representation of UUIDv7 for sorting order |
id_uint | UInt128 | Integer representation of UUIDv7 for sorting order |
function_name | String | Name of the function being called |
variant_name | String | Name of the function variant |
function_type | Enum(‘chat’, ‘json’) | Type of function (chat or json) |
snapshot_hash | Nullable(UInt256) | Configuration snapshot hash for data provenance |
InferenceTag
The InferenceTag table stores tags associated with inferences. Tags are used to categorize and add metadata to inferences, allowing for user-defined filtering later on. Data is inserted into this table by materialized views reading from the ChatInference and JsonInference tables.| Column | Type | Notes |
|---|---|---|
function_name | String | Name of the function the tag is associated with. |
key | String | Key of the tag. |
value | String | Value of the tag. |
inference_id | UUID | UUID referencing the related inference (e.g., ChatInference.id). |
snapshot_hash | Nullable(UInt256) | Configuration snapshot hash for data provenance |
BatchIdByInferenceId
The BatchIdByInferenceId table maps inference IDs to batch IDs, allowing for efficient lookup of which batch an inference belongs to.| Column | Type | Notes |
|---|---|---|
inference_id | UUID | Must be a UUIDv7 |
batch_id | UUID | Must be a UUIDv7 |
TagInference
The TagInference table stores denormalized tag data for efficient queries by tag key and value.
Data is inserted by materialized views reading from the ChatInference and JsonInference tables.| Column | Type | Notes |
|---|---|---|
key | String | Tag key |
value | String | Tag value |
function_name | String | Name of the function |
variant_name | String | Name of the variant |
episode_id | UUID | Episode ID |
inference_id | UUID | Inference ID |
function_type | Enum(‘chat’, ‘json’) | Type of function (chat or json) |
is_deleted | Bool | Soft deletion flag, defaults to false |
updated_at | DateTime64 | When this entry was last updated, defaults to now64() |
snapshot_hash | Nullable(UInt256) | Configuration snapshot hash for data provenance |
ReplacingMergeTree engine ordered by (key, value, inference_id).BooleanMetricFeedbackByTargetId
The BooleanMetricFeedbackByTargetId table indexes boolean metric feedback by target ID, enabling efficient lookup of feedback for a specific target.| Column | Type | Notes |
|---|---|---|
id | UUID | Must be a UUIDv7 |
target_id | UUID | Must be a UUIDv7 |
metric_name | String | Name of the metric (stored as LowCardinality) |
value | Bool | The boolean feedback value |
tags | Map(String, String) | Key-value pairs of tags associated with the feedback |
snapshot_hash | Nullable(UInt256) | Configuration snapshot hash for data provenance |
CommentFeedbackByTargetId
The CommentFeedbackByTargetId table stores text feedback associated with inferences or episodes, enabling efficient lookup of comments by their target ID.| Column | Type | Notes |
|---|---|---|
id | UUID | Must be a UUIDv7 |
target_id | UUID | Must be a UUIDv7 |
target_type | Enum(‘inference’, ‘episode’) | Type of entity this feedback is for |
value | String | The text feedback content |
tags | Map(String, String) | Key-value pairs of tags associated with the feedback |
snapshot_hash | Nullable(UInt256) | Configuration snapshot hash for data provenance |
DemonstrationFeedbackByInferenceId
The DemonstrationFeedbackByInferenceId table stores demonstration feedback associated with inferences, enabling efficient lookup of demonstrations by inference ID.| Column | Type | Notes |
|---|---|---|
id | UUID | Must be a UUIDv7 |
inference_id | UUID | Must be a UUIDv7 |
value | String | The demonstration feedback content |
tags | Map(String, String) | Key-value pairs of tags associated with the feedback |
snapshot_hash | Nullable(UInt256) | Configuration snapshot hash for data provenance |
FloatMetricFeedbackByTargetId
The FloatMetricFeedbackByTargetId table indexes float metric feedback by target ID, enabling efficient lookup of feedback for a specific target.| Column | Type | Notes |
|---|---|---|
id | UUID | Must be a UUIDv7 |
target_id | UUID | Must be a UUIDv7 |
metric_name | String | Name of the metric (stored as LowCardinality) |
value | Float32 | The float feedback value |
tags | Map(String, String) | Key-value pairs of tags associated with the feedback |
snapshot_hash | Nullable(UInt256) | Configuration snapshot hash for data provenance |
EpisodeById
The EpisodeById table aggregates episode data for efficient lookup by episode ID.
Data is inserted by materialized views reading from the ChatInference and JsonInference tables.| Column | Type | Notes |
|---|---|---|
episode_id_uint | UInt128 | Integer representation of UUIDv7 for sorting order |
count | SimpleAggregateFunction(sum, UInt64) | Number of inferences in the episode |
inference_ids | AggregateFunction(groupArray, UUID) | Array of inference IDs in the episode |
min_inference_id_uint | SimpleAggregateFunction(min, UInt128) | Earliest inference ID in the episode |
max_inference_id_uint | SimpleAggregateFunction(max, UInt128) | Latest inference ID in the episode |
AggregatingMergeTree engine ordered by episode_id_uint.DynamicEvaluationRunEpisodeByRunId
The DynamicEvaluationRunEpisodeByRunId table indexes workflow evaluation run episodes by run ID for efficient lookup.Note: This table uses the “DynamicEvaluation” prefix for historical reasons. This feature is externally called “Workflow Evaluations”.| Column | Type | Notes |
|---|---|---|
run_id_uint | UInt128 | UUID encoded as a UInt128 |
episode_id_uint | UInt128 | UUID encoded as a UInt128 |
variant_pins | Map(String, String) | Variant pins for this episode |
tags | Map(String, String) | User-assigned tags |
datapoint_name | Nullable(String) | Name of the task/datapoint being evaluated |
is_deleted | Bool | Soft deletion flag, defaults to false |
updated_at | DateTime64 | When this entry was last updated, defaults to now() |
snapshot_hash | Nullable(UInt256) | Configuration snapshot hash for data provenance |
ReplacingMergeTree engine ordered by (run_id_uint, episode_id_uint).DynamicEvaluationRunByProjectName
The DynamicEvaluationRunByProjectName table indexes workflow evaluation runs by project name for efficient lookup.Note: This table uses the “DynamicEvaluation” prefix for historical reasons. This feature is externally called “Workflow Evaluations”.| Column | Type | Notes |
|---|---|---|
run_id_uint | UInt128 | UUID encoded as a UInt128 |
variant_pins | Map(String, String) | Variant pins for this run |
tags | Map(String, String) | User-assigned tags |
project_name | String | Name of the project |
run_display_name | Nullable(String) | Human-readable display name for the run |
is_deleted | Bool | Soft deletion flag, defaults to false |
updated_at | DateTime64 | When this entry was last updated, defaults to now() |
snapshot_hash | Nullable(UInt256) | Configuration snapshot hash for data provenance |
ReplacingMergeTree engine ordered by (project_name, run_id_uint).FloatMetricFeedbackByVariant
The FloatMetricFeedbackByVariant table denormalizes float metric feedback by joining with inference data to associate feedback with specific function variants.| Column | Type | Notes |
|---|---|---|
function_name | String | Name of the function |
variant_name | String | Name of the variant |
metric_name | String | Name of the metric |
id_uint | UInt128 | Feedback ID encoded as a UInt128 |
target_id_uint | UInt128 | Target ID encoded as a UInt128 |
value | Float32 | The float feedback value |
feedback_tags | Map(String, String) | Tags from the feedback entry |
snapshot_hash | Nullable(UInt256) | Configuration snapshot hash for data provenance |
MergeTree engine ordered by (function_name, metric_name, variant_name, id_uint).BooleanMetricFeedbackByVariant
The BooleanMetricFeedbackByVariant table denormalizes boolean metric feedback by joining with inference data to associate feedback with specific function variants.| Column | Type | Notes |
|---|---|---|
function_name | String | Name of the function |
variant_name | String | Name of the variant |
metric_name | String | Name of the metric |
id_uint | UInt128 | Feedback ID encoded as a UInt128 |
target_id_uint | UInt128 | Target ID encoded as a UInt128 |
value | Bool | The boolean feedback value |
feedback_tags | Map(String, String) | Tags from the feedback entry |
snapshot_hash | Nullable(UInt256) | Configuration snapshot hash for data provenance |
MergeTree engine ordered by (function_name, metric_name, variant_name, id_uint).FeedbackByVariantStatistics
The FeedbackByVariantStatistics table stores pre-aggregated summary statistics about feedback by function, variant, metric, and time.| Column | Type | Notes |
|---|---|---|
function_name | String | Name of the function |
variant_name | String | Name of the variant |
metric_name | String | Name of the metric |
minute | DateTime | Start of the minute bucket |
feedback_mean | AggregateFunction(avg, Float32) | Running mean of feedback values |
feedback_variance | AggregateFunction(varSampStable, Float32) | Running variance of feedback values |
count | SimpleAggregateFunction(sum, UInt64) | Number of feedback entries |
AggregatingMergeTree engine ordered by (function_name, metric_name, variant_name, minute).CumulativeUsage
The CumulativeUsage table stores cumulative token usage counts.| Column | Type | Notes |
|---|---|---|
type | String | Type of count: 'input_tokens', 'output_tokens', or 'model_inferences' |
count | UInt64 | Cumulative count |
SummingMergeTree engine ordered by type.ModelProviderStatistics
The ModelProviderStatistics table stores pre-aggregated performance statistics for model providers by minute.| Column | Type | Notes |
|---|---|---|
model_name | String | Name of the model |
model_provider_name | String | Name of the model provider |
minute | DateTime | Start of the minute bucket |
response_time_ms_quantiles | AggregateFunction(quantilesTDigest, Nullable(UInt32)) | Response time distribution |
ttft_ms_quantiles | AggregateFunction(quantilesTDigest, Nullable(UInt32)) | Time-to-first-token distribution |
total_input_tokens | AggregateFunction(sum, Nullable(UInt32)) | Total input tokens |
total_output_tokens | AggregateFunction(sum, Nullable(UInt32)) | Total output tokens |
count | AggregateFunction(count, UInt32) | Number of inferences |
AggregatingMergeTree engine ordered by (model_name, model_provider_name, minute).