Skip to content

chore: user can request attestations#1207

Merged
MicBun merged 13 commits intomainfrom
chore/get-attestation
Oct 10, 2025
Merged

chore: user can request attestations#1207
MicBun merged 13 commits intomainfrom
chore/get-attestation

Conversation

@outerlook
Copy link
Contributor

@outerlook outerlook commented Oct 9, 2025

Description

Introduces the tn_utils extension, which provides a set of precompiles for dynamic dispatch and canonical serialization. Key features include:

  • call_dispatch(action_name TEXT, args_bytes BYTEA) -> BYTEA: Dispatches to another action while preserving the current engine context.
  • bytea_join(chunks BYTEA[], delimiter BYTEA) -> BYTEA: Concatenates an array of BYTEA chunks with a specified delimiter.
  • bytea_length_prefix(chunk BYTEA) -> BYTEA: Returns a length-prefixed version of the input chunk.
  • encode_uintX(value INT) -> BYTEA: Encodes unsigned integers to big-endian byte arrays.

Related Problem

How Has This Been Tested?

Summary by CodeRabbit

  • New Features

    • Added tn_utils extension providing byte utilities and deterministic encoding helpers.
    • Added attestation schema and a public request_attestation action for canonical, verifiable attestations.
    • Registered tn_utils and attestation extensions into startup.
  • Improvements

    • HTTP sync logging now respects configured log levels for clearer output.
  • Documentation

    • Added tn_utils README with usage examples.
  • Tests

    • Added end-to-end tests validating request_attestation flow and canonical payload storage.

Introduces the tn_utils extension, which provides a set of precompiles for dynamic dispatch and canonical serialization. Key features include:

- `call_dispatch(action_name TEXT, args_bytes BYTEA) -> BYTEA`: Dispatches to another action while preserving the current engine context.
- `bytea_join(chunks BYTEA[], delimiter BYTEA) -> BYTEA`: Concatenates an array of BYTEA chunks with a specified delimiter.
- `bytea_length_prefix(chunk BYTEA) -> BYTEA`: Returns a length-prefixed version of the input chunk.
- `encode_uintX(value INT) -> BYTEA`: Encodes unsigned integers to big-endian byte arrays.

Additionally, comprehensive tests have been added to ensure functionality and reliability.

This extension is registered during node startup to enhance the capabilities of the Kwil framework.
- Modified the attestation table to make signature, validator_pubkey, and signed_height non-nullable.
- Renamed constraints for better readability.
- Updated index names for consistency and clarity in querying.

These changes enhance the schema's structure and maintainability.
… caller normalization

- Updated the return type of the request_attestation action to include attestation_hash.
- Changed variable names for clarity, including replacing @block_height with @height.
- Normalized caller address to bytes for storage.
- Improved the hashing process by building hash material in a canonical order to align with engine-side utilities.

These changes improve the action's clarity and ensure consistent behavior with the signing service.
- Updated the bytea_join method to accept a nullable delimiter, ensuring compatibility with SQL migrations and preserving deterministic ordering.
- Introduced comprehensive tests for the request_attestation action, validating the insertion of canonical payloads and ensuring correct behavior of the attestation process.
- Initialized the tn_utils extension during test setup to support the new functionality.

These changes improve the reliability and clarity of the attestation workflow and precompile methods.
@coderabbitai
Copy link

coderabbitai bot commented Oct 9, 2025

Walkthrough

Adds a new tn_utils extension (precompiles and deterministic serialization), registers and initializes tn_utils and tn_attestation, introduces attestation schema and request_attestation action with deterministic dispatch and canonical payload/hash storage, adds tests for attestation request flow, and introduces a retryable HTTP logger adapter in tn_cache.

Changes

Cohort / File(s) Summary
Extension registration & init
extensions/register.go, internal/migrations/000-extensions.sql, tests/streams/utils/utils.go
Adds tn_utils import and calls tn_utils.InitializeExtension() during init; aliases database-size import as database_size; updates migration USE ordering to include tn_utils and tn_attestation; initializes tn_utils and tn_attestation in test utils.
tn_utils extension (code + docs + tests)
extensions/tn_utils/extension.go, extensions/tn_utils/precompiles.go, extensions/tn_utils/serialization.go, extensions/tn_utils/README.md, extensions/tn_utils/serialization_test.go
New extension bundle tn_utils with ExtensionName and InitializeExtension; precompiles: call_dispatch, bytea_join, bytea_length_prefix, bytea_length_prefix_many, encode_uint{8,16,32,64}; deterministic (de)serialization helpers for action args and canonical query results; README and comprehensive serialization tests.
Attestation schema & action migrations
internal/migrations/023-attestation-schema.sql, internal/migrations/024-attestation-actions.sql
Adds attestations and attestation_actions tables, indexes, bootstrap action IDs (1–5), and a public request_attestation action that validates allowlist/encryption flag, performs deterministic dispatch via tn_utils.call_dispatch, constructs canonical payload, stores unsigned attestation, and queues for signing.
Attestation integration test
tests/streams/attestation/attestation_request_test.go
New integration-style test verifying request_attestation returns expected attestation_hash, inserts canonical payload, and stores expected attestation row state; includes helpers for test scaffolding and payload construction.
Retryable HTTP logging adapter
extensions/tn_cache/syncschecker/sync_checker.go
Adds retryableHTTPLogger adapter implementing Printf to route go-retryablehttp logs through the project's logger at Debug level and configures retryablehttp client to use it.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  participant C as Client
  participant A as Action: request_attestation
  participant TU as Precompile: tn_utils.call_dispatch
  participant E as Engine
  participant DB as Database
  participant QS as Precompile: queue_for_signing

  C->>A: request_attestation(data_provider, stream_id, action_name, args_bytes, encrypt_sig, max_fee)
  A->>A: Validate encrypt_sig=false and action_name allowlist
  A->>A: Get current block height
  A->>TU: call_dispatch(action_name, args_bytes)
  TU->>E: Engine.Call(action_name, decoded args)
  E-->>TU: Query result (rows)
  TU->>TU: EncodeQueryResultCanonical(rows)
  TU-->>A: result_bytes
  A->>A: Build canonical payload (version|algo|height|provider|stream|action_id|args|result)
  A->>A: Compute attestation_hash
  A->>DB: INSERT attestation (unsigned, canonical payload)
  A->>QS: queue_for_signing(attestation_hash)
  A-->>C: attestation_hash
Loading
sequenceDiagram
  autonumber
  participant Caller as Caller
  participant CD as tn_utils.call_dispatch
  participant Ser as tn_utils.serialization
  participant Eng as Engine

  Caller->>CD: call_dispatch(action_name, args_bytes)
  CD->>Ser: DecodeActionArgs(args_bytes)
  Ser-->>CD: []any args
  CD->>Eng: Engine.Call(action_name, args)
  Eng-->>CD: []*Row rows
  CD->>Ser: EncodeQueryResultCanonical(rows)
  Ser-->>CD: result_bytes
  CD-->>Caller: result_bytes
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

Suggested labels

type: chore

Suggested reviewers

  • MicBun
  • williamrusdyputra

Poem

Thump-thump, I hopped through bytes and beams,
I stitched length-prefixes and encoded dreams.
I called a precompile, fetched rows so neat,
Packed canonical payloads, all tidy and sweet.
A rabbit's cheer for utils—attestations complete! 🥕✨

Pre-merge checks and finishing touches

❌ Failed checks (2 warnings)
Check name Status Explanation Resolution
Linked Issues Check ⚠️ Warning The pull request fully implements the schema, allowlist, and request_attestation action with deterministic dispatch and canonical serialization required by issue #1196, but it does not update any project documentation to define or reference the numeric action ID registry mandated by issue #1197. Add or update documentation (for example in a README or dedicated docs file) to include the action ID registry mappings (1 = get_record, 2 = get_index, etc.) and ensure these numeric IDs are explicitly referenced in the request_attestation implementation.
Out of Scope Changes Check ⚠️ Warning The addition of the retryableHTTPLogger in extensions/tn_cache/syncschecker.go and the import alias change for database_size in register.go are unrelated to the attestation and tn_utils objectives defined by the linked issues. Extract the HTTP logging adapter and database_size import alias changes into a separate PR focused on caching and logging improvements to keep this pull request scoped to attestation functionality and tn_utils extensions.
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check ✅ Passed The title clearly and concisely summarizes the primary change by stating that users can request attestations, directly reflecting the core feature implemented in this pull request.
Docstring Coverage ✅ Passed No functions found in the changes. Docstring coverage check skipped.
✨ Finishing touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch chore/get-attestation

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between f1bd2f6 and d73c8ed.

📒 Files selected for processing (1)
  • internal/migrations/023-attestation-schema.sql (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • internal/migrations/023-attestation-schema.sql
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: acceptance-test

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

…tionality

- Renamed the package from utils to tn_utils for clarity.
- Added tn_attestation extension usage in SQL migrations to support attestation signing.
- Updated the initialization process in tests to include tn_attestation.

These changes enhance the organization and functionality of the extensions, supporting the attestation workflow.
- Added a retryableHTTPLogger type to adapt Kwil's logger for use with go-retryablehttp.
- Implemented logging of HTTP requests at Debug level to ensure visibility during health checks.

These changes enhance the logging capabilities of the SyncChecker, improving monitoring and debugging of health check operations.
@outerlook outerlook self-assigned this Oct 9, 2025
- Introduced a TODO comment in the SQL migration for attestation actions, highlighting the need to filter out non-deterministic arguments, such as `use_cache`, before releasing attestations.

This change aims to improve the reliability of the attestation process by addressing potential issues with argument determinism.
@outerlook outerlook changed the title feat: add tn_utils extension for reusable precompiles chore: user can request attestations Oct 9, 2025
- Enhanced the existing TODO comment in the SQL migration for attestation actions to include the idea of storing force_args in whitelisted actions. This aims to improve the handling of non-deterministic arguments before releasing attestations.

These changes contribute to refining the attestation process by addressing potential argument determinism issues.
@outerlook outerlook marked this pull request as ready for review October 9, 2025 18:45
- Added comprehensive comments to the tn_utils precompiles, explaining the purpose and functionality of each method. This includes descriptions for methods like buildPrecompile, callDispatchMethod, byteaJoinMethod, and others, improving code clarity and maintainability.

These changes aim to provide better documentation within the codebase, facilitating easier understanding and usage of the precompiles.
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 9c8e016 and 2013cf0.

📒 Files selected for processing (12)
  • extensions/register.go (1 hunks)
  • extensions/tn_cache/syncschecker/sync_checker.go (3 hunks)
  • extensions/tn_utils/README.md (1 hunks)
  • extensions/tn_utils/extension.go (1 hunks)
  • extensions/tn_utils/precompiles.go (1 hunks)
  • extensions/tn_utils/serialization.go (1 hunks)
  • extensions/tn_utils/serialization_test.go (1 hunks)
  • internal/migrations/000-extensions.sql (1 hunks)
  • internal/migrations/023-attestation-schema.sql (1 hunks)
  • internal/migrations/024-attestation-actions.sql (1 hunks)
  • tests/streams/attestation/attestation_request_test.go (1 hunks)
  • tests/streams/utils/utils.go (2 hunks)
🧰 Additional context used
🧬 Code graph analysis (5)
tests/streams/utils/utils.go (5)
extensions/tn_vacuum/tn_vacuum.go (1)
  • InitializeExtension (13-23)
extensions/tn_digest/tn_digest.go (1)
  • InitializeExtension (18-40)
extensions/leaderwatch/leaderwatch.go (1)
  • InitializeExtension (47-54)
extensions/database-size/init.go (1)
  • InitializeExtension (10-15)
extensions/tn_cache/tn_cache.go (1)
  • InitializeExtension (87-99)
tests/streams/attestation/attestation_request_test.go (4)
tests/streams/utils/runner.go (1)
  • RunSchemaTest (37-116)
internal/migrations/migration.go (1)
  • GetSeedScriptPaths (15-48)
tests/streams/utils/utils.go (1)
  • GetTestOptionsWithCache (75-88)
extensions/tn_utils/serialization.go (2)
  • EncodeActionArgs (44-75)
  • EncodeQueryResultCanonical (144-183)
extensions/register.go (1)
extensions/tn_utils/extension.go (1)
  • InitializeExtension (16-20)
extensions/tn_utils/serialization_test.go (1)
extensions/tn_utils/serialization.go (2)
  • EncodeActionArgs (44-75)
  • DecodeActionArgs (84-129)
extensions/tn_utils/precompiles.go (1)
extensions/tn_utils/serialization.go (2)
  • DecodeActionArgs (84-129)
  • EncodeQueryResultCanonical (144-183)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
  • GitHub Check: lint
  • GitHub Check: acceptance-test

@holdex
Copy link

holdex bot commented Oct 9, 2025

Time Submission Status

Member Status Time Action Last Update
outerlook ✅ Submitted 8h Update time Oct 10, 2025, 1:49 PM
MicBun ✅ Submitted 15min Update time Oct 11, 2025, 8:10 AM

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 9c8e016 and 2013cf0.

📒 Files selected for processing (12)
  • extensions/register.go (1 hunks)
  • extensions/tn_cache/syncschecker/sync_checker.go (3 hunks)
  • extensions/tn_utils/README.md (1 hunks)
  • extensions/tn_utils/extension.go (1 hunks)
  • extensions/tn_utils/precompiles.go (1 hunks)
  • extensions/tn_utils/serialization.go (1 hunks)
  • extensions/tn_utils/serialization_test.go (1 hunks)
  • internal/migrations/000-extensions.sql (1 hunks)
  • internal/migrations/023-attestation-schema.sql (1 hunks)
  • internal/migrations/024-attestation-actions.sql (1 hunks)
  • tests/streams/attestation/attestation_request_test.go (1 hunks)
  • tests/streams/utils/utils.go (2 hunks)
🧰 Additional context used
🧬 Code graph analysis (5)
extensions/tn_utils/serialization_test.go (1)
extensions/tn_utils/serialization.go (2)
  • EncodeActionArgs (44-75)
  • DecodeActionArgs (84-129)
extensions/tn_utils/precompiles.go (1)
extensions/tn_utils/serialization.go (2)
  • DecodeActionArgs (84-129)
  • EncodeQueryResultCanonical (144-183)
tests/streams/utils/utils.go (5)
extensions/tn_vacuum/tn_vacuum.go (1)
  • InitializeExtension (13-23)
extensions/tn_digest/tn_digest.go (1)
  • InitializeExtension (18-40)
extensions/leaderwatch/leaderwatch.go (1)
  • InitializeExtension (47-54)
extensions/database-size/init.go (1)
  • InitializeExtension (10-15)
extensions/tn_cache/tn_cache.go (1)
  • InitializeExtension (87-99)
tests/streams/attestation/attestation_request_test.go (3)
internal/migrations/migration.go (1)
  • GetSeedScriptPaths (15-48)
tests/streams/utils/utils.go (1)
  • GetTestOptionsWithCache (75-88)
extensions/tn_utils/serialization.go (2)
  • EncodeActionArgs (44-75)
  • EncodeQueryResultCanonical (144-183)
extensions/register.go (1)
extensions/tn_utils/extension.go (1)
  • InitializeExtension (16-20)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
  • GitHub Check: lint
  • GitHub Check: acceptance-test
🔇 Additional comments (7)
extensions/tn_cache/syncschecker/sync_checker.go (2)

24-36: LGTM! Clean adapter implementation.

The retryableHTTPLogger correctly implements the go-retryablehttp Logger interface. Logging at Debug level is appropriate for routine health checks, and trimming the trailing newline prevents double newlines in the output.


152-153: LGTM! Proper integration of the logger adapter.

The adapter is correctly assigned to the retryablehttp client, ensuring HTTP logs respect the configured log level. The explanatory comment adds clarity.

tests/streams/utils/utils.go (1)

37-38: Init order covers attestation dependencies

Registering tn_utils ahead of tn_attestation keeps the test harness aligned with the runtime dependency chain, so the new precompiles are ready when attestation spins up. Nicely done.

extensions/register.go (1)

15-20: Global registry wiring looks solid

Adding tn_utils to the extension init sequence before attestation ensures the attestation action can rely on the new precompiles without bootstrap hazards.

extensions/tn_utils/extension.go (1)

15-24: Extension bootstrap is clean

Expose-and-register pattern matches the other extensions, and deferring all build logic to initializePrecompile keeps the initializer trivial and safe.

extensions/tn_utils/serialization_test.go (1)

12-346: Strong coverage for serialization helpers

These tables hit the edge cases (nil, large arrays, error paths, range checks) I was worried about. Great job locking in determinism before wiring attestation onto it.

internal/migrations/023-attestation-schema.sql (1)

9-49: Schema matches attestation requirements

Composite PK, explicit encrypt_sig guard, action ID allowlist, and the supporting indexes all reflect the spec for #1196/#1197. Looks ready for the next migration.

- Replaced Read method with io.ReadFull in DecodeActionArgs and DecodeQueryResultCanonical to ensure complete reads of argument and column bytes. This change enhances error handling and data integrity during deserialization processes.

These updates contribute to more robust serialization functionality within the tn_utils extension.
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (1)
internal/migrations/023-attestation-schema.sql (1)

39-40: Index already flagged: signed_height needs NULLS FIRST ordering.

This index was already flagged in a previous review. Per issue #1196, the leader's unsigned-attestation query requires NULLs to sort first. The current index definition will cause full sorts for those queries.

Apply this diff to fix:

-CREATE INDEX IF NOT EXISTS ix_att_signed_height 
-    ON attestations(signed_height);
+CREATE INDEX IF NOT EXISTS ix_att_signed_height 
+    ON attestations(signed_height ASC NULLS FIRST);
🧹 Nitpick comments (2)
internal/migrations/024-attestation-actions.sql (2)

80-91: Clarify comment: only variable-length fields are length-prefixed.

The comment states "each field length-prefixed", but the code only prefixes variable-length fields (data_provider, stream_id, args_bytes, query_result). Fixed-length fields (version, algo, height, action_id) are not prefixed, which is correct since their lengths are known.

Apply this diff to clarify:

-    -- Canonical payload mirrors Go helpers: each field length-prefixed so the
-    -- validator can recover every component without ambiguity.
+    -- Canonical payload mirrors Go helpers: variable-length fields are length-prefixed
+    -- (data_provider, stream_id, args, result) while fixed-length fields are not
+    -- (version, algo, height, action_id). This allows unambiguous parsing.

94-100: Consider explicit duplicate check for clearer error messages.

The composite primary key (requester, attestation_hash) prevents duplicate attestations, but the error message will be a generic constraint violation. An explicit check could provide better user experience.

Consider adding before the INSERT:

-- Check for duplicate attestation
for $exists in SELECT 1 FROM attestations 
    WHERE requester = $caller_bytes AND attestation_hash = $attestation_hash {
    ERROR('Attestation already requested');
}
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 9c8e016 and 7ece6dd.

📒 Files selected for processing (12)
  • extensions/register.go (1 hunks)
  • extensions/tn_cache/syncschecker/sync_checker.go (3 hunks)
  • extensions/tn_utils/README.md (1 hunks)
  • extensions/tn_utils/extension.go (1 hunks)
  • extensions/tn_utils/precompiles.go (1 hunks)
  • extensions/tn_utils/serialization.go (1 hunks)
  • extensions/tn_utils/serialization_test.go (1 hunks)
  • internal/migrations/000-extensions.sql (1 hunks)
  • internal/migrations/023-attestation-schema.sql (1 hunks)
  • internal/migrations/024-attestation-actions.sql (1 hunks)
  • tests/streams/attestation/attestation_request_test.go (1 hunks)
  • tests/streams/utils/utils.go (2 hunks)
🧰 Additional context used
🧬 Code graph analysis (5)
tests/streams/utils/utils.go (5)
extensions/tn_vacuum/tn_vacuum.go (1)
  • InitializeExtension (13-23)
extensions/tn_digest/tn_digest.go (1)
  • InitializeExtension (18-40)
extensions/leaderwatch/leaderwatch.go (1)
  • InitializeExtension (47-54)
extensions/database-size/init.go (1)
  • InitializeExtension (10-15)
extensions/tn_cache/tn_cache.go (1)
  • InitializeExtension (87-99)
tests/streams/attestation/attestation_request_test.go (4)
tests/streams/utils/runner.go (1)
  • RunSchemaTest (37-116)
internal/migrations/migration.go (1)
  • GetSeedScriptPaths (15-48)
tests/streams/utils/utils.go (1)
  • GetTestOptionsWithCache (75-88)
extensions/tn_utils/serialization.go (2)
  • EncodeActionArgs (44-75)
  • EncodeQueryResultCanonical (144-183)
extensions/tn_utils/serialization_test.go (1)
extensions/tn_utils/serialization.go (2)
  • EncodeActionArgs (44-75)
  • DecodeActionArgs (84-129)
extensions/tn_utils/precompiles.go (1)
extensions/tn_utils/serialization.go (2)
  • DecodeActionArgs (84-129)
  • EncodeQueryResultCanonical (144-183)
extensions/register.go (1)
extensions/tn_utils/extension.go (1)
  • InitializeExtension (16-20)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: acceptance-test
🔇 Additional comments (13)
extensions/tn_cache/syncschecker/sync_checker.go (1)

24-36: LGTM! Clean logger adapter implementation.

The retryableHTTPLogger adapter correctly bridges the kwil logger to go-retryablehttp, logging HTTP requests at Debug level and properly trimming trailing newlines to avoid double newlines in logs.

tests/streams/utils/utils.go (1)

37-38: LGTM! Extension initialization follows established pattern.

The initialization of tn_utils and tn_attestation extensions in the test setup is correct and consistent with how other extensions are registered.

extensions/tn_utils/README.md (1)

1-52: LGTM! Clear and comprehensive documentation.

The README provides clear descriptions of all utility functions with helpful usage examples. The documentation structure makes it easy for developers to understand the extension's capabilities.

internal/migrations/000-extensions.sql (1)

6-8: LGTM! Extension schema activation is correct.

The USE statements properly activate the new extension schemas following the established pattern.

extensions/register.go (1)

15-15: LGTM! Extension initialization order is appropriate.

Initializing tn_utils immediately after leaderwatch ensures the utilities are available for subsequent extensions that may depend on them (such as tn_attestation).

extensions/tn_utils/extension.go (1)

15-24: LGTM! Extension initialization follows established pattern.

The implementation correctly follows the standard extension initialization pattern seen in other extensions. Panicking on registration errors is appropriate for initialization-time failures.

internal/migrations/023-attestation-schema.sql (2)

10-22: LGTM! Attestations table schema is well-designed.

The table structure with composite primary key (requester, attestation_hash) correctly supports per-user attestations, and the constraint enforcing encrypt_sig = false properly guards the MVP limitation.


32-37: LGTM! Query optimization indexes are appropriate.

The indexes on (requester, created_height) and (created_height) will efficiently support the filtering and ordering requirements specified in issue #1196.

internal/migrations/024-attestation-actions.sql (4)

31-33: LGTM! Encryption validation is correct.

The guard properly enforces the MVP constraint that encrypt_sig must be false, with a clear error message.


35-42: LGTM! Action allowlist validation is secure.

The code correctly validates that the action exists in the allowlist before proceeding, preventing unauthorized actions from being attested.


48-49: LGTM! Caller address normalization is correct.

The conversion of the hex-encoded @caller to bytes for storage is properly implemented.


67-78: Confirm external signing service uses the SQL digest() logic
The attestation hash is built only in SQL via digest(...,'sha256') (NULL delimiter, no length prefixes); there’s no Go-side re-implementation. Verify your signing/validation service replicates this exact input ordering and hashing to prevent mismatches.

tests/streams/attestation/attestation_request_test.go (1)

120-135: Excellent canonical payload assertion

Rebuilding the canonical blob from the stored row and verifying a byte-for-byte match gives strong confidence that the request path stays deterministic—nice work capturing the spec directly in the test.

@MicBun
Copy link
Member

MicBun commented Oct 10, 2025

@outerlook is this ready for review?

@outerlook
Copy link
Contributor Author

hey @MicBun, no, the CI fails, I'm still fixing some parts

- Updated serialization tests to use a normalization function for comparing decoded values, ensuring consistent equality checks across various data types.
- Introduced helper functions to normalize decoded slices and values, improving test reliability and clarity.

These changes enhance the robustness of the serialization tests within the tn_utils extension.
- Modified the primary key constraint in the attestation schema to include created_height, enhancing uniqueness.
- Updated comments in the attestation actions SQL migration to reflect changes in the hash calculation process, removing the height from the input parameters.

These changes aim to improve the integrity and clarity of the attestation process.
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (1)
extensions/tn_utils/serialization_test.go (1)

151-163: Remove unused helper function.

The cloneByteSlices function is defined but never called in this test file. Consider removing it to reduce dead code, or if it's intended for future use, add a comment explaining its purpose.

Apply this diff to remove the unused function:

-func cloneByteSlices(in [][]byte) [][]byte {
-	if in == nil {
-		return nil
-	}
-	out := make([][]byte, len(in))
-	for i := range in {
-		if in[i] == nil {
-			continue
-		}
-		out[i] = append([]byte(nil), in[i]...)
-	}
-	return out
-}
-
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 896c1bc and d50e31e.

📒 Files selected for processing (1)
  • extensions/tn_utils/serialization_test.go (1 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
extensions/tn_utils/serialization_test.go (1)
extensions/tn_utils/serialization.go (2)
  • EncodeActionArgs (45-76)
  • DecodeActionArgs (85-130)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: acceptance-test
🔇 Additional comments (3)
extensions/tn_utils/serialization_test.go (3)

13-139: Excellent test coverage for serialization round-trips.

The test suite comprehensively validates encode/decode for basic types, arrays, custom types (Decimal, UUID), edge cases (nil, empty, large values, unicode), and critically tests deterministic encoding. The table-driven approach and clear test names make this maintainable.


263-404: Solid error handling and bytea utility test coverage.

The error handling tests validate decoder robustness against malformed input. The bytea utility tests correctly verify little-endian uint32 length prefixes and edge cases like nil chunks. All test expectations match the expected binary format.


406-463: Thorough testing of uint encoders with boundary validation.

The table-driven tests correctly validate big-endian encoding for all uint widths (8/16/32/64 bits). The error tests appropriately verify overflow and negative value rejection, ensuring the encoders enforce their contracts.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

♻️ Duplicate comments (2)
internal/migrations/023-attestation-schema.sql (1)

39-40: signed_height index: keep as-is; enforce NULL-first ordering in queries

Kwil/Kuneiform doesn’t support NULLS FIRST in CREATE INDEX. The index on (signed_height) is fine; ensure your SELECT uses ORDER BY signed_height NULLS FIRST (and then created_height DESC) to satisfy leader queries.

Based on learnings

internal/migrations/024-attestation-actions.sql (1)

51-56: Deterministic args TODO already tracked.

The non-deterministic args concern is acknowledged and an issue was opened per the prior review. No further action in this PR.

🧹 Nitpick comments (5)
internal/migrations/023-attestation-schema.sql (1)

43-49: Bootstrap insert may still fail on action_id unique conflicts

Using ON CONFLICT (action_name) DO NOTHING ignores only name clashes; a clash on action_id would still error. If the dialect supports it, prefer DO NOTHING without a conflict target to tolerate either constraint; otherwise, consider naming the UNIQUE constraint and using ON CONFLICT ON CONSTRAINT.

Possible adjustments (pick one if supported by your dialect):

-ON CONFLICT (action_name) DO NOTHING;
+ON CONFLICT DO NOTHING;

Or name the unique constraint earlier and target it:

-    action_id INT NOT NULL UNIQUE,
+    CONSTRAINT uq_att_action_id UNIQUE (action_id),
...
-ON CONFLICT (action_name) DO NOTHING;
+ON CONFLICT ON CONSTRAINT uq_att_action_id DO NOTHING;
internal/migrations/024-attestation-actions.sql (4)

92-104: Make INSERT/queue idempotent to handle duplicates safely.

Even with height added, duplicate submits in the same height (or replays) will unique‑violate. Short‑circuit if the row exists so retries are safe; only enqueue when newly inserted.

Apply this guard:

-    -- Store unsigned attestation
-    INSERT INTO attestations (
+    -- Store unsigned attestation (idempotent on duplicates)
+    if EXISTS (
+        SELECT 1 FROM attestations 
+        WHERE requester = $caller_bytes AND attestation_hash = $attestation_hash
+    ) {
+        RETURN $attestation_hash;
+    }
+
+    INSERT INTO attestations (
         attestation_hash, requester, result_canonical, encrypt_sig, 
         created_height, signature, validator_pubkey, signed_height
     ) VALUES (
         $attestation_hash, $caller_bytes, $result_canonical, $encrypt_sig, 
         $created_height, NULL, NULL, NULL
     );
     
     -- Queue for signing (no-op on non-leader validators; handled by precompile)
     tn_attestation.queue_for_signing(encode($attestation_hash, 'hex'));

79-90: Canonical payload comment vs implementation; confirm agreed layout with Go helpers.

Comment says “each field length-prefixed,” but ints are fixed-size (uint8/64/16) while only variable fields are length-prefixed. Ensure this matches tn_utils/Go canonicalization; otherwise align comment or encoding to avoid signature mismatches.


28-29: $max_fee is unused.

Either wire it (enforcement/accounting) or mark TODO so the API surface isn’t misleading.


35-42: Ensure allowlist uniqueness for action_name.

Lookup assumes a single row. Verify there’s a UNIQUE constraint on attestation_actions(action_name); if not, add it to prevent ambiguity.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 896c1bc and f1bd2f6.

📒 Files selected for processing (3)
  • extensions/tn_utils/serialization_test.go (1 hunks)
  • internal/migrations/023-attestation-schema.sql (1 hunks)
  • internal/migrations/024-attestation-actions.sql (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • extensions/tn_utils/serialization_test.go
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2025-10-09T18:55:24.173Z
Learnt from: outerlook
PR: trufnetwork/node#1207
File: internal/migrations/023-attestation-schema.sql:39-0
Timestamp: 2025-10-09T18:55:24.173Z
Learning: In Kwil's Kuneiform SQL dialect, CREATE INDEX statements do not support NULLS FIRST or NULLS LAST clauses. NULL ordering must be specified in the query's ORDER BY clause instead (e.g., ORDER BY column_name NULLS FIRST).

Applied to files:

  • internal/migrations/023-attestation-schema.sql
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
  • GitHub Check: acceptance-test
  • GitHub Check: lint
🔇 Additional comments (1)
internal/migrations/024-attestation-actions.sql (1)

101-103: Confirm queue_for_signing input format.

You pass encode($attestation_hash, 'hex') (text). Validate that tn_attestation.queue_for_signing expects hex string, not raw bytes.

- Removed the unnecessary index on the requester and created_height columns in the attestation schema, simplifying the structure and improving maintainability.

This change enhances the efficiency of the attestation schema by eliminating redundancy.
@outerlook outerlook requested a review from MicBun October 10, 2025 13:33
Copy link
Member

@MicBun MicBun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm

@MicBun MicBun merged commit d9abbe4 into main Oct 10, 2025
7 checks passed
@MicBun MicBun deleted the chore/get-attestation branch October 10, 2025 13:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Problem: No action ID registry for attestation payloads Problem: Users cannot request signed data attestations

2 participants