Skip to content

Adapt to wire-sysio table_id namespace isolation; long table name support#12

Open
heifner wants to merge 1 commit intomasterfrom
feature/wire-sysio-table-id
Open

Adapt to wire-sysio table_id namespace isolation; long table name support#12
heifner wants to merge 1 commit intomasterfrom
feature/wire-sysio-table-id

Conversation

@heifner
Copy link
Copy Markdown
Contributor

@heifner heifner commented Apr 10, 2026

Summary

Wire-sysio PR Wire-Network/wire-sysio#288 widened table_def.name from sysio::name (uint64) to free-form std::string and added table_id (uint16, DJB2 hash of the table name % 65536) and secondary_indexes for KV-table per-table namespace isolation. The binary wire format break means abieos must be updated in lockstep with the chain side or every field after tables in abi_def parses misaligned, and any contract whose table name is longer than 12 chars is unreachable through the C API.

This PR takes the chance to make the long-table-name story end-to-end by switching the internal table_types map and the C API entry point to free-form strings.

Companion PRs:

⚠️ Breaking C API change

abieos_get_type_for_table now takes the table name as a const char* instead of a uint64-encoded sysio name:

// Before
const char* abieos_get_type_for_table(abieos_context*, uint64_t contract, uint64_t table);
// After
const char* abieos_get_type_for_table(abieos_context*, uint64_t contract, const char* table);

The only known consumer of this entry point in the Wire ecosystem is node-abieos (the native binding used by Wire-Network/hyperion-history-api), which was already converting JS strings to uint64 via abieos_string_to_name() before calling this function — that round trip is now redundant and node-abieos can pass the string straight through. node-abieos will be updated in a follow-on PR.

Changes

include/sysio/abi.hpp

  • New index_def struct: {name: string, key_type: string, table_id: uint16} mirroring sysio::chain::index_def
  • table_def: name: name → string, +table_id: uint16, +secondary_indexes: index_def[]
  • abi_def: +protobuf_types as a might_not_exist<std::string> extension to mirror wire-sysio's reflection layout (variants, action_results, enums, protobuf_types). Keeps binary forward-compat for ABIs that include the field even though abieos itself doesn't consume it.
  • abi.table_types: std::map<sysio::name, string>std::map<std::string, std::string, std::less<>>. The std::less<> transparent comparator lets callers do find(string_view) without constructing a temporary std::string per lookup.
  • abi.abi_types likewise gets std::less<>.
  • New aliases: table_type_map, abi_type_map (used in abi.cpp signatures).
  • to_json(abi_def) writes protobuf_types unconditionally for consistency with the surrounding extension fields (variants, action_results, enums) which always emit even when empty.

src/abieos.hpp

  • jobject typedef gets std::less<> for the same transparent-lookup reason — the JSON object map is the hottest map in abieos.

src/abi.cpp

  • convert(abi_def, abi) populates table_types directly with the new string-typed table_def.name; no name conversion needed.
  • Function signatures updated to use abi_type_map alias.

src/abieos.h, src/abieos.cpp

  • abieos_get_type_for_table signature change as described above.
  • Internal lookup uses std::string_view via the std::less<> transparent comparator to avoid an allocation per call.

src/test.cpp

  • tokenHexAbi regenerated for the new binary format. The token ABI shape (sysio::abi/1.0) is unchanged, but the on-wire encoding of tables[].name is now a length-prefixed string instead of a uint64. The new hex was produced from the canonical token ABI JSON via abieos_abi_json_to_bin().

external/wirejs-native

Tests

All four C++ tests pass against the updated layout:

  • test_abieos
  • test_abieos_template
  • test_abieos_key
  • test_abieos_reflect

…port

Wire-sysio PR Wire-Network/wire-sysio#288 widened table_def.name from
sysio::name (uint64) to free-form std::string and added table_id
(uint16, DJB2 hash of the table name % 65536) and secondary_indexes
for KV-table per-table namespace isolation. The binary wire format
break means abieos must be updated in lockstep with the chain side or
every field after `tables` in abi_def parses misaligned, and any
contract whose table name is longer than 12 chars is unreachable
through the C API.

This commit takes the chance to make the long-table-name story
end-to-end by switching the internal table_types map and the C API
entry point to free-form strings.

include/sysio/abi.hpp:
 - new index_def struct (name: string, key_type: string, table_id: uint16)
 - table_def: name → string; +table_id (uint16); +secondary_indexes
 - abi_def: +protobuf_types as a might_not_exist<std::string> extension
   to mirror wire-sysio's reflection layout (variants, action_results,
   enums, protobuf_types). Keeps binary forward-compat for ABIs that
   include the field even though abieos itself doesn't consume it.
 - abi.table_types: std::map<sysio::name, string> →
   std::map<std::string, std::string, std::less<>>. The std::less<>
   transparent comparator lets callers do find(string_view) without
   constructing a temporary std::string per lookup.
 - abi.abi_types likewise gets std::less<>.
 - new aliases: table_type_map, abi_type_map (used in abi.cpp signatures).
 - to_json(abi_def) writes protobuf_types unconditionally for
   consistency with the surrounding extension fields (variants,
   action_results, enums) which always emit even when empty.

src/abieos.hpp:
 - jobject typedef gets std::less<> for the same transparent-lookup
   reason — the JSON object map is the hottest map in abieos.

src/abi.cpp:
 - convert(abi_def, abi) populates table_types directly with the new
   string-typed table_def.name; no name conversion needed.
 - Function signatures updated to use abi_type_map alias.

src/abieos.h, src/abieos.cpp — BREAKING C API change:
 - abieos_get_type_for_table(context, uint64_t contract, const char* table)
   replaces the old uint64_t table parameter. The C API now accepts free-
   form table names directly so long names work end-to-end. Internal
   lookup uses std::string_view via the std::less<> transparent
   comparator to avoid an allocation per call.
 - The only known consumer of this entry point in the Wire ecosystem is
   node-abieos (Wire-Network/hyperion-history-api's native binding),
   which was already converting JS strings to uint64 via
   abieos_string_to_name() before calling this function — that round
   trip is now redundant and node-abieos can pass the string straight
   through. node-abieos will be updated separately.

src/test.cpp:
 - tokenHexAbi regenerated for the new binary format. The token ABI
   shape (sysio::abi/1.0) is unchanged, but the on-wire encoding of
   tables[].name is now a length-prefixed string instead of a uint64.
   The new hex was produced from the canonical token ABI JSON via
   abieos_abi_json_to_bin().

external/wirejs-native bumped to feature/wire-sysio-table-id with the
companion TS schema changes (Wire-Network/wirejs-native#4). All four
abieos C++ tests (test_abieos, test_abieos_template, test_abieos_key,
test_abieos_reflect) pass against the updated layout.

Companion PRs:
 - Wire-Network/wire-sysio#288 (chain-side table_id namespace isolation)
 - Wire-Network/wirejs-native#4 (TS schema mirror of this commit)
heifner added a commit to Wire-Network/node-abieos that referenced this pull request Apr 10, 2026
…rings

Wire-sysio PR Wire-Network/wire-sysio#288 widened table_def.name from
sysio::name (uint64) to free-form std::string and added table_id
(uint16) + secondary_indexes for KV-table per-table namespace
isolation. The companion abieos PR Wire-Network/abieos#12 changed the
C API entry point abieos_get_type_for_table() to take a const char*
table name instead of a uint64 sysio name.

This commit picks up the new abieos and updates the binding to match:

- abieos submodule bumped to feature/wire-sysio-table-id (commit
  9c1000a, the head of Wire-Network/abieos#12). The bump pulls in
  table_def long-name support, the breaking C API signature change,
  the abi.table_types map keyed by std::string with std::less<>
  transparent comparator, and the regenerated tokenHexAbi test
  fixture for the new on-wire encoding.

- src/main.cpp:182 get_type_for_table() — drop the table-name string
  → uint64 round trip via abieos_string_to_name() that the wrapper
  used to do before calling abieos_get_type_for_table(). The round
  trip was lossy: any table name longer than 12 characters or
  containing characters outside the sysio name alphabet was silently
  truncated, making the table type unreachable. The binding now
  passes the JS string straight through. The contract name still
  goes through abieos_string_to_name() because contract identifiers
  ARE sysio::name on both sides of the API.

- examples/ABIs/eosio.token.raw regenerated against the new abieos
  binary table_def encoding. The fixture is base64-encoded binary
  ABI; the old fixture used the legacy table_def shape (uint64 name)
  and produced "Stream overrun" on load against the new abieos. The
  contained ABI shape (eosio::abi/1.1, two tables: accounts/stat) is
  unchanged — only the encoding of tables[].name changes from
  "fixed 8 bytes" to "varint length + bytes".

- lib/abieos.node rebuilt against the new abieos with clang++-18 and
  cmake-js compile --CDSKIP_SUBMODULE_CHECK=1 (the parent submodule
  ref is intentionally ahead of master while abieos PR #12 is in
  flight).

examples/basic.mjs now passes 6/6 positive checks (eosio voteproducer,
eosio.token transfer + stat, eosio.msig approvals2 + proposal,
eosio producers); the seventh check is a deliberate negative test
that looks up a non-existent table on a non-existent contract.

Companion PRs:
 - Wire-Network/wire-sysio#288 — chain-side table_id namespace isolation
 - Wire-Network/abieos#12 — abieos C API + binary format update
 - Wire-Network/wirejs-native#4 — TS schema mirror
heifner added a commit to Wire-Network/wire-libraries-ts that referenced this pull request Apr 10, 2026
Wire-sysio PR Wire-Network/wire-sysio#288 widened table_def.name from
sysio::name (uint64) to a free-form std::string and appended two new
fields — table_id (uint16, DJB2 hash of the table name % 65536) and
secondary_indexes (vector<index_def>) — for KV-table per-table
namespace isolation. The wire format break means any sdk-core caller
that loads a contract ABI from on-chain via get_raw_abi (or any other
binary-encoded source) parses misaligned starting at the first table
field — name reads the wrong bytes, every subsequent field is shifted,
and ricardian_clauses ends up reading garbage.

PR Wire-Network/wire-sysio#290 separately changed the get_table_rows
HTTP response shape for KV-backed tables. Each row used to be the
decoded struct directly (or {data, payer} when show_payer was set);
the unified endpoint now returns {key, value, payer?} per row. The
ChainAPI wrapper used to destructure {data, payer} on show_payer and
then map rows through Serializer.decode for typed callers — both paths
break against the new shape.

This commit takes a hard break on the binary format (no fallback to
the legacy EOSIO 8-byte name encoding) and a backward-compatible
unwrap on the HTTP response shape.

packages/sdk-core/src/chain/Abi.ts:
 - new ABI.Index interface (name, key_type, table_id) mirroring
   sysio::chain::index_def in wire-sysio's
   libraries/chain/include/sysio/chain/abi_def.hpp.
 - ABI.Table gains optional table_id (uint16) and secondary_indexes
   (Index[]) fields. Optional so hand-built test fixtures and ABIs
   constructed programmatically don't need to specify them; the
   encoder defaults to 0 / [] when omitted.
 - fromABI() reads name as a length-prefixed string instead of an
   8-byte sysio::name uint64, then consumes table_id (uint16 LE) and
   secondary_indexes (vector<index_def>) after the existing type
   field. Also reads a trailing protobuf_types extension after enums
   so that any subsequent extension appended to abi_def parses
   cleanly to EOF.
 - toABI() mirrors the parser: writes table_def.name via
   encoder.writeString(String(table.name)) (the String() coercion
   keeps NameType-typed callers compiling), then table_id and
   secondary_indexes, and finally the empty protobuf_types string.

packages/sdk-core/src/api/v1/Chain.ts:
 - get_table_rows() now detects the unified KV row shape — each row
   has both `key` and `value` keys — and unwraps to the inner
   value before downstream processing. The old EOSIO shapes
   (decoded struct directly, or {data, payer} on show_payer) are
   left untouched, so sdk-core remains usable against EOSIO chains.
 - When show_payer is set on a KV row, payer is captured into
   ram_payers in the same way as the legacy {data, payer} path.
   Missing payer fields coerce to an empty Name rather than
   throwing.

Tests (15 new cases, 304/304 sdk-core suite green):

 packages/sdk-core/tests/chain/Abi.test.ts (8 cases)
  - Round-trips a table with table_id + empty secondary_indexes
  - Round-trips a long table name (>12 chars) — the whole reason
    name was widened
  - Round-trips secondary_indexes with checksum256 key_type
    (the case the wire-cdt #49 abigen fix unblocked end-to-end)
  - table_id 0 and 65535 boundary values
  - Missing table_id defaults to 0 on encode + decode
  - protobuf_types extension is consumed without affecting enums
  - Encoder always emits the empty protobuf_types trailer
  - Multi-table ABI end-to-end round-trip (structs + actions +
    secondary_indexes)

 packages/sdk-core/tests/api/v1/Chain.test.ts (7 cases)
  - Unwraps the new {key, value} shape into plain rows
  - Unwraps the new shape with show_payer + captures ram_payers
  - Preserves legacy plain-row shape from EOSIO chains
  - Preserves legacy {data, payer} show_payer shape from EOSIO
  - Empty rows array works for both shapes
  - Missing payer in new shape coerces to empty name (no throw)
  - Uses an in-memory MockProvider so no network access required

Out of scope:
 - packages/sdk-core/src/resources/{Ram,Rex,Powerup}.ts call
   get_table_rows for tables (rammarket, rex_pool, powup_state) that
   do not exist in wire-sysio and have not for some time. These
   resources were already dead on Wire chains regardless of any of
   the in-flight wire-sysio PRs and remain dead after this commit.
 - packages/sdk-core/src/types/SystemContractTypes.ts is auto-
   generated and would benefit from a regen pass after sysio.token's
   migration to kv::scoped_table merges (Wire-Network/wire-sysio#291),
   but the field schemas of the structs it generates do not depend
   on the binary table_def layout.

Companion PRs:
 - Wire-Network/wire-sysio#288 — chain-side table_id namespace isolation
 - Wire-Network/wire-sysio#290 — unified get_table_rows API
 - Wire-Network/wire-sysio#291 — sysio.token migrated to kv::scoped_table
 - Wire-Network/wire-cdt#49 — kv::scoped_table + abigen secondary_indexes
 - Wire-Network/wirejs-native#4 — TS schema mirror for abi_def
 - Wire-Network/abieos#12 — abieos C API + binary format update
 - Wire-Network/node-abieos#8 — bumps abieos + drops the lossy uint64 round-trip
 - Wire-Network/hyperion-history-api#9 — KV delta handler + AbiDefinitions
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant