Skip to content

Conversation

@speeddragon
Copy link

@speeddragon speeddragon commented Nov 11, 2025

This solves the issue where requesting http://localhost:8734/<ID> doesn't return the right content-type (and other fields).

The function hb_cache:store_read executes different actions (different hb_store calls) for each store. This will break the logic when calling type in one store and then list in another store, which always returns the ok tuple (hb_store_lmdb).

  • We call list on the first store, which returns not_found, then call type on the second store, which returns composite.
  • Then we call list on the first store, which always returns (even if empty), not accessing the value we want in the second store.

This PR changes this, so hb_cache:store_read processes the stores like hb_store, to run each logic separately for each store.

The change to lmdb file and the hb_cache:list function is optional, since I didn't detect an error with it, but this behaviour doesn't make sense because it is equal to the hb_cache:store_read.

resolve behaviour

We have two behaviours for resolve function:

  • Return binary (resolved or unresolved path) or not_found atom in hb_store_fs.
  • Return binary (resolved or unresolved path) in hb_store_lmdb
    • This is changed by this PR, to also return not_found.
  • Return binary (resolved or unresolved path) or {error, any()} in hb_store_rocksdb. not_found is never returned.

The different behaviour between them is the reason to have different resolve handling in resolved_type and resolved_list. If we agree on modifying the return to always contain not_found (which may imply reading the object), this might not be needed.

resolved_read not implemented

hb_store_fs, hb_store_lmdb, and hb_store_lru already run resolve when read is called.

  • hb_store_fs and hb_store_lru - read/2 resolves the key before trying to access it.
  • hb_store_lmdb - tries to read first directly. If not successful, it will call resolve_path_links.

We can remove resolve call before each read, and trust the store to call it, or implement the resolved_read to make sure the resolve and read are always called for the same store.

Why not make type and list internally call resolve?

There are a few cases where type and list are used where resolved isn't needed or made in a different place of the function call.

Cases not changed from resolve and list to resolved_list

@speeddragon speeddragon marked this pull request as ready for review November 12, 2025 18:51
@speeddragon
Copy link
Author

Tests pass with the same failures reported in #553

Failed: 4.  Skipped: 0.  Passed: 1373.

@speeddragon speeddragon marked this pull request as draft November 28, 2025 20:32
@speeddragon speeddragon force-pushed the fix_multiple_store_logic branch from fcead56 to 101961c Compare December 5, 2025 11:22
case hb_store:list(Store, << ID/binary, "/commitments">>) of
{ok, []} -> [ID];
{ok, CommitmentIDs} -> CommitmentIDs;
_ -> []
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't see a use case to return an empty array. The LMDB store didn't return not_found, but the hb_store_fs did, so swapping these two stores in the dev_query_test_vectors:simple_blocks_query_test will make the test fail (which shouldn't happen).

Store = hb_opts:get(store, no_viable_store, Opts),
UncommittedID = hb_message:id(Msg, none, Opts#{ linkify_mode => discard }),
%% TODO: Confirm tha this should only look into local stores only
Store = hb_store:scope(hb_opts:get(store, no_viable_store, Opts), local),
Copy link
Author

@speeddragon speeddragon Dec 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On edge, because LMDB store list was returning {ok, []} when it didn't find the information, the not_found case was never executed, making this request to not propagate to other stores.

This can impact performance, since now it will request to the configured stores. By limiting the scope to local we can mitigate this performance impact, but we need to be aware it will always be worse than just requesting to LMDB.


cache_suite_test_() ->
hb_store:generate_test_suite([
{"store ans104 message", fun test_store_ans104_message/1},
Copy link
Author

@speeddragon speeddragon Dec 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test wasn't being used (maybe deleted by mistake?). Re-add it.

),
Msg#{ <<"commitments">> => NewCommitments }.

read_all_commitments_by_store(Msg, Store, Opts) when not is_list(Store) ->
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With read_all_commitments_by_store we make sure all actions only occour in one Store, and not retried in different stores if the item isn't found in the first one.

hb_store:write(Store2, <<"data/final_id2">>, <<"7890">>),
%% Link
%% TODO: Not sure if this structure is possible in HB
hb_store:make_link(Store2, <<"data/final_id2">>, <<"group1/group12/data">>),
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if this structure is possible in HB, but in this scenario, the old resolve followed by list would fail.

@speeddragon speeddragon marked this pull request as ready for review December 26, 2025 16:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant