Skip to content

cache ResultSet metadata in getMetaDataMapByMessageId to reduce per-row metadata lookups#282

Merged
NicoPiel merged 6 commits intoOpenIntegrationEngine:mainfrom
NicoPiel:feat/jdbc-metadata-perf
Apr 2, 2026
Merged

cache ResultSet metadata in getMetaDataMapByMessageId to reduce per-row metadata lookups#282
NicoPiel merged 6 commits intoOpenIntegrationEngine:mainfrom
NicoPiel:feat/jdbc-metadata-perf

Conversation

@NicoPiel
Copy link
Copy Markdown
Collaborator

Cache ResultSetMetaData (column count, types, and names) before iterating rows so getColumnType/getColumnName are not called for every row. Adjust column indexing and decryption logging to use the cached metadata. This reduces repeated metadata calls and improves performance when reading connector metadata from the database.

@NicoPiel NicoPiel force-pushed the feat/jdbc-metadata-perf branch from 1aa2ec3 to e778f04 Compare March 31, 2026 18:38
@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 31, 2026

Test Results

  111 files  ±0    214 suites  ±0   6m 58s ⏱️ + 1m 10s
  654 tests ±0    654 ✅ ±0  0 💤 ±0  0 ❌ ±0 
1 308 runs  ±0  1 308 ✅ ±0  0 💤 ±0  0 ❌ ±0 

Results for commit b385fd4. ± Comparison against base commit a6ae879.

♻️ This comment has been updated with latest results.

Copy link
Copy Markdown
Contributor

@mgaffigan mgaffigan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I doubt this is making actual round-trips, but unopposed to cleaning things up. At a minimum, we should drop some GC pressure from the repeated toUpperCase and fromSqlType.

Also: consider whether the move to 0-indexing is actually adding value. For the cost of two empty array elements, this PR gets a lot smaller and there's less opportunity to screw up the zero-based vs one-based indexing.

Also: missing DCO

…ow metadata lookups

Cache ResultSetMetaData (column count, types, and names) before iterating rows so getColumnType/getColumnName are not called for every row. Adjust column indexing and decryption logging to use the cached metadata. This reduces repeated metadata calls and improves performance when reading connector metadata from the database.

Signed-off-by: Nico Piel <nico.piel@hotmail.de>
Signed-off-by: Nico Piel <nico.piel@hotmail.de>
Cache ResultSetMetaData (column count, types, and names) up front instead of calling getColumnType/getColumnName for every row. This reduces repeated metadata lookups and improves performance when reading metadata rows from the database.

Signed-off-by: Nico Piel <nico.piel@hotmail.de>
Signed-off-by: Nico Piel <nico.piel@hotmail.de>
Signed-off-by: Nico Piel <nico.piel@hotmail.de>
…tadata lookups

Cache ResultSetMetaData (column count, types, and names) before iterating rows and switch to 1-based arrays/loops so resultSet getters can be called directly with the column index. This prevents repeated getColumnType/getColumnName calls for every row and reduces per-row metadata lookups when building connector metadata maps.

Signed-off-by: Nico Piel <nico.piel@hotmail.de>
@NicoPiel NicoPiel force-pushed the feat/jdbc-metadata-perf branch from 311b9d8 to b385fd4 Compare March 31, 2026 19:20
@NicoPiel
Copy link
Copy Markdown
Collaborator Author

Followed your suggestions and added DCO

@NicoPiel NicoPiel requested review from a team, gibson9583, jonbartels, kayyagari, mgaffigan, pacmano1, ssrowe and tonygermano and removed request for a team March 31, 2026 19:21
@NicoPiel NicoPiel self-assigned this Mar 31, 2026
Copy link
Copy Markdown
Contributor

@jonbartels jonbartels left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also did some digging to find out if this is a DB hit or something else. It reduces 1000s of calls to only one call per column. Its a good change. Great find Nico!

Copy link
Copy Markdown

@kayyagari kayyagari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for spotting it.

@NicoPiel NicoPiel merged commit 1cafc2e into OpenIntegrationEngine:main Apr 2, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants