Skip to content

aodMerger: Special option to merge only folders of the same name#15205

Open
sawenzel wants to merge 1 commit intoAliceO2Group:devfrom
sawenzel:swenzel/aod-merger-mod
Open

aodMerger: Special option to merge only folders of the same name#15205
sawenzel wants to merge 1 commit intoAliceO2Group:devfrom
sawenzel:swenzel/aod-merger-mod

Conversation

@sawenzel
Copy link
Copy Markdown
Collaborator

Adding a special merge-by-name option which asks the merger only to merge together dataframe folders of the same name.

This is needed only in special situations where we want to enfore the output structure of the AOD to be the same as that of a reference (data) AOD. An example is MC-DATA embedding.

Adding a special `merge-by-name` option which asks the merger only
to merge together dataframe folders of the same name.

This is needed only in special situations where we want to enfore the output
structure of the AOD to be the same as that of a reference (data) AOD.
An example is MC-DATA embedding.
@sawenzel sawenzel requested a review from a team as a code owner March 23, 2026 12:07
@github-actions
Copy link
Copy Markdown
Contributor

REQUEST FOR PRODUCTION RELEASES:
To request your PR to be included in production software, please add the corresponding labels called "async-" to your PR. Add the labels directly (if you have the permissions) or add a comment of the form (note that labels are separated by a ",")

+async-label <label1>, <label2>, !<label3> ...

This will add <label1> and <label2> and removes <label3>.

The following labels are available
async-2023-pbpb-apass4
async-2023-pp-apass4
async-2024-pp-apass1
async-2022-pp-apass7
async-2024-pp-cpass0
async-2024-PbPb-apass1
async-2024-ppRef-apass1
async-2024-PbPb-apass2
async-2023-PbPb-apass5

@sawenzel
Copy link
Copy Markdown
Collaborator Author

@fjonasALICE : As discussed

@alibuild
Copy link
Copy Markdown
Collaborator

Error while checking build/O2/fullCI_slc9 for d53ad16 at 2026-03-29 14:21:

## sw/BUILD/O2Physics-latest/log
c++: fatal error: Killed signal terminated program cc1plus
ninja: build stopped: subcommand failed.


## sw/BUILD/o2checkcode-latest/log
--
========== List of errors found ==========
++ GRERR=0
++ grep -v clang-diagnostic-error error-log.txt
++ grep ' error:'
++ GRERR=1
++ [[ 1 == 0 ]]
++ mkdir -p /sw/INSTALLROOT/45b2f71616eb78758fe4092e030330b97e548a18/slc9_x86-64/o2checkcode/1.0-local9/etc/modulefiles
++ cat
--

Full log here.

@jgrosseo
Copy link
Copy Markdown
Collaborator

jgrosseo commented Mar 30, 2026

Are you trying to achieve that DF folders are not merged?

In this case the option for that already exists, one has to put --max-size 0. We use it already on Hyperloop. Please don't merge this for now.

@sawenzel
Copy link
Copy Markdown
Collaborator Author

Are you trying to achieve that DF folders are not merged?

In this case the option for that already exists, one has to put --max-size 0. We use it already on Hyperloop. Please don't merge this for now.

Not really, no: I am trying to achieve that only folders with the same name are merged. If I have fews AODs with this structure:

AOD1.root --> DF_xxx
AOD2.root --> DF_xxx
AOD3.root --> DF_xxx
AOD4.root --> DF_yyy
AOD5.root --> DF_yyy

I would like to restrict merging, so that only AOD1,2,3 is merged together into the output (under folder DF_xxx) and AOD4,5 (under folder DF_yyy). With such functionality and I could ensure that MC AODs derived from DATA have the same structure.

Is the --max-size 0 behaviour is already achieving exactly this? If yes, I'll close this PR.

@jgrosseo
Copy link
Copy Markdown
Collaborator

max-size decides if a new DF folder should be merged into the current one or not. If the current size is larger than the provided max-size it closes the current folder and moves to the next one.

I think we never tested the case of identical DF folders in different files but I think it could work out. Could you give it a try with --max-size 0 ?

@jgrosseo
Copy link
Copy Markdown
Collaborator

Of course the files have to be in the right order, it wouldn't work if you have

AOD1.root --> DF_xxx
AOD2.root --> DF_yyy
AOD3.root --> DF_xxx
AOD4.root --> DF_yyy
AOD5.root --> DF_yyy

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants