Skip to content

Process results in series #1843

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
joshqsumner wants to merge 131 commits into
base: v5.0
Choose a base branch
from
Open

Process results in series #1843

joshqsumner wants to merge 131 commits into from

Conversation

@joshqsumner
Copy link
Contributor

@joshqsumner joshqsumner commented Nov 11, 2025

Describe your changes
Moved plantcv.parallel.process_results and plantcv.parallel.json2csv to the main plantcv.plantcv module.

Type of update
This is a feature enhancement/restructuring to allow for clearer use of outputs from jupyter notebooks/non-parallel workflows.

Associated issues
None (whoops), discussed talked about in dev meeting 10/3/2025 and a few times in passing since then.

Additional context
Follows after removing the utils subpackage.

For the reviewer
See this page for instructions on how to review the pull request.

  • PR functionality reviewed in a Jupyter Notebook
  • All tests pass
  • Test coverage remains 100%
  • Documentation tested
  • New documentation pages added to plantcv/mkdocs.yml
  • Changes to function input/output signatures added to updating.md
  • Code reviewed
  • PR approved

@joshqsumner
don't group in parser
dac52fe
@joshqsumner
group in job_builder
76f7977
@joshqsumner
accounting for ungrouped data
952ebf8
@joshqsumner
importing pd for methods
f40fd32
@joshqsumner
save metadata csv
a4e3a02
@joshqsumner
checkpointing column to metadata
7d52f8f
@joshqsumner
adding option to resume using csv metadata
caa5020
@joshqsumner
checkpoint file name
6435363
@joshqsumner
workflow_inputs as a class
349a208
@joshqsumner
using dummy files in temp directory to monitor checkpointing
58154c9
@joshqsumner
minimal working example
16a1ee5
@joshqsumner
fixed mvp
4a198a9
@joshqsumner
flexibility for processing results
648b13c
@joshqsumner
removing files after checkpointing metadata is made
95597cc
@joshqsumner
finishing docstring
cf7c091
@joshqsumner
typo
11e0056
@joshqsumner
informative error if checkpointing files are all completed
3286488
@joshqsumner
cleaning up from "checkpoint" dir instead of checkpoint/tmp
f9655dc
@joshqsumner
refactoring to allow for interim analyses use case
3227966
@joshqsumner
docs and cleaning
e905809
@joshqsumner
flipping logic for easier testing
fd4b73c
@joshqsumner
making name images a function available to WorkflowInputs again
f6d32a1
@joshqsumner
updating tests
773ced7
@joshqsumner
deepsource linting
162d3be
@joshqsumner
covering workflow inputs
865e55a
@joshqsumner
unused argument removal
7d3260d
@joshqsumner
trying to test remaining lines
c9dc571
@joshqsumner
adding a message about excluding files if they are found
a056835
@joshqsumner
adding checkpointed results to removed_df for inspection.
b478a88
@joshqsumner
adding to docs to lessen the chance someone has an important checkpoi…
21b707f 
…nt folder deleted
@joshqsumner
Merge branch 'rename-filepath-to-dir' into process-results-in-series
55d8a29
@joshqsumner
Merge branch 'v5.0' into checkpointing
1b52e74
@joshqsumner
Merge branch 'checkpointing' into parallel-verbosity
32e308b
@joshqsumner
Merge branch 'parallel-verbosity' into output-key-name
82048c1
@joshqsumner
Merge branch 'output-key-name' into remove-utils
2b9f5fb
@joshqsumner
Merge branch 'remove-utils' into rename-filepath-to-dir
cedafb7
@joshqsumner
Merge branch 'rename-filepath-to-dir' into process-results-in-series
18d3b5f
@joshqsumner
keep track of original tmp_dir for checkpointing
c934548
@joshqsumner
walk from original tmp_dir to find checkpointing status
dbc9cf8
@joshqsumner
forcing attribute to exist in helper function
68368ae
@joshqsumner
deepsource
559a44a
@joshqsumner
use chkpt_start_dir in processing results
9515830
@joshqsumner
making expected results directory structure
0051c43
@joshqsumner
pointing into the checkpoint folder
a46c7a9
@joshqsumner
stopping timestamp warnings
0a6a438
@joshqsumner
Merge branch 'checkpointing' into parallel-verbosity
5f37a49
@joshqsumner
Merge branch 'parallel-verbosity' into output-key-name
74b88e7
@joshqsumner
Merge branch 'output-key-name' into remove-utils
f85c857
@joshqsumner
Merge branch 'remove-utils' into rename-filepath-to-dir
355a66d
@joshqsumner
Merge branch 'rename-filepath-to-dir' into process-results-in-series
668911d
@joshqsumner
add _chkpt_start_dir to validated config attributes and hide it
73a965c
@joshqsumner
deepsource does not want me touching a hidden attribute directly
6c56ebd
@joshqsumner
more deepsource preferences
1f0fb8b
@joshqsumner
Merge branch 'parallel-verbosity' into output-key-name
da92da5
@joshqsumner
Merge branch 'output-key-name' into remove-utils
e129fd5
@joshqsumner
Merge branch 'remove-utils' into rename-filepath-to-dir
9707e8e
@joshqsumner
Merge branch 'rename-filepath-to-dir' into process-results-in-series
2720fb7
@joshqsumner
fix unused variable names
2a3f4a0
@joshqsumner
fix conflict
1638d7c
@joshqsumner
deepsource
a554802
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

merge in order Set of PRs that need to be merged sequentially ready to review update Updates an existing feature/method

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@joshqsumner