[Data] Add documentation for Ray Data checkpointing

[bveeramani]

Description

#59409 added job-level checkpointing for Ray Data, but there isn't any documentation.

At the least, we should document it in Execution Configurations and End-to-end: Offline Batch Inference.

Link

No response

[kriyanshii]

Would love to do this.

[anonihunter]

Hello, I saw that job-level checkpointing was added in #59409, and there is an open docs PR (#60250) addressing this. I’d be happy to help improve the documentation further by:

• adding a clear Limitations section (e.g., Read → Write pipelines only),
• including a minimal runnable example based on the example in [Data] Add Checkpointing to Ray Data #59409,
• and adding brief guidance on when to use job-level checkpointing.

Let me know if a small follow-up PR would be helpful.

[bveeramani]

there is an open docs PR (#60250) addressing this

@anonihunter I think the link might be incorrect. The link points to this GitHub Issue

Let me know if a small follow-up PR would be helpful.

Would you mind sharing some more details about your plan? In particular, I'm curious about which sections you'll be editing (or adding)?

[bveeramani]

@kriyanshii Appreciate the enthusiasm! Since you're already assigned to two Issues, I think I'd prefer to give this task to someone else for now. Just want to make sure all newcomers have the opportunity the contribute

[anonihunter]

Thanks for checking! Apologies for the confusion earlier — you’re right, the link pointed back to the issue.

My plan is to make a small, focused follow-up documentation update that complements #60250, without duplicating it. Basically, I’m planning to:

First: Execution Configurations

• Add a short subsection describing job-level checkpointing in Ray Data.
• Document the key configuration options in CheckpointConfig (e.g., id_column, checkpoint_path, and delete_checkpoint_on_success).

Second: End-to-end: Offline Batch Inference

• Add a minimal runnable example based on the example in [Data] Add Checkpointing to Ray Data #59409, showing a failed run and a resumed run from a checkpoint.
• Include a brief “When to use job-level checkpointing” note (long-running batch jobs, fault tolerance, etc.).

Third: Limitations / Notes

• Clearly call out current limitations, such as pipelines needing to start with a Read and end with a Write.

If this sounds reasonable, I'm happy to open a small PR with just these additions.

[bveeramani]

Got it, makes sense to me. Wanna put out a draft PR, and I can give you feedback on that?

[anonihunter]

Hi! I’ve opened a draft PR #60289 documenting the job-level checkpointing introduced in #59409.

It’s currently in draft while review feedback is being addressed.

[bveeramani]

@anonihunter sorry for the delay! Missed your PR.

Just left some comments