A tff.program.FileProgramStateManager is a utility for saving and loading
program state to a file system in a federated program and is used to implement
fault tolerance. In particular, it is intended to only restart the same
simulation and run with the same version of TensorFlow Federated.
Program state is saved to the file system using the SavedModel (see
tf.saved_model) format. When the program state is saved, each
tff.program.MaterializableValueReference is materialized and each
tff.Serializable is serialized. The structure of the program state is
discarded, but is required to load the program state.
A path on the file system to save program state. If this path
does not exist it will be created.
prefix
A string to use as the prefix for filenames.
keep_total
An integer representing the total number of program states to
keep. If the value is zero or smaller, there will be no limitation on
how many program states to keep; if keep_every_k is 1, then all states
will be kept.
keep_first
A boolean indicating if the first program state should be
kept, irrespective of whether it is the oldest program state or not.
This is desirable in settings where you would like to ensure full
reproducibility of the simulation, especially in settings where model
weights or optimizer states are initialized randomly. By loading from
the initial program state, one can avoid re-initializing and obtaining
different results.
keep_every_k
An integer representing how often program states should be
kept. The latest version will always be kept. Defaults to 1. Even when
keep_total is zero or negative, this setting will still be applied. To
keep all states, set this to 1.
A integer representing the version of a saved program state.
structure
The structure of the saved program state for the given
version used to support serialization and deserialization of
user-defined classes in the structure.
Raises
ProgramStateNotFoundError
If there is no program state for the given
version.
Returns the latest saved program state and version or (None, 0).
Args
structure
The structure of the saved program state for the given
version used to support serialization and deserailization of
user-defined classes in the structure.
Returns
A tuple of the latest saved (program state, version) or (None, 0) if
there is no latest saved program state.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Missing the information I need","missingTheInformationINeed","thumb-down"],["Too complicated / too many steps","tooComplicatedTooManySteps","thumb-down"],["Out of date","outOfDate","thumb-down"],["Samples / code issue","samplesCodeIssue","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2024-09-20 UTC."],[],[],null,["# tff.program.FileProgramStateManager\n\n\u003cbr /\u003e\n\n|-------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| [View source on GitHub](https://github.com/tensorflow/federated/blob/v0.87.0 Version 2.0, January 2004 Licensed under the Apache License, Version 2.0 (the) |\n\nA [`tff.program.ProgramStateManager`](../../tff/program/ProgramStateManager) that is backed by a file system.\n\nInherits From: [`ProgramStateManager`](../../tff/program/ProgramStateManager) \n\n tff.program.FileProgramStateManager(\n root_dir: Union[str, os.PathLike[str]],\n prefix: str = 'program_state_',\n keep_total: int = 5,\n keep_first: bool = True,\n keep_every_k: int = 1\n )\n\nA [`tff.program.FileProgramStateManager`](../../tff/program/FileProgramStateManager) is a utility for saving and loading\nprogram state to a file system in a federated program and is used to implement\nfault tolerance. In particular, it is intended to only restart the same\nsimulation and run with the same version of TensorFlow Federated.\n\nProgram state is saved to the file system using the SavedModel (see\n[`tf.saved_model`](https://www.tensorflow.org/api_docs/python/tf/saved_model)) format. When the program state is saved, each\n[`tff.program.MaterializableValueReference`](../../tff/program/MaterializableValueReference) is materialized and each\n`tff.Serializable` is serialized. The structure of the program state is\ndiscarded, but is required to load the program state.\n| **Note:** The SavedModel format can only contain values that can be converted to a [`tf.Tensor`](https://www.tensorflow.org/api_docs/python/tf/Tensor) (see [`tf.convert_to_tensor`](https://www.tensorflow.org/api_docs/python/tf/convert_to_tensor)), releasing any other values will result in an error.\n\nSee \u003chttps://www.tensorflow.org/guide/saved_model\u003e for more information about\nthe SavedModel format.\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n| Args ---- ||\n|----------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| `root_dir` | A path on the file system to save program state. If this path does not exist it will be created. |\n| `prefix` | A string to use as the prefix for filenames. |\n| `keep_total` | An integer representing the total number of program states to keep. If the value is zero or smaller, there will be no limitation on how many program states to keep; if keep_every_k is 1, then all states will be kept. |\n| `keep_first` | A boolean indicating if the first program state should be kept, irrespective of whether it is the oldest program state or not. This is desirable in settings where you would like to ensure full reproducibility of the simulation, especially in settings where model weights or optimizer states are initialized randomly. By loading from the initial program state, one can avoid re-initializing and obtaining different results. |\n| `keep_every_k` | An integer representing how often program states should be kept. The latest version will always be kept. Defaults to 1. Even when keep_total is zero or negative, this setting will still be applied. To keep all states, set this to 1. |\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n| Raises ------ ||\n|--------------|-----------------------------------|\n| `ValueError` | If `root_dir` is an empty string. |\n\n\u003cbr /\u003e\n\nMethods\n-------\n\n### `get_versions`\n\n[View source](https://github.com/tensorflow/federated/blob/v0.87.0\nVersion 2.0, January 2004\nLicensed under the Apache License, Version 2.0 (the) \n\n get_versions()\n\nReturns a list of saved versions or `None`.\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n| Returns ||\n|---|---|\n| A list of saved versions or `None` if there is no saved program state. ||\n\n\u003cbr /\u003e\n\n### `load`\n\n[View source](https://github.com/tensorflow/federated/blob/v0.87.0\nVersion 2.0, January 2004\nLicensed under the Apache License, Version 2.0 (the) \n\n load(\n version, structure\n ) -\u003e ../../tff/program#ProgramStateStructure\n\nReturns the program state for the given `version`.\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n| Args ||\n|-------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| `version` | A integer representing the version of a saved program state. |\n| `structure` | The structure of the saved program state for the given `version` used to support serialization and deserialization of user-defined classes in the structure. |\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n| Raises ||\n|-----------------------------|-------------------------------------------------------|\n| `ProgramStateNotFoundError` | If there is no program state for the given `version`. |\n\n\u003cbr /\u003e\n\n### `load_latest`\n\n[View source](https://github.com/tensorflow/federated/blob/v0.87.0\nVersion 2.0, January 2004\nLicensed under the Apache License, Version 2.0 (the) \n\n load_latest(\n structure\n )\n\nReturns the latest saved program state and version or (`None`, 0).\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n| Args ||\n|-------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| `structure` | The structure of the saved program state for the given `version` used to support serialization and deserailization of user-defined classes in the structure. |\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n| Returns ||\n|---|---|\n| A tuple of the latest saved (program state, version) or (`None`, 0) if there is no latest saved program state. ||\n\n\u003cbr /\u003e\n\n### `remove_all`\n\n[View source](https://github.com/tensorflow/federated/blob/v0.87.0\nVersion 2.0, January 2004\nLicensed under the Apache License, Version 2.0 (the) \n\n remove_all()\n\nRemoves all program states.\n\n### `save`\n\n[View source](https://github.com/tensorflow/federated/blob/v0.87.0\nVersion 2.0, January 2004\nLicensed under the Apache License, Version 2.0 (the) \n\n save(\n program_state, version\n )\n\nSaves `program_state` for the given `version`.\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n| Args ||\n|-----------------|-------------------------------------------------------------------------------------------|\n| `program_state` | A [`tff.program.ProgramStateStructure`](../../tff/program#ProgramStateStructure) to save. |\n| `version` | A strictly increasing integer representing the version of a saved `program_state`. |\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n| Raises ||\n|---------------------------|------------------------------------------------------------|\n| `ProgramStateExistsError` | If there is already program state for the given `version`. |\n\n\u003cbr /\u003e"]]
View source on GitHub