Refactor setup.py to support editable mode

[larryliu0820]

🚀 The feature, motivation and pitch

Blocking #2871

Context:

The current pattern of setup.py in ExecuTorch:

• Build extensions in CustomBuild class (extending build). Run CMake build commands to build pybind extensions. Triggered from cmdclass["build"]
• After extensions are built, run InstallerBuildExt class to install the extension artifacts (.so etc). The class takes a list of ext_modules to learn what is the source path (normally .so files in cmake-out/) and what is the destination path in the package. ext_modules list is also controlled by the same CMake flags for custom build. This step is triggered from cmdclass["build_ext"]
• Lastly, setup.py runsbuild_pyto copy python files to package directories. This step is triggered from cmdclass["build_py"].

This pattern is different from the common pattern, but it works without issue.

Assumption:

We want to support editable mode (pip install -e .). In editable installation we should perform the exact same installation steps as normal mode, respecting the same options as normal mode.

For example, the plain command

pip install -e .

Should install portable_lib extension because by default EXECUTORCH_BUILD_PYBIND=ON

Problem:

The current pattern doesn’t work for editable mode. Here are the reasons:

1. The build step doesn’t even run when editable mode is on. This is causing build_ext to error out because it’s expecting the .so files to be present in the build directory but they are missing.
2. We don’t want to diverge the installation behavior between editable mode on and off as much as possible. This means we don’t want to do if editable mode on do A otherwise do B.

Proposal:

Move the python extension installation step (CMake commands) into one of ext_modules.

Alternatives

No response

Additional context

No response

RFC (Optional)

No response

cc @mergennachin @byjlw @SS-JIA @manuelcandales

[larryliu0820]

cc @dbort

[dbort]

Move the python extension installation step (CMake commands) into one of ext_modules

I'd like to find a different solution if possible, because the build step is where this sort of thing should happen; or at least some of it, like the piece that builds flatc. And doing that requires configuring the cmake build beforehand, which ideally we'd only do once.

We could have a function like "configure cmake if it hasn't already happened" and just call it at the beginning of every build/build_ext step. Then a given build_ext could build with just its own --target.

The build step doesn’t even run when editable mode is on.

Another possibility would be to make the build_ext steps depend on the build step, causing it to happen if it hasn't already. I don't remember the specifics, but I seem to remember that this is possible by making a given Command call another Command.

Also, along the lines of editable mode and the build step, we should look into this note I left:

executorch/setup.py

Lines 509 to 511 in 85231f5

	# TODO(dbort): For editable wheels, may need to update get_source_files(),
	# get_outputs(), and get_output_mapping() to satisfy
	# https://setuptools.pypa.io/en/latest/userguide/extension.html#setuptools.command.build.SubCommand.get_output_mapping

[byjlw]

@dbort I don't have an opinions on how this problem is solved but it should be clear to users what steps they should take so that any changes they make are immediately reflected and debug able without the need for a re-install.

I can't tell if your approach solves that.

[dbort]

@byjlw nothing here should affect that. Editable mode is only relevant for python files, but this issue is about binaries, which are not auto-updated in any form of editable mode without a rebuild.

[mergennachin]

@larryliu0820 - given @dbort 's feedback, how do you want to proceed?

[larryliu0820]

make the build_ext steps depend on the build step, causing it to happen if it hasn't already.

I think this can be a viable solution. I'll give it a try and report back.