Command Line String Limitation on Windows

[Alfystar]

Bug Description:

On Windows-based systems, there exists a limitation on the number of characters for a command in the terminal, approximately 8191 characters source information.

Issue Background:

In the past, I mitigated this limitation by dividing commands into sub-blocks and concatenating the results. However, with the suspension of ANT usage, executing substantial deployments and validating them is no longer possible using the same approach.

Proposed Solution:

I suggest integrating a new mode for providing parameters to the tool through a standard text file. This file would contain all the parameters that would have been entered on the terminal. Alternatively, allowing the tool to receive argv from stdin would also be a viable solution.

Expected result

The tool should handle commands that surpass the character limit by either accepting parameters from a text file or receiving argv from stdin.

Actual result

The current limitation hinders the ability to perform large-scale deployments and validations on Windows systems, affecting the overall usability and functionality of the tool.

System Information

{
  "architecture": "darwin-arm64",
  "cliVersion": "@salesforce/cli/2.23.20",
  "nodeVersion": "node-v20.10.0",
  "osVersion": "Darwin 23.2.0",
  "rootPath": "/Users/ea_enel/.local/share/sf/client/2.23.20-50cd2d8",
  "shell": "zsh",
  "pluginVersions": [
    "@oclif/plugin-autocomplete 3.0.4 (core)",
    "@oclif/plugin-commands 3.1.0 (core)",
    "@oclif/plugin-help 6.0.9 (core)",
    "@oclif/plugin-not-found 3.0.7 (core)",
    "@oclif/plugin-plugins 4.1.14 (core)",
    "@oclif/plugin-search 1.0.10 (core)",
    "@oclif/plugin-update 4.1.7 (core)",
    "@oclif/plugin-version 2.0.11 (core)",
    "@oclif/plugin-warn-if-update-available 3.0.8 (core)",
    "@oclif/plugin-which 3.0.14 (core)",
    "@salesforce/cli 2.23.20 (core)",
    "apex 3.0.14 (core)",
    "auth 3.1.0 (core)",
    "data 3.0.13 (core)",
    "deploy-retrieve 2.2.12 (core)",
    "info 3.0.13 (core)",
    "limits 3.1.0 (core)",
    "marketplace 1.0.15 (core)",
    "org 3.2.0 (core)",
    "packaging 2.0.1 (core)",
    "schema 3.1.0 (core)",
    "settings 2.0.14 (core)",
    "sobject 1.0.7 (core)",
    "source 3.0.10 (core)",
    "telemetry 3.1.8 (core)",
    "templates 56.0.8 (core)",
    "trust 3.2.5 (core)",
    "user 3.2.0 (core)",
    "sfdx-plugin-source-read 1.2.0 (user)"
  ]
}

Additional information

This limitation is particularly impactful in scenarios where extensive deployments and validations are necessary. The proposed solutions would enhance the tool's versatility and usability on Windows platforms.

Thank you for considering this bug report. Let me know if you need any further information or clarification.

Sincerely,
Emanuele

[github-actions]

Thank you for filing this issue. We appreciate your feedback and will review the issue as soon as possible. Remember, however, that GitHub isn't a mechanism for receiving support under any agreement or SLA. If you require immediate assistance, contact Salesforce Customer Support.

[mshanemc]

can you give an example of what you're doing that needs really long commands?

[Alfystar]

In my organization we use to create the Salesforce package to deploy by recovering the sources modified from git during a release, generally a couple of months, we feed all these paths to:

sf project generate manifest --source-dir ...
And then we use the created manifests to deploy/validate on a desired org using ANT.

On Windows, even with about twenty resources you exceed the character limit. And during a release we work with hundreds, sometimes thousands of resources.

We normally use a Linux-based pipeline (bamboo from atlassian), and we don't have such stringent character limits. But recently the need to run from Windows has arisen, so... here I am.

Furthermore, from now on, in addition to the sources, you must also specify the tests to run in the command write in the CLI, whereas previously with ant it could be saved in an xml file, which as you can imagine are also hundreds, which doubles the problem.

From now on, however, the deployment must take place using the sf-cli, and it follows that, even if the manifest is generated first by concatenating the pieces, the run of the test classes must be completely specified in the single command, otherwise it is impossible to deploy the package totally, and not for an ORG limit, but for a stupid terminal limit.

[shetzel]

Maybe you could use this CLI plugin instead which does a similar thing without hitting character limits: https://github.com/scolladon/sfdx-git-delta#readme

jsyk, there are lots of cool plugins like this you can discover by running the CLI command, sf plugins discover

I realize I'm not addressing your issue directly but it might be faster/easier to switch to an existing solution or make other modifications rather than request a new feature. It's possible that even if the CLI supported reading lots of paths or test names from a file that there would be other limits hit like URL when making the metadata API request.

[Alfystar]

We have developed similar solutions locally, but written in python, which take advantage of the standard and officially supported commands of salesforce.

Is it so complex to read argv from the contents of a file?

[shetzel]

Is it so complex to read argv from the contents of a file?

No but new features (which is what this is) are subject to prioritization so I'm suggesting alternatives because I'm not sure when it would be picked up. And like I said, even reading from a file might hit other limits.

[Alfystar]

I don't know what to say about it, it's a limitation that has been present for a while and is now starting to be felt.

The alternative would be to provide for all options that I take a list as input, accept a file containing an element in each line, or whatever. However, I believe that this second option, at least in my opinion, is more complex to implement and is much more impactful.

As for the limits in reading a file, that depends only on the architecture of the OS. And in any case, I highly doubt that we will be able to create commands that exceed 4 Giga characters (for old 32-bit architectures)

In any case, we have waited a long time, surely waiting a little longer will not be the end of the world, but it would be nice to put an end to this limit that is making itself felt more and more time.

[shetzel]

As for the limits in reading a file, that depends only on the architecture of the OS. And in any case, I highly doubt that we will be able to create commands that exceed 4 Giga characters (for old 32-bit architectures)

I'm not talking about file reading limits. By "other limits" I meant URI or any limits enforced by the Metadata API (soap or rest), which is what the CLI uses for deploys and retrieves. Regardless, I am only providing alternatives for consideration while we prioritize this with all other work.

[Alfystar]

Yes, I understand perfectly, in fact forgive me if I have been a little abrupt.

I believe that the limits, at the API level, are the same as currently present with ANT, about 10000 resources, and losing this number is not too problematic (except for some backup scenarios)
The limit on the terminal is certainly much more impactful.