Critical Data Loss: Large files silently truncated when project exceeds 50MB limit

[hanyan009]

Version 5.5.4

Bug Description

When a project's total size exceeds an apparent 50MB limit, large files within the project are silently truncated to 9 bytes. When downloaded, these files only contain the text "Not Found". This leads to irreversible data loss for that version, with no warning given to the user.

The only way to recover is to revert to a previous version from the project history.

Steps to Reproduce

• Create a project and upload files until its size is just under 50MB.
• Upload one more file that pushes the total size over the 50MB threshold.
• Download the project.
• Observe that larger files have been corrupted and are now 9 bytes in size.
• Expected Behavior
• The system should reject any action (e.g., a file upload) that causes the project to exceed the size limit and display a clear error message. User data should never be silently altered or deleted.

Actual Behavior

Existing files are silently corrupted to enforce a size limit, causing critical data loss.

Suggestions

Prevent Data Corruption: Immediately block actions that exceed the size limit instead of truncating files. A clear warning message is essential.
Make the Limit Configurable: The 50MB limit should be an adjustable setting, especially for self-hosted instances, to prevent this issue and allow for more flexibility.

[hanyan009]

I'm not sure if this file truncation was caused by the project exceeding 50MB, by the project being inactive for three days, or a combination of both factors.

[mlevans0]

Hey @hanyan009, I'm sorry to hear you're having this trouble. I've taken an initial look at this and so far haven't been able to recreate this report. As a quick check, can you confirm that you can consistently recreate this and that you have plenty of free disk space? If you're able to recreate the issue with the Frog project and share that with us that would help with our investigation (you can omit the binary files from the zip download -- we can add ones in afterwards -- just let us know the sizes/types).

Note: The 50MB limit is per individual file and not the whole project.

To help identify the issue, please can you provide us with either a copy of your docker-compose.yml file or the output from running bin/doctor, along with a zip or tar of the contents of the /config folder if you are using the Toolkit. This will help us check your configuration.

Important: Please make sure that all sensitive information is redacted from the configuration files, including removing any certificates, before sharing them here.

Please also provide us with redacted copies of the following log files:

• A copy of the web.log. You can get a copy of this log from the container by running this command:
  docker cp sharelatex:/var/log/overleaf/web.log web.log
• A copy of the project-history.log. You can get a copy of this log from the container by running this command:
  docker cp sharelatex:/var/log/overleaf/project-history.log project-history.log
• A copy of the history-v1.log. You can get a copy of this log from the container by running this command:
  docker cp sharelatex:/var/log/overleaf/history-v1.log history-v1.log
• A copy of the docstore.log. You can get a copy of this log from the container by running this command:
  docker cp sharelatex:/var/log/overleaf/docstore.log docstore.log
• A copy of the filestore.log. You can get a copy of this log from the container by running this command:
  docker cp sharelatex:/var/log/overleaf/filestore.log filestore.log

Note: When you send us copies of the log files, please make sure they cover and are limited to the period during which the issue occurred or was recreated, and that you remove any personal or sensitive information.

Once we have this information, we can look into the issue for you and provide some next steps.

[hanyan009]

Thanks for the prompt reply. I'm finding it difficult to reproduce the issue consistently. I'll spend more time on it and will report back once I can pinpoint the specific trigger. Thanks again!