mirror update lacks a way to continue when package import fails due to missing files

[StefanRickli]

Our environment enforces the use of a malware scanner (Microsoft Defender Advanced Threat Protection).

During aptly mirror update, the scanner may immediately delete freshly downloaded .deb files if they are deemed problematic. This happens after the download completes but before aptly imports the file into its pool/db.

When this occurs, aptly aborts the entire mirror update with an error like:

Downloading: http://ch.archive.ubuntu.com/ubuntu/pool/universe/e/ettercap/ettercap-common_0.8.2-10build4_amd64.deb
[...]
ERROR: unable to import file: open /srv/aptly/.aptly/pool/fc/49/6661-493f-4245-b878-5cd1565065daettercap-common_0.8.2-10build4_amd64.deb: no such file or directory

(There are several such packages per distro and pocket)

Minimal reproduction

1. Start a mirror update:
   sudo -u aptly aptly mirror update bionic-orig-main-test
2. Some .deb are downloaded successfully.
3. The problematic .deb file is being downloaded.
4. An external malware scanner deletes the file immediately upon download completion.
5. aptly attempts to import the file and aborts the update.

Observed behavior

• The mirror update stops at the first such incident.
• The order of package downloads appears nondeterministic.
• As a result, repeated invocations of aptly mirror update stop at different points, leaving a large and unpredictable set of packages not downloaded.
• There is currently no way to:
  • continue the update
  • downgrade this specific failure to a warning
  • complete the rest of the download queue

Expected / desired behavior

In environments where files can be externally removed (malware scanners, filesystem policies, etc.), it would be useful if aptly could:

• Continue the mirror update when a downloaded package file is missing at import time
• Report the missing packages at the end of the run (or mark them as failed)
• Avoid aborting the entire update for a single missing file

This could be implemented, for example, as:

• a flag such as --continue-on-import-error
• or a mirror option to treat missing package files as non-fatal

Rationale

In our case, excluding the aptly pool directory from malware scanning is not an option. From a security standpoint, it is acceptable (and even desirable) that certain packages never make it into the mirror. However, the current behavior forces us to repeatedly re-run mirror updates until “by chance” all non-blocked packages are downloaded.

This makes it difficult to maintain a consistent and predictable mirror state in hardened environments.

The current behavior also makes it impossible to detect which packages are systematically blocked by the scanner, because the update stops before the full download queue is processed.