上传115遇到409错误, 重传会报相同错误

[fakedon]

Failed to copy: failed to upload: failed to upload chunk 4 with 268435456 bytes: operation error UploadPart: Error returned by Service.
Http Status Code: 409.
Error Code: PartAlreadyExist.
Request Id: xxxxxxxxx.
Message: For sequential multipart upload, you can't overwrite uploaded parts..
EC: 0042-00000108.

[wiserain]

Haven't experienced this issue before. Is this happening consistently or intermittently? How can I reproduce (command or system environment)?

[fakedon]

ubuntu 22
命令:
rclone copy drive: 115: -Pv --cache-dir=cache --transfers=5 --115-chunk-size=128Mi --ignore-existing --115-pacer-min-sleep=5s --fast-list --drive-acknowledge-abuse --drive-stop-on-download-limit

文件大小20G以下, 5G以上, 不是经常发生, 最近1000文件遇到几十个这样的, 如何复现我也弄不清楚

[wiserain]

Let me understand your reply. Files larger than 5G or smaller than 20G are okay? What is the problematic file size/range of it?

Btw --ignore-existing might be the problem. Would you try without --ignore-existing?

[fakedon]

出问题的文件看不出规律, 由于我这边的文件都是5G到20G的范围, 其他大小的文件未测试

我用不带 --ignore-existing的命令测试下看看结果

[fakedon]

怎么改参数还是会遇到409的问题

rclone copy drive: 115: -Pv --cache-dir=cache --transfers=5 --115-chunk-size=128Mi --drive-acknowledge-abuse --drive-stop-on-download-limit --no-check-dest

https://musetransfer.com/s/lukr349rh
提供一个文件, 如果有空可以测试一下, 谢谢了

[wiserain]

Can you provide me a file, but I suspect it's not because of source...

[fakedon]

我需要等再次遇到才能提供文件
目前来看跟大小没关系, 几百兆的也会遇到409
遇到409之后, 只有这个文件在一定时间内重传会一直报409错误, 具体时间未知
其他文件不受影响

不知道下面这个issue有没有帮助
tickstep/aliyunpan#466

另外问了gpt, 这是给的解答

The error indicates that during a multipart upload to an OSS (Object Storage Service) bucket, the upload failed for part number 7 because the part already exists. In OSS, you cannot overwrite a part in a sequential multipart upload. Here's how to resolve this:

Steps to Fix

1. Verify the Multipart Upload State:

   • Check the parts that have already been uploaded for the given uploadId. You can use the ListParts API to get a list of uploaded parts:

     GET https://fhnfile.oss-cn-shenzhen.aliyuncs.com/xxxxxxxxxxxxxxxxxxxxx?uploadId=00000000000000
     

2. Resume Upload from the Next Part:

   • Once you have verified which parts are already uploaded, start uploading from the next missing part. Do not re-upload any parts that already exist.

3. Abort the Multipart Upload (if necessary):

   • If the upload sequence is broken or incorrect, abort the multipart upload to start over:

     DELETE https://fhnfile.oss-cn-shenzhen.aliyuncs.com/xxxxxxxxxxxxxxxxxxxxx?uploadId=00000000000000
     

4. Re-initiate and Upload Correctly:

   • Re-initiate the multipart upload process and ensure parts are uploaded sequentially without overwriting any uploaded parts.

5. Check for Errors in Upload Logic:

   • Ensure that your upload script or logic properly handles part numbers and does not attempt to re-upload existing parts.

Let me know if you'd like further clarification on any of these steps!

[wiserain]

Could you please try with this release? https://github.com/wiserain/rclone/releases/tag/v1.69.0-beta.9132.4d6488f24

[fakedon]

Could you please try with this release? https://github.com/wiserain/rclone/releases/tag/v1.69.0-beta.9132.4d6488f24

还是会出现409

2025/01/15 04:04:04 ERROR : abc2222.mp4: Failed to copy: failed to upload: failed to upload chunk 4 with 268435456 bytes: operation error UploadPart: Error returned by Service.
Http Status Code: 409.
Error Code: PartAlreadyExist.
Request Id: xxxx.
Message: For sequential multipart upload, you can't overwrite uploaded parts..
EC: 0042-00000108.
Timestamp: 2025-01-14 20:04:00 +0000 UTC.
Request Endpoint: PUT https://fhnfile.oss-cn-shenzhen.aliyuncs.com/xxxx?partNumber=4&uploadId=xxxx.

[wiserain]

Questions

1. Based on your error logs, the error is observed only on chunk 4. Is this accurate?
2. Does this issue occur only with Google Drive as the source? What about local sources or other cloud services?

Please provide full error logs (from command input till the end of process termination) with debugging option -vv.