-
Notifications
You must be signed in to change notification settings - Fork 153
Adding usenet support (Torbox only implementation) #911
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
PR Compliance Guide 🔍Below is a summary of compliance checks for this PR:
Compliance status legend🟢 - Fully Compliant🟡 - Partial Compliant 🔴 - Not Compliant ⚪ - Requires Further Human Verification 🏷️ - Compliance label |
|||||||||||||||||||||||||||||||
PR Code Suggestions ✨Explore these optional code suggestions:
|
||||||||||||||||||||||||
…ting requirements
- Run immediately on entry (good for tests) - Enforces min 10 seconds on loop to prevent high CPU usage on inadvertent low values, delays for the intended wait time as well.
|
Qodo has hallucinated the issue "Ensure NZB files are deleted", it works fine, and the recommendation isn't documented anywhere in TorBox's SDKs or API docs. It's not a valid parameter in that function. |
…ry large amounts of uneeded data through the websocket.
|
@omgbeez I tried this changeset using your fork (version 0.0.17), and it doesn't seem to handle the status messages related with unpack, like the following: How about relaxing the error detection logic a little bit so that it is OK with unpack/verify/recover progress messages? |
|
Tried it as well. The usenet download finished at Torbox but the client only said:
and never managed to continue. There are still a few strings that needs to be changed from torrent to download as well. Didn't write them down but they are there. |
|
@trustin @simonwahlstrom Please note that my test branch has a LOT of other changes I'm testing at the same time, many of which are not part of this PR.
I saw this for the first time last night. I think it's a red herring and the error was probably triggered due to a bug I had in handling ACTIVE_LIMIT responses. I'll investigate if I see it again though.
This isn't actually an error, it shows as one because the mapping of statuses from TorBox to RDTC is incomplete, and not documented, so it falls back to an error status. This is related to a feature I was testing (and disabled this morning) to queue downloads on TB instead of failing to add them when all your download slots are full. TB can be VERY slow at resuming a queued download, that's why I disabled it. Their docs claim an hour for their backend to check queued jobs and start them. If you re-run the update script, latest version is 0.0.17 and should address both problems. |
To be clear, the error status checks are not my code.... but yeah, I noticed this too, and unfortunately the codes are not documented for usenet so there will be some manual discovery to figure them out and improve it |
…rogress calculations
User description
This is an initial version of RDTClient that expands to support usenet via URL or NZB file.
It's possible to test in a ProxMox LXC (or update to this version) using:
bash -c "$(curl -fsSL https://raw.githubusercontent.com/omgbeez/ProxmoxVE/main/ct/rdtclient.sh)"WARNING: ^^ If you do test, this will apply a DB migration that cannot be reverted. Snaphot your current version in case you want to go back to the proper release.
CLOSES #612
PR Type
Enhancement, Tests
Description
Renamed
ITorrentClienttoIDebridClientacross all implementations (AllDebrid, RealDebrid, Premiumize, DebridLink, TorBox) to better reflect support for multiple download types beyond torrentsAdded comprehensive NZB/Usenet support with new methods
AddNzbFileToDebridQueue()andAddNzbLinkToDebridQueue()in the Torrents serviceImplemented SABnzbd API emulator (
Sabnzbdservice andSabnzbdController) for seamless integration with ARR stack (Prowlarr, Sonarr, Radarr) using same port and authentication as qBittorrentAdded
DownloadTypeenum to distinguish between torrent and NZB downloads, with database migration to addTypecolumn to Torrents tableImplemented watch folder support for
.nzbfiles inWatchFolderCheckerbackground serviceAdded SABnzbd authorization handler (
SabnzbdHandler) with support forma_usernameandma_passwordauthenticationUpdated qBittorrent API filtering to exclude NZB downloads and only expose torrent downloads
Added new Web UI endpoints
UploadNzbFileandUploadNzbLinkto TorrentsController for NZB submissionCreated comprehensive test suite with 11+ new test classes covering Sabnzbd service, TorBox client, watch folder checker, NZB operations, and authorization handling
Added utility helpers including
FileSizeHelperfor human-readable byte formatting and hash computation methodsCreated SABnzbd data models (Queue, History, Config, Category, etc.) for API response serialization
Diagram Walkthrough
File Walkthrough
29 files
Torrents.cs
Rename TorrentClient to DebridClient and add NZB supportserver/RdtClient.Service/Services/Torrents.cs
ITorrentClienttoIDebridClientand updated all related classnames and namespaces
AddNzbFileToDebridQueueandAddNzbLinkToDebridQueueDownloadTypeenum parameter to distinguish between torrent andNZB downloads
CopyAddedTorrentmethod to useTorrentobject instead ofseparate parameters
ComputeMd5Hash,ComputeMd5HashFromBytes,ComputeSha1Hash)DequeueFromDebridQueueto handle both torrent and NZB typeswith appropriate API calls
virtualto support testing and mockingTorBoxDebridClient.cs
Add NZB support to TorBox debrid client implementationserver/RdtClient.Service/Services/DebridClients/TorBoxDebridClient.cs
TorBoxTorrentClienttoTorBoxDebridClientandupdated interface from
ITorrentClienttoIDebridClientdownloads to support testing
Mapmethod overload forUsenetInfoResultto handle NZB downloadswith
DownloadType.NzbGetTorrentstoGetDownloadsand updated to return both torrentand NZB results
AddNzbLinkandAddNzbFileGetAvailableFilesto check both torrent and usenetavailability
DeleteandUnrestrictmethods to acceptTorrentobject andhandle both types
GetDownloadInfosto handle NZB downloads with separate logicfor fetching IDs
Sabnzbd.cs
Implement SABnzbd API emulator serviceserver/RdtClient.Service/Services/Sabnzbd.cs
Sabnzbdservice class implementing SABnzbd API emulation for NZBdownloads
GetQueueto return active NZB downloads with progress andtime-left calculations
GetHistoryto return completed NZB downloads with filepaths
AddFileandAddUrlmethods for adding NZB files and linksGetConfigandGetCategoriesfor SABnzbd compatibilityAllDebridDebridClient.cs
Rename AllDebrid client and update method signaturesserver/RdtClient.Service/Services/DebridClients/AllDebridDebridClient.cs
AllDebridTorrentClienttoAllDebridDebridClientandupdated interface
TorrentClientstoDebridClientsGetTorrentstoGetDownloadsAddMagnettoAddTorrentMagnetandAddFiletoAddTorrentFileAddNzbLinkandAddNzbFilethat throwNotSupportedExceptionDeleteandUnrestrictmethod signatures to acceptTorrentobject
TorrentClient*type references toDebridClient*RealDebridDebridClient.cs
Rename RealDebrid client and update method signaturesserver/RdtClient.Service/Services/DebridClients/RealDebridDebridClient.cs
RealDebridTorrentClienttoRealDebridDebridClientand updated interface
TorrentClientstoDebridClientsGetTorrentstoGetDownloadsAddMagnettoAddTorrentMagnetandAddFiletoAddTorrentFileAddNzbLinkandAddNzbFilethat throwNotSupportedExceptionDeleteandUnrestrictmethod signatures to acceptTorrentobject
TorrentClient*type references toDebridClient*TorrentsController.cs
Add NZB upload endpoints to torrents controllerserver/RdtClient.Web/Controllers/TorrentsController.cs
TorrentClienttoDebridClientUploadNzbFileto handle NZB file uploads with formdata
UploadNzbLinkto handle NZB URL submissionsappropriate service methods
VerifyRegexto useDebridClientAvailableFiletypeTorrentControllerUploadNzbLinkRequestclass for NZB linkrequests
DebridLinkTorrentClient.cs
Refactor debrid client interface and add NZB support stubsserver/RdtClient.Service/Services/DebridClients/DebridLinkTorrentClient.cs
TorrentClientstoDebridClientsand updatedinterface from
ITorrentClienttoIDebridClientGetTorrents()toGetDownloads()and return type fromTorrentClientTorrenttoDebridClientTorrentAddMagnet()andAddFile()toAddTorrentMagnet()andAddTorrentFile()respectivelyAddNzbLink()andAddNzbFile()that throwNotSupportedExceptionDelete()andUnrestrict()method signatures to acceptTorrentobject instead of string ID
PremiumizeDebridClient.cs
Rename Premiumize client and implement debrid interface changesserver/RdtClient.Service/Services/DebridClients/PremiumizeDebridClient.cs
PremiumizeTorrentClienttoPremiumizeDebridClientand updated namespace
ITorrentClienttoIDebridClientGetTorrents()toGetDownloads()and updated return types touse
DebridClient*classesAddTorrentMagnet()andAddTorrentFile()AddNzbLink()andAddNzbFile()that throwNotSupportedExceptionDelete()andUnrestrict()to acceptTorrentobjectsSabnzbdController.cs
Add SABnzbd API emulation controller for usenet integrationserver/RdtClient.Web/Controllers/SabnzbdController.cs
endpoints
version,queue,history,get_config,get_cats,addurl,addfile, andfullstatusmodesSabnzbdRequirementpolicySabnzbdservice for business logicQBittorrent.cs
Filter qBittorrent API to exclude NZB downloadsserver/RdtClient.Service/Services/QBittorrent.cs
TorrentInfo()to only return torrents withDownloadType.TorrentTorrentsTopPrio()to validate torrent type before processingWatchFolderChecker.cs
Add NZB file watch folder supportserver/RdtClient.Service/BackgroundServices/WatchFolderChecker.cs
.nzbfile extension in watch folder monitoringAddNzbFileToDebridQueue()IDebridClient.cs
Refactor debrid client interface for torrent and NZB supportserver/RdtClient.Service/Services/DebridClients/IDebridClient.cs
ITorrentClienttoIDebridClientand updatednamespace
GetTorrents()toGetDownloads()and updated return typesAddTorrentMagnet(),AddTorrentFile(),AddNzbLink(), andAddNzbFile()Delete()andUnrestrict()to acceptTorrentobjectsUpdateData()parameter type toDebridClientTorrentTorrent.cs
Add download type property to torrent modelserver/RdtClient.Data/Models/Data/Torrent.cs
Typeproperty of typeDownloadTypeto distinguish betweentorrents and NZBs
Filesproperty to useDebridClientFileinstead ofTorrentClientFileTorrentClienttoDebridClientSabnzbdHandler.cs
Add SABnzbd authorization handler middlewareserver/RdtClient.Service/Middleware/SabnzbdHandler.cs
SabnzbdRequirementauthorization requirementma_usernameandma_passwordparametersTorrentData.cs
Add download type parameter and hash update methodserver/RdtClient.Data/Data/TorrentData.cs
downloadTypeparameter toAdd()method signatureTypeproperty on new torrent entityUpdateHash()method to update torrent hashDebridClientTorrent.cs
Rename torrent client model to debrid client torrentserver/RdtClient.Data/Models/DebridClient/DebridClientTorrent.cs
TorrentClientTorrenttoDebridClientTorrentTorrentClienttoDebridClientTypeproperty with default valueDownloadType.TorrentFilesproperty type toDebridClientFileSabnzbdQueueSlot.cs
Add SABnzbd queue slot data modelserver/RdtClient.Data/Models/Sabnzbd/SabnzbdQueueSlot.cs
category, time left, and priority
DownloadClient.cs
Update download client for debrid client namingserver/RdtClient.Service/Services/DownloadClient.cs
AllDebridTorrentClienttoAllDebridDebridClientTorrentClientstoDebridClientsSabnzbdResponse.cs
Add SABnzbd response data modelserver/RdtClient.Data/Models/Sabnzbd/SabnzbdResponse.cs
IDs, config, and categories
SabnzbdQueue.cs
Add SABnzbd queue status data modelserver/RdtClient.Data/Models/Sabnzbd/SabnzbdQueue.cs
slots
SabnzbdCategory.cs
Add SABnzbd category configuration modelserver/RdtClient.Data/Models/Sabnzbd/SabnzbdCategory.cs
script
SabnzbdHistorySlot.cs
Add SABnzbd history slot data modelserver/RdtClient.Data/Models/Sabnzbd/SabnzbdHistorySlot.cs
storage path
FileSizeHelper.cs
Add file size formatting helper utilityserver/RdtClient.Service/Helpers/FileSizeHelper.cs
SabnzbdModeAttribute.cs
Add SABnzbd mode routing attributeserver/RdtClient.Web/Controllers/SabnzbdModeAttribute.cs
requests
UnpackClient.cs
Update unpack client for debrid client namingserver/RdtClient.Service/Services/UnpackClient.cs
TorBoxTorrentClienttoTorBoxDebridClientTorrentClientstoDebridClientsSabnzbdMisc.cs
Add SABnzbd miscellaneous configuration modelserver/RdtClient.Data/Models/Sabnzbd/SabnzbdMisc.cs
and version
SabnzbdConfig.cs
Add SABnzbd configuration data modelserver/RdtClient.Data/Models/Sabnzbd/SabnzbdConfig.cs
SabnzbdHistory.cs
Add SABnzbd history status data modelserver/RdtClient.Data/Models/Sabnzbd/SabnzbdHistory.cs
Authentication.cs
Make authentication login method virtualserver/RdtClient.Service/Services/Authentication.cs
Login()method virtual to allow mocking in tests11 files
AllDebridDebridClientTest.cs
Update AllDebrid client tests for renamed classserver/RdtClient.Service.Test/Services/TorrentClients/AllDebridDebridClientTest.cs
AllDebridTorrentClientTesttoAllDebridDebridClientTestAllDebridTorrentClienttoAllDebridDebridClientGetTorrentstoGetDownloadsSabnzbdTest.cs
Add Sabnzbd service unit testsserver/RdtClient.Service.Test/Services/SabnzbdTest.cs
Sabnzbdservice with 11 test casesconfiguration
torrents)
TorBoxDebridClientTest.cs
Add TorBox debrid client unit testsserver/RdtClient.Service.Test/Services/TorrentClients/TorBoxDebridClientTest.cs
TorBoxDebridClientwith 8 test casesGetDownloadsto verify correctDownloadTypeassignment fortorrents and NZBs
GetAvailableFilesfor both torrent and usenet file retrievalDeleteandUnrestrictmethods for both download typesAddNzbFilemethod with proper parameter passingSabnzbdControllerTest.cs
Add Sabnzbd controller unit testsserver/RdtClient.Web.Test/Controllers/SabnzbdControllerTest.cs
SabnzbdControllerwith 10 test casesAllowAnonymousattribute verificationWatchFolderCheckerTests.cs
Add watch folder checker background service testsserver/RdtClient.Service.Test/BackgroundServices/WatchFolderCheckerTests.cs
WatchFolderCheckerbackground service with 5 testcases
NzbTorrentsTest.cs
Add unit tests for NZB torrent operationsserver/RdtClient.Service.Test/Services/NzbTorrentsTest.cs
AddNzbLinkToDebridQueue()with valid and invalid linksAddNzbFileToDebridQueue()with valid NZB XML and invalid XMLDownloadType.Nzbis set when adding NZBsSabnzbdHandlerTest.cs
Add authorization handler tests for SABnzbdserver/RdtClient.Web.Test/Controllers/SabnzbdHandlerTest.cs
SabnzbdHandlerauthorization handlerauthenticated, invalid credentials, missing credentials
types
TorrentsTest.cs
Add NZB download type tests to torrents serviceserver/RdtClient.Service.Test/Services/TorrentsTest.cs
AddNzbFileToDebridQueue_ShouldSetDownloadTypeNzb()verifies NZB filehandling
AddNzbLinkToDebridQueue_ShouldSetDownloadTypeNzb()verifies NZB linkhandling
DownloadType.Nzbis correctly setTorrentsControllerNzbTest.cs
Add NZB upload controller testsserver/RdtClient.Web.Test/Controllers/TorrentsControllerNzbTest.cs
TorrentsControllerUploadNzbLink()with valid/invalid requests and empty linksUploadNzbFile()with valid files and missing filesQBittorrentTest.cs
Add qBittorrent filtering testsserver/RdtClient.Service.Test/Services/QBittorrentTest.cs
QBittorrentserviceTorrentInfo()to verify only torrent-type downloads are returnedDownloadHelperTest.cs
Update download helper tests for debrid client modelsserver/RdtClient.Service.Test/Helpers/DownloadHelperTest.cs
TorrentClientFiletoDebridClientFileDebridClientmodels3 files
20251224022148_AddDownloadTypeToTorrent.Designer.cs
Add DownloadType column to Torrent table migrationserver/RdtClient.Data/Migrations/20251224022148_AddDownloadTypeToTorrent.Designer.cs
DownloadTypepropertyto
TorrententityTypecolumn as INTEGER in Torrentstable
DiConfig.cs
Update dependency injection for debrid and SABnzbd servicesserver/RdtClient.Service/DiConfig.cs
DebridClientclass namesSabnzbdservice registrationSabnzbdHandleras authorization handlerTorrentClientstoDebridClients20251224022148_AddDownloadTypeToTorrent.cs
Add download type column to torrents tableserver/RdtClient.Data/Migrations/20251224022148_AddDownloadTypeToTorrent.cs
Typecolumn toTorrentstable19 files