-
-
Notifications
You must be signed in to change notification settings - Fork 4.9k
internxt: add internxt backend support #9110
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
…operations to enhance error handling and improve reliability
… deletion process in Update method
… in problematic calls
…rt thumbnail retries
…rebase-with-rclone-master
…rt thumbnail retries
[_]: chore/rebase with rclone master
|
Hi @jzunigax2 - thank you for the PR :-) Can you tell me the status of the backend tests please? So the results of this from the project root? Do you need the |
…e and enhance configuration instructions
Remove unused mnemonic check in refreshJWTToken since the mnemonic is not needed for token refresh operations. Explicitly ignore error return from pacer.Call in restoreBackupFile cleanup function.
12c90f3 to
b026c15
Compare
Hey @ncw ! I dropped the ultest directory. As for the test results it had been a while since I last ran them and they helped find a small bug :) Still not included on this diff I'm hoping tmrw morning I'll be able to update the branch and attach test results. However I can't seem to get the test |
validate file type before proceeding with existing file checks. H andle specific error cases related to empty file uploads in the Update method.
|
Hey @ncw test results here: also running --- PASS: TestIntegration (431.39s)
--- SKIP: TestIntegration/FsCheckWrap (0.00s)
--- SKIP: TestIntegration/FsCommand (0.00s)
--- PASS: TestIntegration/FsRmdirNotFound (0.71s)
--- PASS: TestIntegration/FsString (0.00s)
--- PASS: TestIntegration/FsName (0.00s)
--- PASS: TestIntegration/FsRoot (0.00s)
--- PASS: TestIntegration/FsRmdirEmpty (1.09s)
--- PASS: TestIntegration/FsMkdir (425.45s)
--- PASS: TestIntegration/FsMkdir/FsMkdirRmdirSubdir (10.58s)
--- PASS: TestIntegration/FsMkdir/FsListEmpty (0.65s)
--- PASS: TestIntegration/FsMkdir/FsListDirEmpty (0.73s)
--- SKIP: TestIntegration/FsMkdir/FsListRDirEmpty (0.00s)
--- PASS: TestIntegration/FsMkdir/FsListDirNotFound (0.46s)
--- SKIP: TestIntegration/FsMkdir/FsListRDirNotFound (0.00s)
--- PASS: TestIntegration/FsMkdir/FsEncoding (148.55s)
--- PASS: TestIntegration/FsMkdir/FsEncoding/control_chars (10.24s)
--- PASS: TestIntegration/FsMkdir/FsEncoding/dot (7.46s)
--- PASS: TestIntegration/FsMkdir/FsEncoding/dot_dot (7.55s)
--- PASS: TestIntegration/FsMkdir/FsEncoding/punctuation (7.46s)
--- PASS: TestIntegration/FsMkdir/FsEncoding/leading_space (7.72s)
--- PASS: TestIntegration/FsMkdir/FsEncoding/leading_tilde (7.38s)
--- PASS: TestIntegration/FsMkdir/FsEncoding/leading_CR (7.57s)
--- PASS: TestIntegration/FsMkdir/FsEncoding/leading_LF (8.05s)
--- PASS: TestIntegration/FsMkdir/FsEncoding/leading_HT (7.78s)
--- PASS: TestIntegration/FsMkdir/FsEncoding/leading_VT (7.62s)
--- PASS: TestIntegration/FsMkdir/FsEncoding/leading_dot (8.62s)
--- PASS: TestIntegration/FsMkdir/FsEncoding/trailing_space (11.15s)
--- PASS: TestIntegration/FsMkdir/FsEncoding/trailing_CR (7.57s)
--- PASS: TestIntegration/FsMkdir/FsEncoding/trailing_LF (8.09s)
--- PASS: TestIntegration/FsMkdir/FsEncoding/trailing_HT (6.88s)
--- PASS: TestIntegration/FsMkdir/FsEncoding/trailing_VT (5.97s)
--- PASS: TestIntegration/FsMkdir/FsEncoding/trailing_dot (6.19s)
--- PASS: TestIntegration/FsMkdir/FsEncoding/invalid_UTF-8 (7.28s)
--- PASS: TestIntegration/FsMkdir/FsEncoding/URL_encoding (7.23s)
--- PASS: TestIntegration/FsMkdir/FsNewObjectNotFound (0.96s)
--- PASS: TestIntegration/FsMkdir/FsPutError (4.20s)
--- SKIP: TestIntegration/FsMkdir/FsPutZeroLength (3.04s)
--- SKIP: TestIntegration/FsMkdir/FsOpenWriterAt (0.00s)
--- SKIP: TestIntegration/FsMkdir/FsOpenChunkWriter (0.00s)
--- SKIP: TestIntegration/FsMkdir/FsChangeNotify (0.00s)
--- PASS: TestIntegration/FsMkdir/FsPutFiles (224.95s)
--- PASS: TestIntegration/FsMkdir/FsPutFiles/FsListDirFile2 (3.71s)
--- SKIP: TestIntegration/FsMkdir/FsPutFiles/FsListRDirFile2 (0.00s)
--- PASS: TestIntegration/FsMkdir/FsPutFiles/FsListR (3.54s)
--- PASS: TestIntegration/FsMkdir/FsPutFiles/FsListRSubdir (3.15s)
--- PASS: TestIntegration/FsMkdir/FsPutFiles/FsListDirRoot (2.10s)
--- SKIP: TestIntegration/FsMkdir/FsPutFiles/FsListRDirRoot (0.00s)
--- PASS: TestIntegration/FsMkdir/FsPutFiles/FsListSubdir (1.74s)
--- SKIP: TestIntegration/FsMkdir/FsPutFiles/FsListRSubdir#01 (0.00s)
--- PASS: TestIntegration/FsMkdir/FsPutFiles/FsListLevel2 (1.80s)
--- SKIP: TestIntegration/FsMkdir/FsPutFiles/FsListRLevel2 (0.00s)
--- PASS: TestIntegration/FsMkdir/FsPutFiles/FsListFile1 (3.35s)
--- PASS: TestIntegration/FsMkdir/FsPutFiles/FsNewObject (0.32s)
--- SKIP: TestIntegration/FsMkdir/FsPutFiles/FsNewObjectCaseInsensitive (0.00s)
--- PASS: TestIntegration/FsMkdir/FsPutFiles/FsListFile1and2 (3.35s)
--- PASS: TestIntegration/FsMkdir/FsPutFiles/FsNewObjectDir (0.29s)
--- SKIP: TestIntegration/FsMkdir/FsPutFiles/FsPurge (0.00s)
--- SKIP: TestIntegration/FsMkdir/FsPutFiles/FsPurgeRoot (0.00s)
--- PASS: TestIntegration/FsMkdir/FsPutFiles/FsListRootedSubdir (21.95s)
--- SKIP: TestIntegration/FsMkdir/FsPutFiles/FsCopy (0.00s)
--- SKIP: TestIntegration/FsMkdir/FsPutFiles/FsMove (0.00s)
--- SKIP: TestIntegration/FsMkdir/FsPutFiles/FsDirMove (0.00s)
--- PASS: TestIntegration/FsMkdir/FsPutFiles/FsRmdirFull (0.35s)
--- PASS: TestIntegration/FsMkdir/FsPutFiles/FsPrecision (0.00s)
--- PASS: TestIntegration/FsMkdir/FsPutFiles/ObjectString (0.31s)
--- PASS: TestIntegration/FsMkdir/FsPutFiles/ObjectFs (0.31s)
--- PASS: TestIntegration/FsMkdir/FsPutFiles/ObjectRemote (0.32s)
--- PASS: TestIntegration/FsMkdir/FsPutFiles/ObjectHashes (0.50s)
--- PASS: TestIntegration/FsMkdir/FsPutFiles/ObjectModTime (0.39s)
--- SKIP: TestIntegration/FsMkdir/FsPutFiles/ObjectMimeType (0.38s)
--- SKIP: TestIntegration/FsMkdir/FsPutFiles/ObjectMetadata (0.35s)
--- SKIP: TestIntegration/FsMkdir/FsPutFiles/ObjectSetMetadata (0.34s)
--- PASS: TestIntegration/FsMkdir/FsPutFiles/ObjectSetModTime (0.39s)
--- PASS: TestIntegration/FsMkdir/FsPutFiles/ObjectSize (0.31s)
--- PASS: TestIntegration/FsMkdir/FsPutFiles/ObjectOpen (1.92s)
--- PASS: TestIntegration/FsMkdir/FsPutFiles/ObjectOpenSeek (1.50s)
--- PASS: TestIntegration/FsMkdir/FsPutFiles/ObjectOpenRange (4.30s)
--- PASS: TestIntegration/FsMkdir/FsPutFiles/ObjectPartialRead (1.09s)
--- PASS: TestIntegration/FsMkdir/FsPutFiles/ObjectUpdate (5.38s)
--- PASS: TestIntegration/FsMkdir/FsPutFiles/ObjectStorable (0.39s)
--- PASS: TestIntegration/FsMkdir/FsPutFiles/FsIsFile (6.28s)
--- PASS: TestIntegration/FsMkdir/FsPutFiles/FsIsFile/FsRoot (0.00s)
--- PASS: TestIntegration/FsMkdir/FsPutFiles/FsIsFileNotFound (3.11s)
--- PASS: TestIntegration/FsMkdir/FsPutFiles/FromRoot (16.60s)
--- PASS: TestIntegration/FsMkdir/FsPutFiles/FromRoot/List (5.45s)
--- PASS: TestIntegration/FsMkdir/FsPutFiles/FromRoot/ListEntries (0.75s)
--- SKIP: TestIntegration/FsMkdir/FsPutFiles/FromRoot/ListR (0.00s)
--- PASS: TestIntegration/FsMkdir/FsPutFiles/FromRoot/Put (9.71s)
--- PASS: TestIntegration/FsMkdir/FsPutFiles/FromRoot/Put/Remove (4.31s)
--- SKIP: TestIntegration/FsMkdir/FsPutFiles/PublicLink (0.00s)
--- SKIP: TestIntegration/FsMkdir/FsPutFiles/SetTier (0.00s)
--- SKIP: TestIntegration/FsMkdir/FsPutFiles/ObjectCheckWrap (0.00s)
--- SKIP: TestIntegration/FsMkdir/FsPutFiles/Bucket (0.00s)
--- PASS: TestIntegration/FsMkdir/FsPutFiles/ObjectRemove (5.37s)
--- PASS: TestIntegration/FsMkdir/FsPutFiles/ObjectAbout (120.73s)
--- SKIP: TestIntegration/FsMkdir/FsPutFiles/FsPutStream (0.00s)
--- SKIP: TestIntegration/FsMkdir/FsPutFiles/Internal (0.00s)
--- SKIP: TestIntegration/FsMkdir/FsPutChunked (0.00s)
--- SKIP: TestIntegration/FsMkdir/FsCopyChunked (0.00s)
--- PASS: TestIntegration/FsMkdir/FsUploadUnknownSize (10.30s)
--- PASS: TestIntegration/FsMkdir/FsUploadUnknownSize/FsPutUnknownSize (3.84s)
--- PASS: TestIntegration/FsMkdir/FsUploadUnknownSize/FsUpdateUnknownSize (6.46s)
--- PASS: TestIntegration/FsMkdir/FsRootCollapse (4.29s)
--- SKIP: TestIntegration/FsMkdir/FsDirSetModTime (0.00s)
--- SKIP: TestIntegration/FsMkdir/FsMkdirMetadata (0.00s)
--- SKIP: TestIntegration/FsMkdir/FsDirectory (0.00s)
--- PASS: TestIntegration/FsShutdown (0.00s)
PASS
ok github.com/rclone/rclone/backend/internxt 431.411s |
ncw
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've given the code a quick once over - see inline!
I tried to give this a go but apparently it doesn't work with the free tier. Can you make it work with the free tier, or document that it doesn't.
2026/01/23 16:08:34 NOTICE: Fatal error: authentication failed: failed to refresh token: refresh token: CLI access not allowed for this user tier (status 402)
I don't really like the backend opening browser windows directly. rclone config is only one UI for configuring rclone - there are many more which use the API or the rclone config state machine. Opening a browser window like this may well break the config on those UIs.
This seems to have re-invented rclone's oauth framework in the auth.go code? Could you use rclone's oauth framework?
However I can't seem to get the test
TestRWFileHandleWriteNoWritein the VFS test suite to pass. I believe it must be related to it swallowing anfs.ErrorCantUploadEmptyFileserror here ?
I think this test may be buggy! All the other backends which use fs.ErrorCantUploadEmptyFiles seem to be skipping this test in config.yaml. Feel free to ignore this test in config.yaml!
Can you fix the notes, rebase this on master, fix the conflicts and squash to a single commit please? Make sure you add the Co-Authored-By so we get the attributions of all the authors.
Thanks
docs/content/_index.md
Outdated
|
|
||
| Users call rclone *"The Swiss army knife of cloud storage"*, and | ||
| *"Technology indistinguishable from magic"*. | ||
| *Technology indistinguishable from magic"*. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Stray edit here
docs/content/internxt.md
Outdated
| --- | ||
| title: "Internxt Drive" | ||
| description: "Rclone docs for Internxt Drive" | ||
| versionIntroduced: "v1.69" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
v1.73
| ### Restricted filename characters | ||
|
|
||
| The Internxt backend replaces the [default restricted characters | ||
| set](/overview/#restricted-characters). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not according to the code!
Default: encoder.EncodeInvalidUtf8 |
encoder.EncodeSlash |
encoder.EncodeBackSlash |
encoder.EncodeRightPeriod |
encoder.EncodeDot |
encoder.EncodeCrLf,
fstest/fstests/fstests.go
Outdated
| SkipDirectoryCheckWrap bool // if set skip DirectoryCheckWrap | ||
| SkipInvalidUTF8 bool // if set skip invalid UTF-8 checks | ||
| SkipLeadingDot bool // if set skip leading dot checks | ||
| SkipTrailingDot bool // if set skip trailing dot checks |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should be done with an encoding EncodeRightPeriod.
Not sure why we ever merged SkipLeadingDot because there is an encoding for that too EncodeLeftPeriod so I don't want to add to the problem!
fstest/test_all/config.yaml
Outdated
| # localonly: true | ||
| - path: cmd/selfupdate | ||
| localonly: true | ||
| - path: backend |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Undo spurious edits please
README.md
Outdated
| @@ -1,6 +1,9 @@ | |||
| <!-- markdownlint-disable-next-line first-line-heading no-inline-html --> | |||
|
|
|||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Undo spurious edits
README.md
Outdated
| # Rclone | ||
|
|
||
| Rclone *("rsync for cloud storage")* is a command-line program to sync files and | ||
| Rclone _("rsync for cloud storage")_ is a command-line program to sync files and |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Undo spurious edit please
| ) | ||
|
|
||
| const ( | ||
| driveWebURL = "https://drive.internxt.com" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I really don't like that the backend has its own oauth server and isn't using rclone's.
backend/internxt/internxt.go
Outdated
| Config: Config, | ||
| Options: []fs.Option{ | ||
| { | ||
| Name: "skipHashValidation", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Option names in snake case please "skip_hash_validation"
…iguration options and clean up test configurations - Changed formatting in README for consistency. - Updated version introduced for Internxt in documentation. - Refactored configuration option name from `skipHashValidation` to `skip_hash_validation` for clarity. - Removed unused trailing dot checks from test options and cleaned up test configuration formatting.

What is the purpose of this change?
New backend, add Internxt support
Was the change discussed in an issue or in the forum before?
also based off of PR: #8556
Checklist