Skip to content

Exclude Directories#1627

Closed
mmattel wants to merge 2 commits intomasterfrom
Exclude_Directories
Closed

Exclude Directories#1627
mmattel wants to merge 2 commits intomasterfrom
Exclude_Directories

Conversation

@mmattel
Copy link
Contributor

@mmattel mmattel commented Oct 5, 2016

Feature PR, references: Exclude Directories (merged, for professional…… Storage use) in owncloud/core/pull/19235

Adds the ability to define directories in config.php which will not be further processed (scanned, shown, created ect). These directories exist in the source filesystem but will not handled in nextcloud.

Usecase: eg. filesystems which have the ability of snapshots have a defined folder naming for these snapshots. Processing them is in most cases not sensful.

For a detailed description pls see also the documentation PR.

… Storage use) owncloud/core/pull/19235

Adds the ability to define directories in config.php which will not be further processed (scanned, shown, created ect). These directories exist but will not handled in nextcloud.
Usecase: eg. filesystem which have the ability of snapshots have a defined naming for them. For details pls see also the documentation PR coming asap.
@mention-bot
Copy link

@mmattel, thanks for your PR! By analyzing the history of the files in this pull request, we identified @icewind1991, @LukasReschke and @DeepDiver1975 to be potential reviewers.

*/
public function createFile($name, $data = null) {

# the check here is necessary, because createFile uses put covered in sabre/file.php
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

# comments are deprecated in php 🙊

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

corrected

Copy link
Member

@LukasReschke LukasReschke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that should be handled on storage system level and not a global setting. I'm not really in favour of that as it is quite an unexpected behaviour for users and likely won't be tested properly since it's a setting for edge cases.

If that would be handled at least on the storage level layer than that would at least only apply to the minimum storages.

Also what's the use-case behind this? What is the hardware in case exactly? Can't it be configured in another way?

cc @icewind1991

# the check here is necessary, because createFile uses put covered in sabre/file.php
# and not touch covered in files/view.php
if (\OC\Files\Filesystem::isForbiddenFileOrDir($name)) {
throw new \Sabre\DAV\Exception\Forbidden();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

either this or the call above is wrong

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

corrected

throw new \Sabre\DAV\Exception\Forbidden();
}

$path = trim($path, '/');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Swap the order of this assignment and the check before, then you can get rid of the second check below

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

corrected

}

// check the path, also called when the path has been entered manually eg via a file explorer
if (\OC\Files\Filesystem::isForbiddenFileOrDir($path)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

drop this one

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

corrected

*
* @param array $data from hook
*/
static public function isBlacklisted($data) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this method might be used by other apps?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, when I originally created the PR in owncloud, I have found one app (but I do not remember anymore...) that was using this call. Therefore I left it and forwarded the call to the new function.

}

/**
/**
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

duplication === error

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

corrected

@mmattel
Copy link
Contributor Author

mmattel commented Oct 5, 2016

@LukasReschke

  • There was a long discussion upfront on this (~2 years) and finally it was agreed to implement/merge this on ownCloud.
  • There are many professional Storage vendors like NetApp, EMC, HDS ect but also public available filesystem like ZFS, BTRFS who are capable of snapshots. Just read the documentation linked. If you have further questions, let me know.
  • No that can´t be done on a storage level because you do not know which storage you will mount and from which provider it will come. In reality you always mount a filesystem which provides folders and files. And snapshots are special folders with a defined but different naming by each vendor.

...quite an unexpected behaviour for users...

  • It is just the other way round. Snapshots are mostly used for backup/restore purposes. Those folders containing the snapshots have in most cases no value for the enduser but the admin. Please just see also the math behind in the documentation referenced.
  • From a company / professional use point of view I can tell you that excluding directories is key for owncloud/nextcloud.

@LukasReschke
Copy link
Member

LukasReschke commented Oct 5, 2016

There was a long discussion upfront on this (~2 years) and finally it was agreed to implement/merge this on ownCloud.

And I happen to have then not been happy with this approach nor am I yet. This adds more static code that is barely unit-test covered. This is not the way we should go.

No that can´t be done on a storage level because you do not know which storage you will mount and from which provider it will come. In reality you always mount a filesystem which provides folders and files. And snapshots are special folders with a defined but different naming by each vendor.

I aimed to refer to StorageWrappers. See https://github.com/nextcloud/files_accesscontrol/blob/master/lib/StorageWrapper.php for example. This should be done using a storage-level approach in Nextcloud and not put directly into the Nextcloud core filesystem classes.

@LukasReschke
Copy link
Member

@icewind1991 Please comment on the storage wrapper approach. THX.

@rullzer
Copy link
Member

rullzer commented Oct 5, 2016

I agree. This solution is way to hacky for my taste. And it is not something that core should take care of. A simple POC to do this with an app in https://github.com/rullzer/files_excludedirs

@codecov-io
Copy link

Current coverage is 30.59% (diff: 37.28%)

Merging #1627 into master will decrease coverage by 0.02%

@@             master      #1627   diff @@
==========================================
  Files          1081       1081          
  Lines         60023      60350   +327   
  Methods        6808       6846    +38   
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          18378      18465    +87   
- Misses        41645      41885   +240   
  Partials          0          0          

Sunburst

Diff Coverage File Path
0% lib/base.php
0% lib/private/Files/Cache/Scanner.php
0% config/config.sample.php
0% lib/private/Files/View.php
0% lib/private/Security/CertificateManager.php
0% apps/dav/lib/Connector/Sabre/ObjectTree.php
0% .../dav/lib/Connector/Sabre/CustomPropertiesBackend.php
0% apps/dav/lib/Connector/Sabre/Directory.php
••••• 50% lib/private/Files/Storage/Common.php
•••••• 67% lib/private/Files/Filesystem.php

Review all 12 files changed

Powered by Codecov. Last update 3f48cb8...76ad849

@LukasReschke
Copy link
Member

Closing as it should be done via an app as the PoC linked from #1627 (comment) shows.

@LukasReschke LukasReschke deleted the Exclude_Directories branch October 7, 2016 11:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants