Inconsistency between filesystem and database - unable to resolve / repair

[msrex]

4.5.2: If you copy a file into the ownCloud or it is modified in anyway in the filesystem, there is no way to trigger a "correction run" or "repair fs cache" run. This leads to very unexpected behaviour.
It is clear that you are not supposed to change anything directly in the FS, but if for some reason there is a difference, a method is needed to get the filecache back in sync.

[RandolfCarter]

This is not specific to 4.5.2 as far as I can tell, it's been the same in all 4.0.x and 4.5.x releases.

There is a way to scan the existing directories, however: use "Rescan" app, or execute "scanFiles" javascript function in Files view.

[karlitschek]

Yes. But we also have to find out What brings the filesystem and the database out of sync. Because the solution for the user is not to run a fix script all the time. So bug reports about ways to bring the database out of sync are helpful.

[MTRichards]

I am running into this problem with users too because people keep copying large sets of files into the fileserver. If they sync, it would take them a week. Copying it manually, it takes a few hours. So, while it is important to not get the two out of sync, there is no question people will still touch the filesystem and things will get out of sync...

[msrex]

Having a "repair" would also fix http://bugs.owncloud.org/thebuggenie/owncloud/issues/oc-379
"oc-379 - symlinks not appearing fully in browser, only some files"

[boxrick]

So if this was working in version 3 and previous to that. How did that particular version keep things in sync differently than 4?

[msrex]

I ran into numerous more cases when testing external mounts where an official way to update the fs_cache is required. Can we please create one, instead of telling people to open the javascipt console?

Especially the case of Matt is really more common than one might think.

[MTRichards]

In the same light of this bug, one from bugzilla: manual "scanFiles()" on javascript-console doesnt update folder-sizes

http://bugs.owncloud.org/thebuggenie/owncloud/issues/oc-2112

Seems there may be a bug in scanFiles(true)

Description:
After syncing my files per samba/Robocopy, I had to clear my fscache (parent=-1 bug) and rescan the filesystem by calling scanFiles() from a javascript console.

All files where displayed correctly now in webinterface, but every folder had the size of only the contained files, not including the contained subfolders (so a folder with only subfolders was shown with 0 bytes).

Problem: lib/filecache.php - function increaseSize() calls self::getParentId with $path, not regarding the needed fullpath

Works fine for me (not sure how this whole github-thingy works, so I just post it here):
348 while($id!=-1 && strlen($path) > 0) {//walk up the filetree increasing the size of all parent folders
(...)
351 $path=dirname($path);
352 $id=self::getId($path,$root);

instead of

348 while($id!=-1) {//walk up the filetree increasing the size of all parent folders
(...)
351 $id=self::getParentId($path);
352 $path=dirname($path);
if the strlen isnt checked, the routine wont stop trying to get the parent of "empty string"...I wasn't able to find out why id!=-1 wasnt enough.

Still a problem: scanFiles() for a specific folder: each call increases the size of parent-folders of the folder to scan.

example:

foo 300MB
foo/foo 150MB
foo/bar 150MB

will be after scanFiles(true,'foo/bar');

foo 450MB
foo/foo 150MB
foo/bar 150MB

Reproduction steps:
clear fscache in your database
rescan by javascript scanFiles(true)
folder-sizes arent updated recursivly (so a folder containing only folders, and no files, will be 0 bytes)

[bartv2]

can you check with the 5.0.3? the file cache is rewritten in OC5

[jancborchardt]

I’m closing this issue because it has been inactive for a few months. This probably means it’s not reproducible or it has been fixed in a newer version.

Please reopen if the error still persists with the latest stable version (currently ownCloud 5.0.9) and then please use the issue template. You an also contribute directly by providing a patch – see the developer manual. :)

Thank you!

[MTRichards]

The problem still exists...when you get inconsistencies between the filecache and the filesystem, it is very difficult to sort it out and fix them. This happens when you have the inconsistencies and migrate to a new version of ownCloud.

It is a fall back, of course, and a fail safe maneuver. But admins need to be able to provide this for a specific user, if something goes sideways (an often requested feature).

[DeepDiver1975]

we have a console script now in master (and soon in stable5 - planned to be shipped with 5.0.10) which allows admins to rescan the filesystem for a given user.

[DeepDiver1975]

#4317
#4285

[MTRichards]

I like it!! :)