Re: [PLUG] large_dir support on ext4

[Matt Mossholder via plug <plug@lists.phillylinux.org>]

Perhaps make a new directory, and move the existing files into it? Then just move the old directory out of the way and rename the new directory to the name of the old directory.

     --Matt

On Fri, Sep 13, 2019 at 9:20 AM Walt Mankowski via plug <plug@lists.phillylinux.org> wrote:

On Fri, Sep 13, 2019 at 09:00:43AM -0400, Rich Freeman via plug wrote:
> On Fri, Sep 13, 2019 at 8:42 AM Walt Mankowski via plug
> <plug@lists.phillylinux.org> wrote:
> >
> > I used tune2fs to add large_dir support to the filesystem in hopes of
> > getting around this problem. It returned very quickly, which makes me
> > suspect it will only be applied to new directories, not existing
> > ones. I'm asking here on the off chance anyone might happen to know
> > for sure.
> >
>
> I don't know the answer to that question, but you might want to
> consider also enabling dir_index.

dir_index is already enabled.

> Now, if you don't actually do name lookups but do something like using
> readdir to iterate through all the inodes in a directory then it won't
> provide benefit.  Anything that actually uses a filename to open a
> file directly would benefit, however, as otherwise the kernel
> basically has to scan the entire directory to find a file which I'm
> guessing is O(N).  I don't know enough about ext4 to know if the
> metadata can get fragmented, but I'm guessing it can, and thus that
> cost would apply to any seeks needed to read the entire directory
> listing.

I don't have any idea what rdiff-backup is doing, but just running

  ls | wc

on this directory took a good 10-15 minutes.

> I suspect that it doesn't work retroactively, so you will want to
> enable this before creating all your files.

Not really an option at this point, seeing as how it's an existing backup.

Walt

___________________________________________________________________________
Philadelphia Linux Users Group         --        http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion  --   http://lists.phillylinux.org/mailman/listinfo/plug

___________________________________________________________________________
Philadelphia Linux Users Group         --        http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion  --   http://lists.phillylinux.org/mailman/listinfo/plug