[v4,00/11] Performance fixes for 9p filesystem

  This is the fourth version of a patch series which adds a number
of features to improve read/write performance in the 9p filesystem.
Mostly it focuses on fixing caching to help utilize the recently
increased MSIZE limits and also fixes some problematic behavior
within the writeback code.

All together, these show roughly 10x speed increases on simple
file transfers over no caching for readahead mode.  Future patch
sets will improve cache consistency and directory caching, which
should benefit loose mode.

This iteration of the patch incorporates an important fix for
writeback which uses a stronger mechanism to flush writeback on
close of files and addresses observed bugs in previous versions of
the patch for writeback, mmap, and loose cache modes.

These patches are also available on github:
https://github.com/v9fs/linux/tree/ericvh/for-next
and on kernel.org:
https://git.kernel.org/pub/scm/linux/kernel/git/ericvh/v9fs.git

Tested against qemu, cpu, and diod with fsx, dbench, and postmark
in every caching mode.

I'm gonna definitely submit the first couple patches as they are
fairly harmless - but would like to submit the whole series to the
upcoming merge window.  Would appreciate reviews.

Eric Van Hensbergen (11):
  net/9p: Adjust maximum MSIZE to account for p9 header
  fs/9p: Expand setup of writeback cache to all levels
  fs/9p: Consolidate file operations and add readahead and writeback
  fs/9p: Remove unnecessary superblock flags
  fs/9p: allow disable of xattr support on mount
  net/9p: fix bug in client create for .L
  9p: Add additional debug flags and open modes
  fs/9p: Add new mount modes
  fs/9p: fix error reporting in v9fs_dir_release
  fs/9p: writeback mode fixes
  fs/9p: Fix revalidate

 Documentation/filesystems/9p.rst |  26 ++--
 fs/9p/fid.c                      |  49 +++-----
 fs/9p/fid.h                      |  33 ++++-
 fs/9p/v9fs.c                     |  49 +++++---
 fs/9p/v9fs.h                     |  10 +-
 fs/9p/v9fs_vfs.h                 |   4 -
 fs/9p/vfs_addr.c                 |  24 ++--
 fs/9p/vfs_dentry.c               |   3 +-
 fs/9p/vfs_dir.c                  |  10 +-
 fs/9p/vfs_file.c                 | 205 +++++++------------------------
 fs/9p/vfs_inode.c                | 102 +++++++--------
 fs/9p/vfs_inode_dotl.c           |  69 ++++++-----
 fs/9p/vfs_super.c                |  28 +++--
 include/net/9p/9p.h              |   5 +
 net/9p/client.c                  |   8 +-
 15 files changed, 284 insertions(+), 341 deletions(-)

--
2.37.2
  

Eric Van Hensbergen wrote on Sat, Feb 18, 2023 at 12:33:12AM +0000:
> I'm gonna definitely submit the first couple patches as they are
> fairly harmless - but would like to submit the whole series to the
> upcoming merge window.

Could you take the three patches I have in my 9p-next branch:
https://github.com/martinetd/linux/commits/9p-next

If you're going to submit some?
The async stuff still isn't good, but there three patches have had
reviews and should be good to go.

(I guess we can just send Linus two pull requests for 9p, but it doesn't
really make sense given the low number of patches)

> Would appreciate reviews.

Just one first review on the form: let's start a new thread for every
new revision of the patchset.

I also used to relink from the pervious cover letter and thought that
made more sense at the time, but I was told to split threads a while ago
and now I'm trying some new tools based on lkml.kernel.org's public
inbox thread view I can agree it's much simpler to grab a batch of patch
if older versions aren't mixed in the thread.
(For the curious, I'm just grabbing the thread to review on an e-ink
reader for my eyes, but there's also b4 that I've been meaning to try at
some point -- https://b4.docs.kernel.org/en/latest/ -- that will likely
work the same)

Anyway, off to look at patches a bit.
  

On Saturday, February 18, 2023 1:33:12 AM CET Eric Van Hensbergen wrote:
> This is the fourth version of a patch series which adds a number
> of features to improve read/write performance in the 9p filesystem.
> Mostly it focuses on fixing caching to help utilize the recently
> increased MSIZE limits and also fixes some problematic behavior
> within the writeback code.
> 
> All together, these show roughly 10x speed increases on simple
> file transfers over no caching for readahead mode.  Future patch
> sets will improve cache consistency and directory caching, which
> should benefit loose mode.
> 
> This iteration of the patch incorporates an important fix for
> writeback which uses a stronger mechanism to flush writeback on
> close of files and addresses observed bugs in previous versions of
> the patch for writeback, mmap, and loose cache modes.
> 
> These patches are also available on github:
> https://github.com/v9fs/linux/tree/ericvh/for-next
> and on kernel.org:
> https://git.kernel.org/pub/scm/linux/kernel/git/ericvh/v9fs.git
> 
> Tested against qemu, cpu, and diod with fsx, dbench, and postmark
> in every caching mode.
> 
> I'm gonna definitely submit the first couple patches as they are
> fairly harmless - but would like to submit the whole series to the
> upcoming merge window.  Would appreciate reviews.

I tested this version thoroughly today (msize=512k in all tests). Good news 
first: the previous problems of v3 are gone. Great! But I'm still trying to 
make sense of the performance numbers I get with these patches.

So when doing some compilations with 9p, performance of mmap, writeback and 
readahead are basically all the same, and only loose being 6x faster than the 
other cache modes. Expected performance results? No errors at least. Good!

Then I tested simple linear file I/O. First linear writing a 12GB file
(time dd if=/dev/zero of=test.data bs=1G count=12):

writeback    3m10s [this series - v4]
readahead    0m11s [this series - v4]
mmap         0m11s [this series - v4]
mmap         0m11s [master]
loose        2m50s [this series - v4]
loose        2m19s [master]

That's a bit surprising. Why is loose and writeback slower?

Next linear reading a 12GB file
(time cat test.data > /dev/null):

writeback    0m24s [this series - v4]
readahead    0m25s [this series - v4]
mmap         0m25s [this series - v4]
mmap         0m9s  [master]
loose        0m24s [this series - v4]
loose        0m24s [master]

mmap degredation sticks out here, and no improvement with the other modes?

I always performed a guest reboot between each run BTW.
  

Glad to hear bugs disappeared.  writeback having a different
performance than mmap is confusing as they should be equivalent.

The huge blocksize on your dd is an interesting choice -- it will
completely get rid of any impact of readahead.  To see impact of
readahead, choose a blocksize of
less than msize (like 4k) to actually see the perf of readahead.  The
mmap degradation is likely due to stricter coherence (open-to-close
consistency means we wait on writeout), but I'd probably need to go in
and trace to verify (which probably isn't a bad idea overall).
probably a similar situation for loose and writeback.  Essentially,
before close consistency it didn't have to wait for the final write to
complete before it returns so you see a faster time (even though data
wasn't actually written all the way through so you aren't measuring
the last little bit of the write (which can be quite large of a big
msize).

I'm going to take a pass through tomorrow making some fixups that
Dominiquee suggested and trying to reproduce/fix the fscache problems.

      -eric

On Sun, Feb 19, 2023 at 3:36 PM Christian Schoenebeck
<linux_oss@crudebyte.com> wrote:
>
> On Saturday, February 18, 2023 1:33:12 AM CET Eric Van Hensbergen wrote:
> > This is the fourth version of a patch series which adds a number
> > of features to improve read/write performance in the 9p filesystem.
> > Mostly it focuses on fixing caching to help utilize the recently
> > increased MSIZE limits and also fixes some problematic behavior
> > within the writeback code.
> >
> > All together, these show roughly 10x speed increases on simple
> > file transfers over no caching for readahead mode.  Future patch
> > sets will improve cache consistency and directory caching, which
> > should benefit loose mode.
> >
> > This iteration of the patch incorporates an important fix for
> > writeback which uses a stronger mechanism to flush writeback on
> > close of files and addresses observed bugs in previous versions of
> > the patch for writeback, mmap, and loose cache modes.
> >
> > These patches are also available on github:
> > https://github.com/v9fs/linux/tree/ericvh/for-next
> > and on kernel.org:
> > https://git.kernel.org/pub/scm/linux/kernel/git/ericvh/v9fs.git
> >
> > Tested against qemu, cpu, and diod with fsx, dbench, and postmark
> > in every caching mode.
> >
> > I'm gonna definitely submit the first couple patches as they are
> > fairly harmless - but would like to submit the whole series to the
> > upcoming merge window.  Would appreciate reviews.
>
> I tested this version thoroughly today (msize=512k in all tests). Good news
> first: the previous problems of v3 are gone. Great! But I'm still trying to
> make sense of the performance numbers I get with these patches.
>
> So when doing some compilations with 9p, performance of mmap, writeback and
> readahead are basically all the same, and only loose being 6x faster than the
> other cache modes. Expected performance results? No errors at least. Good!
>
> Then I tested simple linear file I/O. First linear writing a 12GB file
> (time dd if=/dev/zero of=test.data bs=1G count=12):
>
> writeback    3m10s [this series - v4]
> readahead    0m11s [this series - v4]
> mmap         0m11s [this series - v4]
> mmap         0m11s [master]
> loose        2m50s [this series - v4]
> loose        2m19s [master]
>
> That's a bit surprising. Why is loose and writeback slower?
>
> Next linear reading a 12GB file
> (time cat test.data > /dev/null):
>
> writeback    0m24s [this series - v4]
> readahead    0m25s [this series - v4]
> mmap         0m25s [this series - v4]
> mmap         0m9s  [master]
> loose        0m24s [this series - v4]
> loose        0m24s [master]
>
> mmap degredation sticks out here, and no improvement with the other modes?
>
> I always performed a guest reboot between each run BTW.
>
>
>