mbox series

[v2,0/3] Bitmap percentage flushing

Message ID 20221013224151.300-1-jonathan.derrick@linux.dev
Headers
Series Bitmap percentage flushing |

Message

Jonathan Derrick Oct. 13, 2022, 10:41 p.m. UTC 
  This introduces a percentage-flushing mechanism that works in-tandem to the
mdadm delay timer. The percentage argument is based on the number of chunks
dirty (rather than percentage), due to large drives requiring smaller and
smaller percentages (eg, 32TB drives-> 1% is 320GB).

This set hopes to provide a way to make the bitmap flushing more consistent. It
was observed that a synchronous, random write qd1 workload, could make bitmap
writes easily become almost half of the I/O. And in similar workloads with
different timing, it was several minutes between bitmap updates. This is too
inconsistent to be reliable.

This first and second patches adds the flush_threshold parameter. The default
value of 0 defines the default behavior: unplugging immediately just as before.
With a flush-threshold value of 1, it becomes more consistent and paranoid,
flushing on nearly every I/O, leading to a 40% or greater situation. From
there, the flush_threshold can be defined higher for those situations where
power loss is rare and full resync can be tolerated.

The third patch converts the daemon worker to an actual timer. This makes it
more consistent and removes some ugly code.

Jonathan Derrick (3):
  md/bitmap: Add chunk-threshold unplugging
  md/bitmap: Add sysfs interface for flush threshold
  md/bitmap: Convert daemon_work to proper timer

 Documentation/admin-guide/md.rst |  5 ++
 drivers/md/md-bitmap.c           | 98 +++++++++++++++++++++++++-------
 drivers/md/md-bitmap.h           |  4 +-
 drivers/md/md.c                  |  9 ++-
 drivers/md/md.h                  |  2 +
 5 files changed, 93 insertions(+), 25 deletions(-)

Comments

John Stoffel Oct. 14, 2022, 9:10 p.m. UTC | #1 
>>>>> "Jonathan" == Jonathan Derrick <jonathan.derrick@linux.dev> writes:

> This introduces a percentage-flushing mechanism that works in-tandem to the
> mdadm delay timer. The percentage argument is based on the number of chunks
> dirty (rather than percentage), due to large drives requiring smaller and
> smaller percentages (eg, 32TB drives-> 1% is 320GB).

I've been reading and re-reading this and I still don't understand
what you're saying here.  You say you're adding a percentage based
mechanism, but then you say it's based on chunk counts, not
percentages.  I think you need to clean this up and re-word it.

Maybe you're trying to say that you only take a percentage of the
available write bandwidth per second or something like that? 


> This set hopes to provide a way to make the bitmap flushing more consistent. It
> was observed that a synchronous, random write qd1 workload, could make bitmap
> writes easily become almost half of the I/O. And in similar workloads with
> different timing, it was several minutes between bitmap updates. This is too
> inconsistent to be reliable.

> This first and second patches adds the flush_threshold parameter. The default
> value of 0 defines the default behavior: unplugging immediately just as before.
> With a flush-threshold value of 1, it becomes more consistent and paranoid,
> flushing on nearly every I/O, leading to a 40% or greater situation. From

What situation?  Please be more clear here.  

> there, the flush_threshold can be defined higher for those situations where
> power loss is rare and full resync can be tolerated.

> The third patch converts the daemon worker to an actual timer. This makes it
> more consistent and removes some ugly code.

> Jonathan Derrick (3):
>   md/bitmap: Add chunk-threshold unplugging
>   md/bitmap: Add sysfs interface for flush threshold
>   md/bitmap: Convert daemon_work to proper timer

>  Documentation/admin-guide/md.rst |  5 ++
>  drivers/md/md-bitmap.c           | 98 +++++++++++++++++++++++++-------
>  drivers/md/md-bitmap.h           |  4 +-
>  drivers/md/md.c                  |  9 ++-
>  drivers/md/md.h                  |  2 +
>  5 files changed, 93 insertions(+), 25 deletions(-)

> -- 
> 2.31.1
Jonathan Derrick Oct. 15, 2022, 10:27 p.m. UTC | #2 
On 10/14/2022 3:10 PM, John Stoffel wrote:
>>>>>> "Jonathan" == Jonathan Derrick <jonathan.derrick@linux.dev> writes:
> 
>> This introduces a percentage-flushing mechanism that works in-tandem to the
>> mdadm delay timer. The percentage argument is based on the number of chunks
>> dirty (rather than percentage), due to large drives requiring smaller and
>> smaller percentages (eg, 32TB drives-> 1% is 320GB).
> 
> I've been reading and re-reading this and I still don't understand
> what you're saying here.  You say you're adding a percentage based
> mechanism, but then you say it's based on chunk counts, not
> percentages.  I think you need to clean this up and re-word it.> 
> Maybe you're trying to say that you only take a percentage of the
> available write bandwidth per second or something like that? 
I'll adjust it to chunk-count-based in the cover letter and make sure it
specifies bandwidth. I figured the chunk-count-based was a good way to
cover the desired percentage-based feature [1]. 

[1] https://elixir.bootlin.com/linux/latest/source/drivers/md/md-bitmap.c#L16


> 
> 
>> This set hopes to provide a way to make the bitmap flushing more consistent. It
>> was observed that a synchronous, random write qd1 workload, could make bitmap
>> writes easily become almost half of the I/O. And in similar workloads with
>> different timing, it was several minutes between bitmap updates. This is too
>> inconsistent to be reliable.
> 
>> This first and second patches adds the flush_threshold parameter. The default
>> value of 0 defines the default behavior: unplugging immediately just as before.
>> With a flush-threshold value of 1, it becomes more consistent and paranoid,
>> flushing on nearly every I/O, leading to a 40% or greater situation. From
> 
> What situation?  Please be more clear here.  

40% or more of given workload I/Os being bitmap flushes.
Will be more clear in v3

> 
>> there, the flush_threshold can be defined higher for those situations where
>> power loss is rare and full resync can be tolerated.
> 
>> The third patch converts the daemon worker to an actual timer. This makes it
>> more consistent and removes some ugly code.
> 
>> Jonathan Derrick (3):
>>   md/bitmap: Add chunk-threshold unplugging
>>   md/bitmap: Add sysfs interface for flush threshold
>>   md/bitmap: Convert daemon_work to proper timer
> 
>>  Documentation/admin-guide/md.rst |  5 ++
>>  drivers/md/md-bitmap.c           | 98 +++++++++++++++++++++++++-------
>>  drivers/md/md-bitmap.h           |  4 +-
>>  drivers/md/md.c                  |  9 ++-
>>  drivers/md/md.h                  |  2 +
>>  5 files changed, 93 insertions(+), 25 deletions(-)
> 
>> -- 
>> 2.31.1
>