[4/5] f2fs: update block age info during out of place update
Commit Message
Signed-off-by: qixiaoyu1 <qixiaoyu1@xiaomi.com>
Signed-off-by: xiongping1 <xiongping1@xiaomi.com>
---
fs/f2fs/block_age.c | 89 ++++++++++++++++++++++++++++++++++++++++++++-
fs/f2fs/f2fs.h | 1 +
fs/f2fs/segment.c | 4 ++
3 files changed, 92 insertions(+), 2 deletions(-)
Comments
Hi qixiaoyu1,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on v6.1-rc7]
[also build test WARNING on linus/master next-20221128]
[cannot apply to jaegeuk-f2fs/dev-test jaegeuk-f2fs/dev]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/qixiaoyu1/Support-enhanced-hot-cold-data-separation-for-f2fs/20221128-170808
patch link: https://lore.kernel.org/r/20221128085859.5295-5-qixiaoyu1%40xiaomi.com
patch subject: [PATCH 4/5] f2fs: update block age info during out of place update
config: sparc-allyesconfig
compiler: sparc64-linux-gcc (GCC) 12.1.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# https://github.com/intel-lab-lkp/linux/commit/0ac666d056ce1e58a5ffea1fe153c413dd8b7148
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review qixiaoyu1/Support-enhanced-hot-cold-data-separation-for-f2fs/20221128-170808
git checkout 0ac666d056ce1e58a5ffea1fe153c413dd8b7148
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=sparc SHELL=/bin/bash fs/f2fs/
If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <lkp@intel.com>
All warnings (new ones prefixed by >>):
>> fs/f2fs/block_age.c:505:20: warning: no previous prototype for 'f2fs_get_cur_dblock_allocated' [-Wmissing-prototypes]
505 | unsigned long long f2fs_get_cur_dblock_allocated(struct f2fs_sb_info *sbi)
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
vim +/f2fs_get_cur_dblock_allocated +505 fs/f2fs/block_age.c
504
> 505 unsigned long long f2fs_get_cur_dblock_allocated(struct f2fs_sb_info *sbi)
506 {
507 return atomic64_read(&sbi->total_data_alloc);
508 }
509
Hi qixiaoyu1,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on v6.1-rc7]
[also build test ERROR on linus/master]
[cannot apply to jaegeuk-f2fs/dev-test jaegeuk-f2fs/dev next-20221208]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/qixiaoyu1/Support-enhanced-hot-cold-data-separation-for-f2fs/20221128-170808
patch link: https://lore.kernel.org/r/20221128085859.5295-5-qixiaoyu1%40xiaomi.com
patch subject: [PATCH 4/5] f2fs: update block age info during out of place update
config: xtensa-allyesconfig
compiler: xtensa-linux-gcc (GCC) 12.1.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# https://github.com/intel-lab-lkp/linux/commit/0ac666d056ce1e58a5ffea1fe153c413dd8b7148
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review qixiaoyu1/Support-enhanced-hot-cold-data-separation-for-f2fs/20221128-170808
git checkout 0ac666d056ce1e58a5ffea1fe153c413dd8b7148
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=xtensa SHELL=/bin/bash
If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <lkp@intel.com>
All errors (new ones prefixed by >>):
`.exit.text' referenced in section `__jump_table' of fs/cifs/cifsfs.o: defined in discarded section `.exit.text' of fs/cifs/cifsfs.o
`.exit.text' referenced in section `__jump_table' of fs/cifs/cifsfs.o: defined in discarded section `.exit.text' of fs/cifs/cifsfs.o
`.exit.text' referenced in section `__jump_table' of fs/fuse/inode.o: defined in discarded section `.exit.text' of fs/fuse/inode.o
`.exit.text' referenced in section `__jump_table' of fs/fuse/inode.o: defined in discarded section `.exit.text' of fs/fuse/inode.o
xtensa-linux-ld: fs/f2fs/block_age.o: in function `f2fs_get_cur_dblock_allocated':
>> block_age.c:(.text+0x2bc4): undefined reference to `__udivdi3'
xtensa-linux-ld: fs/f2fs/block_age.o: in function `f2fs_update_data_block_age':
block_age.c:(.text+0x30ce): undefined reference to `__udivdi3'
xtensa-linux-ld: fs/f2fs/block_age.o: in function `f2fs_get_cur_dblock_allocated':
block_age.c:(.text+0x2bd0): undefined reference to `__udivdi3'
xtensa-linux-ld: fs/f2fs/block_age.o: in function `f2fs_update_data_block_age':
block_age.c:(.text+0x3113): undefined reference to `__udivdi3'
`.exit.text' referenced in section `__jump_table' of fs/ceph/super.o: defined in discarded section `.exit.text' of fs/ceph/super.o
`.exit.text' referenced in section `__jump_table' of fs/ceph/super.o: defined in discarded section `.exit.text' of fs/ceph/super.o
`.exit.text' referenced in section `__jump_table' of lib/test_dynamic_debug.o: defined in discarded section `.exit.text' of lib/test_dynamic_debug.o
`.exit.text' referenced in section `__jump_table' of lib/test_dynamic_debug.o: defined in discarded section `.exit.text' of lib/test_dynamic_debug.o
`.exit.text' referenced in section `__jump_table' of drivers/rapidio/rio_cm.o: defined in discarded section `.exit.text' of drivers/rapidio/rio_cm.o
`.exit.text' referenced in section `__jump_table' of drivers/rapidio/rio_cm.o: defined in discarded section `.exit.text' of drivers/rapidio/rio_cm.o
`.exit.text' referenced in section `__jump_table' of drivers/rapidio/switches/idt_gen2.o: defined in discarded section `.exit.text' of drivers/rapidio/switches/idt_gen2.o
`.exit.text' referenced in section `__jump_table' of drivers/rapidio/switches/idt_gen2.o: defined in discarded section `.exit.text' of drivers/rapidio/switches/idt_gen2.o
`.exit.text' referenced in section `__jump_table' of drivers/rapidio/switches/idt_gen2.o: defined in discarded section `.exit.text' of drivers/rapidio/switches/idt_gen2.o
`.exit.text' referenced in section `__jump_table' of drivers/rapidio/switches/idt_gen2.o: defined in discarded section `.exit.text' of drivers/rapidio/switches/idt_gen2.o
`.exit.text' referenced in section `__jump_table' of drivers/rapidio/switches/idt_gen3.o: defined in discarded section `.exit.text' of drivers/rapidio/switches/idt_gen3.o
`.exit.text' referenced in section `__jump_table' of drivers/rapidio/switches/idt_gen3.o: defined in discarded section `.exit.text' of drivers/rapidio/switches/idt_gen3.o
`.exit.text' referenced in section `__jump_table' of drivers/rapidio/switches/idt_gen3.o: defined in discarded section `.exit.text' of drivers/rapidio/switches/idt_gen3.o
`.exit.text' referenced in section `__jump_table' of drivers/rapidio/switches/idt_gen3.o: defined in discarded section `.exit.text' of drivers/rapidio/switches/idt_gen3.o
`.exit.text' referenced in section `__jump_table' of drivers/video/fbdev/vt8623fb.o: defined in discarded section `.exit.text' of drivers/video/fbdev/vt8623fb.o
`.exit.text' referenced in section `__jump_table' of drivers/video/fbdev/vt8623fb.o: defined in discarded section `.exit.text' of drivers/video/fbdev/vt8623fb.o
`.exit.text' referenced in section `__jump_table' of drivers/video/fbdev/s3fb.o: defined in discarded section `.exit.text' of drivers/video/fbdev/s3fb.o
`.exit.text' referenced in section `__jump_table' of drivers/video/fbdev/s3fb.o: defined in discarded section `.exit.text' of drivers/video/fbdev/s3fb.o
`.exit.text' referenced in section `__jump_table' of drivers/video/fbdev/arkfb.o: defined in discarded section `.exit.text' of drivers/video/fbdev/arkfb.o
`.exit.text' referenced in section `__jump_table' of drivers/video/fbdev/arkfb.o: defined in discarded section `.exit.text' of drivers/video/fbdev/arkfb.o
`.exit.text' referenced in section `__jump_table' of drivers/gpu/drm/msm/disp/mdp5/mdp5_kms.o: defined in discarded section `.exit.text' of drivers/gpu/drm/msm/disp/mdp5/mdp5_kms.o
`.exit.text' referenced in section `__jump_table' of drivers/gpu/drm/msm/disp/mdp5/mdp5_kms.o: defined in discarded section `.exit.text' of drivers/gpu/drm/msm/disp/mdp5/mdp5_kms.o
`.exit.text' referenced in section `__jump_table' of drivers/gpu/drm/msm/msm_drv.o: defined in discarded section `.exit.text' of drivers/gpu/drm/msm/msm_drv.o
`.exit.text' referenced in section `__jump_table' of drivers/gpu/drm/msm/msm_drv.o: defined in discarded section `.exit.text' of drivers/gpu/drm/msm/msm_drv.o
`.exit.text' referenced in section `__jump_table' of drivers/gpu/drm/msm/dsi/dsi.o: defined in discarded section `.exit.text' of drivers/gpu/drm/msm/dsi/dsi.o
`.exit.text' referenced in section `__jump_table' of drivers/gpu/drm/msm/dsi/dsi.o: defined in discarded section `.exit.text' of drivers/gpu/drm/msm/dsi/dsi.o
`.exit.text' referenced in section `__jump_table' of drivers/misc/phantom.o: defined in discarded section `.exit.text' of drivers/misc/phantom.o
`.exit.text' referenced in section `__jump_table' of drivers/misc/phantom.o: defined in discarded section `.exit.text' of drivers/misc/phantom.o
`.exit.text' referenced in section `__jump_table' of drivers/misc/habanalabs/common/habanalabs_drv.o: defined in discarded section `.exit.text' of drivers/misc/habanalabs/common/habanalabs_drv.o
`.exit.text' referenced in section `__jump_table' of drivers/misc/habanalabs/common/habanalabs_drv.o: defined in discarded section `.exit.text' of drivers/misc/habanalabs/common/habanalabs_drv.o
`.exit.text' referenced in section `__jump_table' of drivers/scsi/fcoe/fcoe.o: defined in discarded section `.exit.text' of drivers/scsi/fcoe/fcoe.o
`.exit.text' referenced in section `__jump_table' of drivers/scsi/fcoe/fcoe.o: defined in discarded section `.exit.text' of drivers/scsi/fcoe/fcoe.o
`.exit.text' referenced in section `__jump_table' of drivers/scsi/cxgbi/libcxgbi.o: defined in discarded section `.exit.text' of drivers/scsi/cxgbi/libcxgbi.o
`.exit.text' referenced in section `__jump_table' of drivers/scsi/cxgbi/libcxgbi.o: defined in discarded section `.exit.text' of drivers/scsi/cxgbi/libcxgbi.o
`.exit.text' referenced in section `__jump_table' of drivers/target/target_core_configfs.o: defined in discarded section `.exit.text' of drivers/target/target_core_configfs.o
`.exit.text' referenced in section `__jump_table' of drivers/target/target_core_configfs.o: defined in discarded section `.exit.text' of drivers/target/target_core_configfs.o
`.exit.text' referenced in section `__jump_table' of drivers/mtd/maps/pcmciamtd.o: defined in discarded section `.exit.text' of drivers/mtd/maps/pcmciamtd.o
`.exit.text' referenced in section `__jump_table' of drivers/mtd/maps/pcmciamtd.o: defined in discarded section `.exit.text' of drivers/mtd/maps/pcmciamtd.o
`.exit.text' referenced in section `__jump_table' of drivers/net/wireless/purelifi/plfxlc/usb.o: defined in discarded section `.exit.text' of drivers/net/wireless/purelifi/plfxlc/usb.o
`.exit.text' referenced in section `__jump_table' of drivers/net/wireless/purelifi/plfxlc/usb.o: defined in discarded section `.exit.text' of drivers/net/wireless/purelifi/plfxlc/usb.o
`.exit.text' referenced in section `__jump_table' of drivers/net/wireless/zydas/zd1211rw/zd_usb.o: defined in discarded section `.exit.text' of drivers/net/wireless/zydas/zd1211rw/zd_usb.o
`.exit.text' referenced in section `__jump_table' of drivers/net/wireless/zydas/zd1211rw/zd_usb.o: defined in discarded section `.exit.text' of drivers/net/wireless/zydas/zd1211rw/zd_usb.o
`.exit.text' referenced in section `__jump_table' of drivers/net/wireless/ray_cs.o: defined in discarded section `.exit.text' of drivers/net/wireless/ray_cs.o
`.exit.text' referenced in section `__jump_table' of drivers/net/wireless/ray_cs.o: defined in discarded section `.exit.text' of drivers/net/wireless/ray_cs.o
`.exit.text' referenced in section `__jump_table' of drivers/net/wireless/mac80211_hwsim.o: defined in discarded section `.exit.text' of drivers/net/wireless/mac80211_hwsim.o
`.exit.text' referenced in section `__jump_table' of drivers/net/wireless/mac80211_hwsim.o: defined in discarded section `.exit.text' of drivers/net/wireless/mac80211_hwsim.o
`.exit.text' referenced in section `__jump_table' of drivers/usb/gadget/legacy/inode.o: defined in discarded section `.exit.text' of drivers/usb/gadget/legacy/inode.o
`.exit.text' referenced in section `__jump_table' of drivers/usb/gadget/legacy/inode.o: defined in discarded section `.exit.text' of drivers/usb/gadget/legacy/inode.o
`.exit.text' referenced in section `__jump_table' of drivers/usb/gadget/legacy/g_ffs.o: defined in discarded section `.exit.text' of drivers/usb/gadget/legacy/g_ffs.o
`.exit.text' referenced in section `__jump_table' of drivers/usb/gadget/legacy/g_ffs.o: defined in discarded section `.exit.text' of drivers/usb/gadget/legacy/g_ffs.o
`.exit.text' referenced in section `__jump_table' of drivers/media/common/siano/smscoreapi.o: defined in discarded section `.exit.text' of drivers/media/common/siano/smscoreapi.o
`.exit.text' referenced in section `__jump_table' of drivers/media/common/siano/smscoreapi.o: defined in discarded section `.exit.text' of drivers/media/common/siano/smscoreapi.o
`.exit.text' referenced in section `__jump_table' of drivers/staging/vme_user/vme_fake.o: defined in discarded section `.exit.text' of drivers/staging/vme_user/vme_fake.o
`.exit.text' referenced in section `__jump_table' of drivers/staging/vme_user/vme_fake.o: defined in discarded section `.exit.text' of drivers/staging/vme_user/vme_fake.o
`.exit.text' referenced in section `__jump_table' of net/netfilter/nf_conntrack_h323_main.o: defined in discarded section `.exit.text' of net/netfilter/nf_conntrack_h323_main.o
`.exit.text' referenced in section `__jump_table' of net/netfilter/nf_conntrack_h323_main.o: defined in discarded section `.exit.text' of net/netfilter/nf_conntrack_h323_main.o
`.exit.text' referenced in section `__jump_table' of net/netfilter/ipset/ip_set_core.o: defined in discarded section `.exit.text' of net/netfilter/ipset/ip_set_core.o
`.exit.text' referenced in section `__jump_table' of net/netfilter/ipset/ip_set_core.o: defined in discarded section `.exit.text' of net/netfilter/ipset/ip_set_core.o
`.exit.text' referenced in section `__jump_table' of net/rds/rdma_transport.o: defined in discarded section `.exit.text' of net/rds/rdma_transport.o
`.exit.text' referenced in section `__jump_table' of net/rds/rdma_transport.o: defined in discarded section `.exit.text' of net/rds/rdma_transport.o
`.exit.text' referenced in section `__jump_table' of net/rds/rdma_transport.o: defined in discarded section `.exit.text' of net/rds/rdma_transport.o
`.exit.text' referenced in section `__jump_table' of net/rds/rdma_transport.o: defined in discarded section `.exit.text' of net/rds/rdma_transport.o
`.exit.text' referenced in section `__jump_table' of net/ceph/ceph_common.o: defined in discarded section `.exit.text' of net/ceph/ceph_common.o
`.exit.text' referenced in section `__jump_table' of net/ceph/ceph_common.o: defined in discarded section `.exit.text' of net/ceph/ceph_common.o
@@ -9,9 +9,14 @@
#include <linux/f2fs_fs.h>
#include "f2fs.h"
+#include "node.h"
#include "segment.h"
#include <trace/events/f2fs.h>
+
+#define LAST_AGE_WEIGHT 30
+#define SAME_AGE_REGION 1024
+
static struct kmem_cache *age_extent_tree_slab;
static struct kmem_cache *age_extent_node_slab;
@@ -264,8 +269,8 @@ static inline bool __is_age_extent_mergeable(struct age_extent_info *back,
struct age_extent_info *front)
{
return (back->fofs + back->len == front->fofs &&
- back->age == front->age &&
- back->last_blocks == front->last_blocks);
+ abs(back->age - front->age) <= SAME_AGE_REGION &&
+ abs(back->last_blocks - front->last_blocks) <= SAME_AGE_REGION);
}
static inline bool __is_back_age_ext_mergeable(struct age_extent_info *cur,
@@ -497,6 +502,86 @@ void f2fs_truncate_age_extent_cache(struct inode *inode, pgoff_t fofs, unsigned
f2fs_update_age_extent_cache(inode, fofs, len, 0, 0);
}
+unsigned long long f2fs_get_cur_dblock_allocated(struct f2fs_sb_info *sbi)
+{
+ return atomic64_read(&sbi->total_data_alloc);
+}
+
+static unsigned long long calculate_block_age(unsigned long long new,
+ unsigned long long old)
+{
+ if (new >= old)
+ return new - (new - old) * LAST_AGE_WEIGHT / 100;
+ else
+ return new + (old - new) * LAST_AGE_WEIGHT / 100;
+}
+
+void f2fs_update_data_block_age(struct dnode_of_data *dn)
+{
+ struct f2fs_sb_info *sbi = F2FS_I_SB(dn->inode);
+ unsigned long long cur_total_blk_alloced = f2fs_get_cur_dblock_allocated(sbi);
+ pgoff_t fofs;
+ unsigned long long cur_age, new_age;
+ struct age_extent_info ei;
+ bool find;
+ loff_t f_size = i_size_read(dn->inode);
+
+ if (!f2fs_may_age_extent_tree(dn->inode))
+ return;
+
+ fofs = f2fs_start_bidx_of_node(ofs_of_node(dn->node_page), dn->inode) +
+ dn->ofs_in_node;
+
+
+ /* When I/O is not aligned to a PAGE_SIZE, update will happen to the last
+ * file block even in seq write. So don't record age for newly last file
+ * block here.
+ */
+ if ((f_size >> PAGE_SHIFT) == fofs && f_size & (PAGE_SIZE - 1) &&
+ dn->data_blkaddr == NEW_ADDR)
+ return;
+
+ find = f2fs_lookup_age_extent_cache(dn->inode, fofs, &ei);
+ if (find) {
+ if (cur_total_blk_alloced >= ei.last_blocks)
+ cur_age = cur_total_blk_alloced - ei.last_blocks;
+ else
+ /* total_data_alloc overflow */
+ cur_age = ULLONG_MAX - ei.last_blocks + cur_total_blk_alloced;
+
+ if (ei.age)
+ new_age = calculate_block_age(cur_age, ei.age);
+ else
+ new_age = cur_age;
+
+ WARN(new_age > cur_total_blk_alloced,
+ "inode block(%lu: %lu) age changed from: %llu to %llu",
+ dn->inode->i_ino, fofs, ei.age, new_age);
+ } else {
+ f2fs_bug_on(sbi, dn->data_blkaddr == NULL_ADDR);
+
+ if (dn->data_blkaddr == NEW_ADDR)
+ /* the data block was allocated for the first time */
+ new_age = 0;
+ else {
+ if (__is_valid_data_blkaddr(dn->data_blkaddr) &&
+ !f2fs_is_valid_blkaddr(sbi, dn->data_blkaddr,
+ DATA_GENERIC_ENHANCE)) {
+ f2fs_bug_on(sbi, 1);
+ return;
+ }
+
+ /*
+ * init block age with zero, this can happen when the block age extent
+ * was reclaimed due to memory constraint or system reboot
+ */
+ new_age = 0;
+ }
+ }
+
+ f2fs_update_age_extent_cache(dn->inode, fofs, 1, new_age, cur_total_blk_alloced);
+}
+
void f2fs_destroy_age_extent_tree(struct inode *inode)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
@@ -4238,6 +4238,7 @@ bool f2fs_lookup_age_extent_cache(struct inode *inode, pgoff_t pgofs,
void f2fs_update_age_extent_cache(struct inode *inode, pgoff_t fofs,
unsigned int len, u64 age,
unsigned long long cur_blk_alloced);
+void f2fs_update_data_block_age(struct dnode_of_data *dn);
void f2fs_truncate_age_extent_cache(struct inode *inode, pgoff_t fofs,
unsigned int len);
int __init f2fs_create_age_extent_cache(void);
@@ -3417,6 +3417,10 @@ void f2fs_outplace_write_data(struct dnode_of_data *dn,
struct f2fs_summary sum;
f2fs_bug_on(sbi, dn->data_blkaddr == NULL_ADDR);
+#ifdef CONFIG_F2FS_FS_DATA_SEPARATION
+ if (fio->io_type == FS_DATA_IO || fio->io_type == FS_CP_DATA_IO)
+ f2fs_update_data_block_age(dn);
+#endif
set_summary(&sum, dn->nid, dn->ofs_in_node, fio->version);
do_write_page(&sum, fio);
f2fs_update_data_blkaddr(dn, fio->new_blkaddr);