Files
kernel_xiaomi_raphael/include/linux
Joerg Roedel c3cf92fd9b x86/mm: split vmalloc_sync_all()
commit 763802b53a427ed3cbd419dbba255c414fdd9e7c upstream.

Commit 3f8fd02b1bf1 ("mm/vmalloc: Sync unmappings in
__purge_vmap_area_lazy()") introduced a call to vmalloc_sync_all() in
the vunmap() code-path.  While this change was necessary to maintain
correctness on x86-32-pae kernels, it also adds additional cycles for
architectures that don't need it.

Specifically on x86-64 with CONFIG_VMAP_STACK=y some people reported
severe performance regressions in micro-benchmarks because it now also
calls the x86-64 implementation of vmalloc_sync_all() on vunmap().  But
the vmalloc_sync_all() implementation on x86-64 is only needed for newly
created mappings.

To avoid the unnecessary work on x86-64 and to gain the performance
back, split up vmalloc_sync_all() into two functions:

	* vmalloc_sync_mappings(), and
	* vmalloc_sync_unmappings()

Most call-sites to vmalloc_sync_all() only care about new mappings being
synchronized.  The only exception is the new call-site added in the
above mentioned commit.

Shile Zhang directed us to a report of an 80% regression in reaim
throughput.

Fixes: 3f8fd02b1bf1 ("mm/vmalloc: Sync unmappings in __purge_vmap_area_lazy()")
Reported-by: kernel test robot <oliver.sang@intel.com>
Reported-by: Shile Zhang <shile.zhang@linux.alibaba.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Tested-by: Borislav Petkov <bp@suse.de>
Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>	[GHES]
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: <stable@vger.kernel.org>
Link: http://lkml.kernel.org/r/20191009124418.8286-1-joro@8bytes.org
Link: https://lists.01.org/hyperkitty/list/lkp@lists.01.org/thread/4D3JPPHBNOSPFK2KEPC6KGKS6J25AIDB/
Link: http://lkml.kernel.org/r/20191113095530.228959-1-shile.zhang@linux.alibaba.com
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2020-04-02 16:34:20 +02:00
..
2019-12-17 20:39:59 +01:00
2020-01-27 14:46:52 +01:00
2019-12-05 15:38:25 +01:00
2018-04-24 09:36:39 +02:00
2017-07-03 16:56:28 -06:00
2018-02-22 15:42:23 +01:00
2017-08-30 14:40:40 +02:00
2018-08-03 07:50:38 +02:00
2019-03-23 14:35:28 +01:00
2017-08-24 13:23:03 -07:00
2017-08-03 19:09:26 -07:00
2019-11-24 08:22:48 +01:00
2017-08-07 17:22:14 +02:00
2019-12-05 15:38:28 +01:00
2018-06-21 04:02:49 +09:00
2018-02-22 15:42:24 +01:00
2019-12-17 20:38:50 +01:00
2017-08-28 20:51:22 +02:00
2017-09-07 11:53:11 +02:00
2019-12-31 12:37:21 +01:00
2018-02-22 15:42:23 +01:00
2017-07-06 11:30:07 -04:00
2017-07-25 18:05:25 +02:00
2017-09-25 08:56:05 -06:00
2020-03-20 10:54:12 +01:00
2017-08-06 20:55:29 -07:00
2019-06-11 12:21:48 +02:00
2019-12-05 15:38:28 +01:00
2017-07-06 16:24:30 -07:00
2017-08-16 16:28:47 -07:00
2019-04-17 08:37:44 +02:00
2018-11-13 11:15:11 -08:00
2019-06-17 19:52:43 +02:00
2017-07-24 17:50:37 +02:00
2020-04-02 16:34:20 +02:00
2017-08-31 18:50:14 +02:00
2017-08-15 09:02:07 -07:00
2017-08-15 09:02:08 -07:00