Evolution-X-Devices/kernel_google_redbull
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
d4414bc0e93d8da170fd0fc9fef65fe84015677d
kernel_google_redbull/arch/arm64/include/asm
History
Kalesh Singh 924086cdc9 UPSTREAM: arm64: mremap speedup - enable HAVE_MOVE_PUD 
HAVE_MOVE_PUD enables remapping pages at the PUD level if both the source
and destination addresses are PUD-aligned.

With HAVE_MOVE_PUD enabled it can be inferred that there is approximately
a 19x improvement in performance on arm64.  (See data below).

------- Test Results ---------

The following results were obtained using a 5.4 kernel, by remapping a
PUD-aligned, 1GB sized region to a PUD-aligned destination.  The results
from 10 iterations of the test are given below:

Total mremap times for 1GB data on arm64. All times are in nanoseconds.

  Control          HAVE_MOVE_PUD

  1247761          74271
  1219896          46771
  1094792          59687
  1227760          48385
  1043698          76666
  1101771          50365
  1159896          52500
  1143594          75261
  1025833          61354
  1078125          48697

  1134312.6        59395.7    <-- Mean time in nanoseconds

A 1GB mremap completion time drops from ~1.1 milliseconds to ~59
microseconds on arm64.  (~19x speed up).

Link: https://lkml.kernel.org/r/20201014005320.2233162-5-kaleshsingh@google.com
Signed-off-by: Kalesh Singh <kaleshsingh@google.com>
Acked-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will@kernel.org>
Cc: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
Cc: Anshuman Khandual <anshuman.khandual@arm.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Geffon <bgeffon@google.com>
Cc: Christian Brauner <christian.brauner@ubuntu.com>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Frederic Weisbecker <frederic@kernel.org>
Cc: Gavin Shan <gshan@redhat.com>
Cc: Hassan Naveed <hnaveed@wavecomp.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Jia He <justin.he@arm.com>
Cc: John Hubbard <jhubbard@nvidia.com>
Cc: Kees Cook <keescook@chromium.org>
Cc: Krzysztof Kozlowski <krzk@kernel.org>
Cc: Lokesh Gidra <lokeshgidra@google.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Masahiro Yamada <masahiroy@kernel.org>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Mina Almasry <almasrymina@google.com>
Cc: Minchan Kim <minchan@google.com>
Cc: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Ralph Campbell <rcampbell@nvidia.com>
Cc: Ram Pai <linuxram@us.ibm.com>
Cc: Sami Tolvanen <samitolvanen@google.com>
Cc: Sandipan Das <sandipan@linux.ibm.com>
Cc: SeongJae Park <sjpark@amazon.de>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Steven Price <steven.price@arm.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Zi Yan <ziy@nvidia.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
(cherry picked from commit f5308c896d5de211245a9dc73b4e530f75185dd5)

Bug: 151772539
Signed-off-by: Kalesh Singh <kaleshsingh@google.com>
Change-Id: I30590b7375dde84fe345f4920a06f6a2c0b5aa31
2025-09-08 17:19:17 +03:00
..
vdso
BACKPORT: arm64: compat: vdso: Use legacy syscalls as fallback
2022-10-26 09:46:03 +00:00
xen
…
acenv.h
…
acpi.h
arm64: acpi: fix alignment fault in accessing ACPI
2018-07-23 15:34:12 +01:00
alternative.h
Merge 4.19.189 into android-4.19-stable
2021-04-29 09:48:07 +02:00
arch_gicv3.h
drivers: GICv3: remove the rtb logs of gic write and read
2018-07-26 16:47:49 -07:00
arch_timer.h
clocksource/arm_arch_timer: Remove use of workaround static key
2020-05-30 02:29:10 +08:00
arm_dsu_pmu.h
…
arm-cci.h
…
asm-bug.h
arm64: asm-bug: Add .align 2 to the end of __BUG_ENTRY
2024-06-16 13:23:36 +02:00
asm-offsets.h
…
asm-uaccess.h
BACKPORT: arm64: tags: Preserve tags for addresses translated via TTBR1
2019-10-23 18:45:12 +00:00
assembler.h
Merge 4.19.320 into android-4.19-stable
2024-08-28 09:03:40 +00:00
atomic_ll_sc.h
Merge 4.19.270 into android-4.19-stable
2023-01-18 12:02:44 +00:00
atomic_lse.h
Merge 4.19.270 into android-4.19-stable
2023-01-18 12:02:44 +00:00
atomic.h
atomics/treewide: Make conditional inc/dec ops optional
2018-06-21 14:25:24 +02:00
barrier.h
arm64: Add support for SB barrier and patch in over DSB; ISB sequences
2024-08-19 05:32:12 +02:00
bitops.h
locking/atomics/arm64, arm64/bitops: Include <asm-generic/bitops/ext2-atomic-setbit.h>
2018-06-21 12:52:12 +02:00
bitrev.h
…
boot.h
…
brk-imm.h
UPSTREAM: kasan, arm64: add brk handler for inline instrumentation
2019-09-24 17:44:13 -07:00
bug.h
…
cache.h
Merge 4.19.119 into android-4.19
2020-04-29 17:26:17 +02:00
cacheflush.h
Merge 4.19.129 into android-4.19-stable
2020-06-22 10:50:54 +02:00
checksum.h
arm64: csum: Fix handling of bad packets
2020-08-05 10:06:04 +02:00
clocksource.h
BACKPORT: arm64: Introduce asm/vdso/clocksource.h
2020-06-09 17:52:05 +08:00
cmpxchg.h
Revert "arm64: Avoid redundant type conversions in xchg() and cmpxchg()"
2021-03-08 11:06:02 +00:00
compat.h
asm-generic: Move common compat types to asm-generic/compat.h
2024-07-18 11:39:31 +02:00
compiler.h
…
cpu_ops.h
…
cpu.h
arm64: add ID_AA64ISAR2_EL1 sys register
2022-10-17 04:05:18 -07:00
cpucaps.h
Merge tag 'ASB-2024-09-05_4.19-stable' of https://android.googlesource.com/kernel/common into android-msm-pixel-4.19
2024-10-08 21:28:41 +03:00
cpufeature.h
Merge tag 'ASB-2024-08-05_4.19-stable' of https://android.googlesource.com/kernel/common into android-msm-pixel-4.19
2024-09-06 02:00:44 +03:00
cpuidle.h
…
cputype.h
Merge tag 'ASB-2024-12-05_4.19-stable' of https://android.googlesource.com/kernel/common into android-msm-pixel-4.19
2024-12-16 01:51:21 +02:00
current.h
…
daifflags.h
…
dcc.h
…
debug-monitors.h
Merge android-4.19-stable (4.19.294) into android-msm-pixel-4.19-lts
2023-10-17 05:00:19 +00:00
debugv8.h
soc: qcom: Add snapshot of MSM JTAG drivers
2018-12-02 18:40:54 -08:00
device.h
ANDROID: GKI: arm64: dma-mapping: add support for IOMMU mapper
2020-05-01 20:42:11 -07:00
dma-contiguous.h
ANDROID: GKI: arm64: Support early fixup for CMA
2020-05-01 21:57:16 -07:00
dma-iommu.h
ANDROID: GKI: iommu: dma-mapping-fast: Fast ARMv7/v8 Long Descriptor Format
2020-05-01 21:56:54 -07:00
dma-mapping.h
…
dmi.h
…
edac.h
edac: cortex: Snapshot arm64 cortex-a cpu edac driver
2019-09-05 18:56:52 +05:30
efi.h
UPSTREAM: arm64: efi: Make efi_rt_lock a raw_spinlock
2024-01-29 15:46:06 +00:00
elf.h
UPSTREAM: arm64: vdso: Fix population of AT_SYSINFO_EHDR for compat vdso
2020-06-09 17:51:57 +08:00
esr.h
arm64: compat: Add CNTFRQ trap handler
2019-05-20 19:57:14 -07:00
etmv4x.h
soc: qcom: Add snapshot of MSM JTAG drivers
2018-12-02 18:40:54 -08:00
exception.h
…
exec.h
…
extable.h
…
fb.h
…
fixmap.h
arm64: entry: Allow the trampoline text to occupy multiple pages
2022-10-17 04:03:01 -07:00
fpsimd.h
arm64: move sve_user_{enable,disable} to <asm/fpsimd.h>
2018-07-12 14:40:39 +01:00
fpsimdmacros.h
arm64/sve: Write ZCR_EL1 on context switch only if changed
2018-05-17 18:19:53 +01:00
ftrace.h
arm64: ftrace: Fix to enable syscall events on arm64
2018-12-08 12:59:06 +01:00
futex.h
arm64: futex: Avoid copying out uninitialised stack in failed cmpxchg()
2019-07-03 13:14:49 +02:00
hardirq.h
arm64: Fix HCR.TGE status for NMI contexts
2019-03-23 20:10:08 +01:00
hugetlb.h
…
hw_breakpoint.h
perf/hw_breakpoint: Remove default hw_breakpoint_arch_parse()
2018-06-26 09:07:58 +02:00
hwcap.h
…
hypervisor.h
…
insn.h
arm64: insn: Fix ldadd instruction encoding
2019-07-03 13:14:49 +02:00
io.h
Merge android-4.19.21 (6e0411b) into msm-4.19
2019-03-13 10:36:59 -07:00
irq_work.h
…
irq.h
arm64: Use the new GENERIC_IRQ_MULTI_HANDLER
2018-08-03 12:14:09 +02:00
irqflags.h
…
jump_label.h
arm64: jump_label.h: use asm_volatile_goto macro instead of "asm goto"
2018-09-11 11:07:11 +01:00
kasan.h
UPSTREAM: kasan: add tag related helper functions
2019-09-24 17:44:12 -07:00
Kbuild
arm64: locking: Replace ticket lock implementation with qspinlock
2018-07-05 10:05:06 +01:00
kernel-pgtable.h
arm64: head: avoid over-mapping in map_memory
2021-09-22 11:47:59 +02:00
kexec.h
…
kgdb.h
…
kprobes.h
arm64/kprobes: Remove jprobe implementation
2018-06-21 12:33:07 +02:00
kryo-arm64-edac.h
spdx: Modify spdx tag from GPL-2.0 to GPL-2.0-only
2018-12-10 11:09:45 -08:00
kvm_arm.h
KVM: arm64: Disable guest access to trace filter controls
2021-04-16 11:49:30 +02:00
kvm_asm.h
ANDROID: arm64: add __va_function
2025-09-08 16:41:50 +03:00
kvm_coproc.h
…
kvm_emulate.h
KVM: arm64: Assume write fault on S1PTW permission fault on instruction fetch
2020-10-01 13:14:54 +02:00
kvm_host.h
KVM: arm64: Allow SMCCC_ARCH_WORKAROUND_3 to be discovered and migrated
2022-10-17 04:05:04 -07:00
kvm_hyp.h
Merge 4.19.182 into android-4.19-stable
2021-03-20 10:45:47 +01:00
kvm_mmio.h
Merge android-4.19.110:wq(1984fff) into msm-4.19
2020-05-28 18:06:23 +05:30
kvm_mmu.h
KVM: arm64: Add templates for BHB mitigation sequences
2022-10-17 04:04:34 -07:00
linkage.h
…
lse.h
Revert "ANDROID: arm64: lse: fix LSE atomics with LTO"
2021-01-11 17:34:43 -08:00
memblock.h
…
memory.h
ANDROID: arm64: add __va_function
2025-09-08 16:41:50 +03:00
mmu_context.h
Merge LA.UM.9.12.R1.10.00.00.597.032 via branch 'qcom-msm-4.19-7250' into android-msm-pixel-4.19
2020-05-30 02:18:38 +08:00
mmu.h
KVM: arm64: Add templates for BHB mitigation sequences
2022-10-17 04:04:34 -07:00
mmzone.h
…
module.h
arm64/kernel: rename module_emit_adrp_veneer->module_emit_veneer_for_adrp
2018-04-24 19:07:35 +01:00
neon.h
arm64: drop unused kernel_neon_begin_partial() macro
2018-07-31 10:13:50 +01:00
numa.h
arm64/mm: return cpu_all_mask when node is NUMA_NO_NODE
2020-11-05 11:08:42 +01:00
page-def.h
…
page.h
FROMLIST: arm64: define __alloc_zeroed_user_highpage
2020-05-30 02:19:01 +08:00
paravirt.h
…
pci.h
PCI: remove PCI_DMA_BUS_IS_PHYS
2018-05-07 07:15:41 +02:00
percpu.h
arm64: percpu: Initialize ret in the default case
2018-11-27 16:13:01 +01:00
perf_event.h
…
pgalloc.h
BACKPORT: arm64: mm: enable per pmd page table lock
2025-09-08 17:19:11 +03:00
pgtable-hwdef.h
BACKPORT: kasan, arm64: enable top byte ignore for the kernel
2019-09-24 17:44:12 -07:00
pgtable-prot.h
KVM: arm64: Fix definition of PAGE_HYP_DEVICE
2020-07-16 08:17:25 +02:00
pgtable-types.h
…
pgtable.h
UPSTREAM: arm64: mremap speedup - enable HAVE_MOVE_PUD
2025-09-08 17:19:17 +03:00
probes.h
…
proc-fns.h
ANDROID: GKI: Revert "arm64: kill flush_cache_all()"
2020-04-10 16:01:53 +00:00
processor.h
Merge android-4.19-stable (4.19.261) into android-msm-pixel-4.19-lts
2022-10-26 14:55:01 +08:00
ptdump.h
…
ptrace.h
Merge android-4.19.110:wq(1984fff) into msm-4.19
2020-05-28 18:06:23 +05:30
scs.h
FROMLIST: arm64: implement Shadow Call Stack
2020-05-30 02:15:12 +08:00
sdei.h
arm64: Add stack information to on_accessible_stack
2018-07-26 11:36:07 +01:00
seccomp.h
…
sections.h
arm64: entry: Allow the trampoline text to occupy multiple pages
2022-10-17 04:03:01 -07:00
shmparam.h
…
signal32.h
UPSTREAM: arm64: compat: Expose signal related structures
2020-06-09 17:51:53 +08:00
simd.h
arm64: neon: Fix function may_use_simd() return error status
2018-07-11 17:02:02 +01:00
smp_plat.h
…
smp.h
GKI: arm64: smp: Fix smp dependency on CONFIG_MSM_PM
2020-05-30 02:02:51 +08:00
sparsemem.h
arm64: Make section size configurable for memory hotplug
2018-12-04 12:47:01 -08:00
spinlock_types.h
arm64: locking: Replace ticket lock implementation with qspinlock
2018-07-05 10:05:06 +01:00
spinlock.h
arm64: locking: Replace ticket lock implementation with qspinlock
2018-07-05 10:05:06 +01:00
stack_pointer.h
…
stackprotector.h
…
stacktrace.h
FROMLIST: arm64: implement Shadow Call Stack
2020-05-30 02:15:12 +08:00
stage2_pgtable-nopmd.h
…
stage2_pgtable-nopud.h
…
stage2_pgtable.h
…
stat.h
compat: Move compat_timespec/ timeval to compat_time.h
2018-04-19 13:29:54 +02:00
string.h
arm64: lib: use C string functions with KASAN enabled
2019-12-01 09:17:01 +01:00
suspend.h
FROMLIST: arm64: preserve x18 when CPU is suspended
2020-05-30 02:15:11 +08:00
sync_bitops.h
…
syscall_wrapper.h
ANDROID: arm64: fix function types in COND_SYSCALL
2019-07-25 09:36:55 -07:00
syscall.h
arm64: compat: Ensure upper 32 bits of x0 are zero on syscall return
2020-07-22 09:32:13 +02:00
sysreg.h
Merge 4.19.320 into android-4.19-stable
2024-08-28 09:03:40 +00:00
system_misc.h
ANDROID: GKI: ARM/ARM64: Introduce arch_read_hardware_id
2020-04-10 18:15:14 -07:00
thread_info.h
Merge android-4.19-stable.136 (204dd19) into msm-4.19
2020-10-14 20:04:29 +05:30
timex.h
…
tlb.h
BACKPORT: arm64: mm: enable per pmd page table lock
2025-09-08 17:19:11 +03:00
tlbflush.h
UPSTREAM: arm64: tlbflush: Ensure start/end of address range are aligned to stride
2025-09-08 17:19:11 +03:00
topology.h
GKI: arch_topology: compile out walt-specific field when using pelt
2020-05-30 02:28:43 +08:00
traps.h
Merge LA.UM.9.12.R1.10.00.00.597.032 via branch 'qcom-msm-4.19-7250' into android-msm-pixel-4.19
2020-05-30 02:18:38 +08:00
uaccess.h
Merge 4.19.320 into android-4.19-stable
2024-08-28 09:03:40 +00:00
unistd32.h
BACKPORT: arch: add pidfd and io_uring syscalls everywhere
2020-05-30 02:07:28 +08:00
unistd.h
UPSTREAM: arm64: compat: Fix syscall number of compat_clock_getres
2020-06-09 17:52:01 +08:00
uprobes.h
arm64/uprobes: change the uprobe_opcode_t typedef to fix the sparse warning
2024-11-08 16:19:22 +01:00
vdso.h
ANDROID: arm64: Fix off-by-one vdso trampoline return value
2020-11-16 15:38:35 -08:00
vectors.h
arm64: Use the clearbhb instruction in mitigations
2022-10-17 04:05:34 -07:00
virt.h
arm64: capabilities: Change scope of VHE to Boot CPU feature
2018-03-26 18:01:41 +01:00
vmap_stack.h
…
word-at-a-time.h
arm64: fix inline asm in load_unaligned_zeropad()
2021-04-28 13:16:50 +02:00