anierin/msm-5.15
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

ANDROID: mm: fix speculative walk which is unsafe under RCU

Browse Source 
Speculative page fault handling expects MMU_GATHER_RCU_TABLE_FREE to
guarantee that page tables are stable, however tlb_remove_table() has
a slow-path fall-back case when __get_free_page() returns NULL and
tlb_remove_table_one() gets called. The way synchronization is
implemented in that function is not RCU-safe and require IRQs to be
disabled (see the comment in tlb_remove_table_sync_one()).
Fix the invalid assumption to disable IRQs even when
MMU_GATHER_RCU_TABLE_FREE=y.

Bug: 257443051
Change-Id: I227f351607cf73022cb31f6f7a232cab41cf6a5a
Signed-off-by: Suren Baghdasaryan <surenb@google.com>
This commit is contained in:
Suren Baghdasaryan
2022-11-21 14:23:00 -08:00
parent 749aa64ba8
commit 5fcb50b055
1 changed files with 4 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

11
mm/memory.c
View File

@@ -2767,16 +2767,13 @@ EXPORT_SYMBOL_GPL(apply_to_existing_page_range);
* This is similar to what fast GUP does, but fast GUP also needs to
* protect against races with THP page splitting, so it always needs
* to disable interrupts.
* Speculative page faults only need to protect against page table reclamation,
* so rcu_read_lock() is sufficient in the MMU_GATHER_RCU_TABLE_FREE case.
* Speculative page faults need to protect against page table reclamation,
* even with MMU_GATHER_RCU_TABLE_FREE case page table removal slow-path is
* not RCU-safe (see comment inside tlb_remove_table_sync_one), therefore
* we still have to disable IRQs.
*/
#ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE
#define speculative_page_walk_begin() rcu_read_lock()
#define speculative_page_walk_end() rcu_read_unlock()
#else
#define speculative_page_walk_begin() local_irq_disable()
#define speculative_page_walk_end() local_irq_enable()
#endif
bool __pte_map_lock(struct vm_fault *vmf)
{