anierin/msm-5.15
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

FROMLIST: locking/rwsem: Prevent non-first waiter from spinning in down_write() slowpath

Browse Source 
A non-first waiter can potentially spin in the for loop of
rwsem_down_write_slowpath() without sleeping but fail to acquire the
lock even if the rwsem is free if the following sequence happens:

  Non-first waiter       First waiter      Lock holder
  ----------------       ------------      -----------
  Acquire wait_lock
  rwsem_try_write_lock():
    Set handoff bit if RT or
      wait too long
    Set waiter->handoff_set
  Release wait_lock
                         Acquire wait_lock
                         Inherit waiter->handoff_set
                         Release wait_lock
					   Clear owner
                                           Release lock
  if (waiter.handoff_set) {
    rwsem_spin_on_owner(();
    if (OWNER_NULL)
      goto trylock_again;
  }
  trylock_again:
  Acquire wait_lock
  rwsem_try_write_lock():
     if (first->handoff_set && (waiter != first))
     	return false;
  Release wait_lock

It is especially problematic if the non-first waiter is an RT task and
it is running on the same CPU as the first waiter as this can lead to
live lock.

Bug: 252734649

Fixes: d257cc8cb8d5 ("locking/rwsem: Make handoff bit handling more consistent")
Signed-off-by: Waiman Long <longman@redhat.com>
Link: https://lore.kernel.org/lkml/20221012133333.1265281-2-longman@redhat.com/
Signed-off-by: wangting11 <wangting11@xiaomi.com>
Change-Id: I6ac35872e8aae61cb96cd87a365a2d44d66961eb
This commit is contained in:
wangting11
2022-10-13 21:44:22 +08:00
committed by Todd Kjos
parent b5e4b8916f
commit 4ece302f35
1 changed files with 10 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
kernel/locking/rwsem.c
View File

@@ -598,6 +598,11 @@ static inline bool rwsem_try_write_lock(struct rw_semaphore *sem,
new = count;
if (count & RWSEM_LOCK_MASK) {
/*
* A waiter (first or not) can set the handoff bit
* if it is an RT task or wait in the wait queue
* for too long.
*/
if (has_handoff || (!rt_task(waiter->task) &&
!time_after(jiffies, waiter->timeout)))
return false;
@@ -613,11 +618,13 @@ static inline bool rwsem_try_write_lock(struct rw_semaphore *sem,
} while (!atomic_long_try_cmpxchg_acquire(&sem->count, &count, new));
/*
* We have either acquired the lock with handoff bit cleared or
* set the handoff bit.
* We have either acquired the lock with handoff bit cleared or set
* the handoff bit. Only the first waiter can have its handoff_set
* set here to enable optimistic spinning in slowpath loop.
*/
if (new & RWSEM_FLAG_HANDOFF) {
waiter->handoff_set = true;
if (first)
waiter->handoff_set = true;
lockevent_inc(rwsem_wlock_handoff);
return false;
}