Douglas Miller 966d2b04e0
percpu-refcount: fix reference leak during percpu-atomic transition ...
percpu_ref_tryget() and percpu_ref_tryget_live() should return
"true" IFF they acquire a reference. But the return value from
atomic_long_inc_not_zero() is a long and may have high bits set,
e.g. PERCPU_COUNT_BIAS, and the return value of the tryget routines
is bool so the reference may actually be acquired but the routines
return "false" which results in a reference leak since the caller
assumes it does not need to do a corresponding percpu_ref_put().
This was seen when performing CPU hotplug during I/O, as hangs in
blk_mq_freeze_queue_wait where percpu_ref_kill (blk_mq_freeze_queue_start)
raced with percpu_ref_tryget (blk_mq_timeout_work).
Sample stack trace:
__switch_to+0x2c0/0x450
__schedule+0x2f8/0x970
schedule+0x48/0xc0
blk_mq_freeze_queue_wait+0x94/0x120
blk_mq_queue_reinit_work+0xb8/0x180
blk_mq_queue_reinit_prepare+0x84/0xa0
cpuhp_invoke_callback+0x17c/0x600
cpuhp_up_callbacks+0x58/0x150
_cpu_up+0xf0/0x1c0
do_cpu_up+0x120/0x150
cpu_subsys_online+0x64/0xe0
device_online+0xb4/0x120
online_store+0xb4/0xc0
dev_attr_store+0x68/0xa0
sysfs_kf_write+0x80/0xb0
kernfs_fop_write+0x17c/0x250
__vfs_write+0x6c/0x1e0
vfs_write+0xd0/0x270
SyS_write+0x6c/0x110
system_call+0x38/0xe0
Examination of the queue showed a single reference (no PERCPU_COUNT_BIAS,
and __PERCPU_REF_DEAD, __PERCPU_REF_ATOMIC set) and no requests.
However, conditions at the time of the race are count of PERCPU_COUNT_BIAS + 0
and __PERCPU_REF_DEAD and __PERCPU_REF_ATOMIC set.
The fix is to make the tryget routines use an actual boolean internally instead
of the atomic long result truncated to a int.
Fixes: e625305b39https://bugzilla.kernel.org/show_bug.cgi?id=190751
Signed-off-by: Douglas Miller <dougmill@linux.vnet.ibm.com >
Reviewed-by: Jens Axboe <axboe@fb.com >
Signed-off-by: Tejun Heo <tj@kernel.org >
Fixes: e625305b39stable@vger.kernel.org # v3.18+
2017-01-28 07:49:42 -05:00
2016-12-14 20:42:45 -08:00
2016-10-05 10:11:24 -07:00
2016-09-22 10:01:21 +02:00
2016-12-16 11:23:34 -08:00
2016-11-02 20:44:20 +01:00
2016-09-15 21:15:40 +05:30
2016-09-10 16:48:46 +05:30
2016-09-01 14:23:39 -07:00
2016-11-29 15:51:49 -06:00
2016-11-29 09:17:20 +01:00
2017-01-19 09:57:20 +01:00
2016-11-17 23:19:00 +01:00
2016-12-31 14:35:04 +00:00
2016-07-15 14:50:53 -07:00
2016-12-25 11:04:12 +01:00
2016-07-28 09:34:47 +05:30
2016-12-19 08:16:26 -08:00
2016-12-29 14:17:40 -05:00
2016-12-28 14:36:52 -05:00
2016-12-05 14:16:22 +01:00
2016-11-29 18:28:30 -08:00
2016-12-06 21:42:19 +01:00
2016-07-03 10:55:07 +02:00
2016-10-03 19:43:08 -07:00
2016-12-14 10:49:33 -08:00
2016-12-17 20:17:04 -08:00
2016-11-23 23:37:27 +01:00
2016-12-02 12:44:37 -05:00
2016-10-07 09:45:43 -07:00
2016-12-05 11:38:30 +00:00
2016-10-18 20:12:05 +05:30
2016-11-14 21:52:14 -08:00
2016-07-08 16:23:11 +02:00
2016-10-20 08:44:57 +02:00
2016-12-15 16:03:25 -08:00
2016-10-21 12:09:30 +01:00
2017-01-12 15:56:40 -05:00
2016-09-14 20:37:50 +02:00
2016-12-13 11:10:36 -08:00
2016-12-06 16:57:33 +01:00
2016-12-15 12:46:48 -08:00
2016-09-20 14:35:59 -05:00
2016-12-05 14:31:24 -05:00
2016-12-22 22:58:37 -05:00
2016-12-01 14:45:08 +01:00
2016-09-05 12:41:46 +02:00
2016-10-19 14:34:36 -04:00
2016-07-08 17:01:14 +03:00
2016-09-27 12:54:03 +02:00
2016-07-07 09:16:20 +02:00
2016-12-05 19:00:38 -05:00
2016-10-11 15:06:31 -07:00
2016-10-11 15:06:31 -07:00
2016-12-13 10:19:16 -08:00
2016-08-04 14:19:16 -06:00
2016-07-26 16:19:19 -07:00
2016-07-25 16:51:49 +10:00
2016-12-09 08:30:51 -07:00
2016-09-09 12:09:24 +03:00
2016-08-18 10:44:20 +02:00
2016-10-07 18:46:26 -07:00
2016-12-01 07:58:40 -07:00
2016-11-01 09:43:26 -06:00
2016-09-15 08:42:03 -06:00
2016-12-14 10:49:33 -08:00
2017-01-13 15:17:04 -07:00
2016-10-28 08:48:16 -06:00
2016-09-15 18:29:06 -04:00
2016-10-07 18:46:28 -07:00
2016-12-08 13:31:11 -05:00
2016-12-17 11:14:25 -05:00
2017-01-18 17:12:26 -05:00
2016-11-30 10:22:27 -05:00
2016-11-17 20:15:26 -05:00
2016-11-04 14:34:47 -06:00
2016-10-31 13:01:58 -07:00
2016-08-11 09:41:35 -06:00
2016-12-22 09:25:45 -08:00
2016-11-22 13:21:00 -06:00
2016-10-25 11:08:28 +08:00
2016-11-25 16:25:52 -05:00
2016-10-14 12:18:50 -07:00
2016-10-17 15:27:27 -07:00
2016-12-08 16:37:33 -08:00
2016-12-25 11:04:12 +01:00
2016-12-12 18:55:08 -08:00
2016-10-07 18:46:29 -07:00
2016-09-27 21:20:53 -04:00
2016-12-12 18:55:09 -08:00
2016-10-15 10:03:15 -07:00
2016-12-21 10:16:05 -08:00
2016-12-13 11:18:24 -08:00
2016-08-02 16:11:27 -04:00
2017-01-14 19:32:40 -05:00
2016-08-31 13:05:43 +02:00
2016-12-25 10:47:43 +01:00
2016-11-21 14:35:43 +01:00
2017-01-16 13:20:05 +01:00
2016-11-29 14:02:21 +01:00
2016-12-15 11:32:31 +01:00
2016-10-07 18:46:30 -07:00
2016-12-17 16:00:34 -08:00
2016-10-11 15:06:30 -07:00
2016-12-26 20:29:24 -08:00
2016-12-03 20:51:49 -05:00
2016-12-05 19:01:16 -05:00
2016-12-13 11:42:18 -08:00
2016-07-26 16:19:19 -07:00
2016-11-23 10:07:35 +08:00
2016-09-06 13:08:40 +09:00
2016-09-14 13:56:38 -04:00
2016-12-13 11:42:18 -08:00
2016-10-28 08:48:16 -06:00
2016-10-25 14:40:39 +02:00
2016-09-26 22:16:41 +05:30
2016-10-25 14:40:39 +02:00
2016-11-15 12:48:33 +01:00
2016-11-14 16:58:36 +01:00
2016-12-14 16:04:08 -08:00
2016-11-30 08:54:04 +05:30
2016-10-27 16:16:09 -04:00
2016-12-25 11:04:12 +01:00
2016-08-04 08:50:07 -04:00
2016-12-15 08:58:12 -02:00
2017-01-14 16:48:53 +01:00
2016-12-09 09:03:02 -07:00
2016-07-01 16:37:06 +02:00
2016-10-14 14:26:58 -07:00
2016-10-06 09:07:44 -04:00
2016-08-04 10:16:56 +09:30
2016-09-16 08:53:41 +02:00
2016-11-25 10:15:13 -08:00
2016-10-03 09:11:14 -07:00
2016-10-20 14:51:10 -04:00
2016-10-15 10:03:15 -07:00
2016-12-05 19:01:16 -05:00
2017-01-16 14:03:31 -05:00
2016-08-02 19:35:10 -04:00
2016-11-11 08:12:37 -08:00
2016-12-19 08:23:53 -08:00
2016-12-11 16:33:18 -05:00
2016-12-05 15:13:58 +01:00
2017-01-05 23:06:06 -08:00
2016-12-05 18:58:32 -05:00
2016-09-02 12:47:55 -04:00
2016-12-15 13:49:34 -08:00
2016-12-25 17:21:22 +01:00
2016-11-29 15:57:42 +00:00
2016-12-22 11:48:20 -07:00
2016-10-27 16:16:09 -04:00
2017-01-10 18:31:55 -08:00
2016-11-07 16:24:29 -08:00
2016-10-20 14:51:09 -04:00
2016-10-17 14:23:46 +02:00
2016-08-21 20:12:32 +01:00
2016-12-14 10:12:26 +01:00
2016-10-20 14:51:10 -04:00
2016-08-24 15:58:57 +02:00
2016-12-25 17:21:22 +01:00
2016-12-14 16:04:09 -08:00
2016-10-07 18:46:29 -07:00
2016-11-21 22:50:45 +08:00
2016-12-09 21:54:22 -08:00
2016-12-06 11:52:49 +01:00
2016-09-05 13:52:38 +02:00
2016-08-25 22:11:02 +02:00
2016-11-24 16:22:06 +01:00
2017-01-12 20:06:15 +01:00
2016-07-27 14:20:09 -07:00
2016-12-14 16:04:10 -08:00
2016-10-27 16:03:24 +02:00
2016-07-08 13:23:12 +02:00
2016-11-27 20:38:47 -05:00
2016-09-01 22:48:33 -07:00
2016-09-24 08:01:26 -04:00
2016-10-07 23:50:39 -04:00
2016-11-15 12:16:13 -05:00
2016-12-20 09:48:44 -08:00
2016-09-08 16:13:09 -07:00
2016-09-16 09:18:54 +02:00
2016-11-27 16:15:33 -08:00
2016-07-13 17:27:24 -07:00
2016-11-19 09:42:35 -08:00
2016-11-09 08:25:09 +01:00
2016-08-24 09:43:49 +02:00
2016-10-26 15:45:38 +10:00
2016-11-30 14:36:01 +11:00
2016-12-06 17:32:16 +01:00
2016-09-14 09:18:09 -06:00
2016-08-08 14:42:03 -05:00
2016-07-27 10:24:38 -05:00
2016-12-03 23:21:37 -05:00
2016-09-20 23:20:32 +02:00
2016-09-14 15:28:15 +02:00
2016-07-04 12:26:05 +02:00
2016-12-24 11:46:01 -08:00
2016-06-30 11:40:54 -04:00
2016-10-07 18:46:26 -07:00
2017-01-12 14:33:16 +01:00
2016-10-03 13:22:39 -07:00
2016-10-16 11:02:31 +02:00
2016-10-27 18:43:43 -07:00
2016-12-14 16:04:08 -08:00
2016-10-09 12:23:38 -07:00
2016-11-15 09:51:05 +01:00
2017-01-17 10:56:45 -08:00
2016-08-10 11:23:44 -04:00
2016-12-16 09:26:42 -08:00
2016-07-26 16:19:19 -07:00
2016-10-11 15:06:33 -07:00
2016-07-26 16:19:19 -07:00
2016-12-12 18:55:06 -08:00
2016-12-25 17:21:23 +01:00
2016-12-13 15:47:02 -08:00
2016-07-18 08:46:09 +02:00
2016-11-22 12:07:05 +01:00
2016-10-19 14:34:36 -04:00
2016-12-06 16:08:10 -08:00
2016-11-29 12:12:51 -07:00
2016-10-31 13:01:57 -07:00
2016-08-18 23:41:55 +02:00
2016-11-30 14:32:25 +11:00
2016-09-19 13:42:38 -04:00
2016-09-19 12:27:10 +10:00
2016-11-18 13:54:21 -05:00
2016-11-29 18:02:58 +01:00
2016-09-27 12:33:47 +02:00
2017-01-04 13:22:38 -07:00
2016-12-06 11:05:46 +01:00
2016-10-07 18:46:28 -07:00
2017-01-10 18:31:55 -08:00
2017-01-24 16:26:14 -08:00
2016-12-12 18:55:07 -08:00
2016-07-28 16:07:41 -07:00
2017-01-27 11:10:50 -05:00
2016-07-26 16:19:19 -07:00
2016-11-02 15:17:09 -04:00
2016-12-17 20:17:04 -08:00
2017-01-10 18:31:55 -08:00
2016-12-18 11:12:53 -08:00
2017-01-10 18:31:54 -08:00
2016-08-02 19:35:02 -04:00
2016-07-26 16:19:19 -07:00
2017-01-24 16:26:14 -08:00
2016-12-14 16:33:38 +01:00
2016-11-26 11:18:02 -08:00
2016-12-16 16:34:52 -05:00
2016-07-01 23:45:18 +08:00
2016-09-25 23:41:39 -04:00
2016-09-25 23:41:39 -04:00
2016-09-14 22:11:08 +02:00
2016-11-16 10:39:43 +01:00
2016-07-29 12:17:52 -07:00
2016-10-07 09:20:53 -07:00
2016-08-28 23:32:41 -04:00
2017-01-10 21:30:33 -05:00
2016-12-06 21:48:21 +01:00
2016-12-06 21:42:16 +01:00
2016-11-16 18:32:02 -05:00
2016-12-15 20:42:48 -05:00
2016-09-22 15:54:27 -04:00
2016-12-19 17:29:44 -05:00
2016-12-09 23:47:10 -05:00
2017-01-24 16:26:14 -08:00
2016-12-24 11:23:24 -08:00
2016-12-06 10:17:56 +02:00
2016-12-06 10:17:56 +02:00
2016-07-08 08:38:49 -06:00
2016-12-13 10:19:16 -08:00
2016-12-12 18:55:07 -08:00
2016-10-03 23:38:11 +02:00
2016-11-29 15:57:43 +00:00
2016-11-29 23:17:02 -05:00
2016-11-14 15:21:14 -06:00
2016-07-08 18:14:03 -03:00
2016-11-10 17:03:35 +01:00
2016-09-28 08:53:14 +05:30
2016-08-29 11:00:23 +03:00
2016-10-07 18:46:28 -07:00
2016-09-19 21:44:30 +02:00
2016-10-07 18:46:27 -07:00
2016-10-07 18:46:27 -07:00
2016-12-29 11:03:15 -08:00
2016-12-25 11:54:48 -08:00
2016-12-06 10:17:03 +02:00
2016-11-17 18:47:58 -06:00
2016-12-15 12:46:48 -08:00
2016-12-06 13:45:48 -06:00
2016-12-06 13:45:50 -06:00
2016-12-15 12:46:48 -08:00
2017-01-28 07:49:42 -05:00
2016-09-22 15:25:54 +02:00
2017-01-14 11:06:49 +01:00
2017-01-25 14:40:19 -05:00
2017-01-25 14:40:19 -05:00
2016-08-08 14:42:01 -05:00
2016-10-31 16:18:30 -04:00
2016-10-05 18:24:00 -04:00
2016-10-14 11:36:59 -07:00
2016-10-21 22:20:56 +02:00
2016-12-06 02:27:59 +01:00
2016-12-13 11:42:18 -08:00
2016-11-29 18:02:58 +01:00
2016-12-13 11:42:18 -08:00
2016-07-12 19:25:38 -07:00
2016-12-24 11:46:01 -08:00
2016-09-27 21:52:00 -04:00
2016-10-08 11:06:08 -04:00
2016-07-01 22:44:34 +02:00
2016-12-06 10:32:40 +01:00
2016-12-12 18:55:09 -08:00
2016-12-13 16:07:55 -08:00
2016-09-22 19:59:41 -05:00
2016-09-30 15:26:52 -07:00
2016-11-15 16:34:26 -08:00
2016-11-15 16:34:27 -08:00
2016-12-12 19:56:15 -08:00
2016-07-01 05:32:17 -04:00
2016-11-22 12:57:38 -06:00
2016-09-05 16:45:39 +02:00
2016-09-12 20:01:43 +01:00
2016-12-19 08:23:53 -08:00
2016-12-05 19:03:06 -05:00
2017-01-07 18:22:40 -08:00
2016-10-15 10:03:15 -07:00
2016-12-20 09:48:47 -08:00
2016-07-06 10:51:14 +01:00
2016-07-06 10:51:14 +01:00
2016-08-18 15:36:59 +02:00
2016-10-31 13:01:57 -07:00
2017-01-14 21:23:48 -08:00
2016-07-15 10:41:44 +02:00
2016-07-15 10:41:44 +02:00
2016-10-29 12:14:39 -06:00
2016-10-11 15:06:32 -07:00
2016-12-30 03:26:31 -08:00
2016-11-15 08:30:02 +01:00
2016-06-29 23:39:52 +02:00
2016-11-11 18:24:16 +00:00
2016-09-20 04:43:36 -04:00
2016-12-02 00:52:34 +01:00
2016-08-02 19:35:38 -04:00
2016-08-02 19:35:37 -04:00
2016-08-02 19:35:37 -04:00
2016-12-12 18:55:08 -08:00
2016-12-12 11:26:47 -08:00
2016-10-31 15:45:18 -07:00
2016-09-01 16:56:15 -07:00
2016-09-17 08:39:12 -06:00
2017-01-10 18:31:55 -08:00
2016-08-08 12:51:58 -07:00
2016-09-19 12:27:10 +10:00
2016-11-09 20:40:06 -05:00
2016-11-09 20:40:06 -05:00
2016-11-09 20:40:06 -05:00
2016-11-09 20:40:06 -05:00
2016-10-11 15:06:33 -07:00
2016-10-07 18:46:30 -07:00
2016-10-25 14:40:39 +02:00
2016-11-16 10:59:38 +01:00
2016-12-16 00:13:41 +02:00
2016-07-27 14:20:09 -07:00
2016-07-13 09:24:51 +02:00
2016-07-26 16:19:19 -07:00
2016-12-14 16:04:07 -08:00
2016-07-01 05:32:17 -04:00
2017-01-10 18:31:55 -08:00
2016-07-28 16:07:41 -07:00
2017-01-10 18:31:54 -08:00
2016-07-28 16:07:41 -07:00
2016-10-18 14:14:20 -04:00
2016-09-05 13:52:39 +02:00
2016-10-05 18:23:36 -04:00
2016-11-22 17:48:04 -05:00
2016-12-08 13:07:10 -05:00
2017-01-20 03:33:57 +01:00
2017-01-10 18:31:55 -08:00
2017-01-06 13:00:01 -05:00
2016-10-25 14:40:39 +02:00
2016-11-10 10:10:37 +01:00
2016-10-17 09:50:56 -07:00
2016-10-06 09:52:23 -07:00
2016-09-15 09:51:14 -04:00
2017-01-13 12:31:24 -05:00
2016-11-23 10:06:12 +08:00
2016-11-11 18:25:08 +00:00
2016-07-17 19:59:26 +02:00
2016-12-25 17:21:22 +01:00
2016-08-31 14:43:35 -07:00
2016-11-16 09:26:34 +01:00
2016-12-25 11:04:12 +01:00
2016-12-25 11:04:12 +01:00
2016-12-25 11:04:12 +01:00
2016-07-15 10:41:42 +02:00
2017-01-10 18:31:55 -08:00
2016-07-28 16:07:41 -07:00
2016-08-22 10:10:12 -07:00
2016-07-19 17:43:38 +03:00
2016-11-22 17:40:00 -05:00
2016-12-09 09:13:30 -05:00
2016-12-09 09:13:30 -05:00
2016-12-25 11:04:12 +01:00
2016-09-30 11:46:40 +02:00
2016-08-08 13:02:01 -07:00
2016-12-09 22:12:21 -05:00
2016-06-30 18:04:36 -05:00
2016-12-16 10:24:44 -08:00
2016-12-11 18:01:51 -08:00
2016-09-13 08:08:24 +02:00
2016-10-24 14:36:25 +02:00
2016-09-22 20:00:36 -05:00
2016-12-14 16:04:09 -08:00
2016-08-08 14:42:02 -05:00
2016-12-01 10:40:05 -07:00
2016-08-16 18:49:56 +02:00
2016-08-01 21:44:52 +03:00
2017-01-20 11:01:17 -05:00
2016-08-02 02:57:29 +03:00
2016-07-28 16:07:41 -07:00
2016-12-12 18:55:08 -08:00
2016-12-06 11:03:22 +01:00
2016-07-28 16:07:41 -07:00
2016-10-27 16:37:44 +02:00
2016-07-14 10:42:35 +02:00
2016-12-25 17:21:22 +01:00
2016-10-08 10:27:10 +02:00
2016-09-21 00:22:59 -04:00
2016-12-13 12:59:57 -08:00
2016-12-25 11:54:48 -08:00
2016-12-06 10:42:03 +01:00
2016-10-07 21:48:36 -04:00
966d2b04e0