Kirill A. Shutemov 1d798ca3f1
mm: make compound_head() robust ...
Hugh has pointed that compound_head() call can be unsafe in some
context. There's one example:
CPU0 CPU1
isolate_migratepages_block()
page_count()
compound_head()
!!PageTail() == true
put_page()
tail->first_page = NULL
head = tail->first_page
alloc_pages(__GFP_COMP)
prep_compound_page()
tail->first_page = head
__SetPageTail(p);
!!PageTail() == true
<head == NULL dereferencing>
The race is pure theoretical. I don't it's possible to trigger it in
practice. But who knows.
We can fix the race by changing how encode PageTail() and compound_head()
within struct page to be able to update them in one shot.
The patch introduces page->compound_head into third double word block in
front of compound_dtor and compound_order. Bit 0 encodes PageTail() and
the rest bits are pointer to head page if bit zero is set.
The patch moves page->pmd_huge_pte out of word, just in case if an
architecture defines pgtable_t into something what can have the bit 0
set.
hugetlb_cgroup uses page->lru.next in the second tail page to store
pointer struct hugetlb_cgroup. The patch switch it to use page->private
in the second tail page instead. The space is free since ->first_page is
removed from the union.
The patch also opens possibility to remove HUGETLB_CGROUP_MIN_ORDER
limitation, since there's now space in first tail page to store struct
hugetlb_cgroup pointer. But that's out of scope of the patch.
That means page->compound_head shares storage space with:
- page->lru.next;
- page->next;
- page->rcu_head.next;
That's too long list to be absolutely sure, but looks like nobody uses
bit 0 of the word.
page->rcu_head.next guaranteed[1] to have bit 0 clean as long as we use
call_rcu(), call_rcu_bh(), call_rcu_sched(), or call_srcu(). But future
call_rcu_lazy() is not allowed as it makes use of the bit and we can
get false positive PageTail().
[1] http://lkml.kernel.org/g/20150827163634.GD4029@linux.vnet.ibm.com
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com >
Acked-by: Michal Hocko <mhocko@suse.com >
Reviewed-by: Andrea Arcangeli <aarcange@redhat.com >
Cc: Hugh Dickins <hughd@google.com >
Cc: David Rientjes <rientjes@google.com >
Cc: Vlastimil Babka <vbabka@suse.cz >
Acked-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com >
Cc: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com >
Cc: Andi Kleen <ak@linux.intel.com >
Cc: Christoph Lameter <cl@linux.com >
Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com >
Cc: Sergey Senozhatsky <sergey.senozhatsky@gmail.com >
Signed-off-by: Andrew Morton <akpm@linux-foundation.org >
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org >
2015-11-06 17:50:42 -08:00
2015-10-19 17:08:32 +01:00
2015-09-29 10:56:05 +03:00
2015-10-09 07:49:01 -07:00
2015-09-17 20:14:27 +03:00
2015-10-01 12:48:11 -07:00
2015-06-25 11:49:31 +03:00
2015-10-17 21:22:08 -07:00
2015-09-30 19:22:50 +09:00
2015-10-07 18:08:15 +01:00
2015-10-21 18:05:50 -05:00
2015-10-16 22:11:16 +02:00
2015-08-04 17:03:52 -07:00
2015-08-27 20:47:09 +01:00
2015-07-11 17:27:36 -07:00
2015-11-04 18:10:13 -08:00
2015-11-06 10:23:50 -08:00
2015-10-14 19:14:44 -07:00
2015-10-14 19:14:43 -07:00
2015-10-27 10:38:41 +01:00
2015-08-10 23:07:05 -04:00
2015-10-09 07:49:00 -07:00
2015-09-18 21:57:43 +02:00
2015-09-18 21:57:35 +02:00
2015-09-18 21:57:43 +02:00
2015-09-18 21:57:43 +02:00
2015-07-31 15:01:14 +01:00
2015-06-03 14:13:41 +09:00
2015-10-27 11:24:23 +01:00
2015-11-04 21:35:12 -08:00
2015-09-23 17:15:27 +02:00
2015-10-27 19:34:54 -07:00
2015-04-22 08:00:41 +10:00
2015-10-16 17:57:33 +01:00
2015-05-20 15:05:39 -07:00
2015-06-10 15:10:26 +08:00
2015-09-23 09:51:25 +02:00
2015-09-01 13:00:04 -07:00
2015-11-04 11:02:12 +00:00
2015-06-21 21:52:24 +02:00
2015-09-17 18:01:28 -04:00
2015-05-13 12:04:55 -05:00
2015-11-04 21:26:27 -08:00
2015-04-15 16:35:22 -07:00
2015-11-04 18:10:13 -08:00
2015-04-22 17:06:52 +02:00
2015-08-05 13:38:07 +01:00
2015-04-22 08:00:42 +10:00
2015-08-03 12:01:54 -04:00
2015-07-23 18:00:42 -07:00
2015-11-03 17:22:17 -08:00
2015-11-04 08:23:51 -05:00
2015-08-20 14:10:23 -07:00
2015-10-12 10:31:12 -06:00
2015-11-05 14:51:32 -08:00
2015-06-23 15:47:35 +01:00
2015-07-27 15:01:05 +02:00
2015-06-21 21:55:33 +02:00
2015-08-19 14:26:02 -07:00
2015-08-05 09:38:08 +02:00
2015-08-05 09:38:08 +02:00
2015-09-03 12:28:30 -07:00
2015-10-27 15:04:56 +09:00
2015-10-21 14:43:41 -06:00
2015-11-04 21:01:27 -08:00
2015-06-24 17:49:41 -07:00
2015-05-19 08:39:11 +02:00
2015-11-02 22:48:39 -05:00
2015-10-08 04:45:52 -07:00
2015-07-02 01:32:44 -04:00
2015-05-24 12:31:33 -07:00
2015-04-15 16:35:22 -07:00
2015-09-10 18:56:14 -07:00
2015-10-15 16:41:53 -04:00
2015-10-15 16:41:53 -04:00
2015-04-14 16:49:03 -07:00
2015-10-26 12:36:55 -07:00
2015-05-06 11:58:51 +01:00
2015-07-17 16:39:53 -07:00
2015-09-14 11:00:55 +02:00
2015-10-01 02:18:39 +02:00
2015-10-23 17:55:10 +09:00
2015-11-05 19:34:48 -08:00
2015-07-17 16:39:53 -07:00
2015-11-05 19:34:48 -08:00
2015-06-25 17:00:38 -07:00
2015-11-05 19:34:48 -08:00
2015-07-17 16:39:53 -07:00
2015-05-10 19:15:52 +02:00
2015-06-25 17:00:38 -07:00
2015-07-07 10:59:04 +02:00
2015-07-07 10:59:04 +02:00
2015-10-07 17:54:09 +01:00
2015-07-15 13:30:38 +01:00
2015-05-04 21:27:52 -07:00
2015-10-07 16:02:49 -07:00
2015-07-22 09:58:02 +02:00
2015-10-28 09:21:12 +01:00
2015-08-28 15:14:54 +02:00
2015-05-28 11:05:20 +09:30
2015-11-06 17:50:42 -08:00
2015-05-26 15:26:43 +02:00
2015-05-30 22:42:24 -07:00
2015-09-04 16:54:41 -07:00
2015-08-17 16:53:53 +08:00
2015-09-08 15:35:28 -07:00
2015-07-17 16:39:53 -07:00
2015-10-09 07:49:02 -07:00
2015-10-18 10:14:39 -07:00
2015-09-11 14:23:29 +09:00
2015-10-31 19:05:59 -04:00
2015-09-01 19:45:46 -07:00
2015-10-15 13:22:35 +01:00
2015-05-13 14:35:57 +05:30
2015-10-23 17:55:10 +09:00
2015-10-15 16:41:04 +02:00
2015-10-04 12:46:06 +01:00
2015-09-08 15:35:28 -07:00
2015-06-12 11:33:52 +02:00
2015-06-25 09:06:57 +02:00
2015-11-04 21:50:37 -08:00
2015-10-28 12:28:06 +01:00
2015-06-10 08:01:20 -06:00
2015-04-14 16:49:05 -07:00
2015-08-17 12:14:53 -07:00
2015-10-16 08:30:09 +09:00
2015-08-21 22:45:13 -07:00
2015-10-04 11:36:07 +01:00
2015-10-13 10:51:30 -07:00
2015-10-31 16:12:10 -07:00
2015-10-11 04:40:05 -07:00
2015-06-24 17:49:45 -07:00
2015-04-09 11:39:55 -05:00
2015-11-05 23:10:54 -08:00
2015-08-14 16:50:36 -07:00
2015-08-07 22:59:34 -05:00
2015-09-05 20:34:28 -07:00
2015-07-24 13:58:14 -04:00
2015-11-04 18:10:13 -08:00
2015-09-04 16:54:41 -07:00
2015-10-09 07:48:59 -07:00
2015-10-21 14:43:42 -06:00
2015-11-06 17:50:42 -08:00
2015-05-31 11:40:14 +09:00
2015-05-12 10:46:53 +02:00
2015-05-19 08:39:11 +02:00
2015-07-13 14:18:07 -07:00
2015-06-01 14:33:35 +02:00
2015-05-19 08:39:14 +02:00
2015-06-19 15:18:28 +02:00
2015-04-24 15:36:50 +02:00
2015-09-08 15:35:28 -07:00
2015-11-06 17:50:42 -08:00
2015-11-05 19:34:48 -08:00
2015-05-02 09:54:30 +03:00
2015-11-04 16:24:33 +01:00
2015-08-24 14:05:19 +02:00
2015-05-05 13:40:42 -06:00
2015-10-13 10:34:46 +02:00
2015-09-24 20:42:37 +02:00
2015-09-23 14:35:58 -07:00
2015-10-23 05:44:28 -07:00
2015-06-23 06:14:04 -07:00
2015-05-11 10:50:17 -04:00
2015-06-01 16:50:52 -07:00
2015-09-29 16:27:47 -07:00
2015-06-15 19:49:22 -07:00
2015-10-09 07:49:05 -07:00
2015-11-05 14:51:32 -08:00
2015-07-05 23:59:14 -04:00
2015-10-15 15:35:32 +01:00
2015-10-27 08:36:08 +09:00
2015-09-22 12:39:57 +02:00
2015-08-10 23:07:05 -04:00
2015-08-27 19:40:58 -04:00
2015-11-04 11:30:57 -08:00
2015-10-22 00:00:49 +02:00
2015-10-16 22:18:51 +02:00
2015-10-04 12:46:06 +01:00
2015-09-03 15:02:28 -05:00
2015-10-05 02:45:25 -07:00
2015-10-11 11:33:42 +02:00
2015-10-01 15:06:43 +02:00
2015-10-01 02:18:38 +02:00
2015-09-16 15:47:51 +02:00
2015-10-13 19:01:25 +02:00
2015-09-16 15:47:51 +02:00
2015-06-25 12:06:45 +02:00
2015-10-30 10:13:26 +01:00
2015-09-03 12:52:19 -07:00
2015-09-01 14:04:50 -07:00
2015-11-05 14:51:32 -08:00
2015-08-22 14:54:55 +02:00
2015-04-21 09:44:55 -03:00
2015-07-17 16:39:53 -07:00
2015-08-18 15:49:15 -07:00
2015-09-10 13:29:01 -07:00
2015-10-21 15:18:36 +01:00
2015-10-21 15:18:36 +01:00
2015-07-28 08:50:42 +01:00
2015-06-24 17:49:46 -07:00
2015-09-10 13:29:01 -07:00
2015-10-04 11:46:16 +01:00
2015-08-04 10:16:54 +02:00
2015-04-15 16:35:17 -07:00
2015-09-04 16:54:41 -07:00
2015-05-13 10:19:35 +02:00
2015-11-04 16:24:36 +01:00
2015-10-01 15:06:46 +02:00
2015-05-26 12:40:17 +02:00
2015-10-26 09:10:56 +01:00
2015-06-19 10:03:12 +02:00
2015-09-30 17:37:55 -04:00
2015-04-29 17:17:17 -05:00
2015-08-28 23:40:05 -04:00
2015-11-03 09:53:24 -07:00
2015-10-06 11:16:42 -07:00
2015-10-06 11:16:42 -07:00
2015-10-06 11:16:42 -07:00
2015-05-19 23:58:43 +02:00
2015-08-12 11:59:08 +02:00
2015-06-24 15:09:40 -07:00
2015-07-13 13:31:58 -04:00
2015-07-28 13:18:19 +10:00
2015-06-11 22:19:45 +05:30
2015-08-10 14:29:27 +05:30
2015-05-28 12:21:08 +02:00
2015-09-20 19:31:10 -07:00
2015-11-05 19:34:48 -08:00
2015-11-05 23:10:54 -08:00
2015-10-23 14:19:58 +01:00
2015-04-15 16:35:18 -07:00
2015-10-04 12:46:06 +01:00
2015-09-09 17:19:14 -07:00
2015-04-14 16:49:05 -07:00
2015-08-05 10:35:49 -07:00
2015-11-06 17:50:42 -08:00
2015-06-24 17:49:41 -07:00
2015-11-06 17:50:42 -08:00
2015-07-17 16:39:53 -07:00
2015-09-10 13:29:01 -07:00
2015-11-06 17:50:42 -08:00
2015-09-20 19:30:10 -07:00
2015-07-05 23:59:14 -04:00
2015-11-06 17:50:42 -08:00
2015-04-18 11:20:31 -04:00
2015-10-20 22:10:45 +08:00
2015-07-21 10:39:05 -07:00
2015-10-16 13:07:16 +01:00
2015-10-17 21:11:29 -07:00
2015-05-15 01:10:45 -04:00
2015-06-25 04:20:04 -04:00
2015-10-08 05:26:35 -07:00
2015-11-03 11:29:57 -05:00
2015-11-04 11:30:22 -08:00
2015-09-17 17:18:37 -07:00
2015-06-18 21:14:31 +02:00
2015-09-17 17:18:32 -07:00
2015-09-29 20:21:32 +02:00
2015-09-30 01:45:03 -05:00
2015-10-16 18:45:36 +02:00
2015-09-09 21:43:22 -07:00
2015-08-17 13:22:49 -05:00
2015-08-27 19:45:27 -04:00
2015-08-17 13:32:56 -05:00
2015-06-10 18:32:40 -04:00
2015-09-07 12:36:13 -04:00
2015-05-05 17:10:11 -07:00
2015-11-05 19:34:48 -08:00
2015-09-07 15:17:08 -04:00
2015-09-07 15:17:09 -04:00
2015-10-09 10:40:37 -06:00
2015-08-05 13:43:44 -07:00
2015-08-05 13:43:12 -07:00
2015-07-27 08:23:27 -05:00
2015-05-09 17:11:25 +05:30
2015-07-01 19:40:18 -07:00
2015-10-16 22:49:26 +02:00
2015-06-24 11:18:48 +10:00
2015-10-16 13:07:15 +01:00
2015-08-25 11:29:55 -05:00
2015-08-26 01:46:39 +02:00
2015-09-01 09:59:20 +10:00
2015-10-28 10:05:58 -07:00
2015-10-08 05:26:36 -07:00
2015-09-08 15:35:28 -07:00
2015-05-08 12:27:59 +02:00
2015-11-05 19:34:48 -08:00
2015-09-10 13:29:01 -07:00
2015-09-10 13:29:01 -07:00
2015-07-17 16:39:54 -07:00
2015-11-06 17:50:42 -08:00
2015-09-08 15:35:28 -07:00
2015-11-06 17:50:42 -08:00
2015-06-01 07:08:18 +09:00
2015-07-17 16:39:53 -07:00
2015-08-16 13:26:01 -03:00
2015-10-16 22:18:51 +02:00
2015-04-09 14:20:11 -05:00
2015-08-13 15:59:58 -05:00
2015-09-08 15:35:28 -07:00
2015-05-29 07:39:34 +10:00
2015-07-14 17:43:56 -04:00
2015-10-06 11:25:31 -07:00
2015-09-13 11:27:30 +02:00
2015-04-12 21:03:31 +02:00
2015-08-31 14:48:02 -07:00
2015-10-23 05:57:43 -07:00
2015-10-05 05:02:40 +01:00
2015-05-12 23:55:37 +02:00
2015-10-28 04:33:04 +01:00
2015-09-15 02:03:16 +02:00
2015-07-28 08:50:41 +01:00
2015-07-21 23:14:22 +02:00
2015-05-30 02:17:15 +02:00
2015-05-20 01:56:31 +02:00
2015-10-14 02:17:34 +02:00
2015-08-27 19:40:59 -04:00
2015-09-10 13:29:01 -07:00
2015-06-10 16:15:54 +02:00
2015-10-01 09:59:16 -07:00
2015-10-21 14:46:56 -06:00
2015-10-06 17:08:19 +02:00
2015-09-10 13:29:01 -07:00
2015-09-25 02:00:19 +02:00
2015-08-07 14:37:04 +02:00
2015-08-03 12:33:39 +01:00
2015-10-22 08:59:18 -07:00
2015-11-03 11:08:22 -05:00
2015-07-15 11:52:52 -07:00
2015-08-17 15:40:20 +02:00
2015-10-30 11:18:05 +09:00
2015-05-28 10:47:45 -05:00
2015-07-23 20:59:10 +02:00
2015-10-08 05:26:38 -07:00
2015-05-28 11:32:04 +09:30
2015-05-28 11:32:06 +09:30
2015-05-28 11:32:04 +09:30
2015-10-06 11:25:45 -07:00
2015-10-06 11:16:18 -07:00
2015-10-06 11:16:31 -07:00
2015-10-06 11:08:23 -07:00
2015-10-06 11:08:05 -07:00
2015-04-15 16:35:23 -07:00
2015-10-12 19:25:07 +01:00
2015-06-17 09:57:12 +03:00
2015-09-01 09:56:58 +02:00
2015-05-16 18:08:26 -04:00
2015-05-26 15:23:23 +02:00
2015-09-04 16:54:41 -07:00
2015-06-25 01:13:43 +02:00
2015-10-09 07:49:06 -07:00
2015-08-24 14:28:01 -06:00
2015-11-05 23:10:54 -08:00
2015-10-04 12:54:54 +01:00
2015-10-27 19:55:13 -07:00
2015-09-20 17:00:26 -07:00
2015-09-11 15:21:34 -07:00
2015-07-01 10:49:25 -07:00
2015-07-23 17:43:30 -07:00
2015-05-06 22:27:00 +02:00
2015-05-10 19:06:38 +02:00
2015-07-16 10:30:55 -07:00
2015-08-05 08:48:00 +05:30
2015-11-06 17:50:42 -08:00
2015-11-05 19:34:48 -08:00
2015-04-20 09:08:49 -07:00
2015-10-20 10:23:55 +02:00
2015-06-15 19:49:22 -07:00
2015-04-11 15:53:35 -04:00
2015-09-01 08:40:25 -07:00
2015-09-20 19:32:02 -07:00
2015-10-06 11:15:43 -07:00
2015-04-14 09:50:27 -07:00
2015-06-25 17:00:38 -07:00
2015-10-04 20:28:58 +01:00
2015-07-29 00:13:25 -07:00
2015-10-20 10:23:55 +02:00
2015-09-10 13:29:01 -07:00
2015-09-10 15:36:59 -04:00
2015-10-14 02:17:33 +02:00
2015-05-11 15:06:43 +08:00
2015-09-08 15:35:28 -07:00
2015-09-08 15:35:28 -07:00
2015-11-05 19:34:48 -08:00
2015-07-01 10:36:39 -05:00
2015-10-19 01:01:19 +02:00
2015-06-25 17:00:39 -07:00
2015-10-21 14:42:38 -06:00
2015-10-22 06:22:13 -07:00
2015-09-14 07:39:46 -07:00
2015-09-20 22:52:13 -07:00
2015-09-02 10:33:22 +02:00
2015-08-17 11:25:28 -07:00
2015-06-12 11:15:49 +02:00
2015-10-01 09:59:01 -07:00
2015-06-19 15:18:28 +02:00
2015-04-22 17:06:49 +02:00
2015-10-01 09:57:59 -07:00
2015-05-27 15:22:15 +02:00
2015-10-19 01:01:21 +02:00
2015-08-06 15:29:14 -03:00
2015-11-05 19:34:48 -08:00
2015-04-08 09:39:56 -04:00
2015-08-03 15:24:43 -07:00
2015-10-17 21:32:21 -07:00
2015-11-05 19:34:48 -08:00
2015-06-15 16:02:52 -07:00
2015-11-05 19:34:48 -08:00
2015-05-14 17:55:51 -07:00
2015-04-11 22:29:44 -04:00
2015-07-31 10:38:06 +02:00
2015-04-28 12:48:57 +02:00
2015-10-04 10:34:17 +01:00
2015-09-04 16:54:41 -07:00
2015-05-05 17:10:11 -07:00
2015-08-12 17:01:01 +01:00
2015-04-14 18:06:47 -07:00
2015-06-01 15:48:54 +02:00
2015-06-01 15:48:54 +02:00
2015-04-13 21:04:16 +09:30
2015-04-15 12:41:14 +09:30
2015-11-05 19:34:48 -08:00
2015-11-01 17:09:15 -08:00
2015-10-17 22:03:00 -07:00
2015-11-05 19:34:48 -08:00
2015-06-01 15:48:54 +02:00
2015-09-22 15:09:53 -07:00
2015-09-04 16:54:41 -07:00
2015-08-17 15:48:24 -04:00
2015-06-02 08:40:20 -06:00
2015-09-08 15:35:28 -07:00
2015-11-06 17:50:42 -08:00
2015-11-06 17:50:42 -08:00
1d798ca3f1