Peter Zijlstra f105dfec0a
tick/nohz: Evaluate the CPU expression after the static key ...
When tick_nohz_full_cpu() is called with smp_processor_id(), the latter
is unconditionally evaluated whether the static key is on or off. It is
not necessary in the off-case though, so make sure the cpu expression
is executed at the last moment.
Illustrate with the following test function:
int tick_nohz_test(void)
{
return tick_nohz_full_cpu(smp_processor_id());
}
The resulting code before was:
mov %gs:0x7eea92d1(%rip),%eax # smp_processor_id() fetch
nopl 0x0(%rax,%rax,1)
xor %eax,%eax
retq
cmpb $0x0,0x29d393a(%rip) # <tick_nohz_full_running>
je tick_nohz_test+0x29 # jump to below eax clear
mov %eax,%eax
bt %rax,0x29d3936(%rip) # <tick_nohz_full_mask>
setb %al
movzbl %al,%eax
retq
xor %eax,%eax
retq
Now it becomes:
nopl 0x0(%rax,%rax,1)
xor %eax,%eax
retq
cmpb $0x0,0x29d3871(%rip) # <tick_nohz_full_running>
je tick_nohz_test+0x29 # jump to below eax clear
mov %gs:0x7eea91f0(%rip),%eax # smp_processor_id() fetch, after static key
mov %eax,%eax
bt %rax,0x29d3866(%rip) # <tick_nohz_full_mask>
setb %al
movzbl %al,%eax
retq
xor %eax,%eax
retq
Signed-off-by: Peter Zijlstra <peterz@infradead.org >
Signed-off-by: Frederic Weisbecker <frederic@kernel.org >
Signed-off-by: Ingo Molnar <mingo@kernel.org >
Link: https://lore.kernel.org/r/20210512232924.150322-2-frederic@kernel.org
2021-05-13 14:21:20 +02:00
2021-04-26 16:32:11 -07:00
2021-04-22 09:26:22 -07:00
2021-03-30 11:14:45 +02:00
2021-02-16 12:09:52 +01:00
2021-03-24 14:01:58 +01:00
2021-02-09 12:26:40 +01:00
2021-04-27 14:10:15 -07:00
2021-04-30 13:04:30 -07:00
2021-04-06 15:33:32 -05:00
2021-05-05 12:39:29 -07:00
2021-04-07 08:36:38 +01:00
2021-04-26 12:30:36 -07:00
2021-04-28 15:59:13 -07:00
2021-05-01 09:15:05 -07:00
2021-04-23 09:29:49 +02:00
2021-04-16 20:30:54 +02:00
2021-04-26 18:16:56 +02:00
2021-05-03 23:04:01 +02:00
2021-04-26 03:20:07 +02:00
2021-04-26 18:16:56 +02:00
2021-04-26 18:16:56 +02:00
2021-03-15 12:53:12 -07:00
2021-03-17 12:13:19 +05:30
2021-04-22 02:03:01 +02:00
2021-05-07 14:49:18 -07:00
2021-03-21 21:29:26 +01:00
2021-04-08 13:26:22 -07:00
2021-04-23 13:18:35 +01:00
2021-05-05 11:27:26 -07:00
2021-03-29 22:49:54 -05:00
2021-04-06 10:22:02 +05:30
2021-04-28 15:56:51 -07:00
2021-02-08 13:23:42 +02:00
2021-05-07 11:23:41 -07:00
2021-03-08 11:33:28 +01:00
2021-04-29 11:57:23 -07:00
2021-04-26 17:04:27 +02:00
2021-05-06 19:24:11 -07:00
2021-04-07 11:14:55 +02:00
2021-03-22 08:55:28 +05:30
2021-04-07 16:33:20 +01:00
2021-04-09 14:54:23 -07:00
2021-04-14 16:30:30 +03:00
2021-05-07 00:26:33 -07:00
2021-03-08 15:16:30 -08:00
2021-03-08 10:21:11 -05:00
2021-05-08 21:49:48 -06:00
2021-05-07 00:34:51 -07:00
2021-05-06 19:24:12 -07:00
2021-02-08 08:33:16 -07:00
2021-04-21 22:59:17 -04:00
2021-05-09 13:25:14 -07:00
2021-02-23 09:54:51 -07:00
2021-03-17 16:34:11 +01:00
2021-02-26 11:51:48 -08:00
2021-02-26 11:51:47 -08:00
2021-02-26 12:28:03 -08:00
2021-05-03 11:56:23 +02:00
2021-04-03 01:43:49 +02:00
2021-04-29 11:57:23 -07:00
2021-02-16 15:23:23 -08:00
2021-03-26 20:41:51 -07:00
2021-05-05 11:27:24 -07:00
2021-04-15 17:04:24 +01:00
2021-02-26 09:41:03 -08:00
2021-04-08 16:04:20 -07:00
2021-04-04 13:34:46 -04:00
2021-02-17 14:07:48 +01:00
2021-04-07 11:05:44 -07:00
2021-02-22 09:45:23 -08:00
2021-04-07 16:33:20 +01:00
2021-05-01 10:14:08 -07:00
2021-05-05 11:27:24 -07:00
2021-05-05 11:27:24 -07:00
2021-05-07 00:34:51 -07:00
2021-02-14 18:15:15 +00:00
2021-04-08 16:04:20 -07:00
2021-04-08 16:04:20 -07:00
2021-04-30 11:20:35 -07:00
2021-04-25 05:26:10 +09:00
2021-05-07 00:26:34 -07:00
2021-05-05 22:54:12 +02:00
2021-03-08 10:21:11 -05:00
2021-02-11 18:31:48 +01:00
2021-04-06 16:05:38 -06:00
2021-03-10 09:16:09 +01:00
2021-02-04 19:25:47 +01:00
2021-05-04 11:24:46 -07:00
2021-04-07 19:26:44 +02:00
2021-04-29 11:41:43 -07:00
2021-05-06 19:24:12 -07:00
2021-05-06 19:24:11 -07:00
2021-04-15 22:36:45 -04:00
2021-04-16 17:06:34 +02:00
2021-05-07 00:26:32 -07:00
2021-03-09 15:40:16 +09:00
2021-04-19 13:13:26 -04:00
2021-03-23 14:58:10 +01:00
2021-04-22 08:20:15 +02:00
2021-04-08 12:21:13 +02:00
2021-02-25 18:28:24 +05:30
2021-04-07 10:30:47 +02:00
2021-03-15 10:02:31 +01:00
2021-04-02 16:41:08 +02:00
2021-02-04 14:42:00 +01:00
2021-03-19 07:44:28 +01:00
2021-04-25 05:25:42 +09:00
2021-03-22 03:57:39 +01:00
2021-02-17 14:12:43 +01:00
2021-03-14 14:48:02 -07:00
2021-04-16 16:59:20 -07:00
2021-02-16 09:59:41 +01:00
2021-02-08 12:28:07 +01:00
2021-03-15 11:09:38 +09:00
2021-03-12 13:16:42 -08:00
2021-03-16 16:55:05 +01:00
2021-03-12 15:10:03 +01:00
2021-04-16 00:13:04 -04:00
2021-04-12 15:04:23 +02:00
2021-03-30 22:37:06 +02:00
2021-02-26 09:41:04 -08:00
2021-04-08 19:12:20 +02:00
2021-05-07 00:26:34 -07:00
2021-04-23 10:14:32 +01:00
2021-04-23 10:14:32 +01:00
2021-03-16 16:49:31 +01:00
2021-02-07 14:51:11 -08:00
2021-03-23 14:08:18 -04:00
2021-02-24 10:13:55 -08:00
2021-04-08 10:24:36 -06:00
2021-05-07 00:26:34 -07:00
2021-05-07 00:26:34 -07:00
2021-05-07 00:26:33 -07:00
2021-03-17 16:34:08 +01:00
2021-02-23 19:26:43 +01:00
2021-03-11 20:47:09 +00:00
2021-03-11 20:47:09 +00:00
2021-04-29 21:44:07 +02:00
2021-02-24 13:38:32 -08:00
2021-05-05 11:27:27 -07:00
2021-03-31 17:42:14 +02:00
2021-05-05 11:27:21 -07:00
2021-03-25 09:22:55 -07:00
2021-05-05 11:27:22 -07:00
2021-04-28 17:22:10 -07:00
2021-04-10 21:59:21 +02:00
2021-02-23 11:29:52 -08:00
2021-04-19 12:50:15 +02:00
2021-03-23 14:49:05 -07:00
2021-02-11 13:24:44 -08:00
2021-03-10 15:20:24 -08:00
2021-03-15 20:41:58 -07:00
2021-02-23 13:39:45 -08:00
2021-02-16 13:14:06 -08:00
2021-05-06 19:24:11 -07:00
2021-04-08 16:04:20 -07:00
2021-05-07 00:26:33 -07:00
2021-04-08 19:09:02 +02:00
2021-04-07 11:55:47 +02:00
2021-04-07 10:15:19 +02:00
2021-03-17 16:34:07 +01:00
2021-04-11 19:30:41 -06:00
2021-04-30 11:20:39 -07:00
2021-04-07 10:56:53 +02:00
2021-04-30 11:20:40 -07:00
2021-05-04 08:54:29 -07:00
2021-04-16 17:20:45 +02:00
2021-04-26 15:03:23 -07:00
2021-04-07 10:30:47 +02:00
2021-02-23 11:29:52 -08:00
2021-04-26 10:25:03 -07:00
2021-03-22 04:23:14 +01:00
2021-05-05 12:39:29 -07:00
2021-04-05 22:39:38 -04:00
2021-04-08 14:05:19 +02:00
2021-02-08 12:24:04 +01:00
2021-02-24 13:38:31 -08:00
2021-04-30 11:20:41 -07:00
2021-05-09 00:29:45 +09:00
2021-03-06 12:40:21 +01:00
2021-03-08 14:27:43 -08:00
2021-03-08 14:27:43 -08:00
2021-05-08 10:00:11 -07:00
2021-03-08 12:06:29 -07:00
2021-02-22 10:22:54 -08:00
2021-02-26 09:41:02 -08:00
2021-02-26 09:41:05 -08:00
2021-02-26 09:40:59 -08:00
2021-03-08 11:43:35 +01:00
2021-02-26 09:41:03 -08:00
2021-05-05 22:54:12 +02:00
2021-02-19 11:54:08 +01:00
2021-02-19 11:54:08 +01:00
2021-02-19 11:35:28 +01:00
2021-04-13 09:16:12 -06:00
2021-04-08 16:04:22 -07:00
2021-04-28 12:37:53 -07:00
2021-05-01 18:50:44 -07:00
2021-05-01 18:50:44 -07:00
2021-04-20 16:27:54 -07:00
2021-04-12 10:36:00 -06:00
2021-04-27 14:13:24 -07:00
2021-02-09 09:30:16 +01:00
2021-03-25 09:22:55 -07:00
2021-05-05 11:27:23 -07:00
2021-05-05 11:27:26 -07:00
2021-05-05 11:27:26 -07:00
2021-05-05 11:27:26 -07:00
2021-04-09 18:48:22 +05:30
2021-05-05 11:27:26 -07:00
2021-04-04 18:29:37 -04:00
2021-02-24 13:38:33 -08:00
2021-03-13 11:27:30 -08:00
2021-05-07 00:26:35 -07:00
2021-03-25 09:22:55 -07:00
2021-05-07 00:26:33 -07:00
2021-02-24 10:25:37 -08:00
2021-04-08 16:04:20 -07:00
2021-02-27 08:07:12 -08:00
2021-04-09 14:54:23 -07:00
2021-04-20 14:11:22 +01:00
2021-03-29 15:56:48 +02:00
2021-04-08 22:18:36 -07:00
2021-02-23 20:32:11 -07:00
2021-02-11 13:24:44 -08:00
2021-04-07 14:43:28 -07:00
2021-04-23 10:14:32 +01:00
2021-04-19 15:41:28 -07:00
2021-04-13 10:04:05 -04:00
2021-05-07 11:23:41 -07:00
2021-04-13 09:41:16 -04:00
2021-04-13 10:04:05 -04:00
2021-03-22 10:19:00 -04:00
2021-04-06 08:34:39 +02:00
2021-04-02 16:26:33 +02:00
2021-02-12 19:23:57 -06:00
2021-02-11 08:33:41 +01:00
2021-02-17 13:17:49 -08:00
2021-04-13 14:35:02 -07:00
2021-04-21 10:24:41 -05:00
2021-04-26 08:51:23 -07:00
2021-04-01 14:07:41 -07:00
2021-02-24 13:38:29 -08:00
2021-04-30 11:20:42 -07:00
2021-02-26 09:41:01 -08:00
2021-05-05 11:27:19 -07:00
2021-02-26 09:40:59 -08:00
2021-04-30 11:20:41 -07:00
2021-03-10 12:46:04 +00:00
2021-02-03 19:05:50 +01:00
2021-03-12 16:08:59 -06:00
2021-04-16 11:34:17 -05:00
2021-02-23 14:11:13 -06:00
2021-02-23 14:12:01 -06:00
2021-05-05 13:24:11 -07:00
2021-05-01 10:14:08 -07:00
2021-05-07 00:26:34 -07:00
2021-04-20 16:08:02 -07:00
2021-03-15 12:53:12 -07:00
2021-04-02 17:02:31 +02:00
2021-02-24 14:52:41 +01:00
2021-03-16 10:14:11 +05:30
2021-04-08 19:14:46 +02:00
2021-04-07 19:18:23 +02:00
2021-04-28 15:43:58 -07:00
2021-03-24 12:48:38 -07:00
2021-03-17 16:34:08 +01:00
2021-05-06 19:24:11 -07:00
2021-05-06 19:24:11 -07:00
2021-04-05 13:18:22 +02:00
2021-03-06 12:40:22 +01:00
2021-03-06 12:40:22 +01:00
2021-04-21 12:20:04 -04:00
2021-04-07 16:33:07 +01:00
2021-03-12 17:09:34 -08:00
2021-02-26 09:41:03 -08:00
2021-04-09 14:17:03 +02:00
2021-04-12 14:49:31 +02:00
2021-04-08 14:05:19 +02:00
2021-02-17 14:07:44 +01:00
2021-03-15 13:54:54 -07:00
2021-03-08 14:17:35 -08:00
2021-04-28 12:00:13 -07:00
2021-03-24 17:16:15 -07:00
2021-03-22 08:23:48 -07:00
2021-03-31 13:53:38 +01:00
2021-03-18 08:00:51 -05:00
2021-05-05 13:24:11 -07:00
2021-03-16 22:13:11 +01:00
2021-04-01 14:18:32 -04:00
2021-02-26 09:41:01 -08:00
2021-03-17 14:16:15 -05:00
2021-03-29 15:57:03 +02:00
2021-02-04 05:27:42 -05:00
2021-03-22 02:45:52 +01:00
2021-03-18 22:39:30 -04:00
2021-05-05 11:27:26 -07:00
2021-03-30 16:35:16 +01:00
2021-05-01 18:50:44 -07:00
2021-03-18 12:58:27 -04:00
2021-04-27 15:50:15 -07:00
2021-03-10 09:51:45 +01:00
2021-04-22 12:04:26 +02:00
2021-03-10 09:34:10 +01:00
2021-04-11 16:34:56 -07:00
2021-05-05 11:27:23 -07:00
2021-04-28 13:33:57 -07:00
2021-04-01 10:56:13 -07:00
2021-04-25 18:02:32 -07:00
2021-02-26 09:41:02 -08:00
2021-03-08 14:18:46 -08:00
2021-02-26 09:41:02 -08:00
2021-05-09 13:07:03 -07:00
2021-04-26 13:04:19 -07:00
2021-02-26 09:41:04 -08:00
2021-03-10 15:52:31 +01:00
2021-04-19 16:03:15 +02:00
2021-03-11 16:04:39 +01:00
2021-04-28 12:37:53 -07:00
2021-04-22 15:02:40 -07:00
2021-03-13 11:27:30 -08:00
2021-02-26 09:41:04 -08:00
2021-05-06 19:24:11 -07:00
2021-03-19 04:58:25 +00:00
2021-05-01 18:50:44 -07:00
2021-03-25 17:39:33 -07:00
2021-04-11 19:30:25 -06:00
2021-03-01 15:25:24 -08:00
2021-04-22 13:14:09 +02:00
2021-03-16 22:13:10 +01:00
2021-03-18 18:25:31 +03:00
2021-05-13 14:21:20 +02:00
2021-04-16 21:03:50 +02:00
2021-04-07 16:33:20 +01:00
2021-03-22 23:06:48 +01:00
2021-04-14 16:30:30 +03:00
2021-04-01 14:18:32 -04:00
2021-02-02 17:02:05 -05:00
2021-03-23 14:08:18 -04:00
2021-04-10 10:36:34 +02:00
2021-03-10 09:34:06 +01:00
2021-04-15 10:32:17 +02:00
2021-02-13 17:17:53 +01:00
2021-03-10 09:51:45 +01:00
2021-04-03 21:22:18 +11:00
2021-03-30 17:06:49 -07:00
2021-04-26 22:55:12 +01:00
2021-02-15 18:50:01 +01:00
2021-03-17 21:30:15 +01:00
2021-04-09 16:00:00 +02:00
2021-04-29 11:06:13 -07:00
2021-05-05 11:27:22 -07:00
2021-03-19 22:23:19 +01:00
2021-05-03 04:55:53 -04:00
2021-04-06 11:55:11 -06:00
2021-03-22 15:02:14 +01:00
2021-04-06 16:17:57 -07:00
2021-05-04 04:19:59 -04:00
2021-03-14 04:37:35 -04:00
2021-05-05 11:27:25 -07:00
2021-05-07 00:26:35 -07:00
2021-02-09 12:15:07 +01:00
2021-04-30 11:20:37 -07:00
2021-05-03 04:55:53 -04:00
2021-05-05 22:54:11 +02:00
2021-03-08 11:33:39 +01:00
2021-03-19 12:13:10 +01:00
2021-04-16 15:31:02 -07:00
2021-03-30 13:42:33 -04:00
2021-02-26 09:41:01 -08:00
2021-02-26 09:41:01 -08:00
2021-02-08 22:58:55 +01:00
f105dfec0a