Eric Dumazet
7bbe720af7
udp: no longer use SLAB_DESTROY_BY_RCU
...
Tom Herbert would like not touching UDP socket refcnt for encapsulated
traffic. For this to happen, we need to use normal RCU rules, with a grace
period before freeing a socket. UDP sockets are not short lived in the
high usage case, so the added cost of call_rcu() should not be a concern.
This actually removes a lot of complexity in UDP stack.
Multicast receives no longer need to hold a bucket spinlock.
Note that ip early demux still needs to take a reference on the socket.
Same remark for functions used by xt_socket and xt_PROXY netfilter modules,
but this might be changed later.
Performance for a single UDP socket receiving flood traffic from
many RX queues/cpus.
Simple udp_rx using simple recvfrom() loop :
438 kpps instead of 374 kpps : 17 % increase of the peak rate.
v2: Addressed Willem de Bruijn feedback in multicast handling
- keep early demux break in __udp4_lib_demux_lookup()
Signed-off-by: Eric Dumazet <edumazet@google.com >
Cc: Tom Herbert <tom@herbertland.com >
Cc: Willem de Bruijn <willemb@google.com >
Tested-by: Tom Herbert <tom@herbertland.com >
Signed-off-by: David S. Miller <davem@davemloft.net >
Change-Id: I4a8092b7f3adc34bf6f7303d5d23bb3a3fec7a7f
2022-11-15 21:35:32 +01:00
..
2021-03-17 16:07:19 +01:00
2019-09-10 10:29:49 +01:00
2016-12-02 09:24:34 +05:30
2018-08-24 13:27:00 +02:00
2019-12-05 15:26:48 +01:00
2017-08-02 22:47:26 +00:00
2019-11-01 12:32:03 +08:00
2017-05-19 19:49:46 +00:00
2021-08-12 03:09:18 +03:00
2017-03-15 09:57:14 +08:00
2021-09-02 00:24:42 +03:00
2018-04-13 19:50:17 +02:00
2020-06-03 08:12:09 +02:00
2021-08-12 03:09:18 +03:00
2020-11-10 10:22:12 +01:00
2021-07-24 19:28:12 +03:00
2018-10-13 09:11:35 +02:00
2017-06-28 12:35:41 -07:00
2019-09-20 15:48:07 -07:00
2020-04-24 18:27:45 +08:00
2021-10-12 00:11:01 +03:00
2017-07-23 22:16:45 +00:00
2017-04-28 16:56:47 -07:00
2020-04-24 18:27:45 +08:00
2021-08-12 03:09:18 +03:00
2017-05-19 06:14:07 +00:00
2021-07-24 19:28:12 +03:00
2021-02-23 13:58:12 +01:00
2020-12-02 16:20:34 -07:00
2021-01-23 15:36:55 +01:00
2021-08-12 03:09:18 +03:00
2019-12-21 10:35:02 +01:00
2022-03-04 20:17:03 +01:00
2019-11-01 12:32:03 +08:00
2021-02-23 13:58:12 +01:00
2017-02-21 12:41:38 +05:30
2019-09-20 12:22:32 -07:00
2019-06-11 12:23:51 +02:00
2020-01-29 10:21:52 +01:00
2020-06-29 20:07:54 -04:00
2019-04-27 09:33:55 +02:00
2019-05-16 19:45:09 +02:00
2021-08-12 03:09:18 +03:00
2022-03-04 20:16:48 +01:00
2021-08-12 03:09:18 +03:00
2020-05-20 08:11:38 +02:00
2016-10-27 14:58:42 -07:00
2022-03-04 20:16:54 +01:00
2022-11-15 21:35:32 +01:00
2022-11-15 21:35:28 +01:00
2017-12-09 18:42:43 +01:00
2021-08-12 14:44:02 +03:00
2017-07-31 20:36:32 -07:00
2018-01-31 12:06:08 +01:00
2017-01-12 11:22:50 +01:00
2020-05-20 08:11:40 +02:00
2022-03-04 20:16:43 +01:00
2022-11-15 21:35:32 +01:00
2022-03-04 20:17:00 +01:00
2016-12-22 11:41:12 +05:30
2021-08-12 03:09:18 +03:00
2019-08-06 18:28:28 +02:00
2019-08-06 18:28:28 +02:00
2017-02-21 12:41:38 +05:30
2022-11-15 21:35:29 +01:00
2021-07-24 19:25:46 +03:00
2021-04-28 12:05:47 +02:00
2021-04-28 12:05:47 +02:00
2022-11-15 21:35:29 +01:00
2021-05-26 11:27:34 +02:00
2017-07-05 14:37:20 +02:00
2016-10-21 10:53:56 -06:00
2019-12-17 13:28:23 +08:00
2019-11-25 15:54:07 +01:00
2018-07-18 13:21:38 +00:00
2021-08-12 03:09:18 +03:00
2018-12-11 19:41:54 +01:00
2020-05-27 09:57:29 -07:00
2017-10-12 11:27:35 +02:00
2021-10-06 10:22:25 +02:00
2018-06-28 14:24:29 -07:00
2021-08-12 03:09:18 +03:00
2017-05-22 23:19:33 -07:00
2022-11-15 21:35:32 +01:00
2020-04-24 07:57:11 +02:00
2016-10-28 03:01:26 -04:00
2021-07-24 19:19:07 +03:00
2021-09-02 00:24:42 +03:00
2017-03-31 21:24:15 -07:00
2017-01-30 17:54:44 -08:00
2018-08-06 16:24:36 +02:00
2017-01-30 17:54:44 -08:00
2020-02-14 10:51:59 +08:00
2020-01-12 11:22:42 +01:00
2017-02-02 17:35:52 -08:00
2019-11-25 15:54:16 +01:00
2018-07-30 06:57:27 -07:00
2019-08-04 09:34:55 +02:00
2021-10-27 09:32:39 +02:00
2020-06-29 20:07:54 -04:00
2017-03-24 13:22:26 -07:00
2022-11-15 21:35:29 +01:00
2019-05-08 22:00:09 -07:00
2019-12-17 13:28:23 +08:00
2017-07-15 11:57:44 +02:00
2018-02-25 11:03:53 +01:00
2021-12-27 01:17:29 +02:00
2022-11-15 21:35:28 +01:00
2020-10-14 09:46:21 +02:00
2022-03-04 20:16:57 +01:00
2022-03-04 20:16:56 +01:00
2017-06-17 06:39:36 +02:00
2018-01-10 09:27:10 +01:00
2018-06-28 12:18:52 -07:00
2018-06-28 12:19:55 -07:00
2018-04-12 09:58:05 -07:00
2017-10-12 09:35:25 -07:00
2016-09-30 10:18:37 +02:00
2018-02-06 20:39:33 +00:00
2022-11-15 21:35:31 +01:00
2021-02-03 23:16:16 +01:00
2017-12-16 10:33:54 +01:00
2020-06-29 20:07:56 -04:00
2019-02-20 10:13:11 +01:00
2022-11-15 21:35:30 +01:00
2018-04-04 15:03:59 +00:00
2017-03-22 14:39:37 +05:30
2019-09-10 10:29:46 +01:00
2016-10-21 18:00:55 -07:00
2019-05-14 17:56:52 -07:00
2016-11-09 23:51:04 -08:00
2019-02-20 10:13:19 +01:00
2021-12-14 10:35:51 +01:00
2020-11-10 10:22:18 +01:00
2020-02-14 10:51:59 +08:00
2017-07-20 01:24:56 +00:00
2021-07-24 19:28:12 +03:00
2018-10-20 09:52:38 +02:00
2020-09-23 08:44:26 +02:00
2018-11-21 09:27:42 +01:00
2021-03-03 16:44:21 +01:00
2021-07-24 19:28:12 +03:00
2020-01-12 11:22:52 +01:00
2021-03-30 14:45:00 +02:00
2019-11-01 12:32:03 +08:00
2020-12-11 13:36:44 +01:00
2016-10-12 17:34:22 +05:30
2019-12-21 10:35:04 +01:00
2022-11-15 21:35:29 +01:00
2021-07-24 18:05:21 +03:00
2021-02-03 23:16:17 +01:00
2018-05-15 20:14:17 -07:00
2017-03-01 12:02:07 +05:30
2018-09-09 20:04:36 +02:00
2017-07-24 18:01:26 +00:00
2016-10-26 23:36:47 -07:00
2017-01-23 11:03:24 +05:30
2018-01-10 05:52:17 +00:00
2021-12-08 08:44:06 +01:00
2017-06-05 19:11:19 +00:00
2017-06-05 19:11:19 +00:00
2018-02-23 10:03:32 -08:00
2017-04-07 12:27:45 -07:00
2017-04-03 17:43:56 +00:00
2017-04-03 17:43:56 +00:00
2019-12-21 10:34:54 +01:00
2018-04-08 11:51:57 +02:00
2017-01-19 20:17:19 +01:00
2019-05-16 19:45:07 +02:00
2018-02-22 15:45:01 +01:00
2017-05-12 17:22:00 -07:00
2020-10-29 09:03:00 +01:00
2018-05-15 20:14:17 -07:00
2018-11-27 16:07:57 +01:00
2018-01-23 19:50:12 +01:00
2018-05-15 17:49:01 -07:00
2017-08-21 16:28:17 +01:00
2021-08-12 03:09:18 +03:00
2022-03-04 20:16:57 +01:00
2017-10-27 10:23:18 +02:00
2020-06-20 10:23:21 +02:00
2020-02-14 10:51:59 +08:00
2021-12-08 08:44:07 +01:00
2021-08-12 03:09:18 +03:00
2017-02-21 12:38:28 +05:30
2017-06-02 08:01:50 -07:00
2018-01-31 12:06:08 +01:00
2021-06-16 11:34:53 +02:00
2017-05-02 14:54:29 -07:00
2017-05-04 16:43:06 +00:00
2021-11-26 11:58:33 +01:00
2020-01-04 13:34:37 +01:00
2017-03-18 19:09:58 +08:00
2017-09-02 07:06:51 +02:00
2019-06-11 12:24:10 +02:00
2022-03-04 20:16:55 +01:00
2018-09-21 14:51:01 -07:00
2018-04-08 11:51:59 +02:00
2020-09-12 11:45:28 +02:00
2022-03-04 20:19:53 +01:00
2017-08-21 14:08:12 -07:00
2017-05-12 17:22:00 -07:00
2017-10-27 10:23:17 +02:00
2016-09-30 10:21:32 -07:00
2017-02-07 15:18:11 +05:30
2017-09-20 16:42:37 -07:00
2017-02-09 08:02:47 +01:00
2016-11-02 10:21:27 -07:00
2017-02-21 12:41:38 +05:30
2019-01-30 07:50:15 +09:00
2018-12-09 11:41:03 +01:00
2020-08-26 07:32:20 -07:00
2017-12-20 10:04:59 +01:00
2017-12-25 14:22:09 +01:00
2017-12-16 10:33:50 +01:00
2019-03-29 11:30:48 +09:00
2020-06-11 08:44:31 -07:00
2019-08-25 10:53:03 +02:00
2022-11-15 21:35:30 +01:00
2022-11-15 21:35:32 +01:00
2016-11-15 07:46:37 +01:00
2016-11-15 07:46:37 +01:00
2021-08-26 08:37:47 -04:00
2017-03-24 17:46:58 +08:00
2017-02-23 14:06:34 -08:00
2017-02-27 09:16:35 -07:00
2017-04-07 16:05:47 -07:00
2017-06-30 23:17:49 -07:00
2016-12-07 11:17:55 -07:00
2017-02-03 15:04:29 +05:30
2020-05-27 16:40:33 +02:00
2019-02-23 09:05:13 +01:00
2022-03-04 20:17:01 +01:00
2017-12-05 11:22:49 +01:00
2016-11-07 16:23:44 +05:30
2017-02-01 08:30:53 +01:00
2018-07-25 10:18:27 +02:00
2022-03-04 20:16:44 +01:00
2016-10-07 15:23:41 +02:00
2021-01-09 13:34:15 +01:00
2017-12-16 10:33:51 +01:00
2021-04-28 12:05:47 +02:00
2020-05-27 16:40:27 +02:00
2017-11-24 08:32:25 +01:00
2017-02-21 12:38:28 +05:30
2022-11-15 21:35:30 +01:00
2018-07-08 17:37:49 -07:00
2020-06-29 20:07:59 -04:00
2021-09-22 11:41:27 +02:00
2020-04-24 07:57:24 +02:00
2018-01-05 15:44:26 +01:00
2017-02-09 08:02:46 +01:00
2022-03-04 20:17:00 +01:00
2017-04-28 09:50:04 -07:00
2017-12-15 10:33:25 -08:00
2018-01-17 09:35:29 +01:00
2022-11-15 21:35:29 +01:00
2020-03-11 07:51:15 +01:00
2017-03-20 15:56:47 -07:00
2020-05-20 08:11:48 +02:00
2017-02-07 15:21:07 +00:00
2020-05-20 08:11:41 +02:00
2018-12-17 21:55:15 +01:00
2018-05-15 20:25:18 +00:00
2021-07-20 16:22:33 +02:00
2017-11-15 17:13:11 +01:00
2020-06-03 08:12:16 +02:00
2022-11-15 21:35:29 +01:00
2022-03-04 20:16:49 +01:00
2019-11-22 15:15:55 +08:00
2017-09-20 16:42:37 -07:00
2019-05-16 19:45:09 +02:00
2019-06-22 08:18:21 +02:00
2017-02-14 16:44:11 -08:00
2017-03-09 20:03:05 -08:00
2020-01-04 13:34:32 +01:00
2019-12-21 10:35:30 +01:00
2018-12-11 19:41:54 +01:00
2020-08-21 10:52:55 +02:00
2022-11-15 21:35:31 +01:00
2019-04-03 10:24:12 +02:00
2021-08-10 17:39:42 +02:00
2019-12-05 15:26:36 +01:00
2017-08-21 16:04:11 +01:00
2019-02-08 11:25:32 +01:00
2019-04-27 09:33:48 +02:00
2019-01-11 07:53:18 +09:00
2017-01-11 22:31:42 -08:00
2021-12-27 01:17:29 +02:00
2017-02-22 02:06:46 -08:00
2019-03-04 18:14:43 -08:00
2022-03-04 20:19:53 +01:00
2020-04-02 19:02:39 +02:00
2016-10-28 03:01:32 -04:00
2020-12-29 13:42:39 +01:00
2021-08-05 15:18:10 +03:00
2020-10-01 11:11:51 +02:00
2020-02-14 10:51:59 +08:00
2018-01-17 09:35:29 +01:00
2021-12-08 08:44:06 +01:00
2017-05-02 07:50:15 -07:00
2020-01-29 10:21:47 +01:00
2021-12-08 08:44:08 +01:00
2022-11-15 21:35:31 +01:00
2017-12-14 08:20:37 -08:00
2017-12-14 08:19:40 -08:00
2018-12-11 19:36:11 +01:00
2016-09-30 10:18:37 +02:00
2019-06-11 12:23:56 +02:00
2022-03-04 20:16:57 +01:00
2016-11-08 10:23:45 -08:00
2017-12-14 08:19:45 -08:00
2021-08-05 14:01:54 +03:00
2018-07-30 07:07:21 -07:00
2021-08-12 14:44:02 +03:00
2018-08-18 06:56:17 -07:00
2019-07-10 09:56:42 +02:00
2022-11-15 21:35:29 +01:00
2017-09-20 16:42:37 -07:00
2021-03-07 11:37:45 +01:00
2017-09-29 13:15:43 -07:00
2017-03-20 02:28:42 -07:00
2017-06-30 23:02:26 -07:00
2017-06-30 23:02:26 -07:00
2018-11-21 09:27:36 +01:00
2020-07-31 16:43:16 +02:00
2018-02-21 15:40:49 +00:00
2017-02-15 12:05:11 -07:00
2019-12-21 11:36:50 +01:00
2017-07-18 18:18:46 +00:00
2020-11-18 18:24:57 +01:00
2019-12-21 10:35:40 +01:00
2018-05-26 08:49:00 +02:00
2016-12-01 15:18:44 +05:30
2017-04-25 02:10:44 -07:00
2022-03-04 20:16:56 +01:00
2020-12-29 13:42:39 +01:00
2020-08-21 10:52:57 +02:00
2021-05-22 10:38:23 +02:00
2017-09-27 11:00:13 +02:00
2020-12-14 00:49:15 -07:00
2017-01-11 22:31:42 -08:00
2021-03-30 14:44:59 +02:00
2020-09-12 09:47:22 -07:00
2022-11-15 21:35:32 +01:00
2016-12-10 19:07:25 +01:00
2020-09-23 21:48:05 -07:00
2022-03-04 20:16:49 +01:00
2018-01-31 12:06:11 +01:00
2017-07-27 15:06:07 -07:00
2019-05-15 14:34:43 +00:00
2021-03-25 02:03:06 +00:00
2018-05-02 07:53:40 -07:00
2018-12-09 11:41:03 +01:00
2018-11-01 12:34:56 +08:00
2020-04-24 18:27:45 +08:00
2019-04-03 06:23:21 +02:00
2019-07-21 09:07:12 +02:00
2020-04-02 19:02:39 +02:00
2021-12-14 10:03:49 +01:00
2017-04-03 17:44:15 +00:00
2018-07-23 08:12:08 -07:00
2018-08-23 12:00:19 -07:00
2021-08-12 14:44:02 +03:00
2018-08-23 12:00:19 -07:00