Replace all the occurrences of FIELD_SIZEOF() with sizeof_field() except
at places where these are defined. Later patches will remove the unused
definition of FIELD_SIZEOF().
This patch is generated using following script:
EXCLUDE_FILES="include/linux/stddef.h|include/linux/kernel.h"
git grep -l -e "\bFIELD_SIZEOF\b" | while read file;
do
if [[ "$file" =~ $EXCLUDE_FILES ]]; then
continue
fi
sed -i -e 's/\bFIELD_SIZEOF\b/sizeof_field/g' $file;
done
Change-Id: I24296633f28fea05d12618c8e47dc8acb8df18d8
Signed-off-by: Pankaj Bharadiya <pankaj.laxminarayan.bharadiya@intel.com>
Link: https://lore.kernel.org/r/20190924105839.110713-3-pankaj.laxminarayan.bharadiya@intel.com
Co-developed-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Kees Cook <keescook@chromium.org>
Acked-by: David Miller <davem@davemloft.net> # for net
* 'linux-4.19.y-cip' of https://git.kernel.org/pub/scm/linux/kernel/git/cip/linux-cip:
CIP: Bump version suffix to -cip124 after merge from cip/linux-4.19.y-st tree
Update localversion-st, tree is up-to-date with 5.4.298.
f2fs: fix to do sanity check on ino and xnid
squashfs: fix memory leak in squashfs_fill_super
pNFS: Handle RPC size limit for layoutcommits
wifi: iwlwifi: fw: Fix possible memory leak in iwl_fw_dbg_collect
usb: core: usb_submit_urb: downgrade type check
udf: Verify partition map count
f2fs: fix to avoid panic in f2fs_evict_inode
usb: hub: Fix flushing and scheduling of delayed work that tunes runtime pm
Revert "drm/dp: Change AUX DPCD probe address from DPCD_REV to LANE0_1_STATUS"
net: usb: qmi_wwan: add Telit Cinterion LE910C4-WWX new compositions
HID: hid-ntrig: fix unable to handle page fault in ntrig_report_version()
HID: asus: fix UAF via HID_CLAIMED_INPUT validation
efivarfs: Fix slab-out-of-bounds in efivarfs_d_compare
sctp: initialize more fields in sctp_v6_from_sk()
net: stmmac: xgmac: Do not enable RX FIFO Overflow interrupts
net/mlx5e: Set local Xoff after FW update
net: dlink: fix multicast stats being counted incorrectly
atm: atmtcp: Prevent arbitrary write in atmtcp_recv_control().
net/atm: remove the atmdev_ops {get, set}sockopt methods
Bluetooth: hci_event: Detect if HCI_EV_NUM_COMP_PKTS is unbalanced
powerpc/kvm: Fix ifdef to remove build warning
net: ipv4: fix regression in local-broadcast routes
vhost/net: Protect ubufs with rcu read lock in vhost_net_ubuf_put()
scsi: core: sysfs: Correct sysfs attributes access rights
ftrace: Fix potential warning in trace_printk_seq during ftrace_dump
alloc_fdtable(): change calling conventions.
ALSA: usb-audio: Use correct sub-type for UAC3 feature unit validation
net/sched: Make cake_enqueue return NET_XMIT_CN when past buffer_limit
ipv6: sr: validate HMAC algorithm ID in seg6_hmac_info_add
ALSA: usb-audio: Fix size validation in convert_chmap_v3()
scsi: qla4xxx: Prevent a potential error pointer dereference
usb: xhci: Fix slot_id resource race conflict
nfs: fix UAF in direct writes
NFS: Fix up commit deadlocks
Bluetooth: fix use-after-free in device_for_each_child()
selftests: forwarding: tc_actions.sh: add matchall mirror test
codel: remove sch->q.qlen check before qdisc_tree_reduce_backlog()
sch_qfq: make qfq_qlen_notify() idempotent
sch_hfsc: make hfsc_qlen_notify() idempotent
sch_drr: make drr_qlen_notify() idempotent
btrfs: populate otime when logging an inode item
media: venus: hfi: explicitly release IRQ during teardown
f2fs: fix to avoid out-of-boundary access in dnode page
media: venus: protect against spurious interrupts during probe
media: venus: vdec: Clamp param smaller than 1fps and bigger than 240.
drm/dp: Change AUX DPCD probe address from DPCD_REV to LANE0_1_STATUS
media: rainshadow-cec: fix TOCTOU race condition in rain_interrupt()
media: v4l2-ctrls: Don't reset handler's error in v4l2_ctrl_handler_free()
ata: Fix SATA_MOBILE_LPM_POLICY description in Kconfig
usb: musb: omap2430: fix device leak at unbind
NFS: Fix the setting of capabilities when automounting a new filesystem
NFS: Fix up handling of outstanding layoutcommit in nfs_update_inode()
NFSv4: Fix nfs4_bitmap_copy_adjust()
usb: typec: fusb302: cache PD RX state
cdc-acm: fix race between initial clearing halt and open
USB: cdc-acm: do not log successful probe on later errors
nfsd: handle get_client_locked() failure in nfsd4_setclientid_confirm()
tracing: Add down_write(trace_event_sem) when adding trace event
usb: hub: Don't try to recover devices lost during warm reset.
usb: hub: avoid warm port reset during USB3 disconnect
x86/mce/amd: Add default names for MCA banks and blocks
iio: hid-sensor-prox: Fix incorrect OFFSET calculation
mm/zsmalloc: do not pass __GFP_MOVABLE if CONFIG_COMPACTION=n
mm/zsmalloc.c: convert to use kmem_cache_zalloc in cache_alloc_zspage()
net: usbnet: Fix the wrong netif_carrier_on() call
net: usbnet: Avoid potential RCU stall on LINK_CHANGE event
PCI/ACPI: Fix runtime PM ref imbalance on Hot-Plug Capable ports
ACPI: processor: idle: Check acpi_fetch_acpi_dev() return value
kbuild: Add KBUILD_CPPFLAGS to as-option invocation
kbuild: add $(CLANG_FLAGS) to KBUILD_CPPFLAGS
kbuild: Add CLANG_FLAGS to as-instr
mips: Include KBUILD_CPPFLAGS in CHECKFLAGS invocation
kbuild: Update assembler calls to use proper flags and language target
ARM: 9448/1: Use an absolute path to unified.h in KBUILD_AFLAGS
usb: dwc3: Ignore late xferNotReady event to prevent halt timeout
USB: storage: Ignore driver CD mode for Realtek multi-mode Wi-Fi dongles
usb: storage: realtek_cr: Use correct byte order for bcs->Residue
USB: storage: Add unusual-devs entry for Novatek NTK96550-based camera
usb: quirks: Add DELAY_INIT quick for another SanDisk 3.2Gen1 Flash Drive
iio: proximity: isl29501: fix buffered read on big-endian systems
ftrace: Also allocate and copy hash for reading of filter files
fpga: zynq_fpga: Fix the wrong usage of dma_map_sgtable()
fs/buffer: fix use-after-free when call bh_read() helper
drm/amd/display: Fix fractional fb divider in set_pixel_clock_v3
media: venus: Add a check for packet size after reading from shared memory
media: ov2659: Fix memory leaks in ov2659_probe()
media: usbtv: Lock resolution while streaming
media: gspca: Add bounds checking to firmware parser
jbd2: prevent softlockup in jbd2_log_do_checkpoint()
PCI: endpoint: Fix configfs group removal on driver teardown
PCI: endpoint: Fix configfs group list head handling
mtd: rawnand: fsmc: Add missing check after DMA map
wifi: brcmsmac: Remove const from tbl_ptr parameter in wlc_lcnphy_common_read_table()
zynq_fpga: use sgtable-based scatterlist wrappers
ata: libata-scsi: Fix ata_to_sense_error() status handling
ext4: fix reserved gdt blocks handling in fsmap
ext4: fix fsmap end of range reporting with bigalloc
ext4: check fast symlink for ea_inode correctly
Revert "vgacon: Add check for vc_origin address range in vgacon_scroll()"
vt: defkeymap: Map keycodes above 127 to K_HOLE
usb: gadget: udc: renesas_usb3: fix device leak at unbind
usb: atm: cxacru: Merge cxacru_upload_firmware() into cxacru_heavy_init()
m68k: Fix lost column on framebuffer debug console
serial: 8250: fix panic due to PSLVERR
media: uvcvideo: Do not mark valid metadata as invalid
media: uvcvideo: Fix 1-byte out-of-bounds read in uvc_parse_format()
btrfs: fix log tree replay failure due to file with 0 links and extents
thunderbolt: Fix copy+paste error in match_service_id()
misc: rtsx: usb: Ensure mmc child device is active when card is present
scsi: lpfc: Remove redundant assignment to avoid memory leak
rtc: ds1307: remove clear of oscillator stop flag (OSF) in probe
pNFS: Fix uninited ptr deref in block/scsi layout
pNFS: Fix disk addr range check in block/scsi layout
pNFS: Fix stripe mapping in block/scsi layout
ipmi: Fix strcpy source and destination the same
kconfig: lxdialog: fix 'space' to (de)select options
kconfig: gconf: fix potential memory leak in renderer_edited()
kconfig: gconf: avoid hardcoding model2 in on_treeview2_cursor_changed()
scsi: aacraid: Stop using PCI_IRQ_AFFINITY
scsi: Fix sas_user_scan() to handle wildcard and multi-channel scans
kconfig: nconf: Ensure null termination where strncpy is used
kconfig: lxdialog: replace strcpy() with strncpy() in inputbox.c
PCI: pnv_php: Work around switches with broken presence detection
media: uvcvideo: Fix bandwidth issue for Alcor camera
media: dvb-frontends: w7090p: fix null-ptr-deref in w7090p_tuner_write_serpar and w7090p_tuner_read_serpar
media: dvb-frontends: dib7090p: fix null-ptr-deref in dib7090p_rw_on_apb()
media: usb: hdpvr: disable zero-length read messages
media: tc358743: Increase FIFO trigger level to 374
media: tc358743: Return an appropriate colorspace from tc358743_set_fmt
media: tc358743: Check I2C succeeded during probe
pinctrl: stm32: Manage irq affinity settings
scsi: mpt3sas: Correctly handle ATA device errors
RDMA: hfi1: fix possible divide-by-zero in find_hw_thread_mask()
MIPS: Don't crash in stack_top() for tasks without ABI or vDSO
jfs: upper bound check of tree index in dbAllocAG
jfs: Regular file corruption check
jfs: truncate good inode pages when hard link is 0
scsi: bfa: Double-free fix
MIPS: vpe-mt: add missing prototypes for vpe_{alloc,start,stop,free}
watchdog: dw_wdt: Fix default timeout
fs/orangefs: use snprintf() instead of sprintf()
scsi: libiscsi: Initialize iscsi_conn->dd_data only if memory is allocated
ext4: do not BUG when INLINE_DATA_FL lacks system.data xattr
vhost: fail early when __vhost_add_used() fails
uapi: in6: restore visibility of most IPv6 socket options
net: ncsi: Fix buffer overflow in fetching version id
net: dsa: b53: fix b53_imp_vlan_setup for BCM5325
net: vlan: Replace BUG() with WARN_ON_ONCE() in vlan_dev_* stubs
wifi: iwlegacy: Check rate_idx range after addition
netmem: fix skb_frag_address_safe with unreadable skbs
wifi: rtlwifi: fix possible skb memory leak in `_rtl_pci_rx_interrupt()`.
wifi: iwlwifi: dvm: fix potential overflow in rs_fill_link_cmd()
net: fec: allow disable coalescing
(powerpc/512) Fix possible `dma_unmap_single()` on uninitialized pointer
s390/stp: Remove udelay from stp_sync_clock()
wifi: iwlwifi: mvm: fix scan request validation
net: thunderx: Fix format-truncation warning in bgx_acpi_match_id()
net: ipv4: fix incorrect MTU in broadcast routes
wifi: cfg80211: Fix interface type validation
et131x: Add missing check after DMA map
be2net: Use correct byte order and format string for TCP seq and ack_seq
s390/time: Use monotonic clock in get_cycles()
wifi: cfg80211: reject HTC bit for management frames
ktest.pl: Prevent recursion of default variable options
ASoC: codecs: rt5640: Retry DEVICE_ID verification
ALSA: usb-audio: Avoid precedence issues in mixer_quirks macros
ALSA: hda/ca0132: Fix buffer overflow in add_tuning_control
platform/x86: thinkpad_acpi: Handle KCOV __init vs inline mismatches
pm: cpupower: Fix the snapshot-order of tsc,mperf, clock in mperf_stop()
ALSA: intel8x0: Fix incorrect codec index usage in mixer for ICH4
ASoC: hdac_hdmi: Rate limit logging on connection and disconnection
mmc: rtsx_usb_sdmmc: Fix error-path in sd_set_power_mode()
ACPI: processor: fix acpi_object initialization
PM: sleep: console: Fix the black screen issue
thermal: sysfs: Return ENODATA instead of EAGAIN for reads
selftests: tracing: Use mutex_unlock for testing glob filter
ARM: tegra: Use I/O memcpy to write to IRAM
gpio: tps65912: check the return value of regmap_update_bits()
ASoC: soc-dapm: set bias_level if snd_soc_dapm_set_bias_level() was successed
cpufreq: Exit governor when failed to start old governor
usb: xhci: Avoid showing errors during surprise removal
usb: xhci: Set avg_trb_len = 8 for EP0 during Address Device Command
usb: xhci: Avoid showing warnings for dying controller
selftests/futex: Define SYS_futex on 32-bit architectures with 64-bit time_t
usb: xhci: print xhci->xhc_state when queue_command failed
securityfs: don't pin dentries twice, once is enough...
hfs: fix not erasing deleted b-tree node issue
drbd: add missing kref_get in handle_write_conflicts
arm64: Handle KCOV __init vs inline mismatches
hfsplus: don't use BUG_ON() in hfsplus_create_attributes_file()
hfsplus: fix slab-out-of-bounds read in hfsplus_uni2asc()
hfsplus: fix slab-out-of-bounds in hfsplus_bnode_read()
hfs: fix slab-out-of-bounds in hfs_bnode_read()
sctp: linearize cloned gso packets in sctp_rcv
netfilter: ctnetlink: fix refcount leak on table dump
udp: also consider secpath when evaluating ipsec use for checksumming
fs: Prevent file descriptor table allocations exceeding INT_MAX
sunvdc: Balance device refcount in vdc_port_mpgroup_check
NFSD: detect mismatch of file handle and delegation stateid in OPEN op
net: dpaa: fix device leak when querying time stamp info
net: gianfar: fix device leak when querying time stamp info
netlink: avoid infinite retry looping in netlink_unicast()
ALSA: usb-audio: Validate UAC3 cluster segment descriptors
ALSA: usb-audio: Validate UAC3 power domain descriptors, too
usb: gadget : fix use-after-free in composite_dev_cleanup()
MIPS: mm: tlb-r4k: Uniquify TLB entries on init
USB: serial: option: add Foxconn T99W709
vsock: Do not allow binding to VMADDR_PORT_ANY
net/packet: fix a race in packet_set_ring() and packet_notifier()
perf/core: Prevent VMA split of buffer mappings
perf/core: Exit early on perf_mmap() fail
perf/core: Don't leak AUX buffer refcount on allocation failure
pptp: fix pptp_xmit() error path
smb: client: let recv_done() cleanup before notifying the callers.
benet: fix BUG when creating VFs
ipv6: reject malicious packets in ipv6_gso_segment()
pptp: ensure minimal skb length in pptp_xmit()
netpoll: prevent hanging NAPI when netcons gets enabled
NFS: Fix filehandle bounds checking in nfs_fh_to_dentry()
pci/hotplug/pnv-php: Wrap warnings in macro
pci/hotplug/pnv-php: Improve error msg on power state change failure
usb: chipidea: udc: fix sleeping function called from invalid context
f2fs: fix to avoid out-of-boundary access in devs.path
f2fs: fix to avoid UAF in f2fs_sync_inode_meta()
rtc: pcf8563: fix incorrect maximum clock rate handling
rtc: hym8563: fix incorrect maximum clock rate handling
rtc: ds1307: fix incorrect maximum clock rate handling
mtd: rawnand: atmel: set pmecc data setup time
mtd: rawnand: atmel: Fix dma_mapping_error() address
jfs: fix metapage reference count leak in dbAllocCtl
fbdev: imxfb: Check fb_add_videomode to prevent null-ptr-deref
crypto: qat - fix seq_file position update in adf_ring_next()
dmaengine: nbpfaxi: Add missing check after DMA map
dmaengine: mv_xor: Fix missing check after DMA map and missing unmap
fs/orangefs: Allow 2 more characters in do_c_string()
crypto: img-hash - Fix dma_unmap_sg() nents value
scsi: isci: Fix dma_unmap_sg() nents value
scsi: mvsas: Fix dma_unmap_sg() nents value
scsi: ibmvscsi_tgt: Fix dma_unmap_sg() nents value
perf tests bp_account: Fix leaked file descriptor
crypto: ccp - Fix crash when rebind ccp device for ccp.ko
pinctrl: sunxi: Fix memory leak on krealloc failure
power: supply: max14577: Handle NULL pdata when CONFIG_OF is not set
clk: davinci: Add NULL check in davinci_lpsc_clk_register()
mtd: fix possible integer overflow in erase_xfer()
crypto: marvell/cesa - Fix engine load inaccuracy
PCI: rockchip-host: Fix "Unexpected Completion" log message
vrf: Drop existing dst reference in vrf_ip6_input_dst
netfilter: xt_nfacct: don't assume acct name is null-terminated
can: kvaser_usb: Assign netdev.dev_port based on device channel index
wifi: brcmfmac: fix P2P discovery failure in P2P peer due to missing P2P IE
Reapply "wifi: mac80211: Update skb's control block key in ieee80211_tx_dequeue()"
mwl8k: Add missing check after DMA map
wifi: rtl8xxxu: Fix RX skb size for aggregation disabled
net/sched: Restrict conditions for adding duplicating netems to qdisc tree
arch: powerpc: defconfig: Drop obsolete CONFIG_NET_CLS_TCINDEX
netfilter: nf_tables: adjust lockdep assertions handling
drm/amd/pm/powerplay/hwmgr/smu_helper: fix order of mask and value
m68k: Don't unregister boot console needlessly
tcp: fix tcp_ofo_queue() to avoid including too much DUP SACK range
iwlwifi: Add missing check for alloc_ordered_workqueue
wifi: iwlwifi: Fix memory leak in iwl_mvm_init()
wifi: rtl818x: Kill URBs before clearing tx status queue
caif: reduce stack size, again
staging: nvec: Fix incorrect null termination of battery manufacturer
samples: mei: Fix building on musl libc
usb: early: xhci-dbc: Fix early_ioremap leak
Revert "vmci: Prevent the dispatching of uninitialized payloads"
pps: fix poll support
vmci: Prevent the dispatching of uninitialized payloads
staging: fbtft: fix potential memory leak in fbtft_framebuffer_alloc()
ARM: dts: vfxxx: Correctly use two tuples for timer address
ASoC: ops: dynamically allocate struct snd_ctl_elem_value
hfsplus: remove mutex_lock check in hfsplus_free_extents
ASoC: Intel: fix SND_SOC_SOF dependencies
ethernet: intel: fix building with large NR_CPUS
usb: phy: mxs: disconnect line when USB charger is attached
usb: chipidea: udc: protect usb interrupt enable
usb: chipidea: udc: add new API ci_hdrc_gadget_connect
comedi: comedi_test: Fix possible deletion of uninitialized timers
nilfs2: reject invalid file types when reading inodes
i2c: qup: jump out of the loop in case of timeout
net/sched: sch_qfq: Avoid triggering might_sleep in atomic context in qfq_delete_class
net: appletalk: Fix use-after-free in AARP proxy probe
net: appletalk: fix kerneldoc warnings
RDMA/core: Rate limit GID cache warning messages
usb: hub: fix detection of high tier USB3 devices behind suspended hubs
net_sched: sch_sfq: reject invalid perturb period
net_sched: sch_sfq: move the limit validation
net_sched: sch_sfq: use a temporary work area for validating configuration
net_sched: sch_sfq: don't allow 1 packet limit
net_sched: sch_sfq: handle bigger packets
net_sched: sch_sfq: annotate data-races around q->perturb_period
power: supply: bq24190_charger: Fix runtime PM imbalance on error
xhci: Disable stream for xHC controller with XHCI_BROKEN_STREAMS
virtio-net: ensure the received length does not exceed allocated size
usb: dwc3: qcom: Don't leave BCR asserted
usb: musb: fix gadget state on disconnect
net/sched: Return NULL when htb_lookup_leaf encounters an empty rbtree
net: vlan: fix VLAN 0 refcount imbalance of toggling filtering during runtime
Bluetooth: L2CAP: Fix attempting to adjust outgoing MTU
Bluetooth: SMP: Fix using HCI_ERROR_REMOTE_USER_TERM on timeout
Bluetooth: SMP: If an unallowed command is received consider it a failure
Bluetooth: Fix null-ptr-deref in l2cap_sock_resume_cb()
usb: net: sierra: check for no status endpoint
net/sched: sch_qfq: Fix race condition on qfq_aggregate
net: emaclite: Fix missing pointer increment in aligned_read()
comedi: Fix use of uninitialized data in insn_rw_emulate_bits()
comedi: Fix some signed shift left operations
comedi: das6402: Fix bit shift out of bounds
comedi: das16m1: Fix bit shift out of bounds
comedi: aio_iiro_16: Fix bit shift out of bounds
comedi: pcl812: Fix bit shift out of bounds
iio: adc: max1363: Reorder mode_list[] entries
iio: adc: max1363: Fix MAX1363_4X_CHANS/MAX1363_8X_CHANS[]
soc: aspeed: lpc-snoop: Don't disable channels that aren't enabled
soc: aspeed: lpc-snoop: Cleanup resources in stack-order
mmc: sdhci-pci: Quirk for broken command queuing on Intel GLK-based Positivo models
memstick: core: Zero initialize id_reg in h_memstick_read_dev_id()
isofs: Verify inode mode when loading from disk
dmaengine: nbpfaxi: Fix memory corruption in probe()
af_packet: fix soft lockup issue caused by tpacket_snd()
af_packet: fix the SO_SNDTIMEO constraint not effective on tpacked_snd()
phonet/pep: Move call to pn_skb_get_dst_sockaddr() earlier in pep_sock_accept()
HID: core: do not bypass hid_hw_raw_request
HID: core: ensure __hid_request reserves the report ID as the first byte
HID: core: ensure the allocated report buffer can contain the reserved report ID
pch_uart: Fix dma_sync_sg_for_device() nents value
Input: xpad - set correct controller type for Acer NGR200
i2c: stm32: fix the device used for the DMA map
usb: gadget: configfs: Fix OOB read on empty string write
USB: serial: ftdi_sio: add support for NDI EMGUIDE GEMINI
USB: serial: option: add Foxconn T99W640
USB: serial: option: add Telit Cinterion FE910C04 (ECM) composition
dma-mapping: add generic helpers for mapping sgtable objects
usb: renesas_usbhs: Flush the notify_hotplug_work
gpio: rcar: Use raw_spinlock to protect register access
Change-Id: Ia6b8b00918487999c648f298d3550afc7eaaae03
Signed-off-by: bengris32 <bengris32@protonmail.ch>
[ Upstream commit 4f23382841e67174211271a454811dd17c0ef3c5 ]
Enabling RX FIFO Overflow interrupts is counterproductive
and causes an interrupt storm when RX FIFO overflows.
Disabling this interrupt has no side effect and eliminates
interrupt storms when the RX FIFO overflows.
Commit 8a7cb245cf28 ("net: stmmac: Do not enable RX FIFO
overflow interrupts") disables RX FIFO overflow interrupts
for DWMAC4 IP and removes the corresponding handling of
this interrupt. This patch is doing the same thing for
XGMAC IP.
Fixes: 2142754f8b ("net: stmmac: Add MAC related callbacks for XGMAC2")
Signed-off-by: Rohan G Thomas <rohan.g.thomas@altera.com>
Reviewed-by: Matthew Gerlach <matthew.gerlach@altera.com>
Reviewed-by: Andrew Lunn <andrew@lunn.ch>
Link: https://patch.msgid.link/20250825-xgmac-minor-fixes-v3-1-c225fe4444c0@altera.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
Signed-off-by: Ulrich Hecht <uli@kernel.org>
[ Upstream commit aca0c31af61e0d5cf1675a0cbd29460b95ae693c ]
The local Xoff value is being set before the firmware (FW) update.
In case of a failure where the FW is not updated with the new value,
there is no fallback to the previous value.
Update the local Xoff value after the FW has been successfully set.
Fixes: 0696d60853 ("net/mlx5e: Receive buffer configuration")
Signed-off-by: Alexei Lazar <alazar@nvidia.com>
Reviewed-by: Tariq Toukan <tariqt@nvidia.com>
Reviewed-by: Dragos Tatulea <dtatulea@nvidia.com>
Signed-off-by: Mark Bloch <mbloch@nvidia.com>
Link: https://patch.msgid.link/20250825143435.598584-12-mbloch@nvidia.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
Signed-off-by: Ulrich Hecht <uli@kernel.org>
[ Upstream commit 007a5ffadc4fd51739527f1503b7cf048f31c413 ]
`McstFramesRcvdOk` counts the number of received multicast packets, and
it reports the value correctly.
However, reading `McstFramesRcvdOk` clears the register to zero. As a
result, the driver was reporting only the packets since the last read,
instead of the accumulated total.
Fix this by updating the multicast statistics accumulatively instaed of
instantaneously.
Fixes: 1da177e4c3 ("Linux-2.6.12-rc2")
Tested-on: D-Link DGE-550T Rev-A3
Signed-off-by: Yeounsu Moon <yyyynoom@gmail.com>
Reviewed-by: Andrew Lunn <andrew@lunn.ch>
Link: https://patch.msgid.link/20250823182927.6063-3-yyyynoom@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
Signed-off-by: Ulrich Hecht <uli@kernel.org>
Add an efficient ingress to ingress netns switch that can be used out of tc BPF
programs in order to redirect traffic from host ns ingress into a container
veth device ingress without having to go via CPU backlog queue [0]. For local
containers this can also be utilized and path via CPU backlog queue only needs
to be taken once, not twice. On a high level this borrows from ipvlan which does
similar switch in __netif_receive_skb_core() and then iterates via another_round.
This helps to reduce latency for mentioned use cases.
Pod to remote pod with redirect(), TCP_RR [1]:
# percpu_netperf 10.217.1.33
RT_LATENCY: 122.450 (per CPU: 122.666 122.401 122.333 122.401 )
MEAN_LATENCY: 121.210 (per CPU: 121.100 121.260 121.320 121.160 )
STDDEV_LATENCY: 120.040 (per CPU: 119.420 119.910 125.460 115.370 )
MIN_LATENCY: 46.500 (per CPU: 47.000 47.000 47.000 45.000 )
P50_LATENCY: 118.500 (per CPU: 118.000 119.000 118.000 119.000 )
P90_LATENCY: 127.500 (per CPU: 127.000 128.000 127.000 128.000 )
P99_LATENCY: 130.750 (per CPU: 131.000 131.000 129.000 132.000 )
TRANSACTION_RATE: 32666.400 (per CPU: 8152.200 8169.842 8174.439 8169.897 )
Pod to remote pod with redirect_peer(), TCP_RR:
# percpu_netperf 10.217.1.33
RT_LATENCY: 44.449 (per CPU: 43.767 43.127 45.279 45.622 )
MEAN_LATENCY: 45.065 (per CPU: 44.030 45.530 45.190 45.510 )
STDDEV_LATENCY: 84.823 (per CPU: 66.770 97.290 84.380 90.850 )
MIN_LATENCY: 33.500 (per CPU: 33.000 33.000 34.000 34.000 )
P50_LATENCY: 43.250 (per CPU: 43.000 43.000 43.000 44.000 )
P90_LATENCY: 46.750 (per CPU: 46.000 47.000 47.000 47.000 )
P99_LATENCY: 52.750 (per CPU: 51.000 54.000 53.000 53.000 )
TRANSACTION_RATE: 90039.500 (per CPU: 22848.186 23187.089 22085.077 21919.130 )
[0] https://linuxplumbersconf.org/event/7/contributions/674/attachments/568/1002/plumbers_2020_cilium_load_balancer.pdf
[1] https://github.com/borkmann/netperf_scripts/blob/master/percpu_netperf
Change-Id: I17d75ffbb776ea4e36326b8fdd04b71441a1982b
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20201010234006.7075-3-daniel@iogearbox.net
The veth driver can run XDP in "native" mode in it's own NAPI
handler, and since commit 9fc8d518d9 ("veth: Handle xdp_frames in
xdp napi ring") packets can come in two forms either xdp_frame or
skb, calling respectively veth_xdp_rcv_one() or veth_xdp_rcv_skb().
For packets to arrive in xdp_frame format, they will have been
redirected from an XDP native driver. In case of XDP_PASS or no
XDP-prog attached, the veth driver will allocate and create an SKB.
The current code in veth_xdp_rcv_one() xdp_frame case, had to guess
the frame truesize of the incoming xdp_frame, when using
veth_build_skb(). With xdp_frame->frame_sz this is not longer
necessary.
Calculating the frame_sz in veth_xdp_rcv_skb() skb case, is done
similar to the XDP-generic handling code in net/core/dev.c.
Change-Id: I7dcc457a555583478390d025b5cb2e555ff9a855
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Reviewed-by: Lorenzo Bianconi <lorenzo@kernel.org>
Acked-by: Toke Høiland-Jørgensen <toke@redhat.com>
Acked-by: Toshiaki Makita <toshiaki.makita1@gmail.com>
Link: https://lore.kernel.org/bpf/158945338840.97035.935897116345700902.stgit@firesoul
Rely on 'remote' veth_rq to account ndo_xdp_xmit ethtool counters.
Move XDP_TX accounting to veth_xdp_flush_bq routine.
Remove 'rx' prefix in rx xdp ethool counters
Change-Id: Ia2d6c694994c260336a67d5f674a0d133bdae4ea
Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Acked-by: Toshiaki Makita <toshiaki.makita1@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Substitute net_device point with veth_rq one in veth_xdp_flush_bq,
veth_xdp_flush and veth_xdp_tx signature. This is a preliminary patch
to account xdp_xmit counter on 'receiving' veth_rq
Acked-by: Toshiaki Makita <toshiaki.makita1@gmail.com>
Change-Id: Ia4db081a1e3215b920879ac2c1f4bc2f9a329a02
Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Remove atomic64_add from veth_xdp_xmit hotpath and rely on
xdp_xmit_err/xdp_tx_err counters
Change-Id: Ib4d6b80474ce959017982b5d55f01538984d8270
Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Introduce xdp_xmit counter in order to distinguish between XDP_TX and
ndo_xdp_xmit stats. Introduce the following ethtool counters:
- rx_xdp_tx
- rx_xdp_tx_errors
- tx_xdp_xmit
- tx_xdp_xmit_errors
- rx_xdp_redirect
Change-Id: I49eb2307b982646d957b825ef900524d7a6daef0
Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Distinguish between rx_drops and xdp_drops since the latter is already
reported in rx_packets. Report xdp_drops in ethtool statistics
Change-Id: I5ec1941f947b6a182280f23d0ba68567036cb125
Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Introduce xdp_tx, xdp_redirect and rx_drops counters in veth_stats data
structure. Move stats accounting in veth_poll. Remove xdp_xmit variable
in veth_xdp_rcv_one/veth_xdp_rcv_skb and rely on veth_stats counters.
This is a preliminary patch to align veth xdp statistics to mlx, intel
and marvell xdp implementation
Change-Id: I094db83835e4271b40ad563b589d928451b4c2e1
Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Move xdp stats in veth_stats data structure. This is a preliminary patch
to align xdp statistics to mlx5, ixgbe and mvneta drivers
Change-Id: I587c4a52dafaa6a083a4e759f2dae62185962191
Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
When local NET_RX backlog is full due to traffic overrun,
peer veth tx_dropped counter increases. At that time, list
local veth stats, rx_dropped has double value of peer
tx_dropped, even bigger than transmit packets by peer.
In NET_RX softirq process, if any packet drop case happens,
it increases dev's rx_dropped counter and returns NET_RX_DROP.
At veth tx side, it records any error returned from peer netif_rx
into local dev tx_dropped counter.
In veth get stats process, it puts local dev rx_dropped and
peer dev tx_dropped into together as local rx_drpped value.
So that it shows double value of real dropped packets number in
this case.
This patch ignores peer tx_dropped when counting local rx_dropped,
since peer tx_dropped is duplicated to local rx_dropped at most cases.
Change-Id: I47d34357134dddba463a18afbba1df7dcffb64c1
Signed-off-by: Jiang Lidong <jianglidong3@jd.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This cleanup will ease u64_stats_t adoption in a single location.
Change-Id: I46fc0b015355a3b259394af87235c4ab2142371f
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Many network drivers use hand-coded implementation of the same thing,
let's factorize things so that u64_stats_t adoption is done once.
Change-Id: I694032b38d89421892aec589291421b8ed82342b
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Many network drivers need it and hand-coded the same function.
In order to ease u64_stats_t adoption, it is time to factorize.
Change-Id: I31489e072d0a26d8099544576f662b2531caf5d3
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Like cpumap use xdp_release_frame() when an xdp_frame got
converted into an SKB and send towars the network stack.
Change-Id: I0471b6478aa4fec6b4d115b8509195ca094fb02d
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
For reporting the common set of SW timestamping capabilities, use
ethtool_op_get_ts_info() instead of re-implementing it.
Change-Id: Ibb30bb4a72700a024f427cc4ef934a8bb8ea9648
Signed-off-by: Julian Wiedmann <jwi@linux.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Provide a precision hint to snprintf() in order to eliminate a
-Wformat-truncation warning provided below. A maximum of 11 characters
is allowed to reach a maximum of 32 - 1 characters given a possible
maximum value of queues using up to UINT_MAX which occupies 10
characters. Incidentally 11 is the number of characters for
"xdp_packets" which is the largest string we append.
drivers/net/veth.c: In function 'veth_get_strings':
drivers/net/veth.c:118:47: warning: '%s' directive output may be
truncated writing up to 31 bytes into a region of size between 12 and 21
[-Wformat-truncation=]
snprintf(p, ETH_GSTRING_LEN, "rx_queue_%u_%s",
^~
drivers/net/veth.c:118:5: note: 'snprintf' output between 12 and 52
bytes into a destination of size 32
snprintf(p, ETH_GSTRING_LEN, "rx_queue_%u_%s",
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
i, veth_rq_stats_desc[j].desc);
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Change-Id: Ie6768f456b617e9dfc9943706f8cb274d9d40f01
Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
In preparation to enabling -Wimplicit-fallthrough, mark switch
cases where we are expecting to fall through.
Warning level 3 was used: -Wimplicit-fallthrough=3
This patch is part of the ongoing efforts to enabling
-Wimplicit-fallthrough.
Change-Id: I598ef85ab3c789f9080a25800a3830d5e3462cf1
Signed-off-by: Gustavo A. R. Silva <gustavo@embeddedor.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Add extack arg to rtnl_create_link and add messages for invalid
number of Tx or Rx queues.
Change-Id: If1a654feb3174b337676d8689fa3fc27b5580eef
Signed-off-by: David Ahern <dsahern@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Expose per-queue stats for ethtool -S.
As there are only rx queues, and rx queues are used only when XDP is
used, per-queue counters are only rx XDP ones.
Example:
$ ethtool -S veth0
NIC statistics:
peer_ifindex: 11
rx_queue_0_xdp_packets: 28601434
rx_queue_0_xdp_bytes: 1716086040
rx_queue_0_xdp_drops: 28601434
rx_queue_1_xdp_packets: 17873050
rx_queue_1_xdp_bytes: 1072383000
rx_queue_1_xdp_drops: 17873050
Change-Id: I745c08220a892740061c4c429c71ff66d98b49d8
Signed-off-by: Toshiaki Makita <makita.toshiaki@lab.ntt.co.jp>
Signed-off-by: David S. Miller <davem@davemloft.net>
Provide a software TX timestamp as well as the ethtool query interface
and report the software timestamp capabilities.
Tested with "ethtool -T" and two linuxptp instances each bound to a
tunnel endpoint.
Change-Id: I9e71b03af396ab1a2e9a87400a5721dff5d28216
Signed-off-by: Michael Walle <michael@walle.cc>
Signed-off-by: David S. Miller <davem@davemloft.net>
XDP_TX is similar to XDP_REDIRECT as it essentially redirects packets to
the device itself. XDP_REDIRECT has bulk transmit mechanism to avoid the
heavy cost of indirect call but it also reduces lock acquisition on the
destination device that needs locks like veth and tun.
XDP_TX does not use indirect calls but drivers which require locks can
benefit from the bulk transmit for XDP_TX as well.
This patch introduces bulk transmit mechanism in veth using bulk queue
on stack, and improves XDP_TX performance by about 9%.
Here are single-core/single-flow XDP_TX test results. CPU consumptions
are taken from "perf report --no-child".
- Before:
7.26 Mpps
_raw_spin_lock 7.83%
veth_xdp_xmit 12.23%
- After:
7.94 Mpps
_raw_spin_lock 1.08%
veth_xdp_xmit 6.10%
v2:
- Use stack for bulk queue instead of a global variable.
Change-Id: Iacdbeb7f437782093281e4d3d49bbc712bb68765
Signed-off-by: Toshiaki Makita <toshiaki.makita1@gmail.com>
Acked-by: Jesper Dangaard Brouer <brouer@redhat.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
On XDP path veth has napi handler so we can collect statistics on
per-queue basis for XDP.
By this change now we can collect XDP_DROP drop count as well as packets
and bytes coming through ndo_xdp_xmit. Packet counters shown by
"ip -s link", sysfs stats or /proc/net/dev is now correct for XDP.
Change-Id: I8f6e22abc07d44f9efad1d222a009aa09c01f477
Signed-off-by: Toshiaki Makita <makita.toshiaki@lab.ntt.co.jp>
Signed-off-by: David S. Miller <davem@davemloft.net>
struct pcpu_vstats and pcpu_lstats have same members and
usage, and pcpu_lstats is used in many files, so rename
pcpu_vstats as pcpu_lstats to reduce duplicate definition
Change-Id: I0153bd1d2cfe722b78eeffaf263fb6317f37dc30
Signed-off-by: Zhang Yu <zhangyu31@baidu.com>
Signed-off-by: Li RongQing <lirongqing@baidu.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
A common theme in the output path is looking up a neigh entry for a
nexthop, either the gateway in an rtable or a fallback to the daddr
in the skb:
nexthop = (__force u32)rt_nexthop(rt, ip_hdr(skb)->daddr);
neigh = __ipv4_neigh_lookup_noref(dev, nexthop);
if (unlikely(!neigh))
neigh = __neigh_create(&arp_tbl, &nexthop, dev, false);
To allow the nexthop to be an IPv6 address we need to consider the
family of the nexthop and then call __ipv{4,6}_neigh_lookup_noref based
on it.
To make this simpler, add a ip_neigh_gw4 helper similar to ip_neigh_gw6
added in an earlier patch which handles:
neigh = __ipv4_neigh_lookup_noref(dev, nexthop);
if (unlikely(!neigh))
neigh = __neigh_create(&arp_tbl, &nexthop, dev, false);
And then add a second one, ip_neigh_for_gw, that calls either
ip_neigh_gw4 or ip_neigh_gw6 based on the address family of the gateway.
Update the output paths in the VRF driver and core v4 code to use
ip_neigh_for_gw simplifying the family based lookup and making both
ready for a v6 nexthop.
ipv4_neigh_lookup has a different need - the potential to resolve a
passed in address in addition to any gateway in the rtable or skb. Since
this is a one-off, add ip_neigh_gw4 and ip_neigh_gw6 diectly. The
difference between __neigh_create used by the helpers and neigh_create
called by ipv4_neigh_lookup is taking a refcount, so add rcu_read_lock_bh
and bump the refcnt on the neigh entry.
Change-Id: Id5eb472dec5ecbe65758259fab01ed3c56bf7844
Signed-off-by: David Ahern <dsahern@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
To allow the gateway to be either an IPv4 or IPv6 address, remove
rt_uses_gateway from rtable and replace with rt_gw_family. If
rt_gw_family is set it implies rt_uses_gateway. Rename rt_gateway
to rt_gw4 to represent the IPv4 version.
Change-Id: I6e8fa98d38bf26fa1bac1ee241694539458e27dd
Signed-off-by: David Ahern <dsahern@gmail.com>
Reviewed-by: Ido Schimmel <idosch@mellanox.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Rename fib_nh entries that will be moved to a fib_nh_common struct.
Specifically, the device, oif, gateway, flags, scope, lwtstate,
nh_weight and nh_upper_bound are common with all nexthop definitions.
In the process shorten fib_nh_lwtstate to fib_nh_lws to avoid really
long lines.
Rename only; no functional change intended.
Change-Id: I2adcbe39cea5801be8f4578e5d5619284158997f
Signed-off-by: David Ahern <dsahern@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
A later patch allows an IPv6 gateway with an IPv4 route. The neighbor
entry will exist in the v6 ndisc table and the cached header will contain
the ipv6 protocol which is wrong for an IPv4 packet. For an IPv4 packet to
use the v6 neighbor entry, neigh_output needs to skip the cached header
and just use the output callback for the neigh entry.
A future patchset can look at expanding the hh_cache to handle 2
protocols. For now, IPv6 gateways with an IPv4 route will take the
extra overhead of generating the header.
Change-Id: Icab365a1c8d98726b8423dafbbf29ade7d2be8a4
Signed-off-by: David Ahern <dsahern@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
The number of stubs is growing and has nothing to do with addrconf.
Move the definition of the stubs to a separate header file and update
users. In the move, drop the vxlan specific comment before ipv6_stub.
Code move only; no functional change intended.
Change-Id: I77b025a0cfdcd9921dbbf29419b7a94beb620e96
Signed-off-by: David Ahern <dsahern@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
To support modifying the used_maps array, we use a mutex to protect
the use of the counter and the array. The mutex is initialized right
after the prog aux is allocated, and destroyed right before prog
aux is freed. This way we guarantee it's initialized for both cBPF
and eBPF.
Change-Id: I5a5a46193377945bf8bded1a5d8e2effc426555e
Signed-off-by: YiFei Zhu <zhuyifei@google.com>
Signed-off-by: Stanislav Fomichev <sdf@google.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Andrii Nakryiko <andriin@fb.com>
Cc: YiFei Zhu <zhuyifei1999@gmail.com>
Link: https://lore.kernel.org/bpf/20200915234543.3220146-2-sdf@google.com
Cater to devices which:
(a) may want to sleep in the callbacks;
(b) only have IPv4 support;
(c) need all the programming to happen while the netdev is up.
Drivers attach UDP tunnel offload info struct to their netdevs,
where they declare how many UDP ports of various tunnel types
they support. Core takes care of tracking which ports to offload.
Use a fixed-size array since this matches what almost all drivers
do, and avoids a complexity and uncertainty around memory allocations
in an atomic context.
Make sure that tunnel drivers don't try to replay the ports when
new NIC netdev is registered. Automatic replays would mess up
reference counting, and will be removed completely once all drivers
are converted.
v4:
- use a #define NULL to avoid build issues with CONFIG_INET=n.
Change-Id: I0f68142abd1924a25cbd6e6333f66254ef1e958f
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Now that we depend on rcu_call() and synchronize_rcu() to also wait
for preempt_disabled region to complete the rcu read critical section
in __dev_map_flush() is no longer required. Except in a few special
cases in drivers that need it for other reasons.
These originally ensured the map reference was safe while a map was
also being free'd. And additionally that bpf program updates via
ndo_bpf did not happen while flush updates were in flight. But flush
by new rules can only be called from preempt-disabled NAPI context.
The synchronize_rcu from the map free path and the rcu_call from the
delete path will ensure the reference there is safe. So lets remove
the rcu_read_lock and rcu_read_unlock pair to avoid any confusion
around how this is being protected.
If the rcu_read_lock was required it would mean errors in the above
logic and the original patch would also be wrong.
Now that we have done above we put the rcu_read_lock in the driver
code where it is needed in a driver dependent way. I think this
helps readability of the code so we know where and why we are
taking read locks. Most drivers will not need rcu_read_locks here
and further XDP drivers already have rcu_read_locks in their code
paths for reading xdp programs on RX side so this makes it symmetric
where we don't have half of rcu critical sections define in driver
and the other half in devmap.
Change-Id: I5c58307c1e21d94a1d4db1724ed6532a5f1d9ab1
Signed-off-by: John Fastabend <john.fastabend@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Jesper Dangaard Brouer <brouer@redhat.com>
Link: https://lore.kernel.org/bpf/1580084042-11598-4-git-send-email-john.fastabend@gmail.com
Use existing atomic drop counter. Since drop path is really an
exceptional case here, I'm thinking atomic ops would not hurt the
performance.
XDP packets and bytes are not counted in ndo_xdp_xmit, but will be
accounted on rx side by the following commit.
Change-Id: If6cc40e15e8d10b2df3908bb3ec198119f0f0514
Signed-off-by: Toshiaki Makita <makita.toshiaki@lab.ntt.co.jp>
Signed-off-by: David S. Miller <davem@davemloft.net>
Since the bulk queue used by XDP_REDIRECT now lives in struct net_device,
we can re-use the bulking for the non-map version of the bpf_redirect()
helper. This is a simple matter of having xdp_do_redirect_slow() queue the
frame on the bulk queue instead of sending it out with __bpf_tx_xdp().
Unfortunately we can't make the bpf_redirect() helper return an error if
the ifindex doesn't exit (as bpf_redirect_map() does), because we don't
have a reference to the network namespace of the ingress device at the time
the helper is called. So we have to leave it as-is and keep the device
lookup in xdp_do_redirect_slow().
Since this leaves less reason to have the non-map redirect code in a
separate function, so we get rid of the xdp_do_redirect_slow() function
entirely. This does lose us the tracepoint disambiguation, but fortunately
the xdp_redirect and xdp_redirect_map tracepoints use the same tracepoint
entry structures. This means both can contain a map index, so we can just
amend the tracepoint definitions so we always emit the xdp_redirect(_err)
tracepoints, but with the map ID only populated if a map is present. This
means we retire the xdp_redirect_map(_err) tracepoints entirely, but keep
the definitions around in case someone is still listening for them.
With this change, the performance of the xdp_redirect sample program goes
from 5Mpps to 8.4Mpps (a 68% increase).
Since the flush functions are no longer map-specific, rename the flush()
functions to drop _map from their names. One of the renamed functions is
the xdp_do_flush_map() callback used in all the xdp-enabled drivers. To
keep from having to update all drivers, use a #define to keep the old name
working, and only update the virtual drivers in this patch.
Change-Id: I693b2b7799c05cad5bd69bf315a25ec30f62e608
Signed-off-by: Toke Høiland-Jørgensen <toke@redhat.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: John Fastabend <john.fastabend@gmail.com>
Link: https://lore.kernel.org/bpf/157918768505.1458396.17518057312953572912.stgit@toke.dk
Similarly to bpf_map's refcnt/usercnt, convert bpf_prog's refcnt to atomic64
and remove artificial 32k limit. This allows to make bpf_prog's refcounting
non-failing, simplifying logic of users of bpf_prog_add/bpf_prog_inc.
Validated compilation by running allyesconfig kernel build.
Suggested-by: Daniel Borkmann <daniel@iogearbox.net>
Change-Id: Ica9b8722150fd349affe714cf27c4e1554084e49
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20191117172806.2195367-3-andriin@fb.com
92117d8443 ("bpf: fix refcnt overflow") turned refcounting of bpf_map into
potentially failing operation, when refcount reaches BPF_MAX_REFCNT limit
(32k). Due to using 32-bit counter, it's possible in practice to overflow
refcounter and make it wrap around to 0, causing erroneous map free, while
there are still references to it, causing use-after-free problems.
But having a failing refcounting operations are problematic in some cases. One
example is mmap() interface. After establishing initial memory-mapping, user
is allowed to arbitrarily map/remap/unmap parts of mapped memory, arbitrarily
splitting it into multiple non-contiguous regions. All this happening without
any control from the users of mmap subsystem. Rather mmap subsystem sends
notifications to original creator of memory mapping through open/close
callbacks, which are optionally specified during initial memory mapping
creation. These callbacks are used to maintain accurate refcount for bpf_map
(see next patch in this series). The problem is that open() callback is not
supposed to fail, because memory-mapped resource is set up and properly
referenced. This is posing a problem for using memory-mapping with BPF maps.
One solution to this is to maintain separate refcount for just memory-mappings
and do single bpf_map_inc/bpf_map_put when it goes from/to zero, respectively.
There are similar use cases in current work on tcp-bpf, necessitating extra
counter as well. This seems like a rather unfortunate and ugly solution that
doesn't scale well to various new use cases.
Another approach to solve this is to use non-failing refcount_t type, which
uses 32-bit counter internally, but, once reaching overflow state at UINT_MAX,
stays there. This utlimately causes memory leak, but prevents use after free.
But given refcounting is not the most performance-critical operation with BPF
maps (it's not used from running BPF program code), we can also just switch to
64-bit counter that can't overflow in practice, potentially disadvantaging
32-bit platforms a tiny bit. This simplifies semantics and allows above
described scenarios to not worry about failing refcount increment operation.
In terms of struct bpf_map size, we are still good and use the same amount of
space:
BEFORE (3 cache lines, 8 bytes of padding at the end):
struct bpf_map {
const struct bpf_map_ops * ops __attribute__((__aligned__(64))); /* 0 8 */
struct bpf_map * inner_map_meta; /* 8 8 */
void * security; /* 16 8 */
enum bpf_map_type map_type; /* 24 4 */
u32 key_size; /* 28 4 */
u32 value_size; /* 32 4 */
u32 max_entries; /* 36 4 */
u32 map_flags; /* 40 4 */
int spin_lock_off; /* 44 4 */
u32 id; /* 48 4 */
int numa_node; /* 52 4 */
u32 btf_key_type_id; /* 56 4 */
u32 btf_value_type_id; /* 60 4 */
/* --- cacheline 1 boundary (64 bytes) --- */
struct btf * btf; /* 64 8 */
struct bpf_map_memory memory; /* 72 16 */
bool unpriv_array; /* 88 1 */
bool frozen; /* 89 1 */
/* XXX 38 bytes hole, try to pack */
/* --- cacheline 2 boundary (128 bytes) --- */
atomic_t refcnt __attribute__((__aligned__(64))); /* 128 4 */
atomic_t usercnt; /* 132 4 */
struct work_struct work; /* 136 32 */
char name[16]; /* 168 16 */
/* size: 192, cachelines: 3, members: 21 */
/* sum members: 146, holes: 1, sum holes: 38 */
/* padding: 8 */
/* forced alignments: 2, forced holes: 1, sum forced holes: 38 */
} __attribute__((__aligned__(64)));
AFTER (same 3 cache lines, no extra padding now):
struct bpf_map {
const struct bpf_map_ops * ops __attribute__((__aligned__(64))); /* 0 8 */
struct bpf_map * inner_map_meta; /* 8 8 */
void * security; /* 16 8 */
enum bpf_map_type map_type; /* 24 4 */
u32 key_size; /* 28 4 */
u32 value_size; /* 32 4 */
u32 max_entries; /* 36 4 */
u32 map_flags; /* 40 4 */
int spin_lock_off; /* 44 4 */
u32 id; /* 48 4 */
int numa_node; /* 52 4 */
u32 btf_key_type_id; /* 56 4 */
u32 btf_value_type_id; /* 60 4 */
/* --- cacheline 1 boundary (64 bytes) --- */
struct btf * btf; /* 64 8 */
struct bpf_map_memory memory; /* 72 16 */
bool unpriv_array; /* 88 1 */
bool frozen; /* 89 1 */
/* XXX 38 bytes hole, try to pack */
/* --- cacheline 2 boundary (128 bytes) --- */
atomic64_t refcnt __attribute__((__aligned__(64))); /* 128 8 */
atomic64_t usercnt; /* 136 8 */
struct work_struct work; /* 144 32 */
char name[16]; /* 176 16 */
/* size: 192, cachelines: 3, members: 21 */
/* sum members: 154, holes: 1, sum holes: 38 */
/* forced alignments: 2, forced holes: 1, sum forced holes: 38 */
} __attribute__((__aligned__(64)));
This patch, while modifying all users of bpf_map_inc, also cleans up its
interface to match bpf_map_put with separate operations for bpf_map_inc and
bpf_map_inc_with_uref (to match bpf_map_put and bpf_map_put_with_uref,
respectively). Also, given there are no users of bpf_map_inc_not_zero
specifying uref=true, remove uref flag and default to uref=false internally.
Change-Id: Ib745228b950cb1a6dda082e969cf1e6ee1ce6e7a
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Song Liu <songliubraving@fb.com>
Link: https://lore.kernel.org/bpf/20191117172806.2195367-2-andriin@fb.com
Current fib_multipath_hash_policy can make hash based on the L3 or
L4. But it only work on the outer IP. So a specific tunnel always
has the same hash value. But a specific tunnel may contain so many
inner connections.
This patch provide a generic multipath_hash in floi_common. It can
make a user-define hash which can mix with L3 or L4 hash.
Change-Id: I9abee7b7bf21b22e2fcc2303358eff9034319736
Signed-off-by: wenxu <wenxu@ucloud.cn>
Signed-off-by: David S. Miller <davem@davemloft.net>
This makes the age information for cfg80211 scan results more accurate
and fixes issues with wpa_supplicant dropping "old" scan results (e.g.,
"wlan0: Own scan request started a scan in 0.000456 seconds") that
looked like would have been received before a scan started due to the
inaccuracy of the default timing mechanism for calculating the BSS entry
age. This makes hwsim test cases significantly more robust to run.
Change-Id: I0518fad6cbe0d56a4ce816f021351b1167ddb75e
Signed-off-by: Jouni Malinen <j@w1.fi>
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
