Evolution-X-Devices/kernel_samsung_a71
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge branch 'android-4.14' of https://github.com/pascua28/android_kernel_samsung_sm7150 into 16.0

Browse Source 
Change-Id: I70dd6d9d8f57595d95c2b8ce0cb76a866c43d949
This commit is contained in:
Samuel Pascua
2025-10-02 22:19:27 +08:00
parent 90c1db580b d66524a400
commit fb2e9a4f1d
795 changed files with 44514 additions and 17659 deletions
Download Patch File Download Diff File Expand all files Collapse all files

17
Documentation/ABI/testing/sysfs-kernel-btf Normal file
View File

@@ -0,0 +1,17 @@
What: /sys/kernel/btf
Date: Aug 2019
KernelVersion: 5.5
Contact: bpf@vger.kernel.org
Description:
Contains BTF type information and related data for kernel and
kernel modules.
What: /sys/kernel/btf/vmlinux
Date: Aug 2019
KernelVersion: 5.5
Contact: bpf@vger.kernel.org
Description:
Read-only binary attribute exposing kernel's own BTF type
information with description of all internal kernel types. See
Documentation/bpf/btf.rst for detailed description of format
itself.

50
Documentation/RCU/Design/Requirements/Requirements.html
View File

@@ -2393,30 +2393,9 @@ when invoked from a CPU-hotplug notifier.
<p>
RCU depends on the scheduler, and the scheduler uses RCU to
protect some of its data structures.
This means the scheduler is forbidden from acquiring
the runqueue locks and the priority-inheritance locks
in the middle of an outermost RCU read-side critical section unless either
(1)&nbsp;it releases them before exiting that same
RCU read-side critical section, or
(2)&nbsp;interrupts are disabled across
that entire RCU read-side critical section.
This same prohibition also applies (recursively!) to any lock that is acquired
while holding any lock to which this prohibition applies.
Adhering to this rule prevents preemptible RCU from invoking
<tt>rcu_read_unlock_special()</tt> while either runqueue or
priority-inheritance locks are held, thus avoiding deadlock.
<p>
Prior to v4.4, it was only necessary to disable preemption across
RCU read-side critical sections that acquired scheduler locks.
In v4.4, expedited grace periods started using IPIs, and these
IPIs could force a <tt>rcu_read_unlock()</tt> to take the slowpath.
Therefore, this expedited-grace-period change required disabling of
interrupts, not just preemption.
<p>
For RCU's part, the preemptible-RCU <tt>rcu_read_unlock()</tt>
implementation must be written carefully to avoid similar deadlocks.
The preemptible-RCU <tt>rcu_read_unlock()</tt>
implementation must therefore be written carefully to avoid deadlocks
involving the scheduler's runqueue and priority-inheritance locks.
In particular, <tt>rcu_read_unlock()</tt> must tolerate an
interrupt where the interrupt handler invokes both
<tt>rcu_read_lock()</tt> and <tt>rcu_read_unlock()</tt>.
@@ -2425,7 +2404,7 @@ negative nesting levels to avoid destructive recursion via
interrupt handler's use of RCU.
<p>
This pair of mutual scheduler-RCU requirements came as a
This scheduler-RCU requirement came as a
<a href="https://lwn.net/Articles/453002/">complete surprise</a>.
<p>
@@ -2436,9 +2415,28 @@ when running context-switch-heavy workloads when built with
<tt>CONFIG_NO_HZ_FULL=y</tt>
<a href="http://www.rdrop.com/users/paulmck/scalability/paper/BareMetal.2015.01.15b.pdf">did come as a surprise [PDF]</a>.
RCU has made good progress towards meeting this requirement, even
for context-switch-have <tt>CONFIG_NO_HZ_FULL=y</tt> workloads,
for context-switch-heavy <tt>CONFIG_NO_HZ_FULL=y</tt> workloads,
but there is room for further improvement.
<p>
In the past, it was forbidden to disable interrupts across an
<tt>rcu_read_unlock()</tt> unless that interrupt-disabled region
of code also included the matching <tt>rcu_read_lock()</tt>.
Violating this restriction could result in deadlocks involving the
scheduler's runqueue and priority-inheritance spinlocks.
This restriction was lifted when interrupt-disabled calls to
<tt>rcu_read_unlock()</tt> started deferring the reporting of
the resulting RCU-preempt quiescent state until the end of that
interrupts-disabled region.
This deferred reporting means that the scheduler's runqueue and
priority-inheritance locks cannot be held while reporting an RCU-preempt
quiescent state, which lifts the earlier restriction, at least from
a deadlock perspective.
Unfortunately, real-time systems using RCU priority boosting may
need this restriction to remain in effect because deferred
quiescent-state reporting also defers deboosting, which in turn
degrades real-time latencies.
<h3><a name="Tracing and RCU">Tracing and RCU</a></h3>
<p>

2
Documentation/filesystems/Locking
View File

@@ -119,6 +119,7 @@ set: yes
--------------------------- super_operations ---------------------------
prototypes:
struct inode *(*alloc_inode)(struct super_block *sb);
void (*free_inode)(struct inode *);
void (*destroy_inode)(struct inode *);
void (*dirty_inode) (struct inode *, int flags);
int (*write_inode) (struct inode *, struct writeback_control *wbc);
@@ -140,6 +141,7 @@ locking rules:
All may block [not true, see below]
s_umount
alloc_inode:
free_inode: called from RCU callback
destroy_inode:
dirty_inode:
write_inode:

39
Documentation/filesystems/porting
View File

@@ -607,9 +607,38 @@ in your dentry operations instead.
to specify the fields and sync type requested by statx. Filesystems not
supporting any statx-specific features may ignore the new arguments.
--
[mandatory]
[recommended]
->lookup() instances doing an equivalent of
if (IS_ERR(inode))
return ERR_CAST(inode);
return d_splice_alias(inode, dentry);
don't need to bother with the check - d_splice_alias() will do the
right thing when given ERR_PTR(...) as inode. Moreover, passing NULL
inode to d_splice_alias() will also do the right thing (equivalent of
d_add(dentry, NULL); return NULL;), so that kind of special cases
also doesn't need a separate treatment.
--
[strongly recommended]
take the RCU-delayed parts of ->destroy_inode() into a new method -
->free_inode(). If ->destroy_inode() becomes empty - all the better,
just get rid of it. Synchronous work (e.g. the stuff that can't
be done from an RCU callback, or any WARN_ON() where we want the
stack trace) *might* be movable to ->evict_inode(); however,
that goes only for the things that are not needed to balance something
done by ->alloc_inode(). IOW, if it's cleaning up the stuff that
might have accumulated over the life of in-core inode, ->evict_inode()
might be a fit.
[should've been added in 2016] stale comment in finish_open()
nonwithstanding, failure exits in ->atomic_open() instances should
*NOT* fput() the file, no matter what. Everything is handled by the
caller.
Rules for inode destruction:
* if ->destroy_inode() is non-NULL, it gets called
* if ->free_inode() is non-NULL, it gets scheduled by call_rcu()
* combination of NULL ->destroy_inode and NULL ->free_inode is
treated as NULL/free_inode_nonrcu, to preserve the compatibility.
Note that the callback (be it via ->free_inode() or explicit call_rcu()
in ->destroy_inode()) is *NOT* ordered wrt superblock destruction;
as the matter of fact, the superblock and all associated structures
might be already gone. The filesystem driver is guaranteed to be still
there, but that's it. Freeing memory in the callback is fine; doing
more than that is possible, but requires a lot of care and is best
avoided.

1
Documentation/filesystems/proc.txt
View File

@@ -251,7 +251,6 @@ Table 1-2: Contents of the status files (as of 4.8)
VmExe size of text segment
VmLib size of shared library code
VmPTE size of page table entries
VmPMD size of second level page tables
VmSwap amount of swap used by anonymous private data
(shmem swap usage is not included)
HugetlbPages size of hugetlb memory portions

5
Documentation/media/kapi/rc-core.rst
View File

@@ -7,8 +7,3 @@ Remote Controller core
.. kernel-doc:: include/media/rc-core.h
.. kernel-doc:: include/media/rc-map.h
LIRC
~~~~
.. kernel-doc:: include/media/lirc_dev.h

31
Documentation/media/lirc.h.rst.exceptions
View File

@@ -28,6 +28,36 @@ ignore define LIRC_CAN_SEND_MASK
ignore define LIRC_CAN_REC_MASK
ignore define LIRC_CAN_SET_REC_DUTY_CYCLE
# Obsolete ioctls
ignore ioctl LIRC_GET_LENGTH
# rc protocols
ignore symbol RC_PROTO_UNKNOWN
ignore symbol RC_PROTO_OTHER
ignore symbol RC_PROTO_RC5
ignore symbol RC_PROTO_RC5X_20
ignore symbol RC_PROTO_RC5_SZ
ignore symbol RC_PROTO_JVC
ignore symbol RC_PROTO_SONY12
ignore symbol RC_PROTO_SONY15
ignore symbol RC_PROTO_SONY20
ignore symbol RC_PROTO_NEC
ignore symbol RC_PROTO_NECX
ignore symbol RC_PROTO_NEC32
ignore symbol RC_PROTO_SANYO
ignore symbol RC_PROTO_MCIR2_KBD
ignore symbol RC_PROTO_MCIR2_MSE
ignore symbol RC_PROTO_RC6_0
ignore symbol RC_PROTO_RC6_6A_20
ignore symbol RC_PROTO_RC6_6A_24
ignore symbol RC_PROTO_RC6_6A_32
ignore symbol RC_PROTO_RC6_MCE
ignore symbol RC_PROTO_SHARP
ignore symbol RC_PROTO_XMP
ignore symbol RC_PROTO_CEC
# Undocumented macros
ignore define PULSE_BIT
@@ -40,3 +70,4 @@ ignore define LIRC_VALUE_MASK
ignore define LIRC_MODE2_MASK
ignore define LIRC_MODE_RAW
ignore define LIRC_MODE_LIRCCODE

2
Documentation/media/uapi/rc/keytable.c.rst
View File

@@ -7,7 +7,7 @@ file: uapi/v4l/keytable.c
/* keytable.c - This program allows checking/replacing keys at IR
Copyright (C) 2006-2009 Mauro Carvalho Chehab <mchehab@infradead.org>
Copyright (C) 2006-2009 Mauro Carvalho Chehab <mchehab@kernel.org>
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by

76
Documentation/media/uapi/rc/lirc-dev-intro.rst
View File

@@ -6,19 +6,19 @@
Introduction
************
The LIRC device interface is a bi-directional interface for transporting
raw IR data between userspace and kernelspace. Fundamentally, it is just
a chardev (/dev/lircX, for X = 0, 1, 2, ...), with a number of standard
struct file_operations defined on it. With respect to transporting raw
IR data to and fro, the essential fops are read, write and ioctl.
LIRC stands for Linux Infrared Remote Control. The LIRC device interface is
a bi-directional interface for transporting raw IR and decoded scancodes
data between userspace and kernelspace. Fundamentally, it is just a chardev
(/dev/lircX, for X = 0, 1, 2, ...), with a number of standard struct
file_operations defined on it. With respect to transporting raw IR and
decoded scancodes to and fro, the essential fops are read, write and ioctl.
Example dmesg output upon a driver registering w/LIRC:
.. code-block:: none
$ dmesg |grep lirc_dev
lirc_dev: IR Remote Control driver registered, major 248
rc rc0: lirc_dev: driver ir-lirc-codec (mceusb) registered at minor = 0
rc rc0: lirc_dev: driver mceusb registered at minor = 0, raw IR receiver, raw IR transmitter
What you should see for a chardev:
@@ -36,6 +36,43 @@ LIRC modes
LIRC supports some modes of receiving and sending IR codes, as shown
on the following table.
.. _lirc-mode-scancode:
.. _lirc-scancode-flag-toggle:
.. _lirc-scancode-flag-repeat:
``LIRC_MODE_SCANCODE``
This mode is for both sending and receiving IR.
For transmitting (aka sending), create a ``struct lirc_scancode`` with
the desired scancode set in the ``scancode`` member, :c:type:`rc_proto`
set the IR protocol, and all other members set to 0. Write this struct to
the lirc device.
For receiving, you read ``struct lirc_scancode`` from the lirc device,
with ``scancode`` set to the received scancode and the IR protocol
:c:type:`rc_proto`. If the scancode maps to a valid key code, this is set
in the ``keycode`` field, else it is set to ``KEY_RESERVED``.
The ``flags`` can have ``LIRC_SCANCODE_FLAG_TOGGLE`` set if the toggle
bit is set in protocols that support it (e.g. rc-5 and rc-6), or
``LIRC_SCANCODE_FLAG_REPEAT`` for when a repeat is received for protocols
that support it (e.g. nec).
In the Sanyo and NEC protocol, if you hold a button on remote, rather than
repeating the entire scancode, the remote sends a shorter message with
no scancode, which just means button is held, a "repeat". When this is
received, the ``LIRC_SCANCODE_FLAG_REPEAT`` is set and the scancode and
keycode is repeated.
With nec, there is no way to distinguish "button hold" from "repeatedly
pressing the same button". The rc-5 and rc-6 protocols have a toggle bit.
When a button is released and pressed again, the toggle bit is inverted.
If the toggle bit is set, the ``LIRC_SCANCODE_FLAG_TOGGLE`` is set.
The ``timestamp`` field is filled with the time nanoseconds
(in ``CLOCK_MONOTONIC``) when the scancode was decoded.
.. _lirc-mode-mode2:
``LIRC_MODE_MODE2``
@@ -72,21 +109,6 @@ on the following table.
this packet will be sent, with the number of microseconds with
no IR.
.. _lirc-mode-lirccode:
``LIRC_MODE_LIRCCODE``
This mode can be used for IR receive and send.
The IR signal is decoded internally by the receiver, or encoded by the
transmitter. The LIRC interface represents the scancode as byte string,
which might not be a u32, it can be any length. The value is entirely
driver dependent. This mode is used by some older lirc drivers.
The length of each code depends on the driver, which can be retrieved
with :ref:`lirc_get_length`. This length is used both
for transmitting and receiving IR.
.. _lirc-mode-pulse:
``LIRC_MODE_PULSE``
@@ -99,3 +121,13 @@ on the following table.
of entries.
This mode is used only for IR send.
**************************
Remote Controller protocol
**************************
An enum :c:type:`rc_proto` in the :ref:`lirc_header` lists all the
supported IR protocols:
.. kernel-doc:: include/uapi/linux/lirc.h

2
Documentation/media/uapi/rc/lirc-func.rst
View File

@@ -17,8 +17,8 @@ LIRC Function Reference
lirc-get-rec-resolution
lirc-set-send-duty-cycle
lirc-get-timeout
lirc-get-rec-timeout
lirc-set-rec-timeout
lirc-get-length
lirc-set-rec-carrier
lirc-set-rec-carrier-range
lirc-set-send-carrier

25
Documentation/media/uapi/rc/lirc-get-features.rst
View File

@@ -55,15 +55,24 @@ LIRC features
``LIRC_CAN_REC_MODE2``
The driver is capable of receiving using
:ref:`LIRC_MODE_MODE2 <lirc-mode-MODE2>`.
This is raw IR driver for receiving. This means that
:ref:`LIRC_MODE_MODE2 <lirc-mode-MODE2>` is used. This also implies
that :ref:`LIRC_MODE_SCANCODE <lirc-mode-SCANCODE>` is also supported,
as long as the kernel is recent enough. Use the
:ref:`lirc_set_rec_mode` to switch modes.
.. _LIRC-CAN-REC-LIRCCODE:
``LIRC_CAN_REC_LIRCCODE``
The driver is capable of receiving using
:ref:`LIRC_MODE_LIRCCODE <lirc-mode-LIRCCODE>`.
Unused. Kept just to avoid breaking uAPI.
.. _LIRC-CAN-REC-SCANCODE:
``LIRC_CAN_REC_SCANCODE``
This is a scancode driver for receiving. This means that
:ref:`LIRC_MODE_SCANCODE <lirc-mode-SCANCODE>` is used.
.. _LIRC-CAN-SET-SEND-CARRIER:
@@ -157,7 +166,10 @@ LIRC features
``LIRC_CAN_SEND_PULSE``
The driver supports sending (also called as IR blasting or IR TX) using
:ref:`LIRC_MODE_PULSE <lirc-mode-pulse>`.
:ref:`LIRC_MODE_PULSE <lirc-mode-pulse>`. This implies that
:ref:`LIRC_MODE_SCANCODE <lirc-mode-SCANCODE>` is also supported for
transmit, as long as the kernel is recent enough. Use the
:ref:`lirc_set_send_mode` to switch modes.
.. _LIRC-CAN-SEND-MODE2:
@@ -170,8 +182,7 @@ LIRC features
``LIRC_CAN_SEND_LIRCCODE``
The driver supports sending (also called as IR blasting or IR TX) using
:ref:`LIRC_MODE_LIRCCODE <lirc-mode-LIRCCODE>`.
Unused. Kept just to avoid breaking uAPI.
Return Value

44
Documentation/media/uapi/rc/lirc-get-length.rst
View File

@@ -1,44 +0,0 @@
.. -*- coding: utf-8; mode: rst -*-
.. _lirc_get_length:
*********************
ioctl LIRC_GET_LENGTH
*********************
Name
====
LIRC_GET_LENGTH - Retrieves the code length in bits.
Synopsis
========
.. c:function:: int ioctl( int fd, LIRC_GET_LENGTH, __u32 *length )
:name: LIRC_GET_LENGTH
Arguments
=========
``fd``
File descriptor returned by open().
``length``
length, in bits
Description
===========
Retrieves the code length in bits (only for
:ref:`LIRC_MODE_LIRCCODE <lirc-mode-lirccode>`).
Reads on the device must be done in blocks matching the bit count.
The bit could should be rounded up so that it matches full bytes.
Return Value
============
On success 0 is returned, on error -1 and the ``errno`` variable is set
appropriately. The generic error codes are described at the
:ref:`Generic Error Codes <gen-errors>` chapter.

5
Documentation/media/uapi/rc/lirc-get-rec-mode.rst
View File

@@ -34,9 +34,8 @@ Description
===========
Get/set supported receive modes. Only :ref:`LIRC_MODE_MODE2 <lirc-mode-mode2>`
and :ref:`LIRC_MODE_LIRCCODE <lirc-mode-lirccode>` are supported for IR
receive. Use :ref:`lirc_get_features` to find out which modes the driver
supports.
and :ref:`LIRC_MODE_SCANCODE <lirc-mode-scancode>` are supported.
Use :ref:`lirc_get_features` to find out which modes the driver supports.
Return Value
============

2
Documentation/media/uapi/rc/lirc-get-send-mode.rst
View File

@@ -37,7 +37,7 @@ Description
Get/set current transmit mode.
Only :ref:`LIRC_MODE_PULSE <lirc-mode-pulse>` and
:ref:`LIRC_MODE_LIRCCODE <lirc-mode-lirccode>` is supported by for IR send,
:ref:`LIRC_MODE_SCANCODE <lirc-mode-scancode>` are supported by for IR send,
depending on the driver. Use :ref:`lirc_get_features` to find out which
modes the driver supports.

15
Documentation/media/uapi/rc/lirc-read.rst
View File

@@ -45,13 +45,20 @@ descriptor ``fd`` into the buffer starting at ``buf``. If ``count`` is zero,
is greater than ``SSIZE_MAX``, the result is unspecified.
The exact format of the data depends on what :ref:`lirc_modes` a driver
uses. Use :ref:`lirc_get_features` to get the supported mode.
uses. Use :ref:`lirc_get_features` to get the supported mode, and use
:ref:`lirc_set_rec_mode` set the current active mode.
The generally preferred mode for receive is
:ref:`LIRC_MODE_MODE2 <lirc-mode-mode2>`,
in which packets containing an int value describing an IR signal are
The mode :ref:`LIRC_MODE_MODE2 <lirc-mode-mode2>` is for raw IR,
in which packets containing an unsigned int value describing an IR signal are
read from the chardev.
Alternatively, :ref:`LIRC_MODE_SCANCODE <lirc-mode-scancode>` can be available,
in this mode scancodes which are either decoded by software decoders, or
by hardware decoders. The :c:type:`rc_proto` member is set to the
protocol used for transmission, and ``scancode`` to the decoded scancode,
and the ``keycode`` set to the keycode or ``KEY_RESERVED``.
Return Value
============

14
Documentation/media/uapi/rc/lirc-set-rec-timeout.rst
View File

@@ -1,19 +1,23 @@
.. -*- coding: utf-8; mode: rst -*-
.. _lirc_set_rec_timeout:
.. _lirc_get_rec_timeout:
**************************
ioctl LIRC_SET_REC_TIMEOUT
**************************
***************************************************
ioctl LIRC_GET_REC_TIMEOUT and LIRC_SET_REC_TIMEOUT
***************************************************
Name
====
LIRC_SET_REC_TIMEOUT - sets the integer value for IR inactivity timeout.
LIRC_GET_REC_TIMEOUT/LIRC_SET_REC_TIMEOUT - Get/set the integer value for IR inactivity timeout.
Synopsis
========
.. c:function:: int ioctl( int fd, LIRC_GET_REC_TIMEOUT, __u32 *timeout )
:name: LIRC_GET_REC_TIMEOUT
.. c:function:: int ioctl( int fd, LIRC_SET_REC_TIMEOUT, __u32 *timeout )
:name: LIRC_SET_REC_TIMEOUT
@@ -30,7 +34,7 @@ Arguments
Description
===========
Sets the integer value for IR inactivity timeout.
Get and set the integer value for IR inactivity timeout.
If supported by the hardware, setting it to 0 disables all hardware timeouts
and data should be reported as soon as possible. If the exact value

19
Documentation/media/uapi/rc/lirc-write.rst
View File

@@ -42,21 +42,32 @@ Description
referenced by the file descriptor ``fd`` from the buffer starting at
``buf``.
The exact format of the data depends on what mode a driver uses, use
:ref:`lirc_get_features` to get the supported mode.
The exact format of the data depends on what mode a driver is in, use
:ref:`lirc_get_features` to get the supported modes and use
:ref:`lirc_set_send_mode` set the mode.
When in :ref:`LIRC_MODE_PULSE <lirc-mode-PULSE>` mode, the data written to
the chardev is a pulse/space sequence of integer values. Pulses and spaces
are only marked implicitly by their position. The data must start and end
with a pulse, therefore, the data must always include an uneven number of
samples. The write function must block until the data has been transmitted
samples. The write function blocks until the data has been transmitted
by the hardware. If more data is provided than the hardware can send, the
driver returns ``EINVAL``.
When in :ref:`LIRC_MODE_SCANCODE <lirc-mode-scancode>` mode, one
``struct lirc_scancode`` must be written to the chardev at a time, else
``EINVAL`` is returned. Set the desired scancode in the ``scancode`` member,
and the protocol in the :c:type:`rc_proto`: member. All other members must be
set to 0, else ``EINVAL`` is returned. If there is no protocol encoder
for the protocol or the scancode is not valid for the specified protocol,
``EINVAL`` is returned. The write function blocks until the scancode
is transmitted by the hardware.
Return Value
============
On success, the number of bytes read is returned. It is not an error if
On success, the number of bytes written is returned. It is not an error if
this number is smaller than the number of bytes requested, or the amount
of data required for one frame. On error, -1 is returned, and the ``errno``
variable is set appropriately. The generic error codes are described at the

312
Documentation/networking/af_xdp.rst Normal file
View File

@@ -0,0 +1,312 @@
.. SPDX-License-Identifier: GPL-2.0
======
AF_XDP
======
Overview
========
AF_XDP is an address family that is optimized for high performance
packet processing.
This document assumes that the reader is familiar with BPF and XDP. If
not, the Cilium project has an excellent reference guide at
http://cilium.readthedocs.io/en/latest/bpf/.
Using the XDP_REDIRECT action from an XDP program, the program can
redirect ingress frames to other XDP enabled netdevs, using the
bpf_redirect_map() function. AF_XDP sockets enable the possibility for
XDP programs to redirect frames to a memory buffer in a user-space
application.
An AF_XDP socket (XSK) is created with the normal socket()
syscall. Associated with each XSK are two rings: the RX ring and the
TX ring. A socket can receive packets on the RX ring and it can send
packets on the TX ring. These rings are registered and sized with the
setsockopts XDP_RX_RING and XDP_TX_RING, respectively. It is mandatory
to have at least one of these rings for each socket. An RX or TX
descriptor ring points to a data buffer in a memory area called a
UMEM. RX and TX can share the same UMEM so that a packet does not have
to be copied between RX and TX. Moreover, if a packet needs to be kept
for a while due to a possible retransmit, the descriptor that points
to that packet can be changed to point to another and reused right
away. This again avoids copying data.
The UMEM consists of a number of equally sized chunks. A descriptor in
one of the rings references a frame by referencing its addr. The addr
is simply an offset within the entire UMEM region. The user space
allocates memory for this UMEM using whatever means it feels is most
appropriate (malloc, mmap, huge pages, etc). This memory area is then
registered with the kernel using the new setsockopt XDP_UMEM_REG. The
UMEM also has two rings: the FILL ring and the COMPLETION ring. The
fill ring is used by the application to send down addr for the kernel
to fill in with RX packet data. References to these frames will then
appear in the RX ring once each packet has been received. The
completion ring, on the other hand, contains frame addr that the
kernel has transmitted completely and can now be used again by user
space, for either TX or RX. Thus, the frame addrs appearing in the
completion ring are addrs that were previously transmitted using the
TX ring. In summary, the RX and FILL rings are used for the RX path
and the TX and COMPLETION rings are used for the TX path.
The socket is then finally bound with a bind() call to a device and a
specific queue id on that device, and it is not until bind is
completed that traffic starts to flow.
The UMEM can be shared between processes, if desired. If a process
wants to do this, it simply skips the registration of the UMEM and its
corresponding two rings, sets the XDP_SHARED_UMEM flag in the bind
call and submits the XSK of the process it would like to share UMEM
with as well as its own newly created XSK socket. The new process will
then receive frame addr references in its own RX ring that point to
this shared UMEM. Note that since the ring structures are
single-consumer / single-producer (for performance reasons), the new
process has to create its own socket with associated RX and TX rings,
since it cannot share this with the other process. This is also the
reason that there is only one set of FILL and COMPLETION rings per
UMEM. It is the responsibility of a single process to handle the UMEM.
How is then packets distributed from an XDP program to the XSKs? There
is a BPF map called XSKMAP (or BPF_MAP_TYPE_XSKMAP in full). The
user-space application can place an XSK at an arbitrary place in this
map. The XDP program can then redirect a packet to a specific index in
this map and at this point XDP validates that the XSK in that map was
indeed bound to that device and ring number. If not, the packet is
dropped. If the map is empty at that index, the packet is also
dropped. This also means that it is currently mandatory to have an XDP
program loaded (and one XSK in the XSKMAP) to be able to get any
traffic to user space through the XSK.
AF_XDP can operate in two different modes: XDP_SKB and XDP_DRV. If the
driver does not have support for XDP, or XDP_SKB is explicitly chosen
when loading the XDP program, XDP_SKB mode is employed that uses SKBs
together with the generic XDP support and copies out the data to user
space. A fallback mode that works for any network device. On the other
hand, if the driver has support for XDP, it will be used by the AF_XDP
code to provide better performance, but there is still a copy of the
data into user space.
Concepts
========
In order to use an AF_XDP socket, a number of associated objects need
to be setup.
Jonathan Corbet has also written an excellent article on LWN,
"Accelerating networking with AF_XDP". It can be found at
https://lwn.net/Articles/750845/.
UMEM
----
UMEM is a region of virtual contiguous memory, divided into
equal-sized frames. An UMEM is associated to a netdev and a specific
queue id of that netdev. It is created and configured (chunk size,
headroom, start address and size) by using the XDP_UMEM_REG setsockopt
system call. A UMEM is bound to a netdev and queue id, via the bind()
system call.
An AF_XDP is socket linked to a single UMEM, but one UMEM can have
multiple AF_XDP sockets. To share an UMEM created via one socket A,
the next socket B can do this by setting the XDP_SHARED_UMEM flag in
struct sockaddr_xdp member sxdp_flags, and passing the file descriptor
of A to struct sockaddr_xdp member sxdp_shared_umem_fd.
The UMEM has two single-producer/single-consumer rings, that are used
to transfer ownership of UMEM frames between the kernel and the
user-space application.
Rings
-----
There are a four different kind of rings: Fill, Completion, RX and
TX. All rings are single-producer/single-consumer, so the user-space
application need explicit synchronization of multiple
processes/threads are reading/writing to them.
The UMEM uses two rings: Fill and Completion. Each socket associated
with the UMEM must have an RX queue, TX queue or both. Say, that there
is a setup with four sockets (all doing TX and RX). Then there will be
one Fill ring, one Completion ring, four TX rings and four RX rings.
The rings are head(producer)/tail(consumer) based rings. A producer
writes the data ring at the index pointed out by struct xdp_ring
producer member, and increasing the producer index. A consumer reads
the data ring at the index pointed out by struct xdp_ring consumer
member, and increasing the consumer index.
The rings are configured and created via the _RING setsockopt system
calls and mmapped to user-space using the appropriate offset to mmap()
(XDP_PGOFF_RX_RING, XDP_PGOFF_TX_RING, XDP_UMEM_PGOFF_FILL_RING and
XDP_UMEM_PGOFF_COMPLETION_RING).
The size of the rings need to be of size power of two.
UMEM Fill Ring
~~~~~~~~~~~~~~
The Fill ring is used to transfer ownership of UMEM frames from
user-space to kernel-space. The UMEM addrs are passed in the ring. As
an example, if the UMEM is 64k and each chunk is 4k, then the UMEM has
16 chunks and can pass addrs between 0 and 64k.
Frames passed to the kernel are used for the ingress path (RX rings).
The user application produces UMEM addrs to this ring. Note that the
kernel will mask the incoming addr. E.g. for a chunk size of 2k, the
log2(2048) LSB of the addr will be masked off, meaning that 2048, 2050
and 3000 refers to the same chunk.
UMEM Completetion Ring
~~~~~~~~~~~~~~~~~~~~~~
The Completion Ring is used transfer ownership of UMEM frames from
kernel-space to user-space. Just like the Fill ring, UMEM indicies are
used.
Frames passed from the kernel to user-space are frames that has been
sent (TX ring) and can be used by user-space again.
The user application consumes UMEM addrs from this ring.
RX Ring
~~~~~~~
The RX ring is the receiving side of a socket. Each entry in the ring
is a struct xdp_desc descriptor. The descriptor contains UMEM offset
(addr) and the length of the data (len).
If no frames have been passed to kernel via the Fill ring, no
descriptors will (or can) appear on the RX ring.
The user application consumes struct xdp_desc descriptors from this
ring.
TX Ring
~~~~~~~
The TX ring is used to send frames. The struct xdp_desc descriptor is
filled (index, length and offset) and passed into the ring.
To start the transfer a sendmsg() system call is required. This might
be relaxed in the future.
The user application produces struct xdp_desc descriptors to this
ring.
XSKMAP / BPF_MAP_TYPE_XSKMAP
----------------------------
On XDP side there is a BPF map type BPF_MAP_TYPE_XSKMAP (XSKMAP) that
is used in conjunction with bpf_redirect_map() to pass the ingress
frame to a socket.
The user application inserts the socket into the map, via the bpf()
system call.
Note that if an XDP program tries to redirect to a socket that does
not match the queue configuration and netdev, the frame will be
dropped. E.g. an AF_XDP socket is bound to netdev eth0 and
queue 17. Only the XDP program executing for eth0 and queue 17 will
successfully pass data to the socket. Please refer to the sample
application (samples/bpf/) in for an example.
Usage
=====
In order to use AF_XDP sockets there are two parts needed. The
user-space application and the XDP program. For a complete setup and
usage example, please refer to the sample application. The user-space
side is xdpsock_user.c and the XDP side xdpsock_kern.c.
Naive ring dequeue and enqueue could look like this::
// struct xdp_rxtx_ring {
// __u32 *producer;
// __u32 *consumer;
// struct xdp_desc *desc;
// };
// struct xdp_umem_ring {
// __u32 *producer;
// __u32 *consumer;
// __u64 *desc;
// };
// typedef struct xdp_rxtx_ring RING;
// typedef struct xdp_umem_ring RING;
// typedef struct xdp_desc RING_TYPE;
// typedef __u64 RING_TYPE;
int dequeue_one(RING *ring, RING_TYPE *item)
{
__u32 entries = *ring->producer - *ring->consumer;
if (entries == 0)
return -1;
// read-barrier!
*item = ring->desc[*ring->consumer & (RING_SIZE - 1)];
(*ring->consumer)++;
return 0;
}
int enqueue_one(RING *ring, const RING_TYPE *item)
{
u32 free_entries = RING_SIZE - (*ring->producer - *ring->consumer);
if (free_entries == 0)
return -1;
ring->desc[*ring->producer & (RING_SIZE - 1)] = *item;
// write-barrier!
(*ring->producer)++;
return 0;
}
For a more optimized version, please refer to the sample application.
Sample application
==================
There is a xdpsock benchmarking/test application included that
demonstrates how to use AF_XDP sockets with both private and shared
UMEMs. Say that you would like your UDP traffic from port 4242 to end
up in queue 16, that we will enable AF_XDP on. Here, we use ethtool
for this::
ethtool -N p3p2 rx-flow-hash udp4 fn
ethtool -N p3p2 flow-type udp4 src-port 4242 dst-port 4242 \
action 16
Running the rxdrop benchmark in XDP_DRV mode can then be done
using::
samples/bpf/xdpsock -i p3p2 -q 16 -r -N
For XDP_SKB mode, use the switch "-S" instead of "-N" and all options
can be displayed with "-h", as usual.
Credits
=======
- Björn Töpel (AF_XDP core)
- Magnus Karlsson (AF_XDP core)
- Alexander Duyck
- Alexei Starovoitov
- Daniel Borkmann
- Jesper Dangaard Brouer
- John Fastabend
- Jonathan Corbet (LWN coverage)
- Michael S. Tsirkin
- Qi Z Zhang
- Willem de Bruijn

15
Documentation/networking/filter.txt
View File

@@ -857,7 +857,7 @@ Three LSB bits store instruction class which is one of:
BPF_STX 0x03 BPF_STX 0x03
BPF_ALU 0x04 BPF_ALU 0x04
BPF_JMP 0x05 BPF_JMP 0x05
BPF_RET 0x06 [ class 6 unused, for future if needed ]
BPF_RET 0x06 BPF_JMP32 0x06
BPF_MISC 0x07 BPF_ALU64 0x07
When BPF_CLASS(code) == BPF_ALU or BPF_JMP, 4th bit encodes source operand ...
@@ -894,9 +894,9 @@ If BPF_CLASS(code) == BPF_ALU or BPF_ALU64 [ in eBPF ], BPF_OP(code) is one of:
BPF_ARSH 0xc0 /* eBPF only: sign extending shift right */
BPF_END 0xd0 /* eBPF only: endianness conversion */
If BPF_CLASS(code) == BPF_JMP, BPF_OP(code) is one of:
If BPF_CLASS(code) == BPF_JMP or BPF_JMP32 [ in eBPF ], BPF_OP(code) is one of:
BPF_JA 0x00
BPF_JA 0x00 /* BPF_JMP only */
BPF_JEQ 0x10
BPF_JGT 0x20
BPF_JGE 0x30
@@ -904,8 +904,8 @@ If BPF_CLASS(code) == BPF_JMP, BPF_OP(code) is one of:
BPF_JNE 0x50 /* eBPF only: jump != */
BPF_JSGT 0x60 /* eBPF only: signed '>' */
BPF_JSGE 0x70 /* eBPF only: signed '>=' */
BPF_CALL 0x80 /* eBPF only: function call */
BPF_EXIT 0x90 /* eBPF only: function return */
BPF_CALL 0x80 /* eBPF BPF_JMP only: function call */
BPF_EXIT 0x90 /* eBPF BPF_JMP only: function return */
BPF_JLT 0xa0 /* eBPF only: unsigned '<' */
BPF_JLE 0xb0 /* eBPF only: unsigned '<=' */
BPF_JSLT 0xc0 /* eBPF only: signed '<' */
@@ -928,8 +928,9 @@ Classic BPF wastes the whole BPF_RET class to represent a single 'ret'
operation. Classic BPF_RET | BPF_K means copy imm32 into return register
and perform function exit. eBPF is modeled to match CPU, so BPF_JMP | BPF_EXIT
in eBPF means function exit only. The eBPF program needs to store return
value into register R0 before doing a BPF_EXIT. Class 6 in eBPF is currently
unused and reserved for future use.
value into register R0 before doing a BPF_EXIT. Class 6 in eBPF is used as
BPF_JMP32 to mean exactly the same operations as BPF_JMP, but with 32-bit wide
operands for the comparisons instead.
For load and store instructions the 8-bit 'code' field is divided as:

1
Documentation/networking/index.rst
View File

@@ -6,6 +6,7 @@ Contents:
.. toctree::
:maxdepth: 2
af_xdp
batman-adv
kapi
z8530book

32
Documentation/networking/ip-sysctl.txt
View File

@@ -612,6 +612,7 @@ tcp_fastopen_blackhole_timeout_sec - INTEGER
This time period will grow exponentially when more blackhole issues
get detected right after Fastopen is re-enabled and will reset to
initial value when the blackhole issue goes away.
0 to disable the blackhole detection.
By default, it is set to 1hr.
tcp_fwmark_accept - BOOLEAN
@@ -1389,6 +1390,13 @@ flowlabel_reflect - BOOLEAN
FALSE: disabled
Default: FALSE
fib_multipath_hash_policy - INTEGER
Controls which hash policy to use for multipath routes.
Default: 0 (Layer 3)
Possible values:
0 - Layer 3 (source and destination addresses plus flow label)
1 - Layer 4 (standard 5-tuple)
anycast_src_echo_reply - BOOLEAN
Controls the use of anycast addresses as source addresses for ICMPv6
echo reply
@@ -1412,6 +1420,30 @@ mld_qrv - INTEGER
Default: 2 (as specified by RFC3810 9.1)
Minimum: 1 (as specified by RFC6636 4.5)
max_dst_opts_cnt - INTEGER
Maximum number of non-padding TLVs allowed in a Destination
options extension header. If this value is less than zero
then unknown options are disallowed and the number of known
TLVs allowed is the absolute value of this number.
Default: 8
max_hbh_opts_cnt - INTEGER
Maximum number of non-padding TLVs allowed in a Hop-by-Hop
options extension header. If this value is less than zero
then unknown options are disallowed and the number of known
TLVs allowed is the absolute value of this number.
Default: 8
max dst_opts_len - INTEGER
Maximum length allowed for a Destination options extension
header.
Default: INT_MAX (unlimited)
max hbh_opts_len - INTEGER
Maximum length allowed for a Hop-by-Hop options extension
header.
Default: INT_MAX (unlimited)
IPv6 Fragmentation:
ip6frag_high_thresh - INTEGER

34
Documentation/printk-formats.txt
View File

@@ -61,41 +61,31 @@ Symbols/Function Pointers
::
%pS versatile_init+0x0/0x110
%ps versatile_init
%pF versatile_init+0x0/0x110
%pf versatile_init
%pS versatile_init+0x0/0x110
%pSR versatile_init+0x9/0x110
(with __builtin_extract_return_addr() translation)
%ps versatile_init
%pB prev_fn_of_versatile_init+0x88/0x88
The ``F`` and ``f`` specifiers are for printing function pointers,
for example, f->func, &gettimeofday. They have the same result as
``S`` and ``s`` specifiers. But they do an extra conversion on
ia64, ppc64 and parisc64 architectures where the function pointers
are actually function descriptors.
The ``S`` and ``s`` specifiers are used for printing a pointer in symbolic
format. They result in the symbol name with (``S``) or without (``s``)
offsets. If KALLSYMS are disabled then the symbol address is printed instead.
The ``S`` and ``s`` specifiers can be used for printing symbols
from direct addresses, for example, __builtin_return_address(0),
(void *)regs->ip. They result in the symbol name with (``S``) or
without (``s``) offsets. If KALLSYMS are disabled then the symbol
address is printed instead.
Note, that the ``F`` and ``f`` specifiers are identical to ``S`` (``s``)
and thus deprecated. We have ``F`` and ``f`` because on ia64, ppc64 and
parisc64 function pointers are indirect and, in fact, are function
descriptors, which require additional dereferencing before we can lookup
the symbol. As of now, ``S`` and ``s`` perform dereferencing on those
platforms (when needed), so ``F`` and ``f`` exist for compatibility
reasons only.
The ``B`` specifier results in the symbol name with offsets and should be
used when printing stack backtraces. The specifier takes into
consideration the effect of compiler optimisations which may occur
when tail-call``s are used and marked with the noreturn GCC attribute.
Examples::
printk("Going to call: %pF\n", gettimeofday);
printk("Going to call: %pF\n", p->func);
printk("%s: called from %pS\n", __func__, (void *)_RET_IP_);
printk("%s: called from %pS\n", __func__,
(void *)__builtin_return_address(0));
printk("Faulted at %pS\n", (void *)regs->ip);
printk(" %s%pB\n", (reliable ? "" : "? "), (void *)*stack);
Kernel Pointers
===============

2
Documentation/sysctl/vm.txt
View File

@@ -641,7 +641,7 @@ oom_dump_tasks
Enables a system-wide task dump (excluding kernel threads) to be produced
when the kernel performs an OOM-killing and includes such information as
pid, uid, tgid, vm size, rss, nr_ptes, nr_pmds, swapents, oom_score_adj
pid, uid, tgid, vm size, rss, pgtables_bytes, swapents, oom_score_adj
score, and name. This is helpful to determine why the OOM killer was
invoked, to identify the rogue task that caused it, and to determine why
the OOM killer chose the task it did to kill.

32
MAINTAINERS
View File

@@ -2498,7 +2498,6 @@ F: Documentation/devicetree/bindings/sound/axentia,*
F: sound/soc/atmel/tse850-pcm5142.c
AZ6007 DVB DRIVER
M: Mauro Carvalho Chehab <mchehab@s-opensource.com>
M: Mauro Carvalho Chehab <mchehab@kernel.org>
L: linux-media@vger.kernel.org
W: https://linuxtv.org
@@ -3040,7 +3039,6 @@ F: include/linux/btrfs*
F: include/uapi/linux/btrfs*
BTTV VIDEO4LINUX DRIVER
M: Mauro Carvalho Chehab <mchehab@s-opensource.com>
M: Mauro Carvalho Chehab <mchehab@kernel.org>
L: linux-media@vger.kernel.org
W: https://linuxtv.org
@@ -3763,7 +3761,6 @@ S: Maintained
F: drivers/media/dvb-frontends/cx24120*
CX88 VIDEO4LINUX DRIVER
M: Mauro Carvalho Chehab <mchehab@s-opensource.com>
M: Mauro Carvalho Chehab <mchehab@kernel.org>
L: linux-media@vger.kernel.org
W: https://linuxtv.org
@@ -4910,7 +4907,6 @@ F: drivers/edac/thunderx_edac*
EDAC-CORE
M: Borislav Petkov <bp@alien8.de>
M: Mauro Carvalho Chehab <mchehab@s-opensource.com>
M: Mauro Carvalho Chehab <mchehab@kernel.org>
L: linux-edac@vger.kernel.org
T: git git://git.kernel.org/pub/scm/linux/kernel/git/bp/bp.git for-next
@@ -4939,7 +4935,6 @@ S: Maintained
F: drivers/edac/fsl_ddr_edac.*
EDAC-GHES
M: Mauro Carvalho Chehab <mchehab@s-opensource.com>
M: Mauro Carvalho Chehab <mchehab@kernel.org>
L: linux-edac@vger.kernel.org
S: Maintained
@@ -4956,21 +4951,18 @@ S: Maintained
F: drivers/edac/i5000_edac.c
EDAC-I5400
M: Mauro Carvalho Chehab <mchehab@s-opensource.com>
M: Mauro Carvalho Chehab <mchehab@kernel.org>
L: linux-edac@vger.kernel.org
S: Maintained
F: drivers/edac/i5400_edac.c
EDAC-I7300
M: Mauro Carvalho Chehab <mchehab@s-opensource.com>
M: Mauro Carvalho Chehab <mchehab@kernel.org>
L: linux-edac@vger.kernel.org
S: Maintained
F: drivers/edac/i7300_edac.c
EDAC-I7CORE
M: Mauro Carvalho Chehab <mchehab@s-opensource.com>
M: Mauro Carvalho Chehab <mchehab@kernel.org>
L: linux-edac@vger.kernel.org
S: Maintained
@@ -5020,7 +5012,6 @@ S: Maintained
F: drivers/edac/r82600_edac.c
EDAC-SBRIDGE
M: Mauro Carvalho Chehab <mchehab@s-opensource.com>
M: Mauro Carvalho Chehab <mchehab@kernel.org>
L: linux-edac@vger.kernel.org
S: Maintained
@@ -5073,7 +5064,6 @@ S: Maintained
F: drivers/net/ethernet/ibm/ehea/
EM28XX VIDEO4LINUX DRIVER
M: Mauro Carvalho Chehab <mchehab@s-opensource.com>
M: Mauro Carvalho Chehab <mchehab@kernel.org>
L: linux-media@vger.kernel.org
W: https://linuxtv.org
@@ -6758,6 +6748,13 @@ M: James Hogan <jhogan@kernel.org>
S: Maintained
F: drivers/media/rc/img-ir/
IMON SOUNDGRAPH USB IR RECEIVER
M: Sean Young <sean@mess.org>
L: linux-media@vger.kernel.org
S: Maintained
F: drivers/media/rc/imon_raw.c
F: drivers/media/rc/imon.c
IMS TWINTURBO FRAMEBUFFER DRIVER
L: linux-fbdev@vger.kernel.org
S: Orphan
@@ -8594,7 +8591,6 @@ S: Maintained
F: drivers/media/dvb-frontends/stv6111*
MEDIA INPUT INFRASTRUCTURE (V4L/DVB)
M: Mauro Carvalho Chehab <mchehab@s-opensource.com>
M: Mauro Carvalho Chehab <mchehab@kernel.org>
P: LinuxTV.org Project
L: linux-media@vger.kernel.org
@@ -11766,7 +11762,6 @@ S: Odd Fixes
F: drivers/media/i2c/saa6588*
SAA7134 VIDEO4LINUX DRIVER
M: Mauro Carvalho Chehab <mchehab@s-opensource.com>
M: Mauro Carvalho Chehab <mchehab@kernel.org>
L: linux-media@vger.kernel.org
W: https://linuxtv.org
@@ -12254,7 +12249,6 @@ S: Maintained
F: drivers/media/radio/si4713/radio-usb-si4713.c
SIANO DVB DRIVER
M: Mauro Carvalho Chehab <mchehab@s-opensource.com>
M: Mauro Carvalho Chehab <mchehab@kernel.org>
L: linux-media@vger.kernel.org
W: https://linuxtv.org
@@ -13126,7 +13120,6 @@ S: Maintained
F: drivers/media/i2c/tda9840*
TEA5761 TUNER DRIVER
M: Mauro Carvalho Chehab <mchehab@s-opensource.com>
M: Mauro Carvalho Chehab <mchehab@kernel.org>
L: linux-media@vger.kernel.org
W: https://linuxtv.org
@@ -13135,7 +13128,6 @@ S: Odd fixes
F: drivers/media/tuners/tea5761.*
TEA5767 TUNER DRIVER
M: Mauro Carvalho Chehab <mchehab@s-opensource.com>
M: Mauro Carvalho Chehab <mchehab@kernel.org>
L: linux-media@vger.kernel.org
W: https://linuxtv.org
@@ -13542,7 +13534,6 @@ F: Documentation/networking/tlan.txt
F: drivers/net/ethernet/ti/tlan.*
TM6000 VIDEO4LINUX DRIVER
M: Mauro Carvalho Chehab <mchehab@s-opensource.com>
M: Mauro Carvalho Chehab <mchehab@kernel.org>
L: linux-media@vger.kernel.org
W: https://linuxtv.org
@@ -14708,7 +14699,6 @@ S: Maintained
F: arch/x86/entry/vdso/
XC2028/3028 TUNER DRIVER
M: Mauro Carvalho Chehab <mchehab@s-opensource.com>
M: Mauro Carvalho Chehab <mchehab@kernel.org>
L: linux-media@vger.kernel.org
W: https://linuxtv.org
@@ -14716,6 +14706,14 @@ T: git git://linuxtv.org/media_tree.git
S: Maintained
F: drivers/media/tuners/tuner-xc2028.*
XDP SOCKETS (AF_XDP)
M: Björn Töpel <bjorn.topel@intel.com>
M: Magnus Karlsson <magnus.karlsson@intel.com>
L: netdev@vger.kernel.org
S: Maintained
F: kernel/bpf/xskmap.c
F: net/xdp/
XEN BLOCK SUBSYSTEM
M: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
M: Roger Pau Monné <roger.pau@citrix.com>

3
Makefile
View File

@@ -408,6 +408,7 @@ OBJCOPY = $(CROSS_COMPILE)objcopy
OBJDUMP = $(CROSS_COMPILE)objdump
READELF = $(CROSS_COMPILE)readelf
endif
PAHOLE = pahole
AWK = awk
GENKSYMS = scripts/genksyms/genksyms
INSTALLKERNEL := installkernel
@@ -472,7 +473,7 @@ GCC_PLUGINS_CFLAGS :=
CLANG_FLAGS :=
export ARCH SRCARCH CONFIG_SHELL HOSTCC HOSTCFLAGS CROSS_COMPILE LD CC
export CPP AR NM STRIP OBJCOPY OBJDUMP READELF HOSTLDFLAGS HOST_LOADLIBES
export CPP AR NM STRIP OBJCOPY OBJDUMP PAHOLE READELF HOSTLDFLAGS HOST_LOADLIBES
export MAKE AWK GENKSYMS INSTALLKERNEL PERL PYTHON UTS_MACHINE
export HOSTCXX HOSTCXXFLAGS LDFLAGS_MODULE CHECK CHECKFLAGS

7
arch/Kconfig
View File

@@ -199,6 +199,9 @@ config HAVE_OPTPROBES
config HAVE_KPROBES_ON_FTRACE
bool
config HAVE_FUNCTION_ERROR_INJECTION
bool
config HAVE_NMI
bool
@@ -240,6 +243,10 @@ config ARCH_HAS_SET_MEMORY
config ARCH_HAS_CPU_FINALIZE_INIT
bool
# Select if arch has all set_direct_map_invalid/default() functions
config ARCH_HAS_SET_DIRECT_MAP
bool
# Select if arch init_task initializer is different to init/init_task.c
config ARCH_INIT_TASK
bool

4
arch/alpha/include/asm/atomic.h
View File

@@ -193,7 +193,7 @@ ATOMIC_OPS(xor, xor)
#define atomic_xchg(v, new) (xchg(&((v)->counter), new))
/**
* __atomic_add_unless - add unless the number is a given value
* atomic_fetch_add_unless - add unless the number is a given value
* @v: pointer of type atomic_t
* @a: the amount to add to v...
* @u: ...unless v is equal to u.
@@ -201,7 +201,7 @@ ATOMIC_OPS(xor, xor)
* Atomically adds @a to @v, so long as it was not @u.
* Returns the old value of @v.
*/
static __inline__ int __atomic_add_unless(atomic_t *v, int a, int u)
static __inline__ int atomic_fetch_add_unless(atomic_t *v, int a, int u)
{
int c, new, old;
smp_mb();

4
arch/arc/include/asm/atomic.h
View File

@@ -309,7 +309,7 @@ ATOMIC_OPS(xor, ^=, CTOP_INST_AXOR_DI_R2_R2_R3)
#undef ATOMIC_OP
/**
* __atomic_add_unless - add unless the number is a given value
* atomic_fetch_add_unless - add unless the number is a given value
* @v: pointer of type atomic_t
* @a: the amount to add to v...
* @u: ...unless v is equal to u.
@@ -317,7 +317,7 @@ ATOMIC_OPS(xor, ^=, CTOP_INST_AXOR_DI_R2_R2_R3)
* Atomically adds @a to @v, so long as it was not @u.
* Returns the old value of @v
*/
#define __atomic_add_unless(v, a, u) \
#define atomic_fetch_add_unless(v, a, u) \
({ \
int c, old; \
\

4
arch/arm/include/asm/atomic.h
View File

@@ -130,7 +130,7 @@ static inline int atomic_cmpxchg_relaxed(atomic_t *ptr, int old, int new)
}
#define atomic_cmpxchg_relaxed atomic_cmpxchg_relaxed
static inline int __atomic_add_unless(atomic_t *v, int a, int u)
static inline int atomic_fetch_add_unless(atomic_t *v, int a, int u)
{
int oldval, newval;
unsigned long tmp;
@@ -215,7 +215,7 @@ static inline int atomic_cmpxchg(atomic_t *v, int old, int new)
return ret;
}
static inline int __atomic_add_unless(atomic_t *v, int a, int u)
static inline int atomic_fetch_add_unless(atomic_t *v, int a, int u)
{
int c, old;

1
arch/arm/mach-omap2/Makefile
View File

@@ -78,7 +78,6 @@ endif
omap-4-5-pm-common = omap-mpuss-lowpower.o
obj-$(CONFIG_ARCH_OMAP4) += $(omap-4-5-pm-common)
obj-$(CONFIG_SOC_OMAP5) += $(omap-4-5-pm-common)
obj-$(CONFIG_OMAP_PM_NOOP) += omap-pm-noop.o
ifeq ($(CONFIG_PM),y)
obj-$(CONFIG_ARCH_OMAP2) += pm24xx.o

7
arch/arm/mach-omap2/display.c
View File

@@ -32,7 +32,6 @@
#include <linux/platform_data/omapdss.h>
#include "omap_hwmod.h"
#include "omap_device.h"
#include "omap-pm.h"
#include "common.h"
#include "soc.h"
@@ -131,11 +130,6 @@ static void omap_dsi_disable_pads(int dsi_id, unsigned lane_mask)
omap4_dsi_mux_pads(dsi_id, 0);
}
static int omap_dss_set_min_bus_tput(struct device *dev, unsigned long tput)
{
return omap_pm_set_min_bus_tput(dev, OCP_INITIATOR_AGENT, tput);
}
static enum omapdss_version __init omap_display_get_version(void)
{
if (cpu_is_omap24xx())
@@ -174,7 +168,6 @@ static int __init omapdss_init_fbdev(void)
static struct omap_dss_board_info board_data = {
.dsi_enable_pads = omap_dsi_enable_pads,
.dsi_disable_pads = omap_dsi_disable_pads,
.set_min_bus_tput = omap_dss_set_min_bus_tput,
};
struct device_node *node;
int r;

1
arch/arm/mach-omap2/hsmmc.c
View File

@@ -20,7 +20,6 @@
#include "soc.h"
#include "omap_device.h"
#include "omap-pm.h"
#include "hsmmc.h"
#include "control.h"

1
arch/arm/mach-omap2/i2c.c
View File

@@ -22,7 +22,6 @@
#include "soc.h"
#include "omap_hwmod.h"
#include "omap_device.h"
#include "omap-pm.h"
#include "prm.h"
#include "common.h"

3
arch/arm/mach-omap2/io.c
View File

@@ -37,7 +37,6 @@
#include "clock.h"
#include "clock2xxx.h"
#include "clock3xxx.h"
#include "omap-pm.h"
#include "sdrc.h"
#include "control.h"
#include "serial.h"
@@ -421,8 +420,6 @@ static void __init __maybe_unused omap_hwmod_init_postsetup(void)
postsetup_state = _HWMOD_STATE_ENABLED;
#endif
omap_hwmod_for_each(_set_hwmod_postsetup_state, &postsetup_state);
omap_pm_if_early_init();
}
static void __init __maybe_unused omap_common_late_init(void)

176
arch/arm/mach-omap2/omap-pm-noop.c
View File

@@ -1,176 +0,0 @@
// SPDX-License-Identifier: GPL-2.0
/*
* omap-pm-noop.c - OMAP power management interface - dummy version
*
* This code implements the OMAP power management interface to
* drivers, CPUIdle, CPUFreq, and DSP Bridge. It is strictly for
* debug/demonstration use, as it does nothing but printk() whenever a
* function is called (when DEBUG is defined, below)
*
* Copyright (C) 2008-2009 Texas Instruments, Inc.
* Copyright (C) 2008-2009 Nokia Corporation
* Paul Walmsley
*
* Interface developed by (in alphabetical order):
* Karthik Dasu, Tony Lindgren, Rajendra Nayak, Sakari Poussa, Veeramanikandan
* Raju, Anand Sawant, Igor Stoppa, Paul Walmsley, Richard Woodruff
*/
#undef DEBUG
#include <linux/init.h>
#include <linux/cpufreq.h>
#include <linux/device.h>
#include <linux/platform_device.h>
#include "omap_device.h"
#include "omap-pm.h"
static bool off_mode_enabled;
static int dummy_context_loss_counter;
/*
* Device-driver-originated constraints (via board-*.c files)
*/
int omap_pm_set_max_mpu_wakeup_lat(struct device *dev, long t)
{
if (!dev || t < -1) {
WARN(1, "OMAP PM: %s: invalid parameter(s)", __func__);
return -EINVAL;
}
if (t == -1)
pr_debug("OMAP PM: remove max MPU wakeup latency constraint: dev %s\n",
dev_name(dev));
else
pr_debug("OMAP PM: add max MPU wakeup latency constraint: dev %s, t = %ld usec\n",
dev_name(dev), t);
/*
* For current Linux, this needs to map the MPU to a
* powerdomain, then go through the list of current max lat
* constraints on the MPU and find the smallest. If
* the latency constraint has changed, the code should
* recompute the state to enter for the next powerdomain
* state.
*
* TI CDP code can call constraint_set here.
*/
return 0;
}
int omap_pm_set_min_bus_tput(struct device *dev, u8 agent_id, unsigned long r)
{
if (!dev || (agent_id != OCP_INITIATOR_AGENT &&
agent_id != OCP_TARGET_AGENT)) {
WARN(1, "OMAP PM: %s: invalid parameter(s)", __func__);
return -EINVAL;
}
if (r == 0)
pr_debug("OMAP PM: remove min bus tput constraint: dev %s for agent_id %d\n",
dev_name(dev), agent_id);
else
pr_debug("OMAP PM: add min bus tput constraint: dev %s for agent_id %d: rate %ld KiB\n",
dev_name(dev), agent_id, r);
/*
* This code should model the interconnect and compute the
* required clock frequency, convert that to a VDD2 OPP ID, then
* set the VDD2 OPP appropriately.
*
* TI CDP code can call constraint_set here on the VDD2 OPP.
*/
return 0;
}
/*
* DSP Bridge-specific constraints
*/
/**
* omap_pm_enable_off_mode - notify OMAP PM that off-mode is enabled
*
* Intended for use only by OMAP PM core code to notify this layer
* that off mode has been enabled.
*/
void omap_pm_enable_off_mode(void)
{
off_mode_enabled = true;
}
/**
* omap_pm_disable_off_mode - notify OMAP PM that off-mode is disabled
*
* Intended for use only by OMAP PM core code to notify this layer
* that off mode has been disabled.
*/
void omap_pm_disable_off_mode(void)
{
off_mode_enabled = false;
}
/*
* Device context loss tracking
*/
#ifdef CONFIG_ARCH_OMAP2PLUS
int omap_pm_get_dev_context_loss_count(struct device *dev)
{
struct platform_device *pdev = to_platform_device(dev);
int count;
if (WARN_ON(!dev))
return -ENODEV;
if (dev->pm_domain == &omap_device_pm_domain) {
count = omap_device_get_context_loss_count(pdev);
} else {
WARN_ONCE(off_mode_enabled, "omap_pm: using dummy context loss counter; device %s should be converted to omap_device",
dev_name(dev));
count = dummy_context_loss_counter;
if (off_mode_enabled) {
count++;
/*
* Context loss count has to be a non-negative value.
* Clear the sign bit to get a value range from 0 to
* INT_MAX.
*/
count &= INT_MAX;
dummy_context_loss_counter = count;
}
}
pr_debug("OMAP PM: context loss count for dev %s = %d\n",
dev_name(dev), count);
return count;
}
#else
int omap_pm_get_dev_context_loss_count(struct device *dev)
{
return dummy_context_loss_counter;
}
#endif
/* Should be called before clk framework init */
int __init omap_pm_if_early_init(void)
{
return 0;
}
/* Must be called after clock framework is initialized */
int __init omap_pm_if_init(void)
{
return 0;
}

161
arch/arm/mach-omap2/omap-pm.h
View File

@@ -1,161 +0,0 @@
/* SPDX-License-Identifier: GPL-2.0 */
/*
* omap-pm.h - OMAP power management interface
*
* Copyright (C) 2008-2010 Texas Instruments, Inc.
* Copyright (C) 2008-2010 Nokia Corporation
* Paul Walmsley
*
* Interface developed by (in alphabetical order): Karthik Dasu, Jouni
* Högander, Tony Lindgren, Rajendra Nayak, Sakari Poussa,
* Veeramanikandan Raju, Anand Sawant, Igor Stoppa, Paul Walmsley,
* Richard Woodruff
*/
#ifndef ASM_ARM_ARCH_OMAP_OMAP_PM_H
#define ASM_ARM_ARCH_OMAP_OMAP_PM_H
#include <linux/device.h>
#include <linux/cpufreq.h>
#include <linux/clk.h>
#include <linux/pm_opp.h>
/*
* agent_id values for use with omap_pm_set_min_bus_tput():
*
* OCP_INITIATOR_AGENT is only valid for devices that can act as
* initiators -- it represents the device's L3 interconnect
* connection. OCP_TARGET_AGENT represents the device's L4
* interconnect connection.
*/
#define OCP_TARGET_AGENT 1
#define OCP_INITIATOR_AGENT 2
/**
* omap_pm_if_early_init - OMAP PM init code called before clock fw init
* @mpu_opp_table: array ptr to struct omap_opp for MPU
* @dsp_opp_table: array ptr to struct omap_opp for DSP
* @l3_opp_table : array ptr to struct omap_opp for CORE
*
* Initialize anything that must be configured before the clock
* framework starts. The "_if_" is to avoid name collisions with the
* PM idle-loop code.
*/
int __init omap_pm_if_early_init(void);
/**
* omap_pm_if_init - OMAP PM init code called after clock fw init
*
* The main initialization code. OPP tables are passed in here. The
* "_if_" is to avoid name collisions with the PM idle-loop code.
*/
int __init omap_pm_if_init(void);
/*
* Device-driver-originated constraints (via board-*.c files, platform_data)
*/
/**
* omap_pm_set_max_mpu_wakeup_lat - set the maximum MPU wakeup latency
* @dev: struct device * requesting the constraint
* @t: maximum MPU wakeup latency in microseconds
*
* Request that the maximum interrupt latency for the MPU to be no
* greater than @t microseconds. "Interrupt latency" in this case is
* defined as the elapsed time from the occurrence of a hardware or
* timer interrupt to the time when the device driver's interrupt
* service routine has been entered by the MPU.
*
* It is intended that underlying PM code will use this information to
* determine what power state to put the MPU powerdomain into, and
* possibly the CORE powerdomain as well, since interrupt handling
* code currently runs from SDRAM. Advanced PM or board*.c code may
* also configure interrupt controller priorities, OCP bus priorities,
* CPU speed(s), etc.
*
* This function will not affect device wakeup latency, e.g., time
* elapsed from when a device driver enables a hardware device with
* clk_enable(), to when the device is ready for register access or
* other use. To control this device wakeup latency, use
* omap_pm_set_max_dev_wakeup_lat()
*
* Multiple calls to omap_pm_set_max_mpu_wakeup_lat() will replace the
* previous t value. To remove the latency target for the MPU, call
* with t = -1.
*
* XXX This constraint will be deprecated soon in favor of the more
* general omap_pm_set_max_dev_wakeup_lat()
*
* Returns -EINVAL for an invalid argument, -ERANGE if the constraint
* is not satisfiable, or 0 upon success.
*/
int omap_pm_set_max_mpu_wakeup_lat(struct device *dev, long t);
/**
* omap_pm_set_min_bus_tput - set minimum bus throughput needed by device
* @dev: struct device * requesting the constraint
* @tbus_id: interconnect to operate on (OCP_{INITIATOR,TARGET}_AGENT)
* @r: minimum throughput (in KiB/s)
*
* Request that the minimum data throughput on the OCP interconnect
* attached to device @dev interconnect agent @tbus_id be no less
* than @r KiB/s.
*
* It is expected that the OMAP PM or bus code will use this
* information to set the interconnect clock to run at the lowest
* possible speed that satisfies all current system users. The PM or
* bus code will adjust the estimate based on its model of the bus, so
* device driver authors should attempt to specify an accurate
* quantity for their device use case, and let the PM or bus code
* overestimate the numbers as necessary to handle request/response
* latency, other competing users on the system, etc. On OMAP2/3, if
* a driver requests a minimum L4 interconnect speed constraint, the
* code will also need to add an minimum L3 interconnect speed
* constraint,
*
* Multiple calls to omap_pm_set_min_bus_tput() will replace the
* previous rate value for this device. To remove the interconnect
* throughput restriction for this device, call with r = 0.
*
* Returns -EINVAL for an invalid argument, -ERANGE if the constraint
* is not satisfiable, or 0 upon success.
*/
int omap_pm_set_min_bus_tput(struct device *dev, u8 agent_id, unsigned long r);
/*
* CPUFreq-originated constraint
*
* In the future, this should be handled by custom OPP clocktype
* functions.
*/
/*
* Device context loss tracking
*/
/**
* omap_pm_get_dev_context_loss_count - return count of times dev has lost ctx
* @dev: struct device *
*
* This function returns the number of times that the device @dev has
* lost its internal context. This generally occurs on a powerdomain
* transition to OFF. Drivers use this as an optimization to avoid restoring
* context if the device hasn't lost it. To use, drivers should initially
* call this in their context save functions and store the result. Early in
* the driver's context restore function, the driver should call this function
* again, and compare the result to the stored counter. If they differ, the
* driver must restore device context. If the number of context losses
* exceeds the maximum positive integer, the function will wrap to 0 and
* continue counting. Returns the number of context losses for this device,
* or negative value upon error.
*/
int omap_pm_get_dev_context_loss_count(struct device *dev);
void omap_pm_enable_off_mode(void);
void omap_pm_disable_off_mode(void);
#endif

14
arch/arm/mach-omap2/pdata-quirks.c
View File

@@ -25,7 +25,6 @@
#include <linux/platform_data/iommu-omap.h>
#include <linux/platform_data/wkup_m3.h>
#include <linux/platform_data/pwm_omap_dmtimer.h>
#include <linux/platform_data/media/ir-rx51.h>
#include <linux/platform_data/asoc-ti-mcbsp.h>
#include <plat/dmtimer.h>
@@ -33,7 +32,6 @@
#include "common-board-devices.h"
#include "control.h"
#include "omap_device.h"
#include "omap-pm.h"
#include "omap-secure.h"
#include "soc.h"
#include "hsmmc.h"
@@ -411,18 +409,6 @@ static struct pwm_omap_dmtimer_pdata pwm_dmtimer_pdata = {
};
#endif
static struct ir_rx51_platform_data __maybe_unused rx51_ir_data = {
.set_max_mpu_wakeup_lat = omap_pm_set_max_mpu_wakeup_lat,
};
static struct platform_device __maybe_unused rx51_ir_device = {
.name = "ir_rx51",
.id = -1,
.dev = {
.platform_data = &rx51_ir_data,
},
};
#if IS_ENABLED(CONFIG_SND_OMAP_SOC_MCBSP)
static struct omap_mcbsp_platform_data mcbsp_pdata;
static void __init omap3_mcbsp_init(void)

5
arch/arm/mach-omap2/pm-debug.c
View File

@@ -31,7 +31,6 @@
#include "clock.h"
#include "powerdomain.h"
#include "clockdomain.h"
#include "omap-pm.h"
#include "soc.h"
#include "cm2xxx_3xxx.h"
@@ -240,10 +239,6 @@ static int option_set(void *data, u64 val)
*option = val;
if (option == &enable_off_mode) {
if (val)
omap_pm_enable_off_mode();
else
omap_pm_disable_off_mode();
if (cpu_is_omap34xx())
omap3_pm_off_mode_enable(val);
}

10
arch/arm/mach-omap2/pm.c
View File

@@ -16,11 +16,11 @@
#include <linux/pm_opp.h>
#include <linux/export.h>
#include <linux/suspend.h>
#include <linux/clk.h>
#include <linux/cpu.h>
#include <asm/system_misc.h>
#include "omap-pm.h"
#include "omap_device.h"
#include "common.h"
@@ -230,14 +230,6 @@ static void __init omap4_init_voltages(void)
omap2_set_init_voltage("iva", "dpll_iva_m5x2_ck", "iva");
}
static int __init omap2_common_pm_init(void)
{
omap_pm_if_init();
return 0;
}
omap_postcore_initcall(omap2_common_pm_init);
int __init omap2_common_pm_late_init(void)
{
/* Init the voltage layer */

1
arch/arm/mach-omap2/timer.c
View File

@@ -50,7 +50,6 @@
#include "omap_device.h"
#include <plat/counter-32k.h>
#include <plat/dmtimer.h>
#include "omap-pm.h"
#include "soc.h"
#include "common.h"

2
arch/arm/mm/pgd.c
View File

@@ -142,7 +142,7 @@ void pgd_free(struct mm_struct *mm, pgd_t *pgd_base)
pte = pmd_pgtable(*pmd);
pmd_clear(pmd);
pte_free(mm, pte);
atomic_long_dec(&mm->nr_ptes);
mm_dec_nr_ptes(mm);
no_pmd:
pud_clear(pud);
pmd_free(mm, pmd);

3
arch/arm/net/bpf_jit_32.c
View File

@@ -1545,6 +1545,9 @@ exit:
}
break;
}
/* speculation barrier */
case BPF_ST | BPF_NOSPEC:
break;
/* ST: *(size *)(dst + off) = imm */
case BPF_ST | BPF_MEM | BPF_W:
case BPF_ST | BPF_MEM | BPF_H:

10
arch/arm/plat-omap/Kconfig
View File

@@ -121,16 +121,6 @@ config OMAP_SERIAL_WAKE
to data on the serial RX line. This allows you to wake the
system from serial console.
choice
prompt "OMAP PM layer selection"
depends on ARCH_OMAP
default OMAP_PM_NOOP
config OMAP_PM_NOOP
bool "No-op/debug PM layer"
endchoice
endmenu
endif

2
arch/arm64/include/asm/atomic.h
View File

@@ -125,7 +125,7 @@
#define atomic_dec_and_test(v) (atomic_dec_return(v) == 0)
#define atomic_sub_and_test(i, v) (atomic_sub_return((i), (v)) == 0)
#define atomic_add_negative(i, v) (atomic_add_return((i), (v)) < 0)
#define __atomic_add_unless(v, a, u) ___atomic_add_unless(v, a, u,)
#define atomic_fetch_add_unless(v, a, u) ___atomic_add_unless(v, a, u,)
#define atomic_andnot atomic_andnot
/*

82
arch/arm64/net/bpf_jit_comp.c
View File

@@ -99,6 +99,20 @@ static inline void emit_a64_mov_i64(const int reg, const u64 val,
}
}
static inline void emit_addr_mov_i64(const int reg, const u64 val,
struct jit_ctx *ctx)
{
u64 tmp = val;
int shift = 0;
emit(A64_MOVZ(1, reg, tmp & 0xffff, shift), ctx);
for (;shift < 48;) {
tmp >>= 16;
shift += 16;
emit(A64_MOVK(1, reg, tmp & 0xffff, shift), ctx);
}
}
static inline void emit_a64_mov_i(const int is64, const int reg,
const s32 val, struct jit_ctx *ctx)
{
@@ -606,7 +620,10 @@ emit_cond_jmp:
const u8 r0 = bpf2a64[BPF_REG_0];
const u64 func = (u64)__bpf_call_base + imm;
emit_a64_mov_i64(tmp, func, ctx);
if (ctx->prog->is_func)
emit_addr_mov_i64(tmp, func, ctx);
else
emit_a64_mov_i64(tmp, func, ctx);
emit(A64_BLR(tmp), ctx);
emit(A64_MOV(1, r0, A64_R(0)), ctx);
break;
@@ -661,6 +678,19 @@ emit_cond_jmp:
}
break;
/* speculation barrier */
case BPF_ST | BPF_NOSPEC:
/*
* Nothing required here.
*
* In case of arm64, we rely on the firmware mitigation of
* Speculative Store Bypass as controlled via the ssbd kernel
* parameter. Whenever the mitigation is enabled, it works
* for all of the kernel code with no need to provide any
* additional instructions.
*/
break;
/* ST: *(size *)(dst + off) = imm */
case BPF_ST | BPF_MEM | BPF_W:
case BPF_ST | BPF_MEM | BPF_H:
@@ -847,11 +877,19 @@ static inline void bpf_flush_icache(void *start, void *end)
flush_icache_range((unsigned long)start, (unsigned long)end);
}
struct arm64_jit_data {
struct bpf_binary_header *header;
u8 *image;
struct jit_ctx ctx;
};
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
{
struct bpf_prog *tmp, *orig_prog = prog;
struct bpf_binary_header *header;
struct arm64_jit_data *jit_data;
bool tmp_blinded = false;
bool extra_pass = false;
struct jit_ctx ctx;
int image_size;
u8 *image_ptr;
@@ -870,13 +908,30 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
prog = tmp;
}
jit_data = prog->aux->jit_data;
if (!jit_data) {
jit_data = kzalloc(sizeof(*jit_data), GFP_KERNEL);
if (!jit_data) {
prog = orig_prog;
goto out;
}
prog->aux->jit_data = jit_data;
}
if (jit_data->ctx.offset) {
ctx = jit_data->ctx;
image_ptr = jit_data->image;
header = jit_data->header;
extra_pass = true;
image_size = sizeof(u32) * ctx.idx;
goto skip_init_ctx;
}
memset(&ctx, 0, sizeof(ctx));
ctx.prog = prog;
ctx.offset = kcalloc(prog->len, sizeof(int), GFP_KERNEL);
if (ctx.offset == NULL) {
prog = orig_prog;
goto out;
goto out_off;
}
/* 1. Initial fake pass to compute ctx->idx. */
@@ -907,6 +962,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
/* 2. Now, the actual pass. */
ctx.image = (__le32 *)image_ptr;
skip_init_ctx:
ctx.idx = 0;
build_prologue(&ctx);
@@ -932,7 +988,21 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
bpf_flush_icache(header, ctx.image + ctx.idx);
bpf_jit_binary_lock_ro(header);
if (!prog->is_func || extra_pass) {
if (extra_pass && ctx.idx != jit_data->ctx.idx) {
pr_err_once("multi-func JIT bug %d != %d\n",
ctx.idx, jit_data->ctx.idx);
bpf_jit_binary_free(header);
prog->bpf_func = NULL;
prog->jited = 0;
goto out_off;
}
bpf_jit_binary_lock_ro(header);
} else {
jit_data->ctx = ctx;
jit_data->image = image_ptr;
jit_data->header = header;
}
prog->bpf_func = (void *)ctx.image;
prog->jited = 1;
prog->jited_len = image_size;
@@ -940,8 +1010,12 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
uh_call(UH_APP_RKP, RKP_BFP_LOAD, (u64)header, (u64)(header->pages * PAGE_SIZE), RKP_BPF_JIT_LOAD, 0);
#endif
if (!prog->is_func || extra_pass) {
out_off:
kfree(ctx.offset);
kfree(ctx.offset);
kfree(jit_data);
prog->aux->jit_data = NULL;
}
out:
if (tmp_blinded)
bpf_jit_prog_release_other(prog, prog == orig_prog ?

2
arch/frv/include/asm/atomic.h
View File

@@ -146,7 +146,7 @@ static inline void atomic64_dec(atomic64_t *v)
#define atomic64_cmpxchg(v, old, new) (__cmpxchg_64(old, new, &(v)->counter))
#define atomic64_xchg(v, new) (__xchg_64(new, &(v)->counter))
static __inline__ int __atomic_add_unless(atomic_t *v, int a, int u)
static __inline__ int atomic_fetch_add_unless(atomic_t *v, int a, int u)
{
int c, old;
c = atomic_read(v);

2
arch/h8300/include/asm/atomic.h
View File

@@ -94,7 +94,7 @@ static inline int atomic_cmpxchg(atomic_t *v, int old, int new)
return ret;
}
static inline int __atomic_add_unless(atomic_t *v, int a, int u)
static inline int atomic_fetch_add_unless(atomic_t *v, int a, int u)
{
int ret;
h8300flags flags;

4
arch/hexagon/include/asm/atomic.h
View File

@@ -164,7 +164,7 @@ ATOMIC_OPS(xor)
#undef ATOMIC_OP
/**
* __atomic_add_unless - add unless the number is a given value
* atomic_fetch_add_unless - add unless the number is a given value
* @v: pointer to value
* @a: amount to add
* @u: unless value is equal to u
@@ -173,7 +173,7 @@ ATOMIC_OPS(xor)
*
*/
static inline int __atomic_add_unless(atomic_t *v, int a, int u)
static inline int atomic_fetch_add_unless(atomic_t *v, int a, int u)
{
int __oldval;
register int tmp;

2
arch/ia64/include/asm/atomic.h
View File

@@ -237,7 +237,7 @@ ATOMIC64_FETCH_OP(xor, ^)
(cmpxchg(&((v)->counter), old, new))
#define atomic64_xchg(v, new) (xchg(&((v)->counter), new))
static __inline__ int __atomic_add_unless(atomic_t *v, int a, int u)
static __inline__ int atomic_fetch_add_unless(atomic_t *v, int a, int u)
{
int c, old;
c = atomic_read(v);

4
arch/m32r/include/asm/atomic.h
View File

@@ -249,7 +249,7 @@ static __inline__ int atomic_dec_return(atomic_t *v)
#define atomic_xchg(v, new) (xchg(&((v)->counter), new))
/**
* __atomic_add_unless - add unless the number is a given value
* atomic_fetch_add_unless - add unless the number is a given value
* @v: pointer of type atomic_t
* @a: the amount to add to v...
* @u: ...unless v is equal to u.
@@ -257,7 +257,7 @@ static __inline__ int atomic_dec_return(atomic_t *v)
* Atomically adds @a to @v, so long as it was not @u.
* Returns the old value of @v.
*/
static __inline__ int __atomic_add_unless(atomic_t *v, int a, int u)
static __inline__ int atomic_fetch_add_unless(atomic_t *v, int a, int u)
{
int c, old;
c = atomic_read(v);

2
arch/m68k/include/asm/atomic.h
View File

@@ -211,7 +211,7 @@ static inline int atomic_add_negative(int i, atomic_t *v)
return c != 0;
}
static __inline__ int __atomic_add_unless(atomic_t *v, int a, int u)
static __inline__ int atomic_fetch_add_unless(atomic_t *v, int a, int u)
{
int c, old;
c = atomic_read(v);

2
arch/metag/include/asm/atomic_lnkget.h
View File

@@ -154,7 +154,7 @@ static inline int atomic_xchg(atomic_t *v, int new)
return old;
}
static inline int __atomic_add_unless(atomic_t *v, int a, int u)
static inline int atomic_fetch_add_unless(atomic_t *v, int a, int u)
{
int result, temp;

2
arch/metag/include/asm/atomic_lock1.h
View File

@@ -122,7 +122,7 @@ static inline int atomic_cmpxchg(atomic_t *v, int old, int new)
#define atomic_xchg(v, new) (xchg(&((v)->counter), new))
static inline int __atomic_add_unless(atomic_t *v, int a, int u)
static inline int atomic_fetch_add_unless(atomic_t *v, int a, int u)
{
int ret;
unsigned long flags;

4
arch/mips/include/asm/atomic.h
View File

@@ -275,7 +275,7 @@ static __inline__ int atomic_sub_if_positive(int i, atomic_t * v)
#define atomic_xchg(v, new) (xchg(&((v)->counter), (new)))
/**
* __atomic_add_unless - add unless the number is a given value
* atomic_fetch_add_unless - add unless the number is a given value
* @v: pointer of type atomic_t
* @a: the amount to add to v...
* @u: ...unless v is equal to u.
@@ -283,7 +283,7 @@ static __inline__ int atomic_sub_if_positive(int i, atomic_t * v)
* Atomically adds @a to @v, so long as it was not @u.
* Returns the old value of @v.
*/
static __inline__ int __atomic_add_unless(atomic_t *v, int a, int u)
static __inline__ int atomic_fetch_add_unless(atomic_t *v, int a, int u)
{
int c, old;
c = atomic_read(v);

5
arch/mips/net/ebpf_jit.c
View File

@@ -1433,6 +1433,9 @@ ld_skb_common:
}
break;
case BPF_ST | BPF_NOSPEC: /* speculation barrier */
break;
case BPF_ST | BPF_B | BPF_MEM:
case BPF_ST | BPF_H | BPF_MEM:
case BPF_ST | BPF_W | BPF_MEM:
@@ -1871,7 +1874,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
unsigned int image_size;
u8 *image_ptr;
if (!bpf_jit_enable || !cpu_has_mips64r2)
if (!prog->jit_requested || !cpu_has_mips64r2)
return prog;
tmp = bpf_jit_blind_constants(prog);

2
arch/mn10300/include/asm/atomic.h
View File

@@ -144,7 +144,7 @@ static inline void atomic_dec(atomic_t *v)
#define atomic_dec_and_test(v) (atomic_sub_return(1, (v)) == 0)
#define atomic_inc_and_test(v) (atomic_add_return(1, (v)) == 0)
#define __atomic_add_unless(v, a, u) \
#define atomic_fetch_add_unless(v, a, u) \
({ \
int c, old; \
c = atomic_read(v); \

4
arch/openrisc/include/asm/atomic.h
View File

@@ -100,7 +100,7 @@ ATOMIC_OP(xor)
*
* This is often used through atomic_inc_not_zero()
*/
static inline int __atomic_add_unless(atomic_t *v, int a, int u)
static inline int atomic_fetch_add_unless(atomic_t *v, int a, int u)
{
int old, tmp;
@@ -119,7 +119,7 @@ static inline int __atomic_add_unless(atomic_t *v, int a, int u)
return old;
}
#define __atomic_add_unless __atomic_add_unless
#define atomic_fetch_add_unless atomic_fetch_add_unless
#include <asm-generic/atomic.h>

4
arch/parisc/include/asm/atomic.h
View File

@@ -78,7 +78,7 @@ static __inline__ int atomic_read(const atomic_t *v)
#define atomic_xchg(v, new) (xchg(&((v)->counter), new))
/**
* __atomic_add_unless - add unless the number is a given value
* atomic_fetch_add_unless - add unless the number is a given value
* @v: pointer of type atomic_t
* @a: the amount to add to v...
* @u: ...unless v is equal to u.
@@ -86,7 +86,7 @@ static __inline__ int atomic_read(const atomic_t *v)
* Atomically adds @a to @v, so long as it was not @u.
* Returns the old value of @v.
*/
static __inline__ int __atomic_add_unless(atomic_t *v, int a, int u)
static __inline__ int atomic_fetch_add_unless(atomic_t *v, int a, int u)
{
int c, old;
c = atomic_read(v);

8
arch/powerpc/include/asm/atomic.h
View File

@@ -218,7 +218,7 @@ static __inline__ int atomic_dec_return_relaxed(atomic_t *v)
#define atomic_xchg_relaxed(v, new) xchg_relaxed(&((v)->counter), (new))
/**
* __atomic_add_unless - add unless the number is a given value
* atomic_fetch_add_unless - add unless the number is a given value
* @v: pointer of type atomic_t
* @a: the amount to add to v...
* @u: ...unless v is equal to u.
@@ -226,13 +226,13 @@ static __inline__ int atomic_dec_return_relaxed(atomic_t *v)
* Atomically adds @a to @v, so long as it was not @u.
* Returns the old value of @v.
*/
static __inline__ int __atomic_add_unless(atomic_t *v, int a, int u)
static __inline__ int atomic_fetch_add_unless(atomic_t *v, int a, int u)
{
int t;
__asm__ __volatile__ (
PPC_ATOMIC_ENTRY_BARRIER
"1: lwarx %0,0,%1 # __atomic_add_unless\n\
"1: lwarx %0,0,%1 # atomic_fetch_add_unless\n\
cmpw 0,%0,%3 \n\
beq 2f \n\
add %0,%2,%0 \n"
@@ -538,7 +538,7 @@ static __inline__ int atomic64_add_unless(atomic64_t *v, long a, long u)
__asm__ __volatile__ (
PPC_ATOMIC_ENTRY_BARRIER
"1: ldarx %0,0,%1 # __atomic_add_unless\n\
"1: ldarx %0,0,%1 # atomic_fetch_add_unless\n\
cmpd 0,%0,%3 \n\
beq 2f \n\
add %0,%2,%0 \n"

6
arch/powerpc/kernel/vmlinux.lds.S
View File

@@ -332,12 +332,6 @@ SECTIONS
*(.branch_lt)
}
#ifdef CONFIG_DEBUG_INFO_BTF
.BTF : AT(ADDR(.BTF) - LOAD_OFFSET) {
*(.BTF)
}
#endif
.opd : AT(ADDR(.opd) - LOAD_OFFSET) {
*(.opd)
}

1
arch/powerpc/mm/hugetlbpage.c
View File

@@ -437,6 +437,7 @@ static void hugetlb_free_pud_range(struct mmu_gather *tlb, pgd_t *pgd,
pud = pud_offset(pgd, start);
pgd_clear(pgd);
pud_free_tlb(tlb, pud, start);
mm_dec_nr_puds(tlb->mm);
}
/*

2
arch/powerpc/net/Makefile
View File

@@ -3,7 +3,7 @@
# Arch-specific network modules
#
ifeq ($(CONFIG_PPC64),y)
obj-$(CONFIG_BPF_JIT) += bpf_jit_asm64.o bpf_jit_comp64.o
obj-$(CONFIG_BPF_JIT) += bpf_jit_comp64.o
else
obj-$(CONFIG_BPF_JIT) += bpf_jit_asm.o bpf_jit_comp.o
endif

37
arch/powerpc/net/bpf_jit64.h
View File

@@ -20,7 +20,7 @@
* with our redzone usage.
*
* [ prev sp ] <-------------
* [ nv gpr save area ] 8*8 |
* [ nv gpr save area ] 6*8 |
* [ tail_call_cnt ] 8 |
* [ local_tmp_var ] 8 |
* fp (r31) --> [ ebpf stack space ] 512 |
@@ -28,8 +28,8 @@
* sp (r1) ---> [ stack pointer ] --------------
*/
/* for gpr non volatile registers BPG_REG_6 to 10, plus skb cache registers */
#define BPF_PPC_STACK_SAVE (8*8)
/* for gpr non volatile registers BPG_REG_6 to 10 */
#define BPF_PPC_STACK_SAVE (6*8)
/* for bpf JIT code internal usage */
#define BPF_PPC_STACK_LOCALS 16
/* Ensure this is quadword aligned */
@@ -39,10 +39,8 @@
#ifndef __ASSEMBLY__
/* BPF register usage */
#define SKB_HLEN_REG (MAX_BPF_JIT_REG + 0)
#define SKB_DATA_REG (MAX_BPF_JIT_REG + 1)
#define TMP_REG_1 (MAX_BPF_JIT_REG + 2)
#define TMP_REG_2 (MAX_BPF_JIT_REG + 3)
#define TMP_REG_1 (MAX_BPF_JIT_REG + 0)
#define TMP_REG_2 (MAX_BPF_JIT_REG + 1)
/* BPF to ppc register mappings */
static const int b2p[] = {
@@ -63,28 +61,12 @@ static const int b2p[] = {
[BPF_REG_FP] = 31,
/* eBPF jit internal registers */
[BPF_REG_AX] = 2,
[SKB_HLEN_REG] = 25,
[SKB_DATA_REG] = 26,
[TMP_REG_1] = 9,
[TMP_REG_2] = 10
};
/* PPC NVR range -- update this if we ever use NVRs below r24 */
#define BPF_PPC_NVR_MIN 24
/* Assembly helpers */
#define DECLARE_LOAD_FUNC(func) u64 func(u64 r3, u64 r4); \
u64 func##_negative_offset(u64 r3, u64 r4); \
u64 func##_positive_offset(u64 r3, u64 r4);
DECLARE_LOAD_FUNC(sk_load_word);
DECLARE_LOAD_FUNC(sk_load_half);
DECLARE_LOAD_FUNC(sk_load_byte);
#define CHOOSE_LOAD_FUNC(imm, func) \
(imm < 0 ? \
(imm >= SKF_LL_OFF ? func##_negative_offset : func) : \
func##_positive_offset)
/* PPC NVR range -- update this if we ever use NVRs below r27 */
#define BPF_PPC_NVR_MIN 27
/*
* WARNING: These can use TMP_REG_2 if the offset is not at word boundary,
@@ -108,15 +90,14 @@ DECLARE_LOAD_FUNC(sk_load_byte);
#define SEEN_FUNC 0x1000 /* might call external helpers */
#define SEEN_STACK 0x2000 /* uses BPF stack */
#define SEEN_SKB 0x4000 /* uses sk_buff */
#define SEEN_TAILCALL 0x8000 /* uses tail calls */
#define SEEN_TAILCALL 0x4000 /* uses tail calls */
struct codegen_context {
/*
* This is used to track register usage as well
* as calls to external helpers.
* - register usage is tracked with corresponding
* bits (r3-r10 and r25-r31)
* bits (r3-r10 and r27-r31)
* - rest of the bits can be used to track other
* things -- for now, we use bits 16 to 23
* encoded in SEEN_* macros above

180
arch/powerpc/net/bpf_jit_asm64.S
View File

@@ -1,180 +0,0 @@
/*
* bpf_jit_asm64.S: Packet/header access helper functions
* for PPC64 BPF compiler.
*
* Copyright 2016, Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
* IBM Corporation
*
* Based on bpf_jit_asm.S by Matt Evans
*
* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License
* as published by the Free Software Foundation; version 2
* of the License.
*/
#include <asm/ppc_asm.h>
#include <asm/ptrace.h>
#include "bpf_jit64.h"
/*
* All of these routines are called directly from generated code,
* with the below register usage:
* r27 skb pointer (ctx)
* r25 skb header length
* r26 skb->data pointer
* r4 offset
*
* Result is passed back in:
* r8 data read in host endian format (accumulator)
*
* r9 is used as a temporary register
*/
#define r_skb r27
#define r_hlen r25
#define r_data r26
#define r_off r4
#define r_val r8
#define r_tmp r9
_GLOBAL_TOC(sk_load_word)
cmpdi r_off, 0
blt bpf_slow_path_word_neg
b sk_load_word_positive_offset
_GLOBAL_TOC(sk_load_word_positive_offset)
/* Are we accessing past headlen? */
subi r_tmp, r_hlen, 4
cmpd r_tmp, r_off
blt bpf_slow_path_word
/* Nope, just hitting the header. cr0 here is eq or gt! */
LWZX_BE r_val, r_data, r_off
blr /* Return success, cr0 != LT */
_GLOBAL_TOC(sk_load_half)
cmpdi r_off, 0
blt bpf_slow_path_half_neg
b sk_load_half_positive_offset
_GLOBAL_TOC(sk_load_half_positive_offset)
subi r_tmp, r_hlen, 2
cmpd r_tmp, r_off
blt bpf_slow_path_half
LHZX_BE r_val, r_data, r_off
blr
_GLOBAL_TOC(sk_load_byte)
cmpdi r_off, 0
blt bpf_slow_path_byte_neg
b sk_load_byte_positive_offset
_GLOBAL_TOC(sk_load_byte_positive_offset)
cmpd r_hlen, r_off
ble bpf_slow_path_byte
lbzx r_val, r_data, r_off
blr
/*
* Call out to skb_copy_bits:
* Allocate a new stack frame here to remain ABI-compliant in
* stashing LR.
*/
#define bpf_slow_path_common(SIZE) \
mflr r0; \
std r0, PPC_LR_STKOFF(r1); \
stdu r1, -(STACK_FRAME_MIN_SIZE + BPF_PPC_STACK_LOCALS)(r1); \
mr r3, r_skb; \
/* r4 = r_off as passed */ \
addi r5, r1, STACK_FRAME_MIN_SIZE; \
li r6, SIZE; \
bl skb_copy_bits; \
nop; \
/* save r5 */ \
addi r5, r1, STACK_FRAME_MIN_SIZE; \
/* r3 = 0 on success */ \
addi r1, r1, STACK_FRAME_MIN_SIZE + BPF_PPC_STACK_LOCALS; \
ld r0, PPC_LR_STKOFF(r1); \
mtlr r0; \
cmpdi r3, 0; \
blt bpf_error; /* cr0 = LT */
bpf_slow_path_word:
bpf_slow_path_common(4)
/* Data value is on stack, and cr0 != LT */
LWZX_BE r_val, 0, r5
blr
bpf_slow_path_half:
bpf_slow_path_common(2)
LHZX_BE r_val, 0, r5
blr
bpf_slow_path_byte:
bpf_slow_path_common(1)
lbzx r_val, 0, r5
blr
/*
* Call out to bpf_internal_load_pointer_neg_helper
*/
#define sk_negative_common(SIZE) \
mflr r0; \
std r0, PPC_LR_STKOFF(r1); \
stdu r1, -STACK_FRAME_MIN_SIZE(r1); \
mr r3, r_skb; \
/* r4 = r_off, as passed */ \
li r5, SIZE; \
bl bpf_internal_load_pointer_neg_helper; \
nop; \
addi r1, r1, STACK_FRAME_MIN_SIZE; \
ld r0, PPC_LR_STKOFF(r1); \
mtlr r0; \
/* R3 != 0 on success */ \
cmpldi r3, 0; \
beq bpf_error_slow; /* cr0 = EQ */
bpf_slow_path_word_neg:
lis r_tmp, -32 /* SKF_LL_OFF */
cmpd r_off, r_tmp /* addr < SKF_* */
blt bpf_error /* cr0 = LT */
b sk_load_word_negative_offset
_GLOBAL_TOC(sk_load_word_negative_offset)
sk_negative_common(4)
LWZX_BE r_val, 0, r3
blr
bpf_slow_path_half_neg:
lis r_tmp, -32 /* SKF_LL_OFF */
cmpd r_off, r_tmp /* addr < SKF_* */
blt bpf_error /* cr0 = LT */
b sk_load_half_negative_offset
_GLOBAL_TOC(sk_load_half_negative_offset)
sk_negative_common(2)
LHZX_BE r_val, 0, r3
blr
bpf_slow_path_byte_neg:
lis r_tmp, -32 /* SKF_LL_OFF */
cmpd r_off, r_tmp /* addr < SKF_* */
blt bpf_error /* cr0 = LT */
b sk_load_byte_negative_offset
_GLOBAL_TOC(sk_load_byte_negative_offset)
sk_negative_common(1)
lbzx r_val, 0, r3
blr
bpf_error_slow:
/* fabricate a cr0 = lt */
li r_tmp, -1
cmpdi r_tmp, 0
bpf_error:
/*
* Entered with cr0 = lt
* Generated code will 'blt epilogue', returning 0.
*/
li r_val, 0
blr

218
arch/powerpc/net/bpf_jit_comp64.c
View File

@@ -59,7 +59,7 @@ static inline bool bpf_has_stack_frame(struct codegen_context *ctx)
* [ prev sp ] <-------------
* [ ... ] |
* sp (r1) ---> [ stack pointer ] --------------
* [ nv gpr save area ] 8*8
* [ nv gpr save area ] 6*8
* [ tail_call_cnt ] 8
* [ local_tmp_var ] 8
* [ unused red zone ] 208 bytes protected
@@ -87,21 +87,6 @@ static int bpf_jit_stack_offsetof(struct codegen_context *ctx, int reg)
BUG();
}
static void bpf_jit_emit_skb_loads(u32 *image, struct codegen_context *ctx)
{
/*
* Load skb->len and skb->data_len
* r3 points to skb
*/
PPC_LWZ(b2p[SKB_HLEN_REG], 3, offsetof(struct sk_buff, len));
PPC_LWZ(b2p[TMP_REG_1], 3, offsetof(struct sk_buff, data_len));
/* header_len = len - data_len */
PPC_SUB(b2p[SKB_HLEN_REG], b2p[SKB_HLEN_REG], b2p[TMP_REG_1]);
/* skb->data pointer */
PPC_BPF_LL(b2p[SKB_DATA_REG], 3, offsetof(struct sk_buff, data));
}
static void bpf_jit_build_prologue(u32 *image, struct codegen_context *ctx)
{
int i;
@@ -144,18 +129,6 @@ static void bpf_jit_build_prologue(u32 *image, struct codegen_context *ctx)
if (bpf_is_seen_register(ctx, i))
PPC_BPF_STL(b2p[i], 1, bpf_jit_stack_offsetof(ctx, b2p[i]));
/*
* Save additional non-volatile regs if we cache skb
* Also, setup skb data
*/
if (ctx->seen & SEEN_SKB) {
PPC_BPF_STL(b2p[SKB_HLEN_REG], 1,
bpf_jit_stack_offsetof(ctx, b2p[SKB_HLEN_REG]));
PPC_BPF_STL(b2p[SKB_DATA_REG], 1,
bpf_jit_stack_offsetof(ctx, b2p[SKB_DATA_REG]));
bpf_jit_emit_skb_loads(image, ctx);
}
/* Setup frame pointer to point to the bpf stack area */
if (bpf_is_seen_register(ctx, BPF_REG_FP))
PPC_ADDI(b2p[BPF_REG_FP], 1,
@@ -171,14 +144,6 @@ static void bpf_jit_emit_common_epilogue(u32 *image, struct codegen_context *ctx
if (bpf_is_seen_register(ctx, i))
PPC_BPF_LL(b2p[i], 1, bpf_jit_stack_offsetof(ctx, b2p[i]));
/* Restore non-volatile registers used for skb cache */
if (ctx->seen & SEEN_SKB) {
PPC_BPF_LL(b2p[SKB_HLEN_REG], 1,
bpf_jit_stack_offsetof(ctx, b2p[SKB_HLEN_REG]));
PPC_BPF_LL(b2p[SKB_DATA_REG], 1,
bpf_jit_stack_offsetof(ctx, b2p[SKB_DATA_REG]));
}
/* Tear down our stack frame */
if (bpf_has_stack_frame(ctx)) {
PPC_ADDI(1, 1, BPF_PPC_STACKFRAME);
@@ -199,7 +164,33 @@ static void bpf_jit_build_epilogue(u32 *image, struct codegen_context *ctx)
PPC_BLR();
}
static void bpf_jit_emit_func_call(u32 *image, struct codegen_context *ctx, u64 func)
static void bpf_jit_emit_func_call_hlp(u32 *image, struct codegen_context *ctx,
u64 func)
{
#ifdef PPC64_ELF_ABI_v1
/* func points to the function descriptor */
PPC_LI64(b2p[TMP_REG_2], func);
/* Load actual entry point from function descriptor */
PPC_BPF_LL(b2p[TMP_REG_1], b2p[TMP_REG_2], 0);
/* ... and move it to LR */
PPC_MTLR(b2p[TMP_REG_1]);
/*
* Load TOC from function descriptor at offset 8.
* We can clobber r2 since we get called through a
* function pointer (so caller will save/restore r2)
* and since we don't use a TOC ourself.
*/
PPC_BPF_LL(2, b2p[TMP_REG_2], 8);
#else
/* We can clobber r12 */
PPC_FUNC_ADDR(12, func);
PPC_MTLR(12);
#endif
PPC_BLRL();
}
static void bpf_jit_emit_func_call_rel(u32 *image, struct codegen_context *ctx,
u64 func)
{
unsigned int i, ctx_idx = ctx->idx;
@@ -304,7 +295,7 @@ static int bpf_jit_emit_tail_call(u32 *image, struct codegen_context *ctx, u32 o
/* Assemble the body code between the prologue & epilogue */
static int bpf_jit_build_body(struct bpf_prog *fp, u32 *image,
struct codegen_context *ctx,
u32 *addrs)
u32 *addrs, bool extra_pass)
{
const struct bpf_insn *insn = fp->insnsi;
int flen = fp->len;
@@ -319,8 +310,9 @@ static int bpf_jit_build_body(struct bpf_prog *fp, u32 *image,
u32 src_reg = b2p[insn[i].src_reg];
s16 off = insn[i].off;
s32 imm = insn[i].imm;
bool func_addr_fixed;
u64 func_addr;
u64 imm64;
u8 *func;
u32 true_cond;
u32 tmp_idx;
@@ -652,6 +644,12 @@ emit_clear:
}
break;
/*
* BPF_ST NOSPEC (speculation barrier)
*/
case BPF_ST | BPF_NOSPEC:
break;
/*
* BPF_ST(X)
*/
@@ -762,29 +760,22 @@ emit_clear:
break;
/*
* Call kernel helper
* Call kernel helper or bpf function
*/
case BPF_JMP | BPF_CALL:
ctx->seen |= SEEN_FUNC;
func = (u8 *) __bpf_call_base + imm;
/* Save skb pointer if we need to re-cache skb data */
if ((ctx->seen & SEEN_SKB) &&
bpf_helper_changes_pkt_data(func))
PPC_BPF_STL(3, 1, bpf_jit_stack_local(ctx));
bpf_jit_emit_func_call(image, ctx, (u64)func);
ret = bpf_jit_get_func_addr(fp, &insn[i], extra_pass,
&func_addr, &func_addr_fixed);
if (ret < 0)
return ret;
if (func_addr_fixed)
bpf_jit_emit_func_call_hlp(image, ctx, func_addr);
else
bpf_jit_emit_func_call_rel(image, ctx, func_addr);
/* move return value from r3 to BPF_REG_0 */
PPC_MR(b2p[BPF_REG_0], 3);
/* refresh skb cache */
if ((ctx->seen & SEEN_SKB) &&
bpf_helper_changes_pkt_data(func)) {
/* reload skb pointer to r3 */
PPC_BPF_LL(3, 1, bpf_jit_stack_local(ctx));
bpf_jit_emit_skb_loads(image, ctx);
}
break;
/*
@@ -901,65 +892,6 @@ cond_branch:
PPC_BCC(true_cond, addrs[i + 1 + off]);
break;
/*
* Loads from packet header/data
* Assume 32-bit input value in imm and X (src_reg)
*/
/* Absolute loads */
case BPF_LD | BPF_W | BPF_ABS:
func = (u8 *)CHOOSE_LOAD_FUNC(imm, sk_load_word);
goto common_load_abs;
case BPF_LD | BPF_H | BPF_ABS:
func = (u8 *)CHOOSE_LOAD_FUNC(imm, sk_load_half);
goto common_load_abs;
case BPF_LD | BPF_B | BPF_ABS:
func = (u8 *)CHOOSE_LOAD_FUNC(imm, sk_load_byte);
common_load_abs:
/*
* Load from [imm]
* Load into r4, which can just be passed onto
* skb load helpers as the second parameter
*/
PPC_LI32(4, imm);
goto common_load;
/* Indirect loads */
case BPF_LD | BPF_W | BPF_IND:
func = (u8 *)sk_load_word;
goto common_load_ind;
case BPF_LD | BPF_H | BPF_IND:
func = (u8 *)sk_load_half;
goto common_load_ind;
case BPF_LD | BPF_B | BPF_IND:
func = (u8 *)sk_load_byte;
common_load_ind:
/*
* Load from [src_reg + imm]
* Treat src_reg as a 32-bit value
*/
PPC_EXTSW(4, src_reg);
if (imm) {
if (imm >= -32768 && imm < 32768)
PPC_ADDI(4, 4, IMM_L(imm));
else {
PPC_LI32(b2p[TMP_REG_1], imm);
PPC_ADD(4, 4, b2p[TMP_REG_1]);
}
}
common_load:
ctx->seen |= SEEN_SKB;
ctx->seen |= SEEN_FUNC;
bpf_jit_emit_func_call(image, ctx, (u64)func);
/*
* Helper returns 'lt' condition on error, and an
* appropriate return value in BPF_REG_0
*/
PPC_BCC(COND_LT, exit_addr);
break;
/*
* Tail call
*/
@@ -988,6 +920,14 @@ common_load:
return 0;
}
struct powerpc64_jit_data {
struct bpf_binary_header *header;
u32 *addrs;
u8 *image;
u32 proglen;
struct codegen_context ctx;
};
struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
{
u32 proglen;
@@ -995,6 +935,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
u8 *image = NULL;
u32 *code_base;
u32 *addrs;
struct powerpc64_jit_data *jit_data;
struct codegen_context cgctx;
int pass;
int flen;
@@ -1002,8 +943,9 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
struct bpf_prog *org_fp = fp;
struct bpf_prog *tmp_fp;
bool bpf_blinded = false;
bool extra_pass = false;
if (!bpf_jit_enable)
if (!fp->jit_requested)
return org_fp;
tmp_fp = bpf_jit_blind_constants(org_fp);
@@ -1015,20 +957,41 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
fp = tmp_fp;
}
jit_data = fp->aux->jit_data;
if (!jit_data) {
jit_data = kzalloc(sizeof(*jit_data), GFP_KERNEL);
if (!jit_data) {
fp = org_fp;
goto out;
}
fp->aux->jit_data = jit_data;
}
flen = fp->len;
addrs = jit_data->addrs;
if (addrs) {
cgctx = jit_data->ctx;
image = jit_data->image;
bpf_hdr = jit_data->header;
proglen = jit_data->proglen;
alloclen = proglen + FUNCTION_DESCR_SIZE;
extra_pass = true;
goto skip_init_ctx;
}
addrs = kzalloc((flen+1) * sizeof(*addrs), GFP_KERNEL);
if (addrs == NULL) {
fp = org_fp;
goto out;
goto out_addrs;
}
memset(&cgctx, 0, sizeof(struct codegen_context));
/* Scouting faux-generate pass 0 */
if (bpf_jit_build_body(fp, 0, &cgctx, addrs)) {
if (bpf_jit_build_body(fp, 0, &cgctx, addrs, false)) {
/* We hit something illegal or unsupported. */
fp = org_fp;
goto out;
goto out_addrs;
}
/*
@@ -1046,9 +1009,10 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
bpf_jit_fill_ill_insns);
if (!bpf_hdr) {
fp = org_fp;
goto out;
goto out_addrs;
}
skip_init_ctx:
code_base = (u32 *)(image + FUNCTION_DESCR_SIZE);
/* Code generation passes 1-2 */
@@ -1056,7 +1020,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
/* Now build the prologue, body code & epilogue for real. */
cgctx.idx = 0;
bpf_jit_build_prologue(code_base, &cgctx);
bpf_jit_build_body(fp, code_base, &cgctx, addrs);
bpf_jit_build_body(fp, code_base, &cgctx, addrs, extra_pass);
bpf_jit_build_epilogue(code_base, &cgctx);
if (bpf_jit_enable > 1)
@@ -1082,10 +1046,20 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
fp->jited_len = alloclen;
bpf_flush_icache(bpf_hdr, (u8 *)bpf_hdr + (bpf_hdr->pages * PAGE_SIZE));
if (!fp->is_func || extra_pass) {
out_addrs:
kfree(addrs);
kfree(jit_data);
fp->aux->jit_data = NULL;
} else {
jit_data->addrs = addrs;
jit_data->ctx = cgctx;
jit_data->proglen = proglen;
jit_data->image = image;
jit_data->header = bpf_hdr;
}
out:
kfree(addrs);
if (bpf_blinded)
bpf_jit_prog_release_other(fp, fp == org_fp ? tmp_fp : org_fp);

2
arch/s390/include/asm/atomic.h
View File

@@ -90,7 +90,7 @@ static inline int atomic_cmpxchg(atomic_t *v, int old, int new)
return __atomic_cmpxchg(&v->counter, old, new);
}
static inline int __atomic_add_unless(atomic_t *v, int a, int u)
static inline int atomic_fetch_add_unless(atomic_t *v, int a, int u)
{
int c, old;
c = atomic_read(v);

4
arch/s390/include/asm/mmu_context.h
View File

@@ -44,6 +44,8 @@ static inline int init_new_context(struct task_struct *tsk,
mm->context.asce_limit = STACK_TOP_MAX;
mm->context.asce = __pa(mm->pgd) | _ASCE_TABLE_LENGTH |
_ASCE_USER_BITS | _ASCE_TYPE_REGION3;
/* pgd_alloc() did not account this pud */
mm_inc_nr_puds(mm);
break;
case -PAGE_SIZE:
/* forked 5-level task, set new asce with new_mm->pgd */
@@ -59,7 +61,7 @@ static inline int init_new_context(struct task_struct *tsk,
/* forked 2-level compat task, set new asce with new mm->pgd */
mm->context.asce = __pa(mm->pgd) | _ASCE_TABLE_LENGTH |
_ASCE_USER_BITS | _ASCE_TYPE_SEGMENT;
/* pgd_alloc() did not increase mm->nr_pmds */
/* pgd_alloc() did not account this pmd */
mm_inc_nr_pmds(mm);
}
crst_table_init((unsigned long *) mm->pgd, pgd_entry_type(mm));

5
arch/s390/net/bpf_jit_comp.c
View File

@@ -931,6 +931,11 @@ static noinline int bpf_jit_insn(struct bpf_jit *jit, struct bpf_prog *fp, int i
break;
}
break;
/*
* BPF_NOSPEC (speculation barrier)
*/
case BPF_ST | BPF_NOSPEC:
break;
/*
* BPF_ST(X)
*/

4
arch/sh/include/asm/atomic.h
View File

@@ -46,7 +46,7 @@
#define atomic_cmpxchg(v, o, n) (cmpxchg(&((v)->counter), (o), (n)))
/**
* __atomic_add_unless - add unless the number is a given value
* atomic_fetch_add_unless - add unless the number is a given value
* @v: pointer of type atomic_t
* @a: the amount to add to v...
* @u: ...unless v is equal to u.
@@ -54,7 +54,7 @@
* Atomically adds @a to @v, so long as it was not @u.
* Returns the old value of @v.
*/
static inline int __atomic_add_unless(atomic_t *v, int a, int u)
static inline int atomic_fetch_add_unless(atomic_t *v, int a, int u)
{
int c, old;
c = atomic_read(v);

2
arch/sparc/include/asm/atomic_32.h
View File

@@ -27,7 +27,7 @@ int atomic_fetch_or(int, atomic_t *);
int atomic_fetch_xor(int, atomic_t *);
int atomic_cmpxchg(atomic_t *, int, int);
int atomic_xchg(atomic_t *, int);
int __atomic_add_unless(atomic_t *, int, int);
int atomic_fetch_add_unless(atomic_t *, int, int);
void atomic_set(atomic_t *, int);
#define atomic_set_release(v, i) atomic_set((v), (i))

2
arch/sparc/include/asm/atomic_64.h
View File

@@ -89,7 +89,7 @@ static inline int atomic_xchg(atomic_t *v, int new)
return xchg(&v->counter, new);
}
static inline int __atomic_add_unless(atomic_t *v, int a, int u)
static inline int atomic_fetch_add_unless(atomic_t *v, int a, int u)
{
int c, old;
c = atomic_read(v);

4
arch/sparc/lib/atomic32.c
View File

@@ -95,7 +95,7 @@ int atomic_cmpxchg(atomic_t *v, int old, int new)
}
EXPORT_SYMBOL(atomic_cmpxchg);
int __atomic_add_unless(atomic_t *v, int a, int u)
int atomic_fetch_add_unless(atomic_t *v, int a, int u)
{
int ret;
unsigned long flags;
@@ -107,7 +107,7 @@ int __atomic_add_unless(atomic_t *v, int a, int u)
spin_unlock_irqrestore(ATOMIC_HASH(v), flags);
return ret;
}
EXPORT_SYMBOL(__atomic_add_unless);
EXPORT_SYMBOL(atomic_fetch_add_unless);
/* Atomic operations are already serializing */
void atomic_set(atomic_t *v, int i)

3
arch/sparc/mm/hugetlbpage.c
View File

@@ -397,7 +397,7 @@ static void hugetlb_free_pte_range(struct mmu_gather *tlb, pmd_t *pmd,
pmd_clear(pmd);
pte_free_tlb(tlb, token, addr);
atomic_long_dec(&tlb->mm->nr_ptes);
mm_dec_nr_ptes(tlb->mm);
}
static void hugetlb_free_pmd_range(struct mmu_gather *tlb, pud_t *pud,
@@ -472,6 +472,7 @@ static void hugetlb_free_pud_range(struct mmu_gather *tlb, pgd_t *pgd,
pud = pud_offset(pgd, start);
pgd_clear(pgd);
pud_free_tlb(tlb, pud, start);
mm_dec_nr_puds(tlb->mm);
}
void hugetlb_free_pgd_range(struct mmu_gather *tlb,

3
arch/sparc/net/bpf_jit_comp_64.c
View File

@@ -1317,6 +1317,9 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx)
emit(opcode | RS1(src) | rs2 | RD(dst), ctx);
break;
}
/* speculation barrier */
case BPF_ST | BPF_NOSPEC:
break;
/* ST: *(size *)(dst + off) = imm */
case BPF_ST | BPF_MEM | BPF_W:
case BPF_ST | BPF_MEM | BPF_H:

4
arch/tile/include/asm/atomic_32.h
View File

@@ -72,7 +72,7 @@ static inline int atomic_add_return(int i, atomic_t *v)
}
/**
* __atomic_add_unless - add unless the number is already a given value
* atomic_fetch_add_unless - add unless the number is already a given value
* @v: pointer of type atomic_t
* @a: the amount to add to v...
* @u: ...unless v is equal to u.
@@ -80,7 +80,7 @@ static inline int atomic_add_return(int i, atomic_t *v)
* Atomically adds @a to @v, so long as @v was not already @u.
* Returns the old value of @v.
*/
static inline int __atomic_add_unless(atomic_t *v, int a, int u)
static inline int atomic_fetch_add_unless(atomic_t *v, int a, int u)
{
smp_mb(); /* barrier for proper semantics */
return _atomic_xchg_add_unless(&v->counter, a, u);

2
arch/tile/include/asm/atomic_64.h
View File

@@ -97,7 +97,7 @@ static inline void atomic_xor(int i, atomic_t *v)
} while (guess != oldval);
}
static inline int __atomic_add_unless(atomic_t *v, int a, int u)
static inline int atomic_fetch_add_unless(atomic_t *v, int a, int u)
{
int guess, oldval = v->counter;
do {

2
arch/unicore32/mm/pgd.c
View File

@@ -97,7 +97,7 @@ void free_pgd_slow(struct mm_struct *mm, pgd_t *pgd)
pte = pmd_pgtable(*pmd);
pmd_clear(pmd);
pte_free(mm, pte);
atomic_long_dec(&mm->nr_ptes);
mm_dec_nr_ptes(mm);
pmd_free(mm, pmd);
mm_dec_nr_pmds(mm);
free:

2
arch/x86/Kconfig
View File

@@ -60,6 +60,7 @@ config X86
select ARCH_HAS_UACCESS_FLUSHCACHE if X86_64
select ARCH_HAS_SET_MEMORY
select ARCH_HAS_SG_CHAIN
select ARCH_HAS_SET_DIRECT_MAP
select ARCH_HAS_STRICT_KERNEL_RWX
select ARCH_HAS_STRICT_MODULE_RWX
select ARCH_HAS_UBSAN_SANITIZE_ALL
@@ -155,6 +156,7 @@ config X86
select HAVE_KERNEL_XZ
select HAVE_KPROBES
select HAVE_KPROBES_ON_FTRACE
select HAVE_FUNCTION_ERROR_INJECTION
select HAVE_KRETPROBES
select HAVE_KVM
select HAVE_LIVEPATCH if X86_64

4
arch/x86/include/asm/atomic.h
View File

@@ -254,7 +254,7 @@ static inline int arch_atomic_fetch_xor(int i, atomic_t *v)
}
/**
* __arch_atomic_add_unless - add unless the number is already a given value
* arch_atomic_fetch_add_unless - add unless the number is already a given value
* @v: pointer of type atomic_t
* @a: the amount to add to v...
* @u: ...unless v is equal to u.
@@ -262,7 +262,7 @@ static inline int arch_atomic_fetch_xor(int i, atomic_t *v)
* Atomically adds @a to @v, so long as @v was not already @u.
* Returns the old value of @v.
*/
static __always_inline int __arch_atomic_add_unless(atomic_t *v, int a, int u)
static __always_inline int arch_atomic_fetch_add_unless(atomic_t *v, int a, int u)
{
int c = arch_atomic_read(v);

13
arch/x86/include/asm/error-injection.h Normal file
View File

@@ -0,0 +1,13 @@
/* SPDX-License-Identifier: GPL-2.0 */
#ifndef _ASM_ERROR_INJECTION_H
#define _ASM_ERROR_INJECTION_H
#include <linux/compiler.h>
#include <linux/linkage.h>
#include <asm/ptrace.h>
#include <asm-generic/error-injection.h>
asmlinkage void just_return_func(void);
void override_function_with_return(struct pt_regs *regs);
#endif /* _ASM_ERROR_INJECTION_H */

4
arch/x86/include/asm/kprobes.h
View File

@@ -67,6 +67,10 @@ extern const int kretprobe_blacklist_size;
void arch_remove_kprobe(struct kprobe *p);
asmlinkage void kretprobe_trampoline(void);
#ifdef CONFIG_KPROBES_ON_FTRACE
extern void arch_ftrace_kprobe_override_function(struct pt_regs *regs);
#endif
/* Architecture specific copy of original instruction*/
struct arch_specific_insn {
/* copy of the original instruction */

5
arch/x86/include/asm/ptrace.h
View File

@@ -109,6 +109,11 @@ static inline unsigned long regs_return_value(struct pt_regs *regs)
return regs->ax;
}
static inline void regs_set_return_value(struct pt_regs *regs, unsigned long rc)
{
regs->ax = rc;
}
/*
* user_mode(regs) determines whether a register set came from user
* mode. On x86_32, this is true if V8086 mode was enabled OR if the

3
arch/x86/include/asm/set_memory.h
View File

@@ -84,6 +84,9 @@ int set_pages_nx(struct page *page, int numpages);
int set_pages_ro(struct page *page, int numpages);
int set_pages_rw(struct page *page, int numpages);
int set_direct_map_invalid_noflush(struct page *page);
int set_direct_map_default_noflush(struct page *page);
extern int kernel_set_to_readonly;
void set_kernel_text_rw(void);
void set_kernel_text_ro(void);

14
arch/x86/kernel/kprobes/ftrace.c
View File

@@ -102,3 +102,17 @@ int arch_prepare_kprobe_ftrace(struct kprobe *p)
p->ainsn.boostable = false;
return 0;
}
asmlinkage void override_func(void);
asm(
".type override_func, @function\n"
"override_func:\n"
" ret\n"
".size override_func, .-override_func\n"
);
void arch_ftrace_kprobe_override_function(struct pt_regs *regs)
{
regs->ip = (unsigned long)&override_func;
}
NOKPROBE_SYMBOL(arch_ftrace_kprobe_override_function);

12
arch/x86/kvm/x86.c
View File

@@ -1610,7 +1610,7 @@ void kvm_write_tsc(struct kvm_vcpu *vcpu, struct msr_data *msr)
raw_spin_lock_irqsave(&kvm->arch.tsc_write_lock, flags);
offset = kvm_compute_tsc_offset(vcpu, data);
ns = ktime_get_boot_ns();
ns = ktime_get_boottime_ns();
elapsed = ns - kvm->arch.last_tsc_nsec;
if (vcpu->arch.virtual_tsc_khz) {
@@ -1927,7 +1927,7 @@ u64 get_kvmclock_ns(struct kvm *kvm)
spin_lock(&ka->pvclock_gtod_sync_lock);
if (!ka->use_master_clock) {
spin_unlock(&ka->pvclock_gtod_sync_lock);
return ktime_get_boot_ns() + ka->kvmclock_offset;
return ktime_get_boottime_ns() + ka->kvmclock_offset;
}
hv_clock.tsc_timestamp = ka->master_cycle_now;
@@ -1943,7 +1943,7 @@ u64 get_kvmclock_ns(struct kvm *kvm)
&hv_clock.tsc_to_system_mul);
ret = __pvclock_read_cycles(&hv_clock, rdtsc());
} else
ret = ktime_get_boot_ns() + ka->kvmclock_offset;
ret = ktime_get_boottime_ns() + ka->kvmclock_offset;
put_cpu();
@@ -2042,7 +2042,7 @@ static int kvm_guest_time_update(struct kvm_vcpu *v)
}
if (!use_master_clock) {
host_tsc = rdtsc();
kernel_ns = ktime_get_boot_ns();
kernel_ns = ktime_get_boottime_ns();
}
tsc_timestamp = kvm_read_l1_tsc(v, host_tsc);
@@ -8172,7 +8172,7 @@ int kvm_arch_hardware_enable(void)
* before any KVM threads can be running. Unfortunately, we can't
* bring the TSCs fully up to date with real time, as we aren't yet far
* enough into CPU bringup that we know how much real time has actually
* elapsed; our helper function, ktime_get_boot_ns() will be using boot
* elapsed; our helper function, ktime_get_boottime_ns() will be using boot
* variables that haven't been updated yet.
*
* So we simply find the maximum observed TSC above, then record the
@@ -8411,7 +8411,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
mutex_init(&kvm->arch.hyperv.hv_lock);
spin_lock_init(&kvm->arch.pvclock_gtod_sync_lock);
kvm->arch.kvmclock_offset = -ktime_get_boot_ns();
kvm->arch.kvmclock_offset = -ktime_get_boottime_ns();
pvclock_update_vm_gtod_copy(kvm);
INIT_DELAYED_WORK(&kvm->arch.kvmclock_update_work, kvmclock_update_fn);

1
arch/x86/lib/Makefile
View File

@@ -38,6 +38,7 @@ lib-y += memcpy_$(BITS).o
lib-$(CONFIG_RWSEM_XCHGADD_ALGORITHM) += rwsem.o
lib-$(CONFIG_INSTRUCTION_DECODER) += insn.o inat.o
lib-$(CONFIG_RANDOMIZE_BASE) += kaslr.o
lib-$(CONFIG_FUNCTION_ERROR_INJECTION) += error-inject.o
lib-$(CONFIG_RETPOLINE) += retpoline.o
obj-y += msr.o msr-reg.o msr-reg-export.o hweight.o

19
arch/x86/lib/error-inject.c Normal file
View File

@@ -0,0 +1,19 @@
// SPDX-License-Identifier: GPL-2.0
#include <linux/error-injection.h>
#include <linux/kprobes.h>
asmlinkage void just_return_func(void);
asm(
".type just_return_func, @function\n"
"just_return_func:\n"
" ret\n"
".size just_return_func, .-just_return_func\n"
);
void override_function_with_return(struct pt_regs *regs)
{
regs->ip = (unsigned long)&just_return_func;
}
NOKPROBE_SYMBOL(override_function_with_return);

14
arch/x86/mm/pageattr.c
View File

@@ -1976,8 +1976,6 @@ int set_pages_rw(struct page *page, int numpages)
return set_memory_rw(addr, numpages);
}
#ifdef CONFIG_DEBUG_PAGEALLOC
static int __set_pages_p(struct page *page, int numpages)
{
unsigned long tempaddr = (unsigned long) page_address(page);
@@ -2016,6 +2014,17 @@ static int __set_pages_np(struct page *page, int numpages)
return __change_page_attr_set_clr(&cpa, 0);
}
int set_direct_map_invalid_noflush(struct page *page)
{
return __set_pages_np(page, 1);
}
int set_direct_map_default_noflush(struct page *page)
{
return __set_pages_p(page, 1);
}
#ifdef CONFIG_DEBUG_PAGEALLOC
void __kernel_map_pages(struct page *page, int numpages, int enable)
{
if (PageHighMem(page))
@@ -2049,7 +2058,6 @@ void __kernel_map_pages(struct page *page, int numpages, int enable)
}
#ifdef CONFIG_HIBERNATION
bool kernel_page_present(struct page *page)
{
unsigned int level;

5
arch/x86/mm/pgtable.c
View File

@@ -114,13 +114,12 @@ static inline void pgd_list_del(pgd_t *pgd)
static void pgd_set_mm(pgd_t *pgd, struct mm_struct *mm)
{
BUILD_BUG_ON(sizeof(virt_to_page(pgd)->index) < sizeof(mm));
virt_to_page(pgd)->index = (pgoff_t)mm;
virt_to_page(pgd)->pt_mm = mm;
}
struct mm_struct *pgd_page_get_mm(struct page *page)
{
return (struct mm_struct *)page->index;
return page->pt_mm;
}
static void pgd_ctor(struct mm_struct *mm, pgd_t *pgd)

10
arch/x86/mm/pti.c
View File

@@ -1,15 +1,7 @@
// SPDX-License-Identifier: GPL-2.0-only
/*
* Copyright(c) 2017 Intel Corporation. All rights reserved.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of version 2 of the GNU General Public License as
* published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but
* WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* General Public License for more details.
*
* This code is based in part on work published here:
*
* https://github.com/IAIK/KAISER

9
arch/x86/net/bpf_jit_comp.c
View File

@@ -750,6 +750,13 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image,
}
break;
/* speculation barrier */
case BPF_ST | BPF_NOSPEC:
if (boot_cpu_has(X86_FEATURE_XMM2))
/* Emit 'lfence' */
EMIT3(0x0F, 0xAE, 0xE8);
break;
/* ST: *(u8*)(dst_reg + off) = imm */
case BPF_ST | BPF_MEM | BPF_B:
if (is_ereg(dst_reg))
@@ -1197,8 +1204,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
addrs[i] = proglen;
}
ctx.cleanup_addr = proglen;
skip_init_addrs:
/* JITed image shrinks with every pass and the loop iterates
* until the image stops shrinking. Very large bpf programs
* may converge on the last pass. In such case do one more

4
arch/xtensa/include/asm/atomic.h
View File

@@ -275,7 +275,7 @@ ATOMIC_OPS(xor)
#define atomic_xchg(v, new) (xchg(&((v)->counter), new))
/**
* __atomic_add_unless - add unless the number is a given value
* atomic_fetch_add_unless - add unless the number is a given value
* @v: pointer of type atomic_t
* @a: the amount to add to v...
* @u: ...unless v is equal to u.
@@ -283,7 +283,7 @@ ATOMIC_OPS(xor)
* Atomically adds @a to @v, so long as it was not @u.
* Returns the old value of @v.
*/
static __inline__ int __atomic_add_unless(atomic_t *v, int a, int u)
static __inline__ int atomic_fetch_add_unless(atomic_t *v, int a, int u)
{
int c, old;
c = atomic_read(v);

4
crypto/lrw.c
View File

@@ -189,7 +189,7 @@ static int post_crypt(struct skcipher_request *req)
if (rctx->dst != sg) {
rctx->dst[0] = *sg;
sg_unmark_end(rctx->dst);
scatterwalk_crypto_chain(rctx->dst, sg_next(sg), 0, 2);
scatterwalk_crypto_chain(rctx->dst, sg_next(sg), 2);
}
rctx->dst[0].length -= offset - sg->offset;
rctx->dst[0].offset = offset;
@@ -266,7 +266,7 @@ static int pre_crypt(struct skcipher_request *req)
if (rctx->src != sg) {
rctx->src[0] = *sg;
sg_unmark_end(rctx->src);
scatterwalk_crypto_chain(rctx->src, sg_next(sg), 0, 2);
scatterwalk_crypto_chain(rctx->src, sg_next(sg), 2);
}
rctx->src[0].length -= offset - sg->offset;
rctx->src[0].offset = offset;

2
crypto/scatterwalk.c
View File

@@ -91,7 +91,7 @@ struct scatterlist *scatterwalk_ffwd(struct scatterlist dst[2],
sg_init_table(dst, 2);
sg_set_page(dst, sg_page(src), src->length - len, src->offset + len);
scatterwalk_crypto_chain(dst, sg_next(src), 0, 2);
scatterwalk_crypto_chain(dst, sg_next(src), 2);
return dst;
}

4
crypto/xts.c
View File

@@ -138,7 +138,7 @@ static int post_crypt(struct skcipher_request *req)
if (rctx->dst != sg) {
rctx->dst[0] = *sg;
sg_unmark_end(rctx->dst);
scatterwalk_crypto_chain(rctx->dst, sg_next(sg), 0, 2);
scatterwalk_crypto_chain(rctx->dst, sg_next(sg), 2);
}
rctx->dst[0].length -= offset - sg->offset;
rctx->dst[0].offset = offset;
@@ -204,7 +204,7 @@ static int pre_crypt(struct skcipher_request *req)
if (rctx->src != sg) {
rctx->src[0] = *sg;
sg_unmark_end(rctx->src);
scatterwalk_crypto_chain(rctx->src, sg_next(sg), 0, 2);
scatterwalk_crypto_chain(rctx->src, sg_next(sg), 2);
}
rctx->src[0].length -= offset - sg->offset;
rctx->src[0].offset = offset;

10
drivers/acpi/nfit/core.c
View File

@@ -1,14 +1,6 @@
// SPDX-License-Identifier: GPL-2.0-only
/*
* Copyright(c) 2013-2015 Intel Corporation. All rights reserved.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of version 2 of the GNU General Public License as
* published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but
* WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* General Public License for more details.
*/
#include <linux/list_sort.h>
#include <linux/libnvdimm.h>

10
drivers/acpi/nfit/mce.c
View File

@@ -1,16 +1,8 @@
// SPDX-License-Identifier: GPL-2.0-only
/*
* NFIT - Machine Check Handler
*
* Copyright(c) 2013-2016 Intel Corporation. All rights reserved.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of version 2 of the GNU General Public License as
* published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but
* WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* General Public License for more details.
*/
#include <linux/notifier.h>
#include <linux/acpi.h>

10
drivers/acpi/nfit/nfit.h
View File

@@ -1,16 +1,8 @@
/* SPDX-License-Identifier: GPL-2.0-only */
/*
* NVDIMM Firmware Interface Table - NFIT
*
* Copyright(c) 2013-2015 Intel Corporation. All rights reserved.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of version 2 of the GNU General Public License as
* published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but
* WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* General Public License for more details.
*/
#ifndef __NFIT_H__
#define __NFIT_H__

2
drivers/block/rbd.c
View File

@@ -60,7 +60,7 @@ static int atomic_inc_return_safe(atomic_t *v)
{
unsigned int counter;
counter = (unsigned int)__atomic_add_unless(v, 1, 0);
counter = (unsigned int)atomic_fetch_add_unless(v, 1, 0);
if (counter <= (unsigned int)INT_MAX)
return (int)counter;

10
drivers/dax/dax-private.h
View File

@@ -1,14 +1,6 @@
/* SPDX-License-Identifier: GPL-2.0-only */
/*
* Copyright(c) 2016 Intel Corporation. All rights reserved.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of version 2 of the GNU General Public License as
* published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but
* WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* General Public License for more details.
*/
#ifndef __DAX_PRIVATE_H__
#define __DAX_PRIVATE_H__

Some files were not shown because too many files have changed in this diff Show More