[ Upstream commit efab25894a41a920d9581183741e7fadba00719c ]
PSIL_EP_NATIVE endpoints may not have PEER registers for BCNT and thus
udma_decrement_byte_counters() should not try to decrement these counters.
This fixes the issue of crypto IPERF testing where the client side (EVM)
hangs without transfer of packets to the server side, seen since this
function was added.
Fixes: 7c94dcfa8fcf ("dmaengine: ti: k3-udma: Reset UDMA_CHAN_RT byte counters to prevent overflow")
Signed-off-by: Jayesh Choudhary <j-choudhary@ti.com>
Acked-by: Peter Ujfalusi <peter.ujfalusi@gmail.com>
Link: https://lore.kernel.org/r/20221128085005.489964-1-j-choudhary@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
[ Upstream commit 7c94dcfa8fcff2dba53915f1dabfee49a3df8b88 ]
UDMA_CHAN_RT_*BCNT_REG stores the real-time channel bytecount statistics.
These registers are 32-bit hardware counters and the driver uses these
counters to monitor the operational progress status for a channel, when
transferring more than 4GB of data it was observed that these counters
overflow and completion calculation of a operation gets affected and the
transfer hangs indefinitely.
This commit adds changes to decrease the byte count for every complete
transaction so that these registers never overflow and the proper byte
count statistics is maintained for ongoing transaction by the RT counters.
Earlier uc->bcnt used to maintain a count of the completed bytes at driver
side, since the RT counters maintain the statistics of current transaction
now, the maintenance of uc->bcnt is not necessary.
Signed-off-by: Vaishnav Achath <vaishnav.a@ti.com>
Acked-by: Peter Ujfalusi <peter.ujfalusi@gmail.com>
Link: https://lore.kernel.org/r/20220802054835.19482-1-vaishnav.a@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
[ Upstream commit f9fdb0b86f087c2b7f6c6168dd0985a3c1eda87e ]
We should call of_node_put() for the reference returned by
of_parse_phandle() in fail path or when it is not used anymore.
Here we only need to move the of_node_put() before the check.
Fixes: d702419134 ("dmaengine: ti: k3-udma: Add glue layer for non DMAengine users")
Signed-off-by: Liang He <windhl@126.com>
Acked-by: Peter Ujfalusi <peter.ujfalusi@gmail.com>
Link: https://lore.kernel.org/r/20220720073234.1255474-1-windhl@126.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
commit eb91224e47ec33a0a32c9be0ec0fcb3433e555fd upstream.
udma_get_*() checks if rchan/tchan/rflow is already allocated by checking
if it has a NON NULL value. For the error cases, rchan/tchan/rflow will
have error value and udma_get_*() considers this as already allocated
(PASS) since the error values are NON NULL. This results in NULL pointer
dereference error while de-referencing rchan/tchan/rflow.
Reset the value of rchan/tchan/rflow to NULL if a channel request fails.
CC: stable@vger.kernel.org
Acked-by: Peter Ujfalusi <peter.ujfalusi@gmail.com>
Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Link: https://lore.kernel.org/r/20211031032411.27235-3-kishon@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
commit 5c6c6d60e4b489308ae4da8424c869f7cc53cd12 upstream.
bcdma_get_*() checks if bchan is already allocated by checking if it
has a NON NULL value. For the error cases, bchan will have error value
and bcdma_get_*() considers this as already allocated (PASS) since the
error values are NON NULL. This results in NULL pointer dereference
error while de-referencing bchan.
Reset the value of bchan to NULL if a channel request fails.
CC: stable@vger.kernel.org
Acked-by: Peter Ujfalusi <peter.ujfalusi@gmail.com>
Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
Link: https://lore.kernel.org/r/20211031032411.27235-2-kishon@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Fix the following fallthrough warning:
drivers/dma/ti/k3-udma.c:4951:2: warning: unannotated fall-through between switch labels [-Wimplicit-fallthrough]
Reported-by: Nathan Chancellor <nathan@kernel.org>
Signed-off-by: Gustavo A. R. Silva <gustavoars@kernel.org>
There's no need to restore DMA context on CPU_CLUSTER_PM_ENTER_FAILED as
the DMA context won't be lost on errors.
Note that this does not cause invalid context restore as we already check
for busy DMA with omap_dma_busy() in CPU_CLUSTER_PM_ENTER, and block any
deeper idle states for the SoC by returning NOTIFY_BAD if busy.
If other drivers block deeper idle states with cpu_pm, we now just do a
pointless restore, but only if dma was not busy on CPU_CLUSTER_PM_ENTER.
Let's update the CPU_CLUSTER_PM_ENTER_FAILED handling for correctness,
and add a comment.
Cc: Aaro Koskinen <aaro.koskinen@iki.fi>
Cc: Adam Ford <aford173@gmail.com>
Cc: Andreas Kemnade <andreas@kemnade.info>
Cc: Peter Ujfalusi <peter.ujfalusi@gmail.com>
Signed-off-by: Tony Lindgren <tony@atomide.com>
Acked-by: Peter Ujfalusi <peter.ujfalusi@gmail.com>
Link: https://lore.kernel.org/r/20210518074347.16908-1-tony@atomide.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The UDMA and BCDMA can provide higher throughput if the burst_size of the
channel is changed from it's default (which is 64 bytes) for Ultra-high
and high capacity channels.
This performance benefit is even more visible when the buffers are aligned
with the burst_size configuration.
The am654 does not have a way to change the burst size, but it is using
64 bytes burst, so increasing the copy_align from 8 bytes to 64 (and
clients taking that into account) can increase the throughput as well.
Numbers gathered on j721e:
echo 8000000 > /sys/module/dmatest/parameters/test_buf_size
echo 2000 > /sys/module/dmatest/parameters/timeout
echo 50 > /sys/module/dmatest/parameters/iterations
echo 1 > /sys/module/dmatest/parameters/max_channels
Prior this patch: ~1.3 GB/s
After this patch: ~1.8 GB/s
with 1 byte alignment: ~1.7 GB/s
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@gmail.com>
Tested-by: Kishon Vijay Abraham I <kishon@ti.com>
Link: https://lore.kernel.org/r/20210113114923.9231-3-peter.ujfalusi@gmail.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
BCDMA RX channels have one flow per channel, therefore set the rflow_cnt
to rchan_cnt.
Without this patch, request for BCDMA RX channel allocation fails as
rflow_cnt is 0 thus fails to reserve a rflow for the channel.
Fixes: 8844898028 ("dmaengine: ti: k3-udma: Add support for BCDMA channel TPL handling")
Signed-off-by: Vignesh Raghavendra <vigneshr@ti.com>
Acked-by: Peter Ujfalusi <peter.ujfalusi@gmail.com>
Link: https://lore.kernel.org/r/20210112141403.30286-1-vigneshr@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Pull dmaengine updates from Vinod Koul:
"The last dmaengine updates for this year :)
This contains couple of new drivers, new device support and updates to
bunch of drivers.
New drivers/devices:
- Qualcomm ADM driver
- Qualcomm GPI driver
- Allwinner A100 DMA support
- Microchip Sama7g5 support
- Mediatek MT8516 apdma
Updates:
- more updates to idxd driver and support for IAX config
- runtime PM support for dw driver
- TI drivers"
* tag 'dmaengine-5.11-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/vkoul/dmaengine: (75 commits)
soc: ti: k3-ringacc: Use correct error casting in k3_ringacc_dmarings_init
dmaengine: ti: k3-udma-glue: Add support for K3 PKTDMA
dmaengine: ti: k3-udma: Initial support for K3 PKTDMA
dmaengine: ti: k3-udma: Add support for BCDMA channel TPL handling
dmaengine: ti: k3-udma: Initial support for K3 BCDMA
soc: ti: k3-ringacc: add AM64 DMA rings support.
dmaengine: ti: Add support for k3 event routers
dmaengine: ti: k3-psil: Add initial map for AM64
dmaengine: ti: k3-psil: Extend psil_endpoint_config for K3 PKTDMA
dt-bindings: dma: ti: Add document for K3 PKTDMA
dt-bindings: dma: ti: Add document for K3 BCDMA
dmaengine: dmatest: Use dmaengine_get_dma_device
dmaengine: doc: client: Update for dmaengine_get_dma_device() usage
dmaengine: Add support for per channel coherency handling
dmaengine: of-dma: Add support for optional router configuration callback
dmaengine: ti: k3-udma-glue: Configure the dma_dev for rings
dmaengine: ti: k3-udma-glue: Get the ringacc from udma_dev
dmaengine: ti: k3-udma-glue: Add function to get device pointer for DMA API
dmaengine: ti: k3-udma: Add support for second resource range from sysfw
dmaengine: ti: k3-udma: Wait for peer teardown completion if supported
...
This commit adds support for PKTDMA in k3-udma glue driver. Use new
psil_endpoint_config struct to get static data for a given channel or a
flow during setup. Make sure that the RX flows being mapped to a RX
channel is within the range of flows that is been allocated to that RX
channel.
Signed-off-by: Vignesh Raghavendra <vigneshr@ti.com>
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20201208090440.31792-21-peter.ujfalusi@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
One of the DMAs introduced with AM64 is the Packet DMA (PKTDMA).
It serves similar purpose as K3 UDMAP channels in packet mode, but with
notable differences, like tflow support and channels being allocated to
service specific peripherals.
The rings for the PKTDMA is integrated within the DMA itself instead of
using rings from the general purpose ringacc.
PKTDMA can be used to service PSI-L peripherals, similarly to
K3 UDMA channels.
Most of the driver code can be reused for PKTDMA tchan/rchan support but
new setup and allocation functions are needed to handle the differences
between the DMAs.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20201208090440.31792-20-peter.ujfalusi@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Unlike UDMAP the BCDMA defines the channel TPL levels per channel type.
In UDMAP the number of high and ultra-high channels applies to both tchan
and rchan.
BCDMA defines the TPL per channel types: bchan, tchan and rchan can have
different number of high and ultra-high channels.
In order to support BCDMA channel TPL we need to move the tpl information
as per channel type property for the DMAs.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20201208090440.31792-19-peter.ujfalusi@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
One of the DMAs introduced with AM64 is the Block Copy DMA (BCDMA).
It serves similar purpose as K3 UDMAP channels in TR mode.
The rings for the BCDMA is integrated within the DMA itself instead of
using rings from the general purpose ringacc.
A BCDMA have two different type of channels:
- Block Copy Channels (bchan)
- Split Channels (tchan and rchan)
tchan and rchan can be used to service PSI-L peripherals, similarly to
K3 UDMA channels.
bchan can be only used for block copy operation (TR type15) like the
paired K3 UDMA tchan/rchan configured in block copy mode.
bchans can be also used to service peripherals directly if an external
trigger is selected for the channel.
Most of the driver code can be reused for BCDMA bchan/tchan/rchan support
but new setup and allocation functions are needed to handle the
differences between the DMAs.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20201208090440.31792-18-peter.ujfalusi@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Rings in RING mode should be using the DMA device for DMA API as in this
mode the ringacc will not access the ring memory in any ways, but the DMA
is.
Fix up the ring configuration and set the dma_dev unconditionally and let
the ringacc driver to select the correct device to use for DMA API.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Reviewed-by: Grygorii Strashko <grygorii.strashko@ti.com>
Link: https://lore.kernel.org/r/20201208090440.31792-7-peter.ujfalusi@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The driver can match only via the DT table so the main table should be
always used and the of_match_ptr does not have any sense (this also
allows ACPI matching via PRP0001, even though it is not relevant here).
The secondary match of_device_id tables (passed to of_match_node) should
be marked as maybe unused to fix compile testing (!CONFIG_OF on x86_64)
warnings:
drivers/dma/ti/dma-crossbar.c:125:34: warning:
‘ti_am335x_master_match’ defined but not used [-Wunused-const-variable=]
drivers/dma/ti/dma-crossbar.c:22:34: warning:
‘ti_dma_xbar_match’ defined but not used [-Wunused-const-variable=]
Signed-off-by: Krzysztof Kozlowski <krzk@kernel.org>
Link: https://lore.kernel.org/r/20201120162303.482126-6-krzk@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
We now use cpu_pm for saving and restoring device context for deeper SoC
idle states. But for omap3, we must also block idle if SDMA is busy.
If we don't block idle when SDMA is busy, we eventually end up saving and
restoring SDMA register state on PER domain idle while SDMA is active and
that causes at least audio playback to fail.
Fixes: 4c74ecf792 ("dmaengine: ti: omap-dma: Add device tree match data and use it for cpu_pm")
Reported-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Tony Lindgren <tony@atomide.com>
Tested-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Acked-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20201109154013.11950-1-tony@atomide.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The NAVSS UDMA will stuck if target IP module is disabled by PM while PSI-L
threads are paired UDMA<->IP and no further transfers is possible. This
could be the case for IPs J721E Main CPSW (cpsw9g).
Hence, to avoid such situation do PSI-L threads pairing only when UDMA
channel is going to be enabled as at this time DMA consumer module expected
to be active already.
Signed-off-by: Grygorii Strashko <grygorii.strashko@ti.com>
Acked-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20201030203000.4281-1-grygorii.strashko@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
gcc warns about a mismatch argument type when passing
'false' into a function that expects an enum:
drivers/dma/ti/k3-udma-private.c: In function 'xudma_tchan_get':
drivers/dma/ti/k3-udma-private.c:86:34: warning: implicit conversion from 'enum <anonymous>' to 'enum udma_tp_level' [-Wenum-conversion]
86 | return __udma_reserve_##res(ud, false, id); \
| ^~~~~
drivers/dma/ti/k3-udma-private.c:95:1: note: in expansion of macro 'XUDMA_GET_PUT_RESOURCE'
95 | XUDMA_GET_PUT_RESOURCE(tchan);
| ^~~~~~~~~~~~~~~~~~~~~~
In this case, false has the same numerical value as
UDMA_TP_NORMAL, so passing that is most likely the correct
way to avoid the warning without changing the behavior.
Fixes: d702419134 ("dmaengine: ti: k3-udma: Add glue layer for non DMAengine users")
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20201026160123.3704531-1-arnd@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Pull ARM SoC-related driver updates from Olof Johansson:
"Various driver updates for platforms. A bulk of this is smaller fixes
or cleanups, but some of the new material this time around is:
- Support for Nvidia Tegra234 SoC
- Ring accelerator support for TI AM65x
- PRUSS driver for TI platforms
- Renesas support for R-Car V3U SoC
- Reset support for Cortex-M4 processor on i.MX8MQ
There are also new socinfo entries for a handful of different SoCs and
platforms"
* tag 'armsoc-drivers' of git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc: (131 commits)
drm/mediatek: reduce clear event
soc: mediatek: cmdq: add clear option in cmdq_pkt_wfe api
soc: mediatek: cmdq: add jump function
soc: mediatek: cmdq: add write_s_mask value function
soc: mediatek: cmdq: add write_s value function
soc: mediatek: cmdq: add read_s function
soc: mediatek: cmdq: add write_s_mask function
soc: mediatek: cmdq: add write_s function
soc: mediatek: cmdq: add address shift in jump
soc: mediatek: mtk-infracfg: Fix kerneldoc
soc: amlogic: pm-domains: use always-on flag
reset: sti: reset-syscfg: fix struct description warnings
reset: imx7: add the cm4 reset for i.MX8MQ
dt-bindings: reset: imx8mq: add m4 reset
reset: Fix and extend kerneldoc
reset: reset-zynqmp: Added support for Versal platform
dt-bindings: reset: Updated binding for Versal reset driver
reset: imx7: Support module build
soc: fsl: qe: Remove unnessesary check in ucc_set_tdm_rxtx_clk
soc: fsl: qman: convert to use be32_add_cpu()
...
Now the K3 UDMA glue layer enable functions perform RMW operation on UDMA
RX/TX RT_CTL registers to set EN bit and enable channel, which is
incorrect, because only EN bit has to be set in those registers to enable
channel (all other bits should be cleared 0).
More over, this causes issues when bootloader leaves UDMA channel RX/TX
RT_CTL registers in incorrect state - TDOWN bit set, for example. As
result, UDMA channel will just perform teardown right after it's enabled.
Hence, fix it by writing correct values (EN=1) directly in UDMA channel
RX/TX RT_CTL registers in k3_udma_glue_enable_tx/rx_chn() functions.
Fixes: d702419134 ("dmaengine: ti: k3-udma: Add glue layer for non DMAengine users")
Signed-off-by: Grygorii Strashko <grygorii.strashko@ti.com>
Acked-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200916120955.7963-1-grygorii.strashko@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The original commit mixed up the forward and completion ring IDs for the
rx flow configuration.
Acked-by: Vinod Koul <vkoul@kernel.org>
Reviewed-by: Grygorii Strashko <grygorii.strashko@ti.com>
Fixes: 4927b1ab20 ("dmaengine: ti: k3-udma: Switch to k3_ringacc_request_rings_pair")
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: Santosh Shilimkar <santosh.shilimkar@oracle.com>
Use separate data for SoC dependent parameters. These parameters depends
on the DMA integration (either in HW or in SYSFW), the DMA controller
itself remains compatible with either the am654 or j721e variant.
j7200 have the same DMA as j721e with different number of channels, which
can be queried from HW, but SYSFW defines different rchan_oes_offset
number for j7200 (0x80) compared to j721e (0x400).
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200910124329.21206-1-peter.ujfalusi@ti.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The TR which needs to be initialized for the next sg entry is indexed by
tr_idx and not by the running i counter.
In case any sub element in the SG needs more than one TR, the code would
corrupt an already configured TR.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
Link: https://lore.kernel.org/r/20200824120108.9178-1-peter.ujfalusi@ti.com
Fixes: 6cf668a4ef ("dmaengine: ti: k3-udma: Use the TR counter helper for slave_sg and cyclic")
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The of_device_id is included unconditionally by of.h header and used
in the driver as well. Remove of_match_ptr to fix W=1 compile test
warning with !CONFIG_OF:
drivers/dma/ti/omap-dma.c:1892:34: warning: 'omap_dma_match' defined but not used [-Wunused-const-variable=]
1892 | static const struct of_device_id omap_dma_match[] = {
Signed-off-by: Krzysztof Kozlowski <krzk@kernel.org>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Link: https://lore.kernel.org/r/20200728170939.28278-1-krzk@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Pull dmaengine updates from Vinod Koul:
"Core:
- Support out of order dma completion
- Support for repeating transaction
New controllers:
- Support for Actions S700 DMA engine
- Renesas R8A774E1, r8a7742 controller binding
- New driver for Xilinx DPDMA controller
Other:
- Support of out of order dma completion in idxd driver
- W=1 warning cleanup of subsystem
- Updates to ti-k3-dma, dw, idxd drivers"
* tag 'dmaengine-5.9-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/vkoul/dmaengine: (68 commits)
dmaengine: dw: Don't include unneeded header to platform data header
dmaengine: Actions: Add support for S700 DMA engine
dmaengine: Actions: get rid of bit fields from dma descriptor
dt-bindings: dmaengine: convert Actions Semi Owl SoCs bindings to yaml
dmaengine: idxd: add missing invalid flags field to completion
dmaengine: dw: Initialize max_sg_burst capability
dmaengine: dw: Introduce max burst length hw config
dmaengine: dw: Initialize min and max burst DMA device capability
dmaengine: dw: Set DMA device max segment size parameter
dmaengine: dw: Take HC_LLP flag into account for noLLP auto-config
dmaengine: Introduce DMA-device device_caps callback
dmaengine: Introduce max SG burst capability
dmaengine: Introduce min burst length capability
dt-bindings: dma: dw: Add max burst transaction length property
dt-bindings: dma: dw: Convert DW DMAC to DT binding
dmaengine: ti: k3-udma: Query throughput level information from hardware
dmaengine: ti: k3-udma: Use defines for capabilities register parsing
dmaengine: xilinx: dpdma: Fix kerneldoc warning
dmaengine: xilinx: dpdma: add missing kernel doc
dmaengine: xilinx: dpdma: remove comparison of unsigned expression
...
