Currently, separate submissions are made for page table
switch and context switch to the ring buffer. However, if
the page table switch succeeds but the context switch fails,
it can lead to use of wrong page table for drawctxt.
To address this issue, submit page table switch and context
switch commands as a single submission to ring buffer.
Also, remove the unnecessary ADRENO_DEVICE_FAULT check and
correctly put the refcount of adreno context during error
cleanup.
Change-Id: I1bb4ee3ebb0ce6ea32f0b6799cfb7fa89c0d09c7
Signed-off-by: Rakesh Naidu Bhaviripudi <quic_rakeshb@quicinc.com>
The timeline fence needs to remain valid for logging purposes. Take an
extra refcount on the timeline dma_fence to make sure it doesn't go
away till we're done with it.
Bug: 339043278
Change-Id: I6670ef7add099a72684c1fe20ed009dff85d1f27
Signed-off-by: Lynus Vaz <quic_lvaz@quicinc.com>
Provenance: 69df853d40
If iommu unmap fails and leaves dma or anon buffers still mapped in the
iommu, do not free them.
Bug: 332315050
Change-Id: I09335639479aa18e47e311a1b066b1694ce3dfb6
Signed-off-by: Lynus Vaz <quic_lvaz@quicinc.com>
Provenance: ab27a52cf1
If sharedmem cannot be unmapped from the mmu, it can still be accessed
by the GPU. Therefore it is not safe to free the backing memory. In the
case that unmap fails, do not free it or return it to the system.
Change-Id: Iad3e86d043f129a4d71cf862865d9033d4a315e3
Signed-off-by: Lynus Vaz <quic_lvaz@quicinc.com>
Signed-off-by: Kaushal Sanadhya <quic_ksanadhy@quicinc.com>
Provenance: 88bb532b43
Bug: 318393843
drawobj_sync_timeline_fence_work() does a cleanup of fence and syncobj
allocations. Doing this cleanup in irq context requires the irq_work
struct to remain valid after the function executes. Avoid this constraint
by deferring this work to the memory workqueue.
Change-Id: Ie0c91e9a9f6f496ae92e50c3bc27442c7390b750
Signed-off-by: Lynus Vaz <quic_lvaz@quicinc.com>
Signed-off-by: SIVA MULLATI <quic_smullati@quicinc.com>
Provenance: 2bac75def2
Bug: 295908146
When setting svm region during the gpuobj import ioctl call for a usermem
address, there is a possibility of a very large input size causing the
region's 64-bit end address to wrap around. This can cause the region
to incorrectly be considered valid, ultimately allowing a use after free
scenario. To prevent this, detect the occurrence of a wrap and reject the
import.
Change-Id: I48b9cf37310943b4e5c3ff0a18deece13b016c03
Signed-off-by: Mohammed Mirza Mandayappurath Manzoor <quic_mmandaya@quicinc.com>
Signed-off-by: Puranam V G Tejaswi <quic_pvgtejas@quicinc.com>
Provenance: d66b799c80
Bug: 299649795
KGSL internally has a limit on the length of the list of syncpoints
submitted in a single AUX command. Enforce this limit so we don't
overwrite memory beyond the structures that track these syncpoints.
Change-Id: I44175744a9a1dc16238df8b3f25303cde266df16
Signed-off-by: Lynus Vaz <quic_lvaz@quicinc.com>
Signed-off-by: Kaushal Sanadhya <quic_ksanadhy@quicinc.com>
Provenance: 1e46e81dbe
Bug: 300941008
Postamble packets are executed in privileged mode by gpu. So we should keep
them in a privileged scratch buffer to block userspace access. For
targets with APRIV feature support, we can mark the preemption scratch
buffer as privileged too to avoid similar issues in future.
Bug: 276750663
Change-Id: Ifda360dda251083f38dfde80ce1b5dc83daae902
Signed-off-by: Akhil P Oommen <quic_akhilpo@quicinc.com>
Signed-off-by: Kaushal Sanadhya <quic_ksanadhy@quicinc.com>
Signed-off-by: Harrison Lingren <hlingren@google.com>
Currently we don't ensure if vma->vm_file is associated with dma_buf. This
can cause issues later when private_data from a non dma_buf file is used as
dma_buf structure. Hence get the fd that is associated with vma->vm_file
and use dma_buf_get() to get pointer to dma_buf structure. dma_buf_get()
ensures that the file from the input fd is associated with dma_buf.
Bug: 271879598
Provenance: 86695e1dd1
Change-Id: If552298291adf8d1cdd8b0e91c5b01f792fc4382
Signed-off-by: Puranam V G Tejaswi <quic_pvgtejas@quicinc.com>
Signed-off-by: Sean Callanan <spyffe@google.com>
Before putting a page back in the pool be sure that it doesn't have
any additional references that would be a signal that somebody else
is looking at the page and that it would be a bad idea to keep it
around and run the risk of accidentally handing it to a different
process.
Bug: 271879644
Provenance: 9968fcdd9d
Change-Id: I4cbd7f2a9159a14014eca8485c00deb2607039c9
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: Kamal Agrawal <quic_kamaagra@quicinc.com>
Signed-off-by: Sean Callanan <spyffe@google.com>
Kgsl shmem pages are always pinned but still endup in the evictable lru
leading to inteference with the reclaim logic making the later spending
more time in scanning the pages. This time can result into the PSI
events generation leading to oom/lmkd logic to take the wrong
decissions. Move such pages to unevictable LRU thus kept out of the
reclaimable list.
Change-Id: Iee82ce101526f04a8f294a10dce9598356701977
Signed-off-by: Charan Teja Kalla <quic_charante@quicinc.com>
Signed-off-by: Srinivasarao Pathipati <quic_c_spathi@quicinc.com>
Currently we don't ensure if vma->vm_file is associated with dma_buf. This
can cause issues later when private_data from a non dma_buf file is used as
dma_buf structure. Hence get the fd that is associated with vma->vm_file
and use dma_buf_get() to get pointer to dma_buf structure. dma_buf_get()
ensures that the file from the input fd is associated with dma_buf.
Change-Id: Ib78aef8b16bedca5ca86d3a132278ff9f07dce73
Signed-off-by: Puranam V G Tejaswi <quic_pvgtejas@quicinc.com>
Before putting a page back in the pool be sure that it doesn't have
any additional references that would be a signal that somebody else
is looking at the page and that it would be a bad idea to keep it
around and run the risk of accidentally handing it to a different
process.
Change-Id: Ic0dedbad0cf2ffb34b76ad23e393c5a911114b82
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: Kamal Agrawal <quic_kamaagra@quicinc.com>
Timeline semaphore ioctls use local implementation
kgsl_copy_struct_from_user() to validate and copy structs from
user to kernel memory. This can cause problem when user size is
greater than reserved kernel struct size.
Replace this with new kernel implementation of copy_struct_from_user()
helper to ensure correct alignment and access permissions before
proceeding forward.
Bug: 261681571
Change-Id: I5891f355fe515e271c7470543822792572a42f3f
Signed-off-by: Rohan Sethi <quic_rohsethi@quicinc.com>
Signed-off-by: JohnnLee <johnnlee@google.com>
Get the dma_buf handle directly from 'vm_file' after
doing necessary checks on the file.
Bug:240973083
Change-Id: Id5eec16588d64e4e28483b32bb52d4d3d9b86b99
Signed-off-by: ravnar <quic_ravnar@quicinc.com>
Signed-off-by: Sanjay Yadav <quic_sanjyada@quicinc.com>
Signed-off-by: Kaushal Sanadhya <quic_ksanadhy@quicinc.com>
Signed-off-by: Joey Lin <linjoey@google.com>
kcalloc returns out of memory in low memory conditions even if memory
is available in non-contiguous manner. This results in failure to
submit commands to ringbuffer. Use kvcalloc in place of kcalloc so
that when kcalloc fails in low memory conditions, commands can be
submitted if kvcalloc can provide enough memory.
Change-Id: If6a20e35983982b5c0888e5f7dabecfa8c026bcb
Signed-off-by: Pranav Patel <quic_pranavp@quicinc.com>
Ensure that thermal limits set by devfreq cooling framework,
limits framework and sysfs nodes (like gpu_max_clock) don't
overwrite each other, instead a consolidated (lowest of all)
limit is used as a final thermal limit. If this fails, fall back
to setting thermal_pwrlevel directly.
Change-Id: Ia6ec60e4881176da064c3cd5cf4ffd8e0a8e1046
Signed-off-by: Pranav Patel <quic_pranavp@quicinc.com>
Currently performance counters are global and can be read by anyone. Change
the behaviour to disable reading global counters as default and add a sysfs
node to enable/disable reads.
Bug: 193434313
Change-Id: Ic3785acd9bd7425c2a844ed103d7b870d9f80adf
Signed-off-by: Mohammed Mirza Mandayappurath Manzoor <quic_mmandaya@quicinc.com>
Signed-off-by: Harshitha Sai Neelati <quic_hsaineel@quicinc.com>
Signed-off-by: linjoey <linjoey@google.com>
Signed-off-by: Wilson Sung <wilsonsung@google.com>
Performance counter values need not be retained across contexts unless
specifically requested for debug. Zap the counters by initialising
perfcounter SRAM with 0's using GPU_RBBM_PERFCTR_SRAM_INIT_CMD.
Add pm4 packets during context switches and add a KMD postamble packet to
clear the counters during preemption. Do not enable perfcounter save and
restore unless requested.
Bug: 193434313
Change-Id: I371779ce659c07a1cc664327f5ecdcf0374201d8
Signed-off-by: Mohammed Mirza Mandayappurath Manzoor <quic_mmandaya@quicinc.com>
Signed-off-by: Harshitha Sai Neelati <quic_hsaineel@quicinc.com>
Signed-off-by: linjoey <linjoey@google.com>
Signed-off-by: Wilson Sung <wilsonsung@google.com>
Get the dma_buf handle directly from 'vm_file' after
doing necessary checks on the file.
Change-Id: Id5eec16588d64e4e28483b32bb52d4d3d9b86b99
Signed-off-by: ravnar <quic_ravnar@quicinc.com>
Signed-off-by: Sanjay Yadav <quic_sanjyada@quicinc.com>
Signed-off-by: Kaushal Sanadhya <quic_ksanadhy@quicinc.com>
Timeline semaphore ioctls use local implementation
kgsl_copy_struct_from_user() to validate and copy structs from
user to kernel memory. This can cause problem when user size is
greater than reserved kernel struct size.
Replace this with new kernel implementation of copy_struct_from_user()
helper to ensure correct alignment and access permissions before
proceeding forward.
Change-Id: I5891f355fe515e271c7470543822792572a42f3f
Signed-off-by: Rohan Sethi <quic_rohsethi@quicinc.com>
Currently there is a chance that the variable ret can be
used uninitialized in kgsl_reclaim_callback. Assign a default
value to ret to avoid this.
Change-Id: I90014370773ee2a6594ee624f890fead955e6f76
Signed-off-by: Puranam V G Tejaswi <quic_pvgtejas@quicinc.com>
Currently there are a few unnecessary NULL checks for pointers to
kgsl_drawobj_sync_event. The pointers will always be valid at
these points and are moreover already accessed in the same functions
prior to the NULL checks. Hence remove the unnecessary NULL checks.
Change-Id: I65197ea8ad4df7b28445a95165bf4b57f735a6d8
Signed-off-by: Puranam V G Tejaswi <quic_pvgtejas@quicinc.com>
Performance counter values need not be retained across contexts unless
specifically requested for debug. Zap the counters by initialising
perfcounter SRAM with 0's using GPU_RBBM_PERFCTR_SRAM_INIT_CMD.
Add pm4 packets during context switches and add a KMD postamble packet to
clear the counters during preemption. Do not enable perfcounter save and
restore unless requested.
Change-Id: I371779ce659c07a1cc664327f5ecdcf0374201d8
Signed-off-by: Mohammed Mirza Mandayappurath Manzoor <quic_mmandaya@quicinc.com>
Signed-off-by: Harshitha Sai Neelati <quic_hsaineel@quicinc.com>
Currently performance counters are global and can be read by anyone. Change
the behaviour to disable reading global counters as default and add a sysfs
node to enable/disable reads.
Change-Id: Ic3785acd9bd7425c2a844ed103d7b870d9f80adf
Signed-off-by: Mohammed Mirza Mandayappurath Manzoor <quic_mmandaya@quicinc.com>
Signed-off-by: Harshitha Sai Neelati <quic_hsaineel@quicinc.com>
Update the IFPC power up reglist to include all the CP Protect
registers.
Bug:201574693
Change-Id: I1b43420c466b8a228892afac8ecf05b11b5a80e6
Signed-off-by: Akhil P Oommen <akhilpo@codeaurora.org>
Signed-off-by: linjoey <linjoey@google.com>
Consider a scenario where user allocates anonymous memory but does not
write to it. Here the physical pages are not yet allocated. Now when this
memory is requested to be imported, a list of newly allocated zero pages
is obtained using get_user_pages(). Currently cache flush is not done for
these pages and hence GPU sees stale data. Fix this by performing cache
flush on these pages.
Change-Id: Id1e8aa20e8a9de112761732ed92f30c01088840b
Signed-off-by: Puranam V G Tejaswi <quic_pvgtejas@quicinc.com>
Currently gpuaddr_in_range() accepts only the gpuaddr & returns
true if it lies in valid range. But this does not mean that the
entire buffer is within range.
Modify the function to accept size as a parameter and check that
both starting & ending points of buffer lie within mmu range.
Change-Id: I1d722295b9a27e746bfdb6d3bf409ffe722193cb
Signed-off-by: Rohan Sethi <quic_rohsethi@quicinc.com>
Currently there is a chance that release for the fence was already called
before we call dma_fence_get during kgsl_timeline_signal and
kgsl_ioctl_timeline_destroy. This can cause use-after-free issue as we can
access fence after release. Fix this by signalling fence only if the last
refcount on the fence was not yet put. This makes sure that release for the
fence will not be called until we are done signalling.
Change-Id: I6bdcefa1f128febb7a0f7aef133757268a3b9ae3
Signed-off-by: Puranam V G Tejaswi <pvgtejas@codeaurora.org>
Signed-off-by: Pranav Patel <quic_pranavp@quicinc.com>
Update the IFPC power up reglist to include all the CP Protect
registers.
Change-Id: I1b43420c466b8a228892afac8ecf05b11b5a80e6
Signed-off-by: Akhil P Oommen <akhilpo@codeaurora.org>
Add some traces to make it easier to debug and track timeline behavior.
Change-Id: Ic0dedbadbfe623c675616a0d2fdd800b49f14705
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: Rohan Sethi <rohsethi@codeaurora.org>
Some graphics APIs want the ability to create and use timelines
with 64 bit sequence numbers to synchronize between threads.
Add support for timelines that can be created, signaled and managed from
user space. Timelines can also be signaled from a draw context command
stream via an auxiliary command and a wait can be added as a sync object
to any GPU command.
Change-Id: Ic0dedbad0ffc99cc309951369eaf85b56727802c
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: Rohan Sethi <rohsethi@codeaurora.org>
Make sure there is enough room in the memory descriptor to store the
entire profiling buffer object.
Change-Id: I1e1c73097bb2bba9645b0a3c66fdbbc71d8ba8fa
Signed-off-by: Kamal Agrawal <kamaagra@codeaurora.org>
The context is currently kept alive until all its fences are
released. This means that we do not destroy a context until all
the fence fds associated with it are closed by userspace, which may
be a long time. Instead, remove the timeline's refcount on its
context when it is detached. This permits the context to be freed
once all its fences signal.
Change-Id: Ifc5dd55980358ddbb4d3f6220d8b5d9f725aed1b
Signed-off-by: Lynus Vaz <lvaz@codeaurora.org>
Signed-off-by: Hareesh Gundu <hareeshg@codeaurora.org>
Signed-off-by: Pankaj Gupta <gpankaj@codeaurora.org>
If refcount on mem entry is the last refcount, we will call
kgsl_mem_entry_destroy and detach it from process list. If
there is no refcount on the process private as well then we
will call kgsl_destroy_process_private to cleanup. During
cleanup, we will try to remove the same sysfs node which is
in use by the current thread and this situation will end up in
a deadlock.
To fix this problem, use a worker to put the refcount on mem
entry.
Change-Id: I69a77c4059489c95de2ca205701571daac4bdc8a
Signed-off-by: Pankaj Gupta <gpankaj@codeaurora.org>
