Commit Graph
129 Commits
Author SHA1 Message Date
Felix Fietkau 130e5de093 Revert "mt76: dma: queue more rx frames internally before passing them to the stack"
This reverts commit 0fc3c5eb61.
2020-09-07 12:08:07 +02:00
Felix Fietkau 0fc3c5eb61 mt76: dma: queue more rx frames internally before passing them to the stack
On MT7615 and MT7915, packets received from DMA can include TX_FREE events,
which free up space for tx. In order to process those packets faster, keep
keep a longer internal queue of packets and pass as many packets to the
stack as the NAPI budget allows

Signed-off-by: Felix Fietkau <[email protected]>
2020-09-04 19:34:57 +02:00
Felix Fietkau 5f080033ec mt76: move txwi handling code to dma.c, since it is mmio specific
This way we can make some functions static

Signed-off-by: Felix Fietkau <[email protected]>
2020-09-04 19:31:36 +02:00
Felix Fietkau 3b946a6dc5 mt76: dma: cache dma map address/len in struct mt76_queue_entry
Accessing them from uncached memory can be expensive, so it's cheaper to
cache them

Signed-off-by: Felix Fietkau <[email protected]>
2020-08-27 07:58:36 +02:00
Felix Fietkau 3218b914a2 mt76: remove struct mt76_sw_queue
All members except for the struct mt76_queue pointer have been removed

Signed-off-by: Felix Fietkau <[email protected]>
2020-08-24 20:59:30 +02:00
Felix Fietkau 780cdabb86 mt76: remove qid argument to drv->tx_complete_skb
It is not needed

Signed-off-by: Felix Fietkau <[email protected]>
2020-08-24 20:59:30 +02:00
Felix Fietkau 3c2cd85803 mt76: unify queue tx cleanup code
Cleanup and preparation for changing tx scheduling behavior

Signed-off-by: Felix Fietkau <[email protected]>
2020-08-24 20:59:30 +02:00
Felix Fietkau 2f1318a06d mt76: convert from tx tasklet to tx worker thread
This should improve performance by allowing the scheduler to move the tx
scheduling work to idle CPUs

Signed-off-by: Felix Fietkau <[email protected]>
2020-08-22 15:56:57 +02:00
Felix Fietkau 55d79ab7fa mt76: dma: update q->queued immediately on cleanup
With threaded NAPI support it becomes more likely that tx cleanup and tx
enqueuing will run in parallel. In order to avoid queue starvation issues
under load, update q->queued immediately.

Signed-off-by: Felix Fietkau <[email protected]>
2020-08-22 15:56:57 +02:00
Felix Fietkau 888990e159 mt76: use threaded NAPI
With threaded NAPI, the rx handler function is no longer bound to the CPU that
fired the interrupt, which significantly helps to spread the workload over
multiple CPUs, especially when multiple devices are using threaded NAPI at the
same time.

Signed-off-by: Felix Fietkau <[email protected]>
2020-08-22 15:56:52 +02:00
Felix Fietkau 825343467d mt76: fix double DMA unmap of the first buffer on 7615/7915
A small part of the first skb buffer is passed to the firmware for parsing
via DMA, while the full buffer is passed as part of the TXP.

Avoid calling DMA unmap on the first part (with a different length than map)

Signed-off-by: Felix Fietkau <[email protected]>
2020-08-05 11:31:29 +02:00
Felix Fietkau 1eb8b7d689 mt76: testmode: fix tx_done accounting on enqueue failures
Signed-off-by: Felix Fietkau <[email protected]>
2020-07-21 13:21:04 +02:00
Felix Fietkau 4e6f4e432d mt76: only iterate over initialized rx queues
Fixes the following reported crash:

[    2.361127] BUG: spinlock bad magic on CPU#0, modprobe/456
[    2.361583]  lock: 0xffffa1287525b3b8, .magic: 00000000, .owner: <none>/-1, .owner_cpu: 0
[    2.362250] CPU: 0 PID: 456 Comm: modprobe Not tainted 4.14.177 #5
[    2.362751] Hardware name: HP Meep/Meep, BIOS Google_Meep.11297.75.0 06/17/2019
[    2.363343] Call Trace:
[    2.363552]  dump_stack+0x97/0xdb
[    2.363826]  ? spin_bug+0xa6/0xb3
[    2.364096]  do_raw_spin_lock+0x6a/0x9a
[    2.364417]  mt76_dma_rx_fill+0x44/0x1de [mt76]
[    2.364787]  ? mt76_dma_kick_queue+0x18/0x18 [mt76]
[    2.365184]  mt76_dma_init+0x53/0x85 [mt76]
[    2.365532]  mt7615_dma_init+0x3d7/0x546 [mt7615e]
[    2.365928]  mt7615_register_device+0xe6/0x1a0 [mt7615e]
[    2.366364]  mt7615_mmio_probe+0x14b/0x171 [mt7615e]
[    2.366771]  mt7615_pci_probe+0x118/0x13b [mt7615e]
[    2.367169]  pci_device_probe+0xaf/0x13d
[    2.367491]  driver_probe_device+0x284/0x2ca
[    2.367840]  __driver_attach+0x7a/0x9e
[    2.368146]  ? driver_attach+0x1f/0x1f
[    2.368451]  bus_for_each_dev+0xa0/0xdb
[    2.368765]  bus_add_driver+0x132/0x204
[    2.369078]  driver_register+0x8e/0xcd
[    2.369384]  do_one_initcall+0x160/0x257
[    2.369706]  ? 0xffffffffc0240000
[    2.369980]  do_init_module+0x60/0x1bb
[    2.370286]  load_module+0x18c2/0x1a2b
[    2.370596]  ? kernel_read_file+0x141/0x1b9
[    2.370937]  ? kernel_read_file_from_fd+0x46/0x71
[    2.371320]  SyS_finit_module+0xcc/0xf0
[    2.371636]  do_syscall_64+0x6b/0xf7
[    2.371930]  entry_SYSCALL_64_after_hwframe+0x3d/0xa2
[    2.372344] RIP: 0033:0x7da218ae4199
[    2.372637] RSP: 002b:00007fffd0608398 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[    2.373252] RAX: ffffffffffffffda RBX: 00005a705449df90 RCX: 00007da218ae4199
[    2.373833] RDX: 0000000000000000 RSI: 00005a7052e73bd8 RDI: 0000000000000006
[    2.374411] RBP: 00007fffd06083e0 R08: 0000000000000000 R09: 00005a705449d540
[    2.374989] R10: 0000000000000006 R11: 0000000000000246 R12: 0000000000000000
[    2.375569] R13: 00005a705449def0 R14: 00005a7052e73bd8 R15: 0000000000000000

Reported-by: Sean Wang <[email protected]>
Fixes: d3377b78cec6 ("mt76: add HE phy modes and hardware queue")
Signed-off-by: Felix Fietkau <[email protected]>
2020-05-24 19:11:50 +02:00
Felix Fietkau 3b4f938409 mt76: add memory barrier to DMA queue kick
Ensure that descriptor memory has been fully written before letting the
hardware read it

Signed-off-by: Felix Fietkau <[email protected]>
2020-03-26 14:24:28 +01:00
Felix Fietkau 8f8e9161b3 mt76: fix array overflow on receiving too many fragments for a packet
If the hardware receives an oversized packet with too many rx fragments,
skb_shinfo(skb)->frags can overflow and corrupt memory of adjacent pages.
This becomes especially visible if it corrupts the freelist pointer of
a slab page.

Cc: [email protected]
Signed-off-by: Felix Fietkau <[email protected]>
2020-02-19 13:55:52 +01:00
Felix Fietkau 14f37f8d86 mt76: dma: do not write cpu_idx on rx queue reset until after refill
The hardware should only start processing the ring after at least one
buffer has been added

Signed-off-by: Felix Fietkau <[email protected]>
2020-01-20 12:11:09 +01:00
Felix Fietkau 65745c5ac5 mt76: fix handling full tx queues in mt76_dma_tx_queue_skb_raw
Fixes a theoretical issue where it could potentially overwrite an existing
descriptor entry (and leaking its skb)

Signed-off-by: Felix Fietkau <[email protected]>
2020-01-20 12:08:32 +01:00
Felix Fietkau 9453dbe921 mt76: set dma-done flag for flushed descriptors
Avoids a theoretical corner case where the hardware could try to process
a stale descriptor after a watchdog reset

Signed-off-by: Felix Fietkau <[email protected]>
2020-01-20 12:07:10 +01:00
Felix Fietkau b3bf83d6da mt76: disable bh in mt76_dma_rx_poll
Fixes potential RCU issues and avoids calling ieee80211_rx_napi with softirq
enabled.

Signed-off-by: Felix Fietkau <[email protected]>
2019-12-05 17:46:42 +01:00
Felix Fietkau fff2c07193 Revert "mt76: use rcu_read_lock_bh in mt76_dma_rx_poll"
This reverts commit 44825e88a4.
Will be replaced by a better fix

Signed-off-by: Felix Fietkau <[email protected]>
2019-12-05 17:46:07 +01:00
Felix Fietkau 88db676f85 mt76: fix rx dma ring descriptor state on reset
To avoid having the hardware potentially write to memory behind stale
descriptors, set the dma-done flag on all of them before cleaning up
allocated rx buffers

Signed-off-by: Felix Fietkau <[email protected]>
2019-12-05 17:40:11 +01:00
Felix Fietkau d246beb233 mt76: keep a set of software tx queues per phy
Allows tracking tx scheduling separately per phy

Signed-off-by: Felix Fietkau <[email protected]>
2019-11-20 12:54:17 +01:00
Felix Fietkau 1d80118714 mt76: add support for an extra wiphy in the tx status path
This is preparation for supporting multiple wiphys per device to support the
concurrent dual-band feature of MT7615D

Signed-off-by: Felix Fietkau <[email protected]>
2019-11-20 12:54:17 +01:00
Felix Fietkau af83ee8de9 mt76: move initialization of some struct members to mt76_alloc_device
Reduces duplication and prepares for further rework

Signed-off-by: Felix Fietkau <[email protected]>
2019-11-20 12:54:17 +01:00
Felix Fietkau 44825e88a4 mt76: use rcu_read_lock_bh in mt76_dma_rx_poll
Fixes potential RCU issues and avoids calling ieee80211_rx_napi with softirq
enabled.

Reported-by: Markus Theil <[email protected]>
Signed-off-by: Felix Fietkau <[email protected]>
2019-11-20 12:37:28 +01:00
zhengbinandFelix Fietkau ea19cd7ac6 mt76: Remove set but not used variable 'idx'
Fixes gcc '-Wunused-but-set-variable' warning:

drivers/net/wireless/mediatek/mt76/dma.c: In function mt76_dma_rx_fill:
drivers/net/wireless/mediatek/mt76/dma.c:377:6: warning: variable idx set but not used [-Wunused-but-set-variable]

It is not used since commit 17f1de56df05 ("mt76:
add common code shared between multiple chipsets")

Reported-by: Hulk Robot <[email protected]>
Signed-off-by: zhengbin <[email protected]>
Signed-off-by: Felix Fietkau <[email protected]>
2019-11-20 12:26:32 +01:00
Lorenzo BianconiandFelix Fietkau f2be00b101 mt76: dma: fix buffer unmap with non-linear skbs
mt76 dma layer is supposed to unmap skb data buffers while keep txwi
mapped on hw dma ring. At the moment mt76 wrongly unmap txwi or does
not unmap data fragments in even positions for non-linear skbs. This
issue may result in hw hangs with A-MSDU if the system relies on IOMMU
or SWIOTLB. Fix this behaviour properly unmapping data fragments on
non-linear skbs.

Fixes: 17f1de56df05 ("mt76: add common code shared between multiple chipsets")
Signed-off-by: Lorenzo Bianconi <[email protected]>
2019-10-28 08:42:32 +01:00
Felix Fietkau 7d8764d320 mt76: avoid enabling interrupt if NAPI poll is still pending
if napi_complete() returns false, it means that polling is still pending.
Interrupts should not fire until the polling is no longer scheduled

Signed-off-by: Felix Fietkau <[email protected]>
2019-10-07 12:33:40 +02:00
Felix Fietkau b0429879ea mt76: rename mt76_driver_ops txwi_flags to drv_flags and include tx aligned4
This reduces the struct size and is useful for adding more flags later

Signed-off-by: Felix Fietkau <[email protected]>
2019-09-26 12:42:08 +02:00
Ryder LeeandFelix Fietkau 37673a4181 mt76: fix some checkpatch warnings
This fixes the following checkpatch warnings:
CHECK: Alignment should match open parenthesis
CHECK: No space is necessary after a cast

Signed-off-by: Ryder Lee <[email protected]>
Signed-off-by: Felix Fietkau <[email protected]>
2019-08-22 09:26:30 +02:00
Ryder LeeandFelix Fietkau f625afcedc mt76: switch to SPDX tag instead of verbose boilerplate text
No functional change intended.

Add SPDX identifiers to all remaining files in /mt76.

Signed-off-by: Ryder Lee <[email protected]>
Signed-off-by: Felix Fietkau <[email protected]>
2019-08-20 15:22:20 +02:00
Felix Fietkau 76aada563b mt76: dma: reset q->rx_buf on rx reset
Fixes rx of the first frame if a fragmented rx was interrupted by the reset

Signed-off-by: Felix Fietkau <[email protected]>
2019-08-12 15:51:00 +02:00
Felix Fietkau 47ddf4b722 mt76: revert support for TX_NEEDS_ALIGNED4_SKBS
This didn't bring the performance benefit that I had hoped for, so the mac80211
patch for this will not make it upstream

Signed-off-by: Felix Fietkau <[email protected]>
2019-06-24 23:56:23 +02:00
Lorenzo BianconiandFelix Fietkau 2cb46830fa mt76: move netif_napi_del in mt76_dma_cleanup
Move netif_napi_del in mt76_dma_cleanup routine since it is done
by all drivers

Signed-off-by: Lorenzo Bianconi <[email protected]>
Signed-off-by: Felix Fietkau <[email protected]>
2019-05-11 15:25:58 +02:00
Lorenzo BianconiandFelix Fietkau 9029560fad mt76: dma: add skb check for dummy pointer
Introduce skb check for dummy address in mt76_dma_tx_cleanup_idx.
This is a preliminary patch to add support for new chipsets (e.g. 7615)

Signed-off-by: Lorenzo Bianconi <[email protected]>
2019-04-25 12:04:34 +02:00
Lorenzo BianconiandFelix Fietkau c47a5685d0 mt76: dma: introduce skb field in mt76_txwi_cache
Introduce skb field in mt76_txwi_cache. Moreover add txwi_flags
to mt76_driver_ops since new chipsets will release mt76_txwi_cache/skbs
at tx completion instead of dma one.
This is a preliminary patch to add mt7615 support

Signed-off-by: Lorenzo Bianconi <[email protected]>
2019-04-25 12:04:34 +02:00
Lorenzo BianconiandFelix Fietkau 5007326555 mt76: add skb pointer to mt76_tx_info
Pass skb pointer to tx_prepare_skb through mt76_tx_info data structure.
This is a preliminary patch to properly support dma error path for
new chipsets (e.g. 7615)

Signed-off-by: Lorenzo Bianconi <[email protected]>
2019-04-25 12:04:34 +02:00
Lorenzo BianconiandFelix Fietkau 3e371ca35c mt76: set txwi_size according to the driver value
Dynamically allocate txwi since new chipsets will use longer txwi
descriptors

Signed-off-by: Lorenzo Bianconi <[email protected]>
2019-04-25 12:04:34 +02:00
Ryder LeeandFelix Fietkau edb2a001b6 mt76: add unlikely() for dma_mapping_error() check
In the tx/rx fastpath, the funciton dma_map_single() rarely fails.
This adds unlikely() optimization to this error check conditional.

Signed-off-by: Ryder Lee <[email protected]>
Signed-off-by: Felix Fietkau <[email protected]>
2019-04-25 12:04:34 +02:00
Felix Fietkau 60e508ed14 mt76: use readl/writel instead of ioread32/iowrite32
readl/writel is faster because it gets rid of unnecessary extra checks

Signed-off-by: Felix Fietkau <[email protected]>
2019-03-23 15:24:58 +01:00
Lorenzo BianconiandFelix Fietkau 3ce8a93ae1 mt76: dma: add static qualifier to mt76_dma_tx_queue_skb
As already done for mt76_dma_tx_queue_skb_raw, add static qualifier to
mt76_dma_tx_queue_skb and introduce mt76_tx_queue_skb in order to run
mt76_dma_tx_queue_skb in driver code

Signed-off-by: Lorenzo Bianconi <[email protected]>
Signed-off-by: Felix Fietkau <[email protected]>
2019-03-16 20:28:17 +01:00
Felix Fietkau 42ce040662 mt76: use TX_NEEDS_ALIGNED4_SKBS
Remove custom header padding insert code

Signed-off-by: Felix Fietkau <[email protected]>
2019-03-15 11:27:33 +01:00
Felix Fietkau 688d708a43 mt76: only schedule txqs from the tx tasklet
Reduces lock contention from the tx path and improves performance

Signed-off-by: Felix Fietkau <[email protected]>
2019-03-14 22:37:30 +01:00
Felix Fietkau b77b9323c6 mt76: reduce locking in mt76_dma_tx_cleanup
q->tail can be safely updated without locking, because there is no
concurrent access. If called from outside of the tasklet (for flushing),
the tasklet is always disabled.
q->queued can be safely read without locking, as long as the decrement
happens within the locked section.
This patch allows cleaning up tx packets outside of the section that holds
the queue lock for improved performance

Signed-off-by: Felix Fietkau <[email protected]>
2019-03-14 22:37:30 +01:00
Felix Fietkau 72fe286f2d mt76: use mac80211 txq scheduling
Signed-off-by: Felix Fietkau <[email protected]>
2019-03-14 22:37:30 +01:00
Lorenzo BianconiandFelix Fietkau f977a924b8 mt76: introduce mt76_tx_info data structure
Add mt76_tx_info as auxiliary data structure to pass values
to tx_prepare_skb pointer. This is a preliminary patch to add
support for new chipsets (e.g. mt7615)

Signed-off-by: Lorenzo Bianconi <[email protected]>
Signed-off-by: Felix Fietkau <[email protected]>
2019-03-14 22:09:32 +01:00
Lorenzo BianconiandFelix Fietkau 0b03f87b22 mt76: move skb dma mapping before running tx_prepare_skb
Move skb dma mapping before configuring txwi since new chipsets (mt7615)
will need skb dma addresses in order to properly configure txwi

Signed-off-by: Lorenzo Bianconi <[email protected]>
Signed-off-by: Felix Fietkau <[email protected]>
2019-03-14 22:09:32 +01:00
Lorenzo BianconiandFelix Fietkau efe9a473af mt76: mmio: move mt76_insert_hdr_pad in mt76_dma_tx_queue_skb
Introduce tx_aligned4_skbs in mt76_driver_ops and move
mt76_insert_hdr_pad in mt76_dma_tx_queue_skb. This is a preliminary
patch in order to unify tx dma mapping for mt76x02 and new chipsets

Signed-off-by: Lorenzo Bianconi <[email protected]>
Signed-off-by: Felix Fietkau <[email protected]>
2019-03-14 22:09:32 +01:00
Lorenzo BianconiandFelix Fietkau 2dc63b0da0 mt76: introduce mt76_txq_id field in mt76_queue_entry
Add mt76_txq_id field to mt76_queue_entry in order to properly
track outstanding frames for mt7615 that relies on a single hw queue

Signed-off-by: Lorenzo Bianconi <[email protected]>
Signed-off-by: Felix Fietkau <[email protected]>
2019-03-13 12:53:23 +01:00
Lorenzo BianconiandFelix Fietkau 06c917faeb mt76: introduce mt76_sw_queue data structure
Introduce mt76_sw_queue data structure in order to support new
chipsets (e.g. mt7615) that have a shared hardware queue for all traffic
identifiers. mt76_sw_queue will be used to track outstanding packets

Signed-off-by: Lorenzo Bianconi <[email protected]>
Signed-off-by: Felix Fietkau <[email protected]>
2019-03-13 12:53:23 +01:00