TencentOS-kernel/kernel/bpf
Björn Töpel 084e0ffc9f bpf: Introduce BPF dispatcher
[upstream commit 75ccbef6369e94ecac696a152a998a978d41376b]

The BPF dispatcher is a multi-way branch code generator, mainly
targeted for XDP programs. When an XDP program is executed via the
bpf_prog_run_xdp(), it is invoked via an indirect call. The indirect
call has a substantial performance impact, when retpolines are
enabled. The dispatcher transform indirect calls to direct calls, and
therefore avoids the retpoline. The dispatcher is generated using the
BPF JIT, and relies on text poking provided by bpf_arch_text_poke().

The dispatcher hijacks a trampoline function it via the __fentry__ nop
of the trampoline. One dispatcher instance currently supports up to 64
dispatch points. A user creates a dispatcher with its corresponding
trampoline with the DEFINE_BPF_DISPATCHER macro.

Signed-off-by: Björn Töpel <bjorn.topel@intel.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20191213175112.30208-3-bjorn.topel@gmail.com
2022-08-30 11:33:19 +08:00
..
Makefile bpf: Introduce BPF dispatcher 2022-08-30 11:33:19 +08:00
arraymap.c bpf: Simplify __bpf_arch_text_poke poke type handling 2022-08-30 11:33:19 +08:00
bpf_lru_list.c bpf_lru_list: Read double-checked variable once without lock 2021-04-12 12:52:15 +08:00
bpf_lru_list.h Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces. 2021-03-16 11:01:34 +08:00
bpf_struct_ops.c bpf: Introduce BPF_MAP_TYPE_STRUCT_OPS 2022-08-30 11:33:19 +08:00
bpf_struct_ops_types.h bpf: tcp: Support tcp_congestion_ops in bpf 2022-08-30 11:33:19 +08:00
btf.c bpf: Introduce function-by-function verification 2022-08-30 11:33:19 +08:00
cgroup.c Revert "bpf/cgroup: Replace rcu_swap_protected() with rcu_replace_pointer()" 2022-08-30 11:33:17 +08:00
core.c bpf: Add poke dependency tracking for prog array maps 2022-08-30 11:33:19 +08:00
cpumap.c cpumap: Avoid warning when CONFIG_DEBUG_PER_CPU_MAPS is enabled 2021-03-16 16:27:10 +08:00
devmap.c devmap: Use bpf_map_area_alloc() for allocating hash buckets 2021-03-16 16:30:57 +08:00
disasm.c bpf: Introduce BPF nospec instruction for mitigating Spectre v4 2022-08-30 11:33:16 +08:00
disasm.h Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces. 2021-03-16 11:01:34 +08:00
dispatcher.c bpf: Introduce BPF dispatcher 2022-08-30 11:33:19 +08:00
hashtab.c bpf: Zero-fill re-used per-cpu map element 2021-03-16 16:40:49 +08:00
helpers.c bpf: Fix helper bpf_map_peek_elem_proto pointing to wrong callback 2021-03-16 16:44:00 +08:00
inode.c bpf: Convert bpf_prog refcnt to atomic64_t 2021-03-16 16:43:58 +08:00
local_storage.c Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces. 2021-03-16 11:01:34 +08:00
lpm_trie.c Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces. 2021-03-16 11:01:34 +08:00
map_in_map.c bpf: Move owner type, jited info into array auxiliary data 2022-08-30 11:33:19 +08:00
map_in_map.h Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces. 2021-03-16 11:01:34 +08:00
offload.c Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces. 2021-03-16 11:01:34 +08:00
percpu_freelist.c Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces. 2021-03-16 11:01:34 +08:00
percpu_freelist.h Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces. 2021-03-16 11:01:34 +08:00
queue_stack_maps.c Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces. 2021-03-16 11:01:34 +08:00
reuseport_array.c bpf, net: Rework cookie generator as per-cpu one 2022-03-02 10:48:06 +08:00
stackmap.c bpf: Check for integer overflow when using roundup_pow_of_two() 2021-04-12 12:52:03 +08:00
syscall.c bpf: Add poke dependency tracking for prog array maps 2022-08-30 11:33:19 +08:00
sysfs_btf.c bpf: Fix sysfs export of empty BTF section 2021-03-16 16:37:55 +08:00
tnum.c Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces. 2021-03-16 11:01:34 +08:00
trampoline.c bpf: Move trampoline JIT image allocation to a function 2022-08-30 11:33:19 +08:00
verifier.c bpf: Constant map key tracking for prog array pokes 2022-08-30 11:33:19 +08:00
xskmap.c bpf: Implement map_gen_lookup() callback for XSKMAP 2022-08-30 11:33:17 +08:00