Martin KaFai Lau 90913d4410 bpf: Stop caching subprog index in the bpf_pseudo_func insn ... [upstream commit 3990ed4c426652fcd469f8c9dc08156294b36c28] This patch is to fix an out-of-bound access issue when jit-ing the bpf_pseudo_func insn (i.e. ld_imm64 with src_reg == BPF_PSEUDO_FUNC) In jit_subprog(), it currently reuses the subprog index cached in insn[1].imm. This subprog index is an index into a few array related to subprogs. For example, in jit_subprog(), it is an index to the newly allocated 'struct bpf_prog **func' array. The subprog index was cached in insn[1].imm after add_subprog(). However, this could become outdated (and too big in this case) if some subprogs are completely removed during dead code elimination (in adjust_subprog_starts_after_remove). The cached index in insn[1].imm is not updated accordingly and causing out-of-bound issue in the later jit_subprog(). Unlike bpf_pseudo_'func' insn, the current bpf_pseudo_'call' insn is handling the DCE properly by calling find_subprog(insn->imm) to figure out the index instead of caching the subprog index. The existing bpf_adj_branches() will adjust the insn->imm whenever insn is added or removed. Instead of having two ways handling subprog index, this patch is to make bpf_pseudo_func works more like bpf_pseudo_call. First change is to stop caching the subprog index result in insn[1].imm after add_subprog(). The verification process will use find_subprog(insn->imm) to figure out the subprog index. Second change is in bpf_adj_branches() and have it to adjust the insn->imm for the bpf_pseudo_func insn also whenever insn is added or removed. Third change is in jit_subprog(). Like the bpf_pseudo_call handling, bpf_pseudo_func temporarily stores the find_subprog() result in insn->off. It is fine because the prog's insn has been finalized at this point. insn->off will be reset back to 0 later to avoid confusing the userspace prog dump tool. Fixes: 69c087ba6225 ("bpf: Add bpf_for_each_map_elem() helper") Signed-off-by: Martin KaFai Lau <kafai@fb.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://lore.kernel.org/bpf/20211106014014.651018-1-kafai@fb.com		2022-06-21 10:27:21 +08:00
..
Makefile	bpf: Implement BPF ring buffer and verifier support for it	2022-06-21 10:27:16 +08:00
arraymap.c	bpf: Add map side support for bpf timers.	2022-06-21 10:27:17 +08:00
bpf_inode_storage.c	bpf: Implement bpf_local_storage for inodes	2022-06-21 10:27:15 +08:00
bpf_iter.c	bpf: Add bpf_for_each_map_elem() helper	2022-06-21 10:27:16 +08:00
bpf_local_storage.c	bpf: Split bpf_local_storage to bpf_sk_storage	2022-06-21 10:27:15 +08:00
bpf_lru_list.c	bpf_lru_list: Read double-checked variable once without lock	2021-04-12 12:52:15 +08:00
bpf_lru_list.h	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
bpf_lsm.c	bpf: Allow local storage to be used from LSM programs	2022-06-21 10:27:15 +08:00
bpf_struct_ops.c	bpf: Set map_btf_{name, id} for all map types	2022-06-21 10:27:13 +08:00
bpf_struct_ops_types.h	bpf: tcp: Support tcp_congestion_ops in bpf	2022-06-21 10:27:05 +08:00
btf.c	bpf: Remember BTF of inner maps.	2022-06-21 10:27:17 +08:00
cgroup.c	bpf: Add support for forced LINK_DETACH command	2022-06-21 10:27:19 +08:00
core.c	bpf: Stop caching subprog index in the bpf_pseudo_func insn	2022-06-21 10:27:21 +08:00
cpumap.c	bpf: Set map_btf_{name, id} for all map types	2022-06-21 10:27:13 +08:00
devmap.c	bpf: Set map_btf_{name, id} for all map types	2022-06-21 10:27:13 +08:00
disasm.c	bpf: Introduce BPF nospec instruction for mitigating Spectre v4	2022-06-21 10:27:02 +08:00
disasm.h	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
dispatcher.c	bpf: Introduce BPF dispatcher	2022-06-21 10:27:06 +08:00
hashtab.c	bpf: Add map side support for bpf timers.	2022-06-21 10:27:17 +08:00
helpers.c	bpf: Implement verifier support for validation of async callbacks.	2022-06-21 10:27:17 +08:00
inode.c	bpf: Create file bpf iterator	2022-06-21 10:27:11 +08:00
local_storage.c	bpf: Add map side support for bpf timers.	2022-06-21 10:27:17 +08:00
lpm_trie.c	bpf: Set map_btf_{name, id} for all map types	2022-06-21 10:27:13 +08:00
map_in_map.c	bpf: Remember BTF of inner maps.	2022-06-21 10:27:17 +08:00
map_in_map.h	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
map_iter.c	bpf: Change uapi for bpf iterator map elements	2022-06-21 10:27:15 +08:00
net_namespace.c	bpf: Add support for forced LINK_DETACH command	2022-06-21 10:27:19 +08:00
offload.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
percpu_freelist.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
percpu_freelist.h	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
prog_iter.c	bpf: Refactor bpf_iter_reg to have separate seq_info member	2022-06-21 10:27:14 +08:00
queue_stack_maps.c	bpf: Set map_btf_{name, id} for all map types	2022-06-21 10:27:13 +08:00
reuseport_array.c	bpf: Set map_btf_{name, id} for all map types	2022-06-21 10:27:13 +08:00
ringbuf.c	bpf: Implement BPF ring buffer and verifier support for it	2022-06-21 10:27:16 +08:00
stackmap.c	bpf: Resolve BTF IDs in vmlinux image	2022-06-21 10:27:14 +08:00
syscall.c	bpf: Fix leak in LINK_UPDATE and enforce empty old_prog_fd	2022-06-21 10:27:19 +08:00
sysfs_btf.c	bpf: Fix sysfs export of empty BTF section	2021-03-16 16:37:55 +08:00
task_iter.c	bpf: Refactor to provide aux info to bpf_iter_init_seq_priv_t	2022-06-21 10:27:14 +08:00
tnum.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
trampoline.c	bpf: lsm: Implement attach, detach and execution	2022-06-21 10:27:08 +08:00
verifier.c	bpf: Stop caching subprog index in the bpf_pseudo_func insn	2022-06-21 10:27:21 +08:00
xskmap.c	bpf: Set map_btf_{name, id} for all map types	2022-06-21 10:27:13 +08:00