Zhiping du 48cd975825 bpf: Add support for BTF pointers to x86 JIT ... [upstream commit 3dec541b2e632d630fe7142ed44f0b3702ef1f8c] Pointer to BTF object is a pointer to kernel object or NULL. Such pointers can only be used by BPF_LDX instructions. The verifier changed their opcode from LDX|MEM|size to LDX|PROBE_MEM|size to make JITing easier. The number of entries in extable is the number of BPF_LDX insns that access kernel memory via "pointer to BTF type". Only these load instructions can fault. Since x86 extable is relative it has to be allocated in the same memory region as JITed code. Allocate it prior to last pass of JITing and let the last pass populate it. Pointer to extable in bpf_prog_aux is necessary to make page fault handling fast. Page fault handling is done in two steps: 1. bpf_prog_kallsyms_find() finds BPF program that page faulted. It's done by walking rb tree. 2. then extable for given bpf program is binary searched. This process is similar to how page faulting is done for kernel modules. The exception handler skips over faulting x86 instruction and initializes destination register with zero. This mimics exact behavior of bpf_probe_read (when probe_kernel_read faults dest is zeroed). JITs for other architectures can add support in similar way. Until then they will reject unknown opcode and fallback to interpreter. Since extable should be aligned and placed near JITed code make bpf_jit_binary_alloc() return 4 byte aligned image offset, so that extable aligning formula in bpf_int_jit_compile() doesn't need to rely on internal implementation of bpf_jit_binary_alloc(). On x86 gcc defaults to 16-byte alignment for regular kernel functions due to better performance. JITed code may be aligned to 16 in the future, but it will use 4 in the meantime. Signed-off-by: Alexei Starovoitov <ast@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Acked-by: Andrii Nakryiko <andriin@fb.com> Acked-by: Martin KaFai Lau <kafai@fb.com> Link: https://lore.kernel.org/bpf/20191016032505.2089704-10-ast@kernel.org		2022-07-21 11:32:30 +08:00
..
bpf	bpf: Add support for BTF pointers to x86 JIT	2022-07-21 11:32:30 +08:00
cgroup	sli/cpu: replace tabs with spaces as seperators	2022-03-02 10:48:07 +08:00
configs	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
debug	kdb: Make memory allocations more robust	2021-04-12 12:52:10 +08:00
dma	swiotlb: fix "x86: Don't panic if can not alloc buffer for swiotlb"	2021-03-16 16:40:57 +08:00
events	perf/core: Fix unconditional security_locked_down() call	2021-05-20 16:07:19 +08:00
gcov	gcov: re-fix clang-11+ support	2021-05-20 16:06:55 +08:00
irq	genirq/matrix: Prevent allocation counter corruption	2021-05-20 16:07:24 +08:00
livepatch	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
locking	locking/qspinlock: Introduce llc aware spinlock	2022-03-02 10:48:05 +08:00
power	PM: EM: postpone creating the debugfs dir till fs_initcall	2021-04-12 12:53:30 +08:00
printk	printk: fix deadlock when kernel panic	2021-04-12 12:52:39 +08:00
rcu	rcu/nocb: Perform deferred wake up before last idle's need_resched() check	2021-04-12 12:52:37 +08:00
sched	rq-qos: fix missed wake-ups in rq_qos_throttle try two	2022-04-12 16:58:30 +08:00
time	posix-timers: Preserve return value in clock_adjtime32()	2021-05-20 16:07:22 +08:00
tkernel	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
trace	bpf: Implement accurate raw_tp context access via BTF	2022-07-21 11:32:30 +08:00
.gitignore	kbuild: update config_data.gz only when the content of .config is changed	2021-05-20 16:07:35 +08:00
Kconfig.freezer	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
Kconfig.hz	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
Kconfig.locks	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
Kconfig.preempt	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
Makefile	kbuild: update config_data.gz only when the content of .config is changed	2021-05-20 16:07:35 +08:00
acct.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
async.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
audit.c	audit: fix a net reference leak in audit_list_rules_send()	2021-03-16 16:29:25 +08:00
audit.h	audit: fix a net reference leak in audit_list_rules_send()	2021-03-16 16:29:25 +08:00
audit_fsnotify.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
audit_tree.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
audit_watch.c	audit: CONFIG_CHANGE don't log internal bookkeeping as an event	2021-03-16 16:36:51 +08:00
auditfilter.c	audit: fix a net reference leak in audit_list_rules_send()	2021-03-16 16:29:25 +08:00
auditsc.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
backtracetest.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
bounds.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
capability.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
compat.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
configs.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
context_tracking.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
cpu.c	kernel/cpu: add arch override for clear_tasks_mm_cpumask() mm handling	2021-03-16 16:42:28 +08:00
cpu_pm.c	kernel/cpu_pm: Fix uninitted local in cpu_pm	2021-03-16 16:29:48 +08:00
crash_core.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
crash_dump.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
cred.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
delayacct.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
dma.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
exec_domain.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
exit.c	don't dump the threads that had been already exiting when zapped.	2021-03-16 16:40:57 +08:00
extable.c	bpf: Add support for BTF pointers to x86 JIT	2022-07-21 11:32:30 +08:00
fail_function.c	fail_function: Remove a redundant mutex unlock	2021-03-16 16:41:23 +08:00
fork.c	exec: Transform exec_update_mutex into a rw_semaphore	2021-04-12 12:51:05 +08:00
freezer.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
futex.c	Revert 337f13046ff0 ("futex: Allow FUTEX_CLOCK_REALTIME with FUTEX_WAIT op")	2021-05-20 16:07:35 +08:00
gen_kheaders.sh	kbuild: add variables for compression tools	2021-03-16 16:35:19 +08:00
groups.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
hung_task.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
iomem.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
irq_work.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
jump_label.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
kallsyms.c	kallsyms: Refactor kallsyms_show_value() to take cred	2021-03-16 16:31:37 +08:00
kcmp.c	exec: Transform exec_update_mutex into a rw_semaphore	2021-04-12 12:51:05 +08:00
kcov.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
kexec.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
kexec_core.c	kernel: kexec: remove the lock operation of system_transition_mutex	2021-04-12 12:51:43 +08:00
kexec_elf.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
kexec_file.c	ima: Free IMA measurement buffer after kexec syscall	2021-04-12 12:52:21 +08:00
kexec_internal.h	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
kheaders.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
kmod.c	kmod: make request_module() return an error when autoloading is disabled	2021-03-16 16:26:15 +08:00
kprobes.c	tracing/kprobe: Fix to support kretprobe events on unloaded modules	2021-04-12 12:51:58 +08:00
ksysfs.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
kthread.c	kthread: Fix PF_KTHREAD vs to_kthread() race	2022-03-02 10:48:07 +08:00
latencytop.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
module-internal.h	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
module.c	kabi: check modver major version and features	2021-06-23 07:24:18 +00:00
module_signature.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
module_signing.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
notifier.c	kernel/notifier.c: intercept duplicate registrations to avoid infinite loops	2021-03-16 16:36:43 +08:00
nsproxy.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
padata.c	padata: add separate cpuhp node for CPUHP_PADATA_DEAD	2021-03-16 16:28:57 +08:00
panic.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
params.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
pid.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
pid_namespace.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
profile.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
ptrace.c	ptrace: Set PF_SUPERPRIV when checking capability	2021-03-16 16:41:32 +08:00
range.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
reboot.c	reboot: fix overflow parsing reboot cpu number	2021-03-16 16:40:55 +08:00
relay.c	kernel/relay.c: fix memleak on destroy relay channel	2021-03-16 16:34:28 +08:00
resource.c	/dev/mem: Revoke mappings when a driver claims the region	2021-03-16 16:30:25 +08:00
rseq.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
seccomp.c	seccomp: Add missing return in non-void function	2021-04-12 12:52:36 +08:00
signal.c	ptrace: fix task_join_group_stop() for the case when current is traced	2021-03-16 16:40:18 +08:00
smp.c	smp: Fix smp_call_function_single_async prototype	2021-05-20 16:08:01 +08:00
smpboot.c	kthread: Extract KTHREAD_IS_PER_CPU	2021-04-12 12:51:51 +08:00
smpboot.h	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
softirq.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
stackleak.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
stacktrace.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
stop_machine.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
sys.c	kernel/sys.c: avoid copying possible padding bytes in copy_to_user	2021-03-16 16:36:43 +08:00
sys_ni.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
sysctl-test.c	kernel/sysctl-test: Add null pointer test for sysctl.c:proc_dointvec()	2021-03-16 16:36:31 +08:00
sysctl.c	Make CFS bandwidth controller burstable	2022-03-02 10:48:06 +08:00
sysctl_binary.c	net: change TCP_TIMEWAIT_LEN on runtime	2021-03-16 16:43:58 +08:00
task_work.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
taskstats.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
test_kprobes.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
torture.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
tracepoint.c	tracepoint: Do not fail unregistering a probe due to memory failure	2021-04-12 12:52:27 +08:00
tsacct.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
ucount.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
uid16.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
uid16.h	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
umh.c	usermodehelper: reset umask to default before executing user process	2021-03-16 16:37:54 +08:00
up.c	smp: Fix smp_call_function_single_async prototype	2021-05-20 16:08:01 +08:00
user-return-notifier.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
user.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
user_namespace.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
utsname.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
utsname_sysctl.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
watchdog.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
watchdog_hld.c	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00
workqueue.c	workqueue: Move the position of debug_work_activate() in __queue_work()	2021-05-20 16:07:02 +08:00
workqueue_internal.h	Init Repo base on linux 5.4.32 long term, and add base tlinux kernel interfaces.	2021-03-16 11:01:34 +08:00