upstream: 51c48b310183ab6ba5419edfc6a8de889cc04521
pci_bridge_check_ranges() determines whether a bridge supports the optional
I/O and prefetchable memory windows and sets the flag bits in the bridge
resources. This *could* be done once during enumeration except that the
resource allocation code completely clears the flag bits, e.g., in the
pci_assign_unassigned_bridge_resources() path.
The problem with pci_bridge_check_ranges() in the resource allocation path
is that we may allocate resources after devices have been claimed by
drivers, and pci_bridge_check_ranges() *changes* the window registers to
determine whether they're writable. This may break concurrent accesses to
devices behind the bridge.
Add a new pci_read_bridge_windows() to determine whether a bridge supports
the optional windows, call it once during enumeration, remember the
results, and change pci_bridge_check_ranges() so it doesn't touch the
bridge windows but sets the flag bits based on those remembered results.
Signed-off-by: Lei Chen <lennychen@tencent.com>
Run an asynchronous function on a many cpus.
smp_call_function_many_async_begin
for_each_csd
smp_call_function_many_async
smp_call_function_many_async_end
Signed-off-by: yongduan <yongduan@tencent.com>
We have limited the length of the inode prealloc list, and here are
some optimizations for the calculation of the length of the list.
Signed-off-by: Chunguang Xu <brookxu@tencent.com>
In the scenario of writing sparse files, the Per-inode prealloc list may
be very long, resulting in high overhead for ext4_mb_use_preallocated().
To circumvent this problem, we limit the maximum length of per-inode
prealloc list to 512 and allow users to modify it.
Signed-off-by: Chunguang Xu <brookxu@tencent.com>
1) switch to default net ns during entry function
2) switch to skb's dev net ns during route
3) DNAT to local ip will set skb->dst to loopback dev
4) rename bpf_mode_on to ipvs_mode
Signed-off-by: jianmingfan <jianmingfan@tencent.com>
upstream: 5424ea27390f ("netns: get more entropy from net_hash_mix()")
struct net are effectively allocated from order-1 pages on x86,
with one object per slab, meaning that the 13 low order bits
of their addresses are zero.
Once shifted by L1_CACHE_SHIFT, this leaves 7 zero-bits,
meaning that net_hash_mix() does not help spreading
objects on various hash tables.
For example, TCP listen table has 32 buckets, meaning that
all netns use the same bucket for port 80 or port 443.
Signed-off-by: Hongbo Li <herberthbli@tencent.com>
[tkernel2 commit c6f2e27f7ad]
add SO_MARK2 to get the MARK of flow
Signed-off-by: brookxu <brookxu@tencent.com>
Signed-off-by: Zhiping Du <zhipingdu@tencent.com>
previously, for clusterIP type service, no SNAT is done.
However, in some case, user may add rs ip outside vpc which may require
SNAT.
To address this issue, a non snat ip table of max 64 entries are added.
Usage:
1. echo -n "0.0.0.0/0" > /proc/net/ip_vs_non_masq_cidrs will make all ip
bypass snat.
2. echo -n ":" > /proc/net/ip_vs_non_masq_cidrs will make all ip do snat.
3. echo "a.b.c.d/24:a.b.c.e/24" > /proc/net/ip_vs_non_masq_cidrs
Test case:
create a cluster with 9 PODS
172.19.0.175 172.19.0.176 172.19.0.177 172.19.0.241 172.19.0.242
172.19.0.243 172.19.0.244 172.19.0.100 172.19.0.101
0)
/proc/net/ip_vs_non_masq_cidrs is empty, curl cluster ip shall do SNAT
result: pass
1)
Three ip/32 in /proc/net/ip_vs_non_masq_cidrs
echo "172.19.0.175/32:172.19.0.176/32:172.19.0.177/32" > /proc/net/ip_vs_non_masq_cidrs
Test: curl the clusterip, and watch the tcpdump log
Expected result:
access to the ip in list no SNAT; access to the ip not in list do SNAT.
Result: pass, no leak.
2) stress test
wrk the clusterip , at the same time, run a program to modify the ip_vs_non_masq_cidrs in a loop
Expected result: curl ok. lo leak
while [ 1 ]
do
echo "172.19.0.175/32:172.19.0.176/32:172.19.0.177/32" > /proc/net/ip_vs_non_masq_cidrs
sleep 1
cat /proc/net/ip_vs_non_masq_cidrs
echo ""
echo "172.19.0.175/32" > /proc/net/ip_vs_non_masq_cidrs
sleep 1
cat /proc/net/ip_vs_non_masq_cidrs
echo ""
done
result: no leak;
3) corner test
write "0.0.0.0/0" to it.
expected result: shall not do SNAT.
result: ok
echo -n ":
expected result: do SNAT
result: ok
Signed-off-by: jianmingfan <jianmingfan@tencent.com>
This commit add cpu.offline to cpu cgroup, echo 1 > cpu.offline would
convert all tasks under this cgroup to offline task. Beside, a new
sysctl sysctl_sched_bt_ignore_cpubind is added, which makes offline
tasks ignoring CPU binding and running on any CPU.
Signed-off-by: Xiaoming Gao <newtongao@tencent.com>
Signed-off-by: Hua Liu <shookliu@tencent.com>
Signed-off-by: Xiaogguang Chen <xiaoggchen@tencent.com>
Signed-off-by: Zhiguang Peng <zgpeng@tencent.com>
Signed-off-by: Bin Fan <tombinfan@tencent.com>
Signed-off-by: He Chen <heddchen@tencent.com>
Since 'commit f719e3754ee2 ("ipvs: drop first packet to
redirect conntrack")', when a new TCP connection meet
the conditions that need reschedule, the first syn packet
is dropped, this cause one second latency for the new
connection, more discussion about this problem can easy
search from google, such as:
1)One second connection delay in masque
https://marc.info/?t=151683118100004&r=1&w=2
2)IPVS low throughput #70747
https://github.com/kubernetes/kubernetes/issues/70747
3)Apache Bench can fill up ipvs service proxy in seconds #544https://github.com/cloudnativelabs/kube-router/issues/544
4)Additional 1s latency in `host -> service IP -> pod`
https://github.com/kubernetes/kubernetes/issues/90854
5)kube-proxy ipvs conn_reuse_mode setting causes errors
with high load from single client
https://github.com/kubernetes/kubernetes/issues/81775
The root cause is when the old session is expired, the
conntrack related to the session is dropped by
ip_vs_conn_drop_conntrack. The code is as follows:
```
static void ip_vs_conn_expire(struct timer_list *t)
{
...
if ((cp->flags & IP_VS_CONN_F_NFCT) &&
!(cp->flags & IP_VS_CONN_F_ONE_PACKET)) {
/* Do not access conntracks during subsys cleanup
* because nf_conntrack_find_get can not be used after
* conntrack cleanup for the net.
*/
smp_rmb();
if (ipvs->enable)
ip_vs_conn_drop_conntrack(cp);
}
...
}
```
As shown in the code, only when condition (cp->flags & IP_VS_CONN_F_NFCT)
is true, the function ip_vs_conn_drop_conntrack will be called.
So we optimize this by following steps (Administrators
can choose the following optimization by setting
net.ipv4.vs.conn_reuse_old_conntrack=1):
1) erase the IP_VS_CONN_F_NFCT flag (it is safely because
no packets will use the old session)
2) call ip_vs_conn_expire_now to release the old session,
then the related conntrack will not be dropped
3) then ipvs unnecessary to drop the first syn packet, it
just continue to pass the syn packet to the next process,
create a new ipvs session, and the new session will related
to the old conntrack(which is reopened by conntrack as a new
one), the next whole things is just as normal as that the old
session isn't used to exist.
The above processing has no problems except for passive FTP,
for passive FTP situation, ipvs can judging from
condition (atomic_read(&cp->n_control)) and condition (cp->control).
So, for other conditions(means not FTP), ipvs should give users
the right to choose，they can choose a high performance one processing
logical by setting net.ipv4.vs.conn_reuse_old_conntrack=1. It is necessary
because most business scenarios (such as kubernetes) are very sensitive
to TCP short connection latency.
This patch has been verified on our thousands of kubernets
node servers on Tencent Inc.
Signed-off-by: YangYuxi <yx.atom1@gmail.com>
year 2015 d752c364571743d696c2a54a449ce77550c35ac5
year 2016 f719e3754ee2f7275437e61a6afd520181fdd43b
current only fix it in bpf mode. It will later be promoted to ipvs mode.
The key is that add a ref count in ct so that old/new ip_vs_conn can share it
without packet loss.
Test case
./wrk http 1.0 test, the cps increases from 1.5K to 30K.
2) solve no route to host bug
when conn_reuse_mode = 0, new connection may be redirect to rs with weight=0
if client port reuse. This cause icmp no route to host if the rs is
terminating
Test case:
1. wrk http1.0 from client
2. set rs to zero on lb, then kill the rs
in ipvs mode, you can see icmp error like
14:17:28.509454 IP 10.0.0.4 > 10.0.0.17: ICMP host 172.16.0.16 unreachable, length 68
in bpf mode, this is fixed.
Signed-off-by: jianmingfan <jianmingfan@tencent.com>
Offline tasks (BT tasks) may have some performance impact to online
tasks.
In this commit, we introduce Intel RDT features to limit offline tasks
L3 cache usage to avoid the influence caused by offline tasks.
Signed-off-by: Xiaoguang Chen <xiaoggchen@tencent.com>
Signed-off-by: He Chen <heddchen@tencent.com>
use lightweight track method to count free objects in
slub, remove heavy logic in spin_lock_irqsave.
Signed-off-by: Xiaoming Gao <newtongao@tencent.com>
commit <aac8d41cd438f25bf3110fc6b98f1d16d7dbc169>
commit <85009b4f5f0399669a44f07cb9a5622c0e71d419>
When someone calls wakeup_flusher_threads() or
wakeup_flusher_threads_bdi(), they schedule writeback of all dirty
pages in the system (or on that bdi). If we are tight on memory, we
can get tons of these queued from kswapd/vmscan. This causes (at
least) two problems:
1) We consume a ton of memory just allocating writeback work items.
We've seen as much as 600 million of these writeback work items
pending. That's a lot of memory to pointlessly hold hostage,
while the box is under memory pressure.
2) We spend so much time processing these work items, that we
introduce a softlockup in writeback processing. This is because
each of the writeback work items don't end up doing any work (it's
hard when you have millions of identical ones coming in to the
flush machinery), so we just sit in a tight loop pulling work
items and deleting/freeing them.
Fix this by adding a 'start_all' bit to the writeback structure, and
set that when someone attempts to flush all dirty pages. The bit is
cleared when we start writeback on that work item. If the bit is
already set when we attempt to queue !nr_pages writeback, then we
simply ignore it.
This provides us one full flush in flight, with one pending as well,
and makes for more efficient handling of this type of writeback.
Signed-off-by: Chunguang Xu <brookxu@tencent.com>
When there is no rs, kernel will print:
ipvs will print IPVS: rr: TCP xxxxxX - no destination available
But this is common in k8s.
So add a sysctl to allow user ignore this error.
Signed-off-by: Hongbo Li <herberthbli@tencent.com>
It does the following
* To bypass netfilter and conntrack in BPF mode
** Add mode=1 switch in modprobe to enable BPF mode
** Register different set of netfilter hooks in BPF mode so that
incoming and response packets are handled in NF PRE-ROUTING
bypassing conntrack hooks.
** Don't access skb->dst if it is null.
** Force route in handle_response
** Call ip_finish_output directly instead of Netfilter api.
** User space shall set /proc/sys/net/ipv4/vs/conntrack to be zero
** Call ip defragment during pre-route when needed!
* To interact with BPF map
** ip_vs_conn_new_bpf add entries into BPF map
** ip_vs_conn_unlink_bpf del them
** in lc,wlc,rr,wrr scheduler, avoid loopback traffic since BPF can't
handle it currently.
* Interface
** Add file ip_vs_bpf_proc.c to pass BPF map id into IPVS.
Add the max_map_count isolation switch kernel.isolate_max_map_count;
The isolation of max_map_count is turned on by default. If you want
to turn it off, set kernel.isolate_max_map_count to 0;
Signed-off-by: Zhiguang Peng <zgpeng@tencent.com>
Kernel now supports cgroup bps and iops hard limit by io-throttling.
But that's not a scalable way, because block devices' bandwith is
always changing over time.
This patch dynamically adjusts bps limit of each blkio cgroup according
to its pre-set weight. Additionally, if a process has not issued any io
request during a time window, its weight will be shared by other processes
until its next io request.
Signed-off-by: Lei Chen <lennychen@tencent.com>
The offline scheduler named BT sched is based on the CFS scheduler. We
also use the rb-tree as the run queue to save the runnable tasks. And the
vruntime concept is also used in the offline scheduler. And the priority
of offline scheduler is from 140 to 179. So now the schedulers in the
kernel are as follows: stop, RT, CFS, BT and idle.
Signed-off-by: Xiaoming Gao <newtongao@tencent.com>
Signed-off-by: Hua Liu <shookliu@tencent.com>
Signed-off-by: Xiaogguang Chen <xiaoggchen@tencent.com>
Signed-off-by: Zhiguang Peng <zgpeng@tencent.com>
Signed-off-by: Bin Fan <tombinfan@tencent.com>
Signed-off-by: He Chen <heddchen@tencent.com>
This patch adds the infrastructure required to support cephfs quotas as it
is currently implemented in the ceph fuse client. Cephfs quotas can be
set on any directory, and can restrict the number of bytes or the number
of files stored beneath that point in the directory hierarchy.
Quotas are set using the extended attributes 'ceph.quota.max_files' and
'ceph.quota.max_bytes', and can be removed by setting these attributes to
'0'.
Link: http://tracker.ceph.com/issues/22372
Signed-off-by: Luis Henriques <lhenriques@suse.com>
Reviewed-by: "Yan, Zheng" <zyan@redhat.com>
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Which control whether to resend requests if timeout. Further more,
the patch adds a debugfs file which can change this option during mount.
Signed-off-by: Liu Yu <allanyuliu@tencent.com>
When osd requests timeout, the defaut action is discarding all timeout
requests and reseting the connection. It then returns IO errors to
userspace.
The patch re-send all timeout requests instead of discarding,
so that make the timeout transparent to userspace.
Signed-off-by: Liu Yu <allanyuliu@tencent.com>
Based on upstream commit:
d63bae0 KVM: X86: Add kvm hypervisor init time platform setup callback
aaffcfd KVM: X86: Implement PV IPIs in linux guest
Implement paravirtual apic hooks to enable PV IPIs for KVM if the "send IPI"
hypercall is available. The hypercall lets a guest send IPIs, with
at most 128 destinations per hypercall in 64-bit mode and 64 vCPUs per
hypercall in 32-bit mode.
Signed-off-by: loobinliu <loobinliu@tencent.com>
The value of the cpuset.stat process field will be abnormal in
some scenarios. The reason for this problem is that the sum of
process_counts may be negative for several cpus.
Signed-off-by: brookxu <brookxu@tencent.com>
tcp_rto_min,tcp_rto_max control the parameters of them.
remember that the min value of tcp_rto_min is 4ms which
be used to be div by RTO_MAX.
Signed-off-by: Shan Wei <davidshan@tencent.com>
Signed-off-by: Fuhai Wang <fuhaiwang@tencent.com>
Signed-off-by: Xiaoming Gao <newtongao@tencent.com>
In order to facilitate each container to obtain its own IO statistics,
we implement per blkcg diskstats and expose some data from the host
into the container such as io_ticks.
Signed-off-by: Chunguang Xu <brookxu@tencent.com>
Lei Chen