In the virtualization scenario, you can use pvipi to optimize the sending of
IPI interrupts and reduce the number of vmexit.
rps_using_pvipi test
Test guest: 32 core, 64G memory, virtio-net 8 queues, rss interrupt is bound
to 24-31 core. Observe the number of vmexit of 31 cores.
1. 1000000pps
100 iperf instances, 10000pps per instance.
disable rps_using_pvipi 68077 vmexit/s
enable rps_using_pvipi 21138 vmexit/s
2. 2000000pps
1000 iperf instances, 2000pps per instance.
disable rps_using_pvipi 40879 vmexit/s
enable rps_using_pvipi 9195 vmexit/s
3. 2000000pps
2000 iperf instances, 1000pps per instance.
disable rps_using_pvipi 24746 vmexit/s
enable rps_using_pvipi 7292 vmexit/s
Enabling rps_using_pvipi can greatly reduce the number of vmexit.
Signed-off-by: yongduan <yongduan@tencent.com>
reason: in ipvs_mode ==1 , skb_rtable is ip_vs_nat_xmit is null.
Test case: test bpf mode nodeport ok!
Signed-off-by: jianmingfan <jianmingfan@tencent.com>
upstream: acf69c9
Using tp_reserve to calculate netoff can overflow as
tp_reserve is unsigned int and netoff is unsigned short.
This may lead to macoff receiving a smaller value then
sizeof(struct virtio_net_hdr), and if po->has_vnet_hdr
is set, an out-of-bounds write will occur when
calling virtio_net_hdr_from_skb.
The bug is fixed by converting netoff to unsigned int
and checking if it exceeds USHRT_MAX.
This addresses CVE-2020-14386
Fixes: 8913336a7e8d ("packet: add PACKET_RESERVE sockopt")
Signed-off-by: Or Cohen <orcohen@paloaltonetworks.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Fuhai Wang <fuhaiwang@tencent.com>
1) switch to default net ns during entry function
2) switch to skb's dev net ns during route
3) DNAT to local ip will set skb->dst to loopback dev
4) rename bpf_mode_on to ipvs_mode
Signed-off-by: jianmingfan <jianmingfan@tencent.com>
cat/proc/net/nf_conntrack may cause long latency.
Add cond_resched_rcu() when reading this proc file.
Signed-off-by: Hongbo Li <herberthbli@tencent.com>
[tkernel2 commit c6f2e27f7ad]
add SO_MARK2 to get the MARK of flow
Signed-off-by: brookxu <brookxu@tencent.com>
Signed-off-by: Zhiping Du <zhipingdu@tencent.com>
previously, for clusterIP type service, no SNAT is done.
However, in some case, user may add rs ip outside vpc which may require
SNAT.
To address this issue, a non snat ip table of max 64 entries are added.
Usage:
1. echo -n "0.0.0.0/0" > /proc/net/ip_vs_non_masq_cidrs will make all ip
bypass snat.
2. echo -n ":" > /proc/net/ip_vs_non_masq_cidrs will make all ip do snat.
3. echo "a.b.c.d/24:a.b.c.e/24" > /proc/net/ip_vs_non_masq_cidrs
Test case:
create a cluster with 9 PODS
172.19.0.175 172.19.0.176 172.19.0.177 172.19.0.241 172.19.0.242
172.19.0.243 172.19.0.244 172.19.0.100 172.19.0.101
0)
/proc/net/ip_vs_non_masq_cidrs is empty, curl cluster ip shall do SNAT
result: pass
1)
Three ip/32 in /proc/net/ip_vs_non_masq_cidrs
echo "172.19.0.175/32:172.19.0.176/32:172.19.0.177/32" > /proc/net/ip_vs_non_masq_cidrs
Test: curl the clusterip, and watch the tcpdump log
Expected result:
access to the ip in list no SNAT; access to the ip not in list do SNAT.
Result: pass, no leak.
2) stress test
wrk the clusterip , at the same time, run a program to modify the ip_vs_non_masq_cidrs in a loop
Expected result: curl ok. lo leak
while [ 1 ]
do
echo "172.19.0.175/32:172.19.0.176/32:172.19.0.177/32" > /proc/net/ip_vs_non_masq_cidrs
sleep 1
cat /proc/net/ip_vs_non_masq_cidrs
echo ""
echo "172.19.0.175/32" > /proc/net/ip_vs_non_masq_cidrs
sleep 1
cat /proc/net/ip_vs_non_masq_cidrs
echo ""
done
result: no leak;
3) corner test
write "0.0.0.0/0" to it.
expected result: shall not do SNAT.
result: ok
echo -n ":
expected result: do SNAT
result: ok
Signed-off-by: jianmingfan <jianmingfan@tencent.com>
Since 'commit f719e3754ee2 ("ipvs: drop first packet to
redirect conntrack")', when a new TCP connection meet
the conditions that need reschedule, the first syn packet
is dropped, this cause one second latency for the new
connection, more discussion about this problem can easy
search from google, such as:
1)One second connection delay in masque
https://marc.info/?t=151683118100004&r=1&w=2
2)IPVS low throughput #70747
https://github.com/kubernetes/kubernetes/issues/70747
3)Apache Bench can fill up ipvs service proxy in seconds #544https://github.com/cloudnativelabs/kube-router/issues/544
4)Additional 1s latency in `host -> service IP -> pod`
https://github.com/kubernetes/kubernetes/issues/90854
5)kube-proxy ipvs conn_reuse_mode setting causes errors
with high load from single client
https://github.com/kubernetes/kubernetes/issues/81775
The root cause is when the old session is expired, the
conntrack related to the session is dropped by
ip_vs_conn_drop_conntrack. The code is as follows:
```
static void ip_vs_conn_expire(struct timer_list *t)
{
...
if ((cp->flags & IP_VS_CONN_F_NFCT) &&
!(cp->flags & IP_VS_CONN_F_ONE_PACKET)) {
/* Do not access conntracks during subsys cleanup
* because nf_conntrack_find_get can not be used after
* conntrack cleanup for the net.
*/
smp_rmb();
if (ipvs->enable)
ip_vs_conn_drop_conntrack(cp);
}
...
}
```
As shown in the code, only when condition (cp->flags & IP_VS_CONN_F_NFCT)
is true, the function ip_vs_conn_drop_conntrack will be called.
So we optimize this by following steps (Administrators
can choose the following optimization by setting
net.ipv4.vs.conn_reuse_old_conntrack=1):
1) erase the IP_VS_CONN_F_NFCT flag (it is safely because
no packets will use the old session)
2) call ip_vs_conn_expire_now to release the old session,
then the related conntrack will not be dropped
3) then ipvs unnecessary to drop the first syn packet, it
just continue to pass the syn packet to the next process,
create a new ipvs session, and the new session will related
to the old conntrack(which is reopened by conntrack as a new
one), the next whole things is just as normal as that the old
session isn't used to exist.
The above processing has no problems except for passive FTP,
for passive FTP situation, ipvs can judging from
condition (atomic_read(&cp->n_control)) and condition (cp->control).
So, for other conditions(means not FTP), ipvs should give users
the right to choose，they can choose a high performance one processing
logical by setting net.ipv4.vs.conn_reuse_old_conntrack=1. It is necessary
because most business scenarios (such as kubernetes) are very sensitive
to TCP short connection latency.
This patch has been verified on our thousands of kubernets
node servers on Tencent Inc.
Signed-off-by: YangYuxi <yx.atom1@gmail.com>
year 2015 d752c364571743d696c2a54a449ce77550c35ac5
year 2016 f719e3754ee2f7275437e61a6afd520181fdd43b
current only fix it in bpf mode. It will later be promoted to ipvs mode.
The key is that add a ref count in ct so that old/new ip_vs_conn can share it
without packet loss.
Test case
./wrk http 1.0 test, the cps increases from 1.5K to 30K.
2) solve no route to host bug
when conn_reuse_mode = 0, new connection may be redirect to rs with weight=0
if client port reuse. This cause icmp no route to host if the rs is
terminating
Test case:
1. wrk http1.0 from client
2. set rs to zero on lb, then kill the rs
in ipvs mode, you can see icmp error like
14:17:28.509454 IP 10.0.0.4 > 10.0.0.17: ICMP host 172.16.0.16 unreachable, length 68
in bpf mode, this is fixed.
Signed-off-by: jianmingfan <jianmingfan@tencent.com>
[upstream commit 4e35c1cb9460]
It is possible that two concurrent packets originating from the same
socket of a connection-less protocol (e.g. UDP) can end up having
different IP_CT_DIR_REPLY tuples which results in one of the packets
being dropped.
To illustrate this, consider the following simplified scenario:
1. Packet A and B are sent at the same time from two different threads
by same UDP socket. No matching conntrack entry exists yet.
Both packets cause allocation of a new conntrack entry.
2. get_unique_tuple gets called for A. No clashing entry found.
conntrack entry for A is added to main conntrack table.
3. get_unique_tuple is called for B and will find that the reply
tuple of B is already taken by A.
It will allocate a new UDP source port for B to resolve the clash.
4. conntrack entry for B cannot be added to main conntrack table
because its ORIGINAL direction is clashing with A and the REPLY
directions of A and B are not the same anymore due to UDP source
port reallocation done in step 3.
This patch modifies nf_conntrack_tuple_taken so it doesn't consider
colliding reply tuples if the IP_CT_DIR_ORIGINAL tuples are equal.
[ Florian: simplify patch to not use .allow_clash setting
and always ignore identical flows ]
Signed-off-by: Martynas Pumputis <martynas@weave.works>
Signed-off-by: Florian Westphal <fw@strlen.de>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Signed-off-by: Zhiping Du <zhipingdu@tencent.com>
When there is no rs, kernel will print:
ipvs will print IPVS: rr: TCP xxxxxX - no destination available
But this is common in k8s.
So add a sysctl to allow user ignore this error.
Signed-off-by: Hongbo Li <herberthbli@tencent.com>
user can assign a dedicated ip address for lb snat. In such case
all ports can be used by lb. So init the local port range as 0,0.
Test case:
1. test ip isolation using k8s, the local port range is 0,0.
2. test port isolation using k8s, the local port range is [32768, 42767]
Result:pass
2. follow iptables nf_nat_l4proto_unique_tuple(),
map port [512, 1023] to [600, 1023]
Signed-off-by: Jianming Fan <jianmingfan@tencent.com>
It does the following
* To bypass netfilter and conntrack in BPF mode
** Add mode=1 switch in modprobe to enable BPF mode
** Register different set of netfilter hooks in BPF mode so that
incoming and response packets are handled in NF PRE-ROUTING
bypassing conntrack hooks.
** Don't access skb->dst if it is null.
** Force route in handle_response
** Call ip_finish_output directly instead of Netfilter api.
** User space shall set /proc/sys/net/ipv4/vs/conntrack to be zero
** Call ip defragment during pre-route when needed!
* To interact with BPF map
** ip_vs_conn_new_bpf add entries into BPF map
** ip_vs_conn_unlink_bpf del them
** in lc,wlc,rr,wrr scheduler, avoid loopback traffic since BPF can't
handle it currently.
* Interface
** Add file ip_vs_bpf_proc.c to pass BPF map id into IPVS.
upstream:
ee60ad219f5c ("route: set the deleted fnhe fnhe_daddr to 0 ")
The race occurs in __mkroute_output() when 2 threads lookup a dst:
CPU A CPU B
find_exception()
find_exception() [fnhe expires]
ip_del_fnhe() [fnhe is deleted]
rt_bind_exception()
In rt_bind_exception() it will bind a deleted fnhe with the new dst, and
this dst will get no chance to be freed. It causes a dev defcnt leak and
consecutive dmesg warnings:
unregister_netdevice: waiting for ethX to become free. Usage count = 1
Especially thanks Jon to identify the issue.
This patch fixes it by setting fnhe_daddr to 0 in ip_del_fnhe() to stop
binding the deleted fnhe with a new dst when checking fnhe's fnhe_daddr
and daddr in rt_bind_exception().
It works as both ip_del_fnhe() and rt_bind_exception() are protected by
fnhe_lock and the fhne is freed by kfree_rcu().
Fixes: deed49df7390 ("route: check and remove route cache when we get route")
Signed-off-by: Hongbo Li <herberthbli@tencent.com>
upstream:
b6653b3629e5 ("tcp: refine memory limit test in tcp_fragment")
b617158dc096 ("tcp: be more careful in tcp_fragment")
tcp_fragment() might be called for skbs in the write queue.
Memory limits might have been exceeded because tcp_sendmsg() only
checks limits at full skb (64KB) boundaries.
Therefore, we need to make sure tcp_fragment() wont punish applications
that might have setup very low SO_SNDBUF values.
75c119afe14f ("tcp: implement rb-tree based retransmit queue")
separate transmit queue to transmit queue and retransmit queue.
b6653b36 adds some check to the transmit queue, b617158dc adds some check
on the rtx queue.
So we only backport the b6653b36 patch. use tcp_send_head(sk) instead of
TCP_FRAG_IN_WRITE_QUEUE.
Signed-off-by: Hongbo Li <herberthbli@tencent.com>
when we namespacify net.ipv4.tcp_max_orphans, the default
value of host and container are both 8192. This patch
change the default value of host to the original value.
Signed-off-by: Hongbo Li <herberthbli@tencent.com>
This patch adds the infrastructure required to support cephfs quotas as it
is currently implemented in the ceph fuse client. Cephfs quotas can be
set on any directory, and can restrict the number of bytes or the number
of files stored beneath that point in the directory hierarchy.
Quotas are set using the extended attributes 'ceph.quota.max_files' and
'ceph.quota.max_bytes', and can be removed by setting these attributes to
'0'.
Link: http://tracker.ceph.com/issues/22372
Signed-off-by: Luis Henriques <lhenriques@suse.com>
Reviewed-by: "Yan, Zheng" <zyan@redhat.com>
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Which control whether to resend requests if timeout. Further more,
the patch adds a debugfs file which can change this option during mount.
Signed-off-by: Liu Yu <allanyuliu@tencent.com>
When osd requests timeout, the defaut action is discarding all timeout
requests and reseting the connection. It then returns IO errors to
userspace.
The patch re-send all timeout requests instead of discarding,
so that make the timeout transparent to userspace.
Signed-off-by: Liu Yu <allanyuliu@tencent.com>
tcp_rto_min,tcp_rto_max control the parameters of them.
remember that the min value of tcp_rto_min is 4ms which
be used to be div by RTO_MAX.
Signed-off-by: Shan Wei <davidshan@tencent.com>
Signed-off-by: Fuhai Wang <fuhaiwang@tencent.com>
Signed-off-by: Xiaoming Gao <newtongao@tencent.com>
[upstream commit 5f3e2bf008c2221478101ee72f5cb4654b9fc363]
Some TCP peers announce a very small MSS option in their SYN and/or
SYN/ACK messages.
This forces the stack to send packets with a very high network/cpu
overhead.
Linux has enforced a minimal value of 48. Since this value includes
the size of TCP options, and that the options can consume up to 40
bytes, this means that each segment can include only 8 bytes of payload.
In some cases, it can be useful to increase the minimal value
to a saner value.
We still let the default to 48 (TCP_MIN_SND_MSS), for compatibility
reasons.
Note that TCP_MAXSEG socket option enforces a minimal value
of (TCP_MIN_MSS). David Miller increased this minimal value
in commit c39508d6f118 ("tcp: Make TCP_MAXSEG minimum more correct.")
from 64 to 88.
We might in the future merge TCP_MIN_SND_MSS and TCP_MIN_MSS.
CVE-2019-11479 -- tcp mss hardcoded to 48
[upstream commit: 967c05aee439e6e5d7d805e195b3a20ef5c433d6]
tcp: enforce tcp_min_snd_mss in tcp_mtu_probing()
If mtu probing is enabled tcp_mtu_probing() could very well end up
with a too small MSS.
Use the new sysctl tcp_min_snd_mss to make sure MSS search
is performed in an acceptable range.
Signed-off-by: Zhiping Du <zhipingdu@tencent.com>
[upstream commit f070ef2ac66716357066b683fb0baf55f8191a2e]
Jonathan Looney reported that a malicious peer can force a sender
to fragment its retransmit queue into tiny skbs, inflating memory
usage and/or overflow 32bit counters.
TCP allows an application to queue up to sk_sndbuf bytes,
so we need to give some allowance for non malicious splitting
of retransmit queue.
A new SNMP counter is added to monitor how many times TCP
did not allow to split an skb if the allowance was exceeded.
Note that this counter might increase in the case applications
use SO_SNDBUF socket option to lower sk_sndbuf.
CVE-2019-11478 : tcp_fragment, prevent fragmenting a packet when the
socket is already using more than half the allowed space
Signed-off-by: Zhiping Du <zhipingdu@tencent.com>
[upstream commit 3b4929f65b0d8249f19a50245cd88ed1a2f78cff]
Jonathan Looney reported that TCP can trigger the following crash
in tcp_shifted_skb() :
BUG_ON(tcp_skb_pcount(skb) < pcount);
This can happen if the remote peer has advertized the smallest
MSS that linux TCP accepts : 48
An skb can hold 17 fragments, and each fragment can hold 32KB
on x86, or 64KB on PowerPC.
This means that the 16bit witdh of TCP_SKB_CB(skb)->tcp_gso_segs
can overflow.
Note that tcp_sendmsg() builds skbs with less than 64KB
of payload, so this problem needs SACK to be enabled.
SACK blocks allow TCP to coalesce multiple skbs in the retransmit
queue, thus filling the 17 fragments to maximal capacity.
CVE-2019-11477 -- u16 overflow of TCP_SKB_CB(skb)->tcp_gso_segs
[upstream commit cd4ffa93f16efea290bb70537f98f518e1927e63]
tcp: fix fack_count accounting on
tcp_shift_skb_data()
v4.15 or since commit 737ff314563 ("tcp: use sequence distance to
detect reordering") had switched from the packet-based FACK tracking
to sequence-based.
v4.14 and older still have the old logic and hence on
tcp_skb_shift_data() needs to retain its original logic and have
@fack_count in sync. In other words, we keep the increment of pcount
with
tcp_skb_pcount(skb) to later used that to update fack_count. To make it
more explicit we track the new skb that gets incremented to pcount in
@next_pcount, and we get to avoid the constant invocation of
tcp_skb_pcount(skb) all together.
Fixes: a5f1faa40101 ("tcp: limit payload size of sacked skbs")
Fixes: 832d11c5cd07 ("tcp: Try to restore large SKBs while SACK processing")
Signed-off-by: Zhiping Du <zhipingdu@tencent.com>
When use-gss-proxy is set to 1, write_gssp will lead to 2 get_net calls
in gssp_rpc_create, the paired put_net calls are supposed to be made
in rpcsec_gss_net_ops->exit i.e. rpcsec_gss_exit_net, but this will
never happen because rpcsec_gss_exit_net (via ops_exit_list) is called
by cleanup_net and cleanup_net will never run unless net->count becomes
0, since the paired put_net calls of gssp_rpc_create are not made yet,
net->count cannot drop to 0. This is dead lock situation.
This fix introduced a new method i.e. evict in struct proc_ns_operations,
which is called in nsfs_evict, moving rpcsec_gss_exit_net into the code
path of nsfs_evict gives it a chance to run.
Signed-off-by: Wenbin Zeng <wenbinzeng@tencent.com>
upstream: ed07d9a021df6da53456663a76999189badc432a
This patch enables the clash resolution for NAT (disabled in
"590b52e10d41") if clashing conntracks match (i.e. both tuples are equal)
and a protocol allows it.
The clash might happen for a connections-less protocol (e.g. UDP) when
two threads in parallel writes to the same socket and consequent calls
to "get_unique_tuple" return the same tuples (incl. reply tuples).
In this case it is safe to perform the resolution, as the losing CT
describes the same mangling as the winning CT, so no modifications to
the packet are needed, and the result of rules traversal for the loser's
packet stays valid.
Signed-off-by: Hongbo Li <herberthbli@tencent.com>
upstream commit: 356d1833b638bd465672aefeb71def3ab93fc17d
Note that when a new netns is created, it inherits its
sysctl_tcp_rmem and sysctl_tcp_wmem from initial netns.
This change is needed so that we can refine TCP rcvbuf autotuning,
to take RTT into consideration.
Signed-off-by: Hongbo Li <herberthbli@tencent.com>
upstream commit: a3dcaf17ee54f1d01d22cc2b22cab0b4f60d78cf
As we want to gradually implement per netns sysctl_rmem and sysctl_wmem
on per protocol basis, add two new fields in struct proto,
and two new helpers : sk_get_wmem0() and sk_get_rmem0()
First user will be TCP. Then UDP and SCTP can be easily converted,
while DECNET probably wont get this support.
Signed-off-by: Hongbo Li <herberthbli@tencent.com>
gxm-newtonf