Compare commits

..

No commits in common. "master" and "shookliu" have entirely different histories.

49 changed files with 255 additions and 976 deletions

View File

@ -1,4 +1,4 @@
![img](images/tencentos-logo.png)
![img](https://github.com/Tencent/TencentOS-kernel/blob/master/images/tencentos-logo.png)
# TencentOS Server kernel
@ -81,7 +81,7 @@
- 通过以下步骤编译内核rpm
```
git tag 4.14.105-19-0008.beta10
cd package/default
cd package/defeault
./generate-rpms.sh -j jobs_num
```
@ -130,17 +130,17 @@ TencentOS server的内核和用户态包的更新也会持续同步至腾讯软
隔离方案如图所示
![img](images/docker-isolation.jpg)
![img](https://github.com/Tencent/TencentOS-kernel/blob/master/images/docker-isolation.jpg)
tlinux内核在cgroup的memorycpuset等子系统中分别添加对应的文件输出然后由用户通过mount bind操作将同名文件绑定到container的proc中。Mount bind操作可以在docker启动container的流程中添加。
例如在memeory子系统对应的container目录中添加meminfo和vmstat文件。
![img](images/docker_isolation_img1.jpg)
![img](https://github.com/Tencent/TencentOS-kernel/blob/master/images/docker_isolation_img1.jpg)
在cpu子系统对应的container下实现cpuinfostat文件。
![img](images/docker_isolation_img2.jpg)
![img](https://github.com/Tencent/TencentOS-kernel/blob/master/images/docker_isolation_img2.jpg)
**3.** **文件接口说明**
@ -180,7 +180,7 @@ tlinux内核在cgroup的memorycpuset等子系统中分别添加对应的文
- blkio.diskstats的通过blkcg_diskstats对象统计当前blkcg对特定设备的io量由于单个blkcg可以访问多个设备因此blkcg会维护一个blkcg_diskstats队列。由于实际blkcg_diskstats队列长度较短同时为了提高blkcg_diskstats搜索效率我们设置了一个cache点用于缓存最近命中的blkcg_diskstats对象的地址。Io统计的基本流程是io提交阶段我们会将bio与blkcg进行绑定因为end_of_io函数的运行上下文非提交io进程的上下文因此我们需要通过bio确定相应的blkcg。如果当前bio可以与plug队列设备dispatch队列或者io调度器内部队列的request合并此时进行io_merged的统计Io完成的时候我们对io_sectorsio_serviced, io_wait_time的统计。in_flightio_ticks, time_in_queue这三个字段与物理设备的处理能力相关因此我们不单独进行统计全部填0然后追加了两个字段将母机侧的io_ticks, time_in_queue的值透传到容器里面。值得注意的是blkio.diskstats入口默认是关闭的用户需要通过echo 1 > blkio.diskstats打开方可获取当前cgroup的io统计。基本框架如下所示
![img](images/docker_blkcg_img1.jpg)
![img](https://github.com/Tencent/TencentOS-kernel/blob/master/images/docker_blkcg_img1.jpg)
@ -302,7 +302,7 @@ NSsid: 1 11126
- page cache在系统中的大致位置如下图所示
![img](images/page_cache_img1.png)
![img](https://github.com/Tencent/TencentOS-kernel/blob/master/images/page_cache_img1.png)
@ -389,7 +389,7 @@ NSsid: 1 11126
内核热补丁技术是一种无需重启服务器即可实现修改内核运行时代码的技术。基于该技术可以在不影响业务正常运行的情况下修复内核bug或者安全漏洞以提高运营效率、底层平台的稳定性和可用性并使得业务运营体验有效提升。
![img](images/hot_patch_img1.png)
![img](https://github.com/Tencent/TencentOS-kernel/blob/master/images/hot_patch_img1.png)
@ -407,12 +407,12 @@ arm64热补丁功能实现包括内核、编译器、用户态工具几部分。
kpatch在内核中是基于ftrace实现内核函数的替换类似于ftrace的动态探测点不过不是统计某些运行数据而是修改函数的运行序列在函数运行某些额外的代码之后略过旧函数代码并跳转至新函数。框架如下图所示
![img](images/hot_patch_img2.png)
![img](https://github.com/Tencent/TencentOS-kernel/blob/master/images/hot_patch_img2.png)
针对arm64架构整个流程可以细化为下图所示
![img](images/hot_patch_img3.png)
![img](https://github.com/Tencent/TencentOS-kernel/blob/master/images/hot_patch_img3.png)
@ -420,7 +420,7 @@ kpatch在内核中是基于ftrace实现内核函数的替换类似于ftrace
![img](images/hot_patch_img4.png)
![img](https://github.com/Tencent/TencentOS-kernel/blob/master/images/hot_patch_img4.png)
@ -436,18 +436,18 @@ kpatch在内核中是基于ftrace实现内核函数的替换类似于ftrace
**x86机器上**
![img](images/hot_patch_img5.png)
![img](https://github.com/Tencent/TencentOS-kernel/blob/master/images/hot_patch_img5.png)
x86机器上如果使用-mfentryelf文件中ftrace跳转指令位于prologue前面在由旧函数跳转到新函数后执行指令流程不会出错。如果使用mcount则在新函数前需要添加stub函数用于处理栈信息等。**arm64机器上**
![img](images/hot_patch_img6.png)
![img](https://github.com/Tencent/TencentOS-kernel/blob/master/images/hot_patch_img6.png)
Arm64只支持mcount功能但是arm64 prologue会对寄存器做修改所以无法使用stub函数来适配。所以采用gcc patchable-function-entry来实现类似于mfentry的功能。使用了GCC 8.2.1版本来编译内核rpm包链接地址https://tlinux-mirror.tencent-cloud.com/tlinux/2.4/arm64/tlinux-sclo/aarch64/tl/devtoolset-8/devtoolset-8-gcc-8.2.1-3.tl2.aarch64.rpm 。
![img](images/hot_patch_img7.png)
![img](https://github.com/Tencent/TencentOS-kernel/blob/master/images/hot_patch_img7.png)
@ -455,7 +455,7 @@ Arm64只支持mcount功能但是arm64 prologue会对寄存器做修改
热补丁中涉及到修改regs参数所以在ftrace跳转时需要将寄存器入栈所以针对arm64实现了ftrace with regs功能为热补丁功能做准备。x0 ~ x30入栈操作如下
![img](images/hot_patch_img8.png)
![img](https://github.com/Tencent/TencentOS-kernel/blob/master/images/hot_patch_img8.png)
@ -464,7 +464,7 @@ Arm64只支持mcount功能但是arm64 prologue会对寄存器做修改
包括ftrace_ops注册删除、模块载入时数据重定位等功能。
重定位简要代码如下:
![img](images/hot_patch_img9.png)
![img](https://github.com/Tencent/TencentOS-kernel/blob/master/images/hot_patch_img9.png)
@ -548,7 +548,7 @@ filling_function 在不同的架构下规则不同在arm64架构中主要
首先需要载入kpatch模块然后载入用户态工具生成的新函数模块。通过lsmod查看模块是否载入成功。同时kpatch提供了sysfs接口可以查看载入新函数模块的信息包括新旧函数地址等。可以通过`/sys/kernel/kpatch/xxx/enabled`来卸载模块,恢复执行原函数。
简要操作流程如下:
![img](images/hot_patch_img10.png)
![img](https://github.com/Tencent/TencentOS-kernel/blob/master/images/hot_patch_img10.png)
@ -668,9 +668,9 @@ int main()
为此好的混部方案就是将离在线业务彻底分开所以在调度算法这一层次就要做区分。基于这种考虑开发了针对离线业务的新调度算法bt该算法可以保证在线业务优先运行。新调度算法的基本算法借鉴于CFS但在CPU选择、抢占、负载均衡、时延处理、CPU带宽控制等多个方面都有自己的特点和要求有特有的处理方式。特别是配有特有的负载均衡策略、CPU带宽控制策略等。
整个的运行机制如下图:
![img](images/bt_sched_img01.png)
![img](https://github.com/Tencent/TencentOS-kernel/blob/master/images/bt_sched_img01.png)
![img](images/bt_sched_img02.png)
![img](https://github.com/Tencent/TencentOS-kernel/blob/master/images/bt_sched_img02.png)
其中蓝色代表使用新离线调度算法bt的离线业务橙色代表在线业务CPU的颜色代表哪种业务在运行。通过运行切换图可以看到1、只有离线业务时如同CFS一样可以均匀的分散到CPU上2、在线业务需要运行时可以及时的抢占离线业务占用的CPU且将离线业务排挤到其它离线业务占用的CPU上这样在线业务及时得到运行且离线也会占用剩余CPU存在个别离线业务无法运行的情况3、在线业务较多时可以均衡合理的占用所有CPU此时离线业务抢不到CPU4、在线业务休眠时离线业务可以及时的占用在线业务释放的CPU。
### 业务场景效果
@ -678,18 +678,18 @@ int main()
- 场景A
如下图所示在A测试场景中模块a一个用于统计频率的模块对时延非常敏感。此业务不能混部整机CPU利用率只有15%左右业务尝试过使用cgroup方案来混部但是cgroup方案混部之后对在线模块a影响太大导致错误次数陡增因此此模块一直不能混部。使用我们提供的方案之后可以发现CPU提升至60%,并且错误次数基本没有变化。
![img](images/bt_sched_img03.png)
![img](images/bt_sched_img04.png)
![img](https://github.com/Tencent/TencentOS-kernel/blob/master/images/bt_sched_img03.png)
![img](https://github.com/Tencent/TencentOS-kernel/blob/master/images/bt_sched_img04.png)
- 场景B
在B测试场景中模块b是一个翻译模块对时延很敏感原本b模块是不能混部的业务尝试过混部但是因为离线混部上去之后对模块b的影响很大时延变长所以一直不能混部。使用我们的方案的效果如下图所示整机CPU利用率从20%提升至50%,并且对模块没有影响,时延基本上没有变化
![img](images/bt_sched_img05.png)
![img](https://github.com/Tencent/TencentOS-kernel/blob/master/images/bt_sched_img05.png)
- 场景C
模块C对时延不像场景AB那么敏感所以在使用我们提供的方案之前利用cgroup方案进行混部CPU最高可以达到40%。但是平台不再敢往上压因为再往上压就会影响到在线c业务。如下图所示使用我们的方案之后平台不断往机器上添加离线业务将机器CPU压至90%的情况下c业务的各项指标还是正常并没有受到影响。
![img](images/bt_sched_img06.png)
![img](https://github.com/Tencent/TencentOS-kernel/blob/master/images/bt_sched_img06.png)
上面列的是腾讯内部使用BT调度算法的效果。有兴趣的同学可以在自己的业务场景中进行适用真实的去体验腾讯离在线混部方案的效果。具体使用方法详见下面的使用指南。
@ -697,27 +697,27 @@ int main()
我们提供了一个启动参数offline_class来支持用户程序使用离线调度。
设置offline_class即使能了离线调度用户可以通过sched_setscheduler函数把一个进程设置成离线调度
![img](images/bt_sched_img07.png)
![img](https://github.com/Tencent/TencentOS-kernel/blob/master/images/bt_sched_img07.png)
其中7表示离线调度。
设置成功后我们可以用top比较下设置前后进程的优先级变化
设置前:
![img](images/bt_sched_img08.png)
![img](https://github.com/Tencent/TencentOS-kernel/blob/master/images/bt_sched_img08.png)
设置成离线调度后:
![img](images/bt_sched_img09.png)
![img](https://github.com/Tencent/TencentOS-kernel/blob/master/images/bt_sched_img09.png)
通过设置kernel.sched_bt_period_us和kernel.sched_bt_runtime_us这两个内核参数我们可以控制离线进程占用的cpu比例。
默认情况下kernel.sched_bt_period_us=1000000kernel.sched_bt_runtime_us=-1表示控制周期是1s离线进程占用cpu不受限制比如我们设置kernel.sched_bt_runtime_us=100000即离线占用10%的cpu
![img](images/bt_sched_img10.png)
![img](https://github.com/Tencent/TencentOS-kernel/blob/master/images/bt_sched_img10.png)
__统计离线进程所占cpu比例__
通过查看/proc/bt_stat文件可以查看系统中离线进程所占用的cpu比例
![img](images/bt_sched_img11.png)
![img](https://github.com/Tencent/TencentOS-kernel/blob/master/images/bt_sched_img11.png)
该文件的结构和/proc/stat类似只是在每个cpu的最后又增加了一列表示该cpu上离线进程运行的时间。
@ -727,7 +727,7 @@ __离线调度对docker的支持__
为了更好的支持docker离线调度在cgroup的cpu目录下会新增几个和离线调度相关的文件
![img](images/bt_sched_img12.png)
![img](https://github.com/Tencent/TencentOS-kernel/blob/master/images/bt_sched_img12.png)
cpu.bt_shares同cpu.shares表示该task group的share比例。
cpuacct.bt_stat,cpuacct.bt_usage,cpuacct.bt_usage_percpu_sys,

View File

@ -113,7 +113,7 @@ static void rwb_wake_all(struct rq_wb *rwb)
for (i = 0; i < WBT_NUM_RWQ; i++) {
struct rq_wait *rqw = &rwb->rq_wait[i];
if (wq_has_sleeper(&rqw->wait))
if (waitqueue_active(&rqw->wait))
wake_up_all(&rqw->wait);
}
}
@ -153,7 +153,7 @@ void __wbt_done(struct rq_wb *rwb, enum wbt_flags wb_acct)
if (inflight && inflight >= limit)
return;
if (wq_has_sleeper(&rqw->wait)) {
if (waitqueue_active(&rqw->wait)) {
int diff = limit - inflight;
if (!inflight || diff >= rwb->wb_background / 2)
@ -480,13 +480,6 @@ static inline unsigned int get_limit(struct rq_wb *rwb, unsigned long rw)
{
unsigned int limit;
/*
* If we got disabled, just return UINT_MAX. This ensures that
* we'll properly inc a new IO, and dec+wakeup at the end.
*/
if (!rwb_enabled(rwb))
return UINT_MAX;
/*
* At this point we know it's a buffered write. If this is
* kswapd trying to free memory, or REQ_SYNC is set, set, then
@ -509,80 +502,50 @@ static inline unsigned int get_limit(struct rq_wb *rwb, unsigned long rw)
return limit;
}
struct wbt_wait_data {
struct wait_queue_entry wq;
struct task_struct *task;
struct rq_wb *rwb;
struct rq_wait *rqw;
unsigned long rw;
bool got_token;
};
static int wbt_wake_function(struct wait_queue_entry *curr, unsigned int mode,
int wake_flags, void *key)
static inline bool may_queue(struct rq_wb *rwb, struct rq_wait *rqw,
wait_queue_entry_t *wait, unsigned long rw)
{
struct wbt_wait_data *data = container_of(curr, struct wbt_wait_data,
wq);
/*
* inc it here even if disabled, since we'll dec it at completion.
* this only happens if the task was sleeping in __wbt_wait(),
* and someone turned it off at the same time.
*/
if (!rwb_enabled(rwb)) {
atomic_inc(&rqw->inflight);
return true;
}
/*
* If we fail to get a budget, return -1 to interrupt the wake up
* loop in __wake_up_common.
* If the waitqueue is already active and we are not the next
* in line to be woken up, wait for our turn.
*/
if (!atomic_inc_below(&data->rqw->inflight, get_limit(data->rwb, data->rw)))
return -1;
if (waitqueue_active(&rqw->wait) &&
rqw->wait.head.next != &wait->entry)
return false;
data->got_token = true;
list_del_init(&curr->entry);
wake_up_process(data->task);
return 1;
return atomic_inc_below(&rqw->inflight, get_limit(rwb, rw));
}
/*
* Block if we will exceed our limit, or if we are currently waiting for
* the timer to kick off queuing again.
*/
static void __wbt_wait(struct rq_wb *rwb, enum wbt_flags wb_acct,
unsigned long rw, spinlock_t *lock)
static void __wbt_wait(struct rq_wb *rwb, unsigned long rw, spinlock_t *lock)
__releases(lock)
__acquires(lock)
{
struct rq_wait *rqw = get_rq_wait(rwb, current_is_kswapd());
struct wbt_wait_data data = {
.wq = {
.func = wbt_wake_function,
.entry = LIST_HEAD_INIT(data.wq.entry),
},
.task = current,
.rwb = rwb,
.rqw = rqw,
.rw = rw,
.got_token = false
};
bool has_sleeper;
DEFINE_WAIT(wait);
has_sleeper = wq_has_sleeper(&rqw->wait);
if (!has_sleeper &&
atomic_inc_below(&rqw->inflight, get_limit(rwb, rw)))
if (may_queue(rwb, rqw, &wait, rw))
return;
prepare_to_wait_exclusive(&rqw->wait, &data.wq, TASK_UNINTERRUPTIBLE);
do {
if (data.got_token)
break;
prepare_to_wait_exclusive(&rqw->wait, &wait,
TASK_UNINTERRUPTIBLE);
if (!has_sleeper &&
atomic_inc_below(&rqw->inflight, get_limit(rwb, rw))) {
finish_wait(&rqw->wait, &data.wq);
/*
* We raced with wbt_wake_function() getting a token,
* which means we now have two. Put our local token
* and wake anyone else potentially waiting for one.
*/
if (data.got_token)
__wbt_done(rwb, wb_acct);
if (may_queue(rwb, rqw, &wait, rw))
break;
}
if (lock) {
spin_unlock_irq(lock);
@ -590,11 +553,9 @@ static void __wbt_wait(struct rq_wb *rwb, enum wbt_flags wb_acct,
spin_lock_irq(lock);
} else
io_schedule();
has_sleeper = false;
} while (1);
finish_wait(&rqw->wait, &data.wq);
finish_wait(&rqw->wait, &wait);
}
static inline bool wbt_should_throttle(struct rq_wb *rwb, struct bio *bio)
@ -616,22 +577,6 @@ static inline bool wbt_should_throttle(struct rq_wb *rwb, struct bio *bio)
return true;
}
static enum wbt_flags bio_to_wbt_flags(struct rq_wb *rwb, struct bio *bio)
{
enum wbt_flags flags = 0;
if (!rwb_enabled(rwb))
return 0;
if (bio_op(bio) == REQ_OP_READ) {
flags = WBT_READ;
} else if (wbt_should_throttle(rwb, bio)) {
if (current_is_kswapd())
flags |= WBT_KSWAPD;
flags |= WBT_TRACKED;
}
return flags;
}
/*
* Returns true if the IO request should be accounted, false if not.
* May sleep, if we have exceeded the writeback limits. Caller can pass
@ -640,7 +585,6 @@ static enum wbt_flags bio_to_wbt_flags(struct rq_wb *rwb, struct bio *bio)
*/
enum wbt_flags wbt_wait(struct rq_wb *rwb, struct bio *bio, spinlock_t *lock)
{
enum wbt_flags flags;
unsigned int ret = 0;
if (!rwb_enabled(rwb))
@ -655,8 +599,7 @@ enum wbt_flags wbt_wait(struct rq_wb *rwb, struct bio *bio, spinlock_t *lock)
return ret;
}
flags = bio_to_wbt_flags(rwb, bio);
__wbt_wait(rwb, flags, bio->bi_opf, lock);
__wbt_wait(rwb, bio->bi_opf, lock);
if (!blk_stat_is_active(rwb->cb))
rwb_arm_timer(rwb);

View File

@ -142,10 +142,8 @@ static int __init hest_parse_ghes_count(struct acpi_hest_header *hest_hdr, void
{
int *count = data;
if ((hest_hdr->type == ACPI_HEST_TYPE_GENERIC_ERROR ||
hest_hdr->type == ACPI_HEST_TYPE_GENERIC_ERROR_V2) &&
(boot_cpu_data.x86_vendor != X86_VENDOR_AMD ||
!(hest_hdr->source_id &0xF000)))
if (hest_hdr->type == ACPI_HEST_TYPE_GENERIC_ERROR ||
hest_hdr->type == ACPI_HEST_TYPE_GENERIC_ERROR_V2)
(*count)++;
return 0;
}
@ -156,10 +154,8 @@ static int __init hest_parse_ghes(struct acpi_hest_header *hest_hdr, void *data)
struct ghes_arr *ghes_arr = data;
int rc, i;
if ((hest_hdr->type != ACPI_HEST_TYPE_GENERIC_ERROR &&
hest_hdr->type != ACPI_HEST_TYPE_GENERIC_ERROR_V2) ||
((hest_hdr->source_id & 0xF000) &&
boot_cpu_data.x86_vendor == X86_VENDOR_AMD))
if (hest_hdr->type != ACPI_HEST_TYPE_GENERIC_ERROR &&
hest_hdr->type != ACPI_HEST_TYPE_GENERIC_ERROR_V2)
return 0;
if (!((struct acpi_hest_generic *)hest_hdr)->enabled)

View File

@ -601,16 +601,6 @@ static void virtblk_initialize_rq(struct request *req)
scsi_req_init(&vbr->sreq);
}
#endif
static enum blk_eh_timer_return virtblk_timeout(struct request *req,
bool reserved)
{
printk_ratelimited(KERN_ERR "%s: timeout error, dev %s, sector %llu\n",
__func__, req->rq_disk ?
req->rq_disk->disk_name : "?",
(unsigned long long)blk_rq_pos(req));
return BLK_EH_RESET_TIMER;
}
static const struct blk_mq_ops virtio_mq_ops = {
.queue_rq = virtio_queue_rq,
@ -620,7 +610,6 @@ static const struct blk_mq_ops virtio_mq_ops = {
.initialize_rq_fn = virtblk_initialize_rq,
#endif
.map_queues = virtblk_map_queues,
.timeout = virtblk_timeout,
};
static unsigned int virtblk_queue_depth;

View File

@ -339,57 +339,6 @@ static void pci_read_bases(struct pci_dev *dev, unsigned int howmany, int rom)
}
}
static void pci_read_bridge_windows(struct pci_dev *bridge)
{
u16 io;
u32 pmem, tmp;
pci_read_config_word(bridge, PCI_IO_BASE, &io);
if (!io) {
pci_write_config_word(bridge, PCI_IO_BASE, 0xe0f0);
pci_read_config_word(bridge, PCI_IO_BASE, &io);
pci_write_config_word(bridge, PCI_IO_BASE, 0x0);
}
if (io)
bridge->io_window = 1;
/*
* DECchip 21050 pass 2 errata: the bridge may miss an address
* disconnect boundary by one PCI data phase. Workaround: do not
* use prefetching on this device.
*/
if (bridge->vendor == PCI_VENDOR_ID_DEC && bridge->device == 0x0001)
return;
pci_read_config_dword(bridge, PCI_PREF_MEMORY_BASE, &pmem);
if (!pmem) {
pci_write_config_dword(bridge, PCI_PREF_MEMORY_BASE,
0xffe0fff0);
pci_read_config_dword(bridge, PCI_PREF_MEMORY_BASE, &pmem);
pci_write_config_dword(bridge, PCI_PREF_MEMORY_BASE, 0x0);
}
if (!pmem)
return;
bridge->pref_window = 1;
if ((pmem & PCI_PREF_RANGE_TYPE_MASK) == PCI_PREF_RANGE_TYPE_64) {
/*
* Bridge claims to have a 64-bit prefetchable memory
* window; verify that the upper bits are actually
* writable.
*/
pci_read_config_dword(bridge, PCI_PREF_BASE_UPPER32, &pmem);
pci_write_config_dword(bridge, PCI_PREF_BASE_UPPER32,
0xffffffff);
pci_read_config_dword(bridge, PCI_PREF_BASE_UPPER32, &tmp);
pci_write_config_dword(bridge, PCI_PREF_BASE_UPPER32, pmem);
if (tmp)
bridge->pref_64_window = 1;
}
}
static void pci_read_bridge_io(struct pci_bus *child)
{
struct pci_dev *dev = child->self;
@ -1571,7 +1520,6 @@ int pci_setup_device(struct pci_dev *dev)
pci_read_irq(dev);
dev->transparent = ((dev->class & 0xff) == 1);
pci_read_bases(dev, 2, PCI_ROM_ADDRESS1);
pci_read_bridge_windows(dev);
set_pcie_hotplug_bridge(dev);
pos = pci_find_capability(dev, PCI_CAP_ID_SSVID);
if (pos) {

View File

@ -3817,150 +3817,6 @@ static int reset_chelsio_generic_dev(struct pci_dev *dev, int probe)
return 0;
}
#define DEFAULT_PCI_RESET_WAIT_TIMEOUT 0
#define MIN_PCI_RESET_WAIT_TIMEOUT DEFAULT_PCI_RESET_WAIT_TIMEOUT
#define MAX_PCI_RESET_WAIT_TIMEOUT 4
struct pci_reset_quirk_config {
unsigned short vendor;
unsigned short device;
union { /* 4 bytes in size. */
unsigned int timeout;
unsigned int reserve;
};
int (*reset)(struct pci_dev *dev, struct pci_reset_quirk_config *cfg, int probe);
};
#define MAX_PCI_RESET_CONFIG_NUM 20
static int pci_reset_quirk_config_num = 0;
static struct pci_reset_quirk_config pci_reset_quirk_config[MAX_PCI_RESET_CONFIG_NUM] = {
{ 0 }
};
static int pci_reset_quirk_no_reset(struct pci_dev *dev,
struct pci_reset_quirk_config *cfg, int probe)
{
pci_info(dev, "do no reset the device\n");
return 0;
}
static void pci_no_reset_quirk_get_opt(char *str)
{
unsigned short vid, did;
struct pci_reset_quirk_config *cfg;
while (str) {
char *k = strchr(str, ',');
if (k)
*k++ = 0;
if (str) {
vid = (unsigned short)simple_strtoul(str, &str, 16);
did = (unsigned short)simple_strtoul(str + 1, &str, 16);
cfg = &pci_reset_quirk_config[pci_reset_quirk_config_num++ % MAX_PCI_RESET_CONFIG_NUM];
cfg->vendor = vid;
cfg->device = did;
cfg->reserve = 0;
cfg->reset = pci_reset_quirk_no_reset;
pr_info("add no reset quirk success: %x:%x\n", vid, did);
}
str = k;
}
}
static int pci_reset_quirk_ssleep_after_sbr(struct pci_dev *dev,
struct pci_reset_quirk_config *cfg, int probe)
{
struct pci_dev *slot = dev->bus->self;
u16 reg, timeout = cfg->timeout;
if (probe)
return -ENOTTY;
if (!timeout)
return -ENOTTY;
pcie_capability_read_word(slot, PCI_EXP_SLTCTL, &reg);
reg &= (~PCI_EXP_SLTCTL_DLLSCE);
pcie_capability_write_word(slot, PCI_EXP_SLTCTL, reg);
pci_reset_secondary_bus(slot);
ssleep(timeout);
pcie_capability_write_word(slot, PCI_EXP_SLTSTA, PCI_EXP_SLTSTA_DLLSC);
pcie_capability_read_word(slot, PCI_EXP_SLTCTL, &reg);
reg |= PCI_EXP_SLTCTL_DLLSCE;
pcie_capability_write_word(slot, PCI_EXP_SLTCTL, reg);
pci_info(dev, "do sleep %ds after device reset\n", timeout);
return 0;
}
static void pci_reset_timeout_quirk_get_opt(char *str)
{
unsigned short vid, did;
unsigned int t;
struct pci_reset_quirk_config *cfg;
while (str) {
char *k = strchr(str, ',');
if (k)
*k++ = 0;
if (str) {
vid = (unsigned short)simple_strtoul(str, &str, 16);
did = (unsigned short)simple_strtoul(str + 1, &str, 16);
t = (unsigned int)simple_strtoul(str + 1, &str, 10);
if (t > MAX_PCI_RESET_WAIT_TIMEOUT
|| t < MIN_PCI_RESET_WAIT_TIMEOUT)
t = MIN_PCI_RESET_WAIT_TIMEOUT;
cfg = &pci_reset_quirk_config[pci_reset_quirk_config_num++ % MAX_PCI_RESET_CONFIG_NUM];
cfg->vendor = vid;
cfg->device = did;
cfg->timeout = t;
cfg->reset = pci_reset_quirk_ssleep_after_sbr;
pr_info("add reset timeout quirk success: %x:%x timeout:%x\n", vid, did, t);
}
str = k;
}
}
/*
* cmdline e.g.: pcireset_quirk=timeout,1ea0:2a16=4,1ea0:2a17=3 pcireset_quirk=noreset,1ea0:2a16,1ea0:2a17
*/
static int __init pci_reset_quirk_setup(char *str)
{
if (!strncmp(str, "timeout,", 8)) {
pci_reset_timeout_quirk_get_opt(str + 8);
} else if (!strncmp(str, "noreset,", 8)) {
pci_no_reset_quirk_get_opt(str + 8);
} else {
pr_err("PCI: Unknown option in pcireset_quirk. '%s'\n", str);
}
return 0;
}
__setup("pcireset_quirk=", pci_reset_quirk_setup);
static int pci_reset_quirk_cmdline(struct pci_dev *dev, int probe)
{
int i;
struct pci_reset_quirk_config *cfg;
for (i = 0; i < pci_reset_quirk_config_num && i < MAX_PCI_RESET_CONFIG_NUM; i++){
cfg = &pci_reset_quirk_config[i];
if (cfg->vendor == dev->vendor &&
cfg->device == dev->device &&
cfg->reset)
return cfg->reset(dev, cfg, probe);
}
return -ENOTTY;
}
#define PCI_DEVICE_ID_INTEL_82599_SFP_VF 0x10ed
#define PCI_DEVICE_ID_INTEL_IVB_M_VGA 0x0156
#define PCI_DEVICE_ID_INTEL_IVB_M2_VGA 0x0166
@ -3974,7 +3830,6 @@ static const struct pci_dev_reset_methods pci_dev_reset_methods[] = {
reset_ivb_igd },
{ PCI_VENDOR_ID_CHELSIO, PCI_ANY_ID,
reset_chelsio_generic_dev },
{ PCI_ANY_ID, PCI_ANY_ID, pci_reset_quirk_cmdline },
{ 0 }
};

View File

@ -740,21 +740,58 @@ int pci_claim_bridge_resource(struct pci_dev *bridge, int i)
base/limit registers must be read-only and read as 0. */
static void pci_bridge_check_ranges(struct pci_bus *bus)
{
u16 io;
u32 pmem;
struct pci_dev *bridge = bus->self;
struct resource *b_res = &bridge->resource[PCI_BRIDGE_RESOURCES];
struct resource *b_res;
b_res = &bridge->resource[PCI_BRIDGE_RESOURCES];
b_res[1].flags |= IORESOURCE_MEM;
if (bridge->io_window)
pci_read_config_word(bridge, PCI_IO_BASE, &io);
if (!io) {
pci_write_config_word(bridge, PCI_IO_BASE, 0xe0f0);
pci_read_config_word(bridge, PCI_IO_BASE, &io);
pci_write_config_word(bridge, PCI_IO_BASE, 0x0);
}
if (io)
b_res[0].flags |= IORESOURCE_IO;
if (bridge->pref_window) {
/* DECchip 21050 pass 2 errata: the bridge may miss an address
disconnect boundary by one PCI data phase.
Workaround: do not use prefetching on this device. */
if (bridge->vendor == PCI_VENDOR_ID_DEC && bridge->device == 0x0001)
return;
pci_read_config_dword(bridge, PCI_PREF_MEMORY_BASE, &pmem);
if (!pmem) {
pci_write_config_dword(bridge, PCI_PREF_MEMORY_BASE,
0xffe0fff0);
pci_read_config_dword(bridge, PCI_PREF_MEMORY_BASE, &pmem);
pci_write_config_dword(bridge, PCI_PREF_MEMORY_BASE, 0x0);
}
if (pmem) {
b_res[2].flags |= IORESOURCE_MEM | IORESOURCE_PREFETCH;
if (bridge->pref_64_window) {
if ((pmem & PCI_PREF_RANGE_TYPE_MASK) ==
PCI_PREF_RANGE_TYPE_64) {
b_res[2].flags |= IORESOURCE_MEM_64;
b_res[2].flags |= PCI_PREF_RANGE_TYPE_64;
}
}
/* double check if bridge does support 64 bit pref */
if (b_res[2].flags & IORESOURCE_MEM_64) {
u32 mem_base_hi, tmp;
pci_read_config_dword(bridge, PCI_PREF_BASE_UPPER32,
&mem_base_hi);
pci_write_config_dword(bridge, PCI_PREF_BASE_UPPER32,
0xffffffff);
pci_read_config_dword(bridge, PCI_PREF_BASE_UPPER32, &tmp);
if (!tmp)
b_res[2].flags &= ~IORESOURCE_MEM_64;
pci_write_config_dword(bridge, PCI_PREF_BASE_UPPER32,
mem_base_hi);
}
}
/* Helper function for sizing routines: find first available

View File

@ -1034,7 +1034,6 @@ struct ext4_inode_info {
struct timespec i_crtime;
/* mballoc */
atomic_t i_prealloc_active;
struct list_head i_prealloc_list;
spinlock_t i_prealloc_lock;
@ -1458,7 +1457,6 @@ struct ext4_sb_info {
unsigned int s_mb_stats;
unsigned int s_mb_order2_reqs;
unsigned int s_mb_group_prealloc;
unsigned int s_mb_max_inode_prealloc;
unsigned int s_max_dir_size_kb;
/* where last allocation was done - for stream allocation */
unsigned long s_mb_last_group;
@ -2444,7 +2442,7 @@ extern int ext4_mb_release(struct super_block *);
extern ext4_fsblk_t ext4_mb_new_blocks(handle_t *,
struct ext4_allocation_request *, int *);
extern int ext4_mb_reserve_blocks(struct super_block *, int);
extern void ext4_discard_preallocations(struct inode *, unsigned int);
extern void ext4_discard_preallocations(struct inode *);
extern int __init ext4_init_mballoc(void);
extern void ext4_exit_mballoc(void);
extern void ext4_free_blocks(handle_t *handle, struct inode *inode,

View File

@ -4511,7 +4511,7 @@ got_allocated_blocks:
/* free data blocks we just allocated */
/* not a good idea to call discard here directly,
* but otherwise we'd need to call it every free() */
ext4_discard_preallocations(inode, 0);
ext4_discard_preallocations(inode);
ext4_free_blocks(handle, inode, NULL, newblock,
EXT4_C2B(sbi, allocated_clusters), fb_flags);
goto out2;
@ -5548,7 +5548,7 @@ int ext4_collapse_range(struct inode *inode, loff_t offset, loff_t len)
}
down_write(&EXT4_I(inode)->i_data_sem);
ext4_discard_preallocations(inode, 0);
ext4_discard_preallocations(inode);
ret = ext4_es_remove_extent(inode, punch_start,
EXT_MAX_BLOCKS - punch_start);
@ -5562,7 +5562,7 @@ int ext4_collapse_range(struct inode *inode, loff_t offset, loff_t len)
up_write(&EXT4_I(inode)->i_data_sem);
goto out_stop;
}
ext4_discard_preallocations(inode, 0);
ext4_discard_preallocations(inode);
ret = ext4_ext_shift_extents(inode, handle, punch_stop,
punch_stop - punch_start, SHIFT_LEFT);
@ -5695,7 +5695,7 @@ int ext4_insert_range(struct inode *inode, loff_t offset, loff_t len)
goto out_stop;
down_write(&EXT4_I(inode)->i_data_sem);
ext4_discard_preallocations(inode, 0);
ext4_discard_preallocations(inode);
path = ext4_find_extent(inode, offset_lblk, NULL, 0);
if (IS_ERR(path)) {

View File

@ -92,7 +92,7 @@ static int ext4_release_file(struct inode *inode, struct file *filp)
!EXT4_I(inode)->i_reserved_data_blocks)
{
down_write(&EXT4_I(inode)->i_data_sem);
ext4_discard_preallocations(inode, 0);
ext4_discard_preallocations(inode);
up_write(&EXT4_I(inode)->i_data_sem);
}
if (is_dx(inode) && filp->private_data)

View File

@ -183,7 +183,7 @@ int ext4_truncate_restart_trans(handle_t *handle, struct inode *inode,
up_write(&EXT4_I(inode)->i_data_sem);
ret = ext4_journal_restart(handle, nblocks);
down_write(&EXT4_I(inode)->i_data_sem);
ext4_discard_preallocations(inode, 0);
ext4_discard_preallocations(inode);
return ret;
}
@ -391,7 +391,7 @@ void ext4_da_update_reserve_space(struct inode *inode,
*/
if ((ei->i_reserved_data_blocks == 0) &&
(atomic_read(&inode->i_writecount) == 0))
ext4_discard_preallocations(inode, 0);
ext4_discard_preallocations(inode);
}
static int __check_block_validity(struct inode *inode, const char *func,
@ -4261,7 +4261,7 @@ int ext4_punch_hole(struct inode *inode, loff_t offset, loff_t length)
if (stop_block > first_block) {
down_write(&EXT4_I(inode)->i_data_sem);
ext4_discard_preallocations(inode, 0);
ext4_discard_preallocations(inode);
ret = ext4_es_remove_extent(inode, first_block,
stop_block - first_block);
@ -4417,7 +4417,7 @@ int ext4_truncate(struct inode *inode)
down_write(&EXT4_I(inode)->i_data_sem);
ext4_discard_preallocations(inode, 0);
ext4_discard_preallocations(inode);
if (ext4_test_inode_flag(inode, EXT4_INODE_EXTENTS))
err = ext4_ext_truncate(handle, inode);

View File

@ -163,7 +163,7 @@ static long swap_inode_boot_loader(struct super_block *sb,
inode_bl->i_generation = sbi->s_next_generation++;
spin_unlock(&sbi->s_next_gen_lock);
ext4_discard_preallocations(inode, 0);
ext4_discard_preallocations(inode);
err = ext4_mark_inode_dirty(handle, inode);
if (err < 0) {

View File

@ -2651,7 +2651,6 @@ int ext4_mb_init(struct super_block *sb)
sbi->s_mb_stats = MB_DEFAULT_STATS;
sbi->s_mb_stream_request = MB_DEFAULT_STREAM_THRESHOLD;
sbi->s_mb_order2_reqs = MB_DEFAULT_ORDER2_REQS;
sbi->s_mb_max_inode_prealloc = MB_DEFAULT_MAX_INODE_PREALLOC;
/*
* The default group preallocation is 512, which for 4k block
* sizes translates to 2 megabytes. However for bigalloc file
@ -3581,26 +3580,6 @@ void ext4_mb_generate_from_pa(struct super_block *sb, void *bitmap,
mb_debug(1, "preallocated %u for group %u\n", preallocated, group);
}
static void ext4_mb_mark_pa_deleted(struct super_block *sb,
struct ext4_prealloc_space *pa)
{
struct ext4_inode_info *ei;
if (pa->pa_deleted) {
ext4_warning(sb, "deleted pa, type:%d, pblk:%llu, lblk:%u, len:%d\n",
pa->pa_type, pa->pa_pstart, pa->pa_lstart,
pa->pa_len);
return;
}
pa->pa_deleted = 1;
if (pa->pa_type == MB_INODE_PA) {
ei = EXT4_I(pa->pa_inode);
atomic_dec(&ei->i_prealloc_active);
}
}
static void ext4_mb_pa_callback(struct rcu_head *head)
{
struct ext4_prealloc_space *pa;
@ -3633,7 +3612,7 @@ static void ext4_mb_put_pa(struct ext4_allocation_context *ac,
return;
}
ext4_mb_mark_pa_deleted(sb, pa);
pa->pa_deleted = 1;
spin_unlock(&pa->pa_lock);
grp_blk = pa->pa_pstart;
@ -3761,7 +3740,6 @@ ext4_mb_new_inode_pa(struct ext4_allocation_context *ac)
spin_lock(pa->pa_obj_lock);
list_add_rcu(&pa->pa_inode_list, &ei->i_prealloc_list);
spin_unlock(pa->pa_obj_lock);
atomic_inc(&ei->i_prealloc_active);
return 0;
}
@ -3983,7 +3961,7 @@ repeat:
}
/* seems this one can be freed ... */
ext4_mb_mark_pa_deleted(sb, pa);
pa->pa_deleted = 1;
/* we can trust pa_free ... */
free += pa->pa_free;
@ -4041,7 +4019,7 @@ out:
*
* FIXME!! Make sure it is valid at all the call sites
*/
void ext4_discard_preallocations(struct inode *inode, unsigned int needed)
void ext4_discard_preallocations(struct inode *inode)
{
struct ext4_inode_info *ei = EXT4_I(inode);
struct super_block *sb = inode->i_sb;
@ -4058,19 +4036,15 @@ void ext4_discard_preallocations(struct inode *inode, unsigned int needed)
}
mb_debug(1, "discard preallocation for inode %lu\n", inode->i_ino);
trace_ext4_discard_preallocations(inode,
atomic_read(&ei->i_prealloc_active), needed);
trace_ext4_discard_preallocations(inode);
INIT_LIST_HEAD(&list);
if (needed == 0)
needed = UINT_MAX;
repeat:
/* first, collect all pa's in the inode */
spin_lock(&ei->i_prealloc_lock);
while (!list_empty(&ei->i_prealloc_list) && needed) {
pa = list_entry(ei->i_prealloc_list.prev,
while (!list_empty(&ei->i_prealloc_list)) {
pa = list_entry(ei->i_prealloc_list.next,
struct ext4_prealloc_space, pa_inode_list);
BUG_ON(pa->pa_obj_lock != &ei->i_prealloc_lock);
spin_lock(&pa->pa_lock);
@ -4087,11 +4061,10 @@ repeat:
}
if (pa->pa_deleted == 0) {
ext4_mb_mark_pa_deleted(sb, pa);
pa->pa_deleted = 1;
spin_unlock(&pa->pa_lock);
list_del_rcu(&pa->pa_inode_list);
list_add(&pa->u.pa_tmp_list, &list);
needed--;
continue;
}
@ -4360,7 +4333,7 @@ ext4_mb_discard_lg_preallocations(struct super_block *sb,
BUG_ON(pa->pa_type != MB_GROUP_PA);
/* seems this one can be freed ... */
ext4_mb_mark_pa_deleted(sb, pa);
pa->pa_deleted = 1;
spin_unlock(&pa->pa_lock);
list_del_rcu(&pa->pa_inode_list);
@ -4457,31 +4430,11 @@ static void ext4_mb_add_n_trim(struct ext4_allocation_context *ac)
return ;
}
/*
* if per-inode prealloc list is too long, trim some PA
*/
static void
ext4_mb_trim_inode_pa(struct inode *inode)
{
struct ext4_inode_info *ei = EXT4_I(inode);
struct ext4_sb_info *sbi = EXT4_SB(inode->i_sb);
int count, delta;
count = atomic_read(&ei->i_prealloc_active);
delta = (sbi->s_mb_max_inode_prealloc >> 2) + 1;
if (count > sbi->s_mb_max_inode_prealloc + delta) {
count -= sbi->s_mb_max_inode_prealloc;
ext4_discard_preallocations(inode, count);
}
}
/*
* release all resource we used in allocation
*/
static int ext4_mb_release_context(struct ext4_allocation_context *ac)
{
struct inode *inode = ac->ac_inode;
struct ext4_inode_info *ei = EXT4_I(inode);
struct ext4_sb_info *sbi = EXT4_SB(ac->ac_sb);
struct ext4_prealloc_space *pa = ac->ac_pa;
if (pa) {
@ -4508,17 +4461,6 @@ static int ext4_mb_release_context(struct ext4_allocation_context *ac)
spin_unlock(pa->pa_obj_lock);
ext4_mb_add_n_trim(ac);
}
if (pa->pa_type == MB_INODE_PA) {
/*
* treat per-inode prealloc list as a lru list, then try
* to trim the least recently used PA.
*/
spin_lock(pa->pa_obj_lock);
list_move(&pa->pa_inode_list, &ei->i_prealloc_list);
spin_unlock(pa->pa_obj_lock);
}
ext4_mb_put_pa(ac, ac->ac_sb, pa);
}
if (ac->ac_bitmap_page)
@ -4528,7 +4470,6 @@ static int ext4_mb_release_context(struct ext4_allocation_context *ac)
if (ac->ac_flags & EXT4_MB_HINT_GROUP_ALLOC)
mutex_unlock(&ac->ac_lg->lg_mutex);
ext4_mb_collect_stats(ac);
ext4_mb_trim_inode_pa(inode);
return 0;
}

View File

@ -77,10 +77,6 @@ do { \
*/
#define MB_DEFAULT_GROUP_PREALLOC 512
/*
* maximum length of inode prealloc list
*/
#define MB_DEFAULT_MAX_INODE_PREALLOC 512
struct ext4_free_data {
/* this links the free block information from sb_info */

View File

@ -698,8 +698,8 @@ ext4_move_extents(struct file *o_filp, struct file *d_filp, __u64 orig_blk,
out:
if (*moved_len) {
ext4_discard_preallocations(orig_inode, 0);
ext4_discard_preallocations(donor_inode, 0);
ext4_discard_preallocations(orig_inode);
ext4_discard_preallocations(donor_inode);
}
ext4_ext_drop_refs(path);

View File

@ -1005,7 +1005,6 @@ static struct inode *ext4_alloc_inode(struct super_block *sb)
ei->vfs_inode.i_version = 1;
spin_lock_init(&ei->i_raw_lock);
atomic_set(&ei->i_prealloc_active, 0);
INIT_LIST_HEAD(&ei->i_prealloc_list);
spin_lock_init(&ei->i_prealloc_lock);
ext4_es_init_tree(&ei->i_es_tree);
@ -1098,7 +1097,7 @@ void ext4_clear_inode(struct inode *inode)
invalidate_inode_buffers(inode);
clear_inode(inode);
dquot_drop(inode);
ext4_discard_preallocations(inode, 0);
ext4_discard_preallocations(inode);
ext4_es_remove_extent(inode, 0, EXT_MAX_BLOCKS);
if (EXT4_I(inode)->jinode) {
jbd2_journal_release_jbd_inode(EXT4_JOURNAL(inode),

View File

@ -177,7 +177,6 @@ EXT4_RW_ATTR_SBI_UI(mb_min_to_scan, s_mb_min_to_scan);
EXT4_RW_ATTR_SBI_UI(mb_order2_req, s_mb_order2_reqs);
EXT4_RW_ATTR_SBI_UI(mb_stream_req, s_mb_stream_request);
EXT4_RW_ATTR_SBI_UI(mb_group_prealloc, s_mb_group_prealloc);
EXT4_RW_ATTR_SBI_UI(mb_max_inode_prealloc, s_mb_max_inode_prealloc);
EXT4_RW_ATTR_SBI_UI(extent_max_zeroout_kb, s_extent_max_zeroout_kb);
EXT4_ATTR(trigger_fs_error, 0200, trigger_test_error);
EXT4_RW_ATTR_SBI_UI(err_ratelimit_interval_ms, s_err_ratelimit_state.interval);
@ -206,7 +205,6 @@ static struct attribute *ext4_attrs[] = {
ATTR_LIST(mb_order2_req),
ATTR_LIST(mb_stream_req),
ATTR_LIST(mb_group_prealloc),
ATTR_LIST(mb_max_inode_prealloc),
ATTR_LIST(max_writeback_mb_bump),
ATTR_LIST(extent_max_zeroout_kb),
ATTR_LIST(trigger_fs_error),

View File

@ -828,7 +828,6 @@ int ext4_get_inode_usage(struct inode *inode, qsize_t *usage)
bh = ext4_sb_bread(inode->i_sb, EXT4_I(inode)->i_file_acl, REQ_PRIO);
if (IS_ERR(bh)) {
ret = PTR_ERR(bh);
bh = NULL;
goto out;
}
@ -1822,11 +1821,8 @@ ext4_xattr_block_find(struct inode *inode, struct ext4_xattr_info *i,
if (EXT4_I(inode)->i_file_acl) {
/* The inode already has an extended attribute block. */
bs->bh = ext4_sb_bread(sb, EXT4_I(inode)->i_file_acl, REQ_PRIO);
if (IS_ERR(bs->bh)) {
error = PTR_ERR(bs->bh);
bs->bh = NULL;
return error;
}
if (IS_ERR(bs->bh))
return PTR_ERR(bs->bh);
ea_bdebug(bs->bh, "b_count=%d, refcount=%d",
atomic_read(&(bs->bh->b_count)),
le32_to_cpu(BHDR(bs->bh)->h_refcount));
@ -2909,7 +2905,6 @@ int ext4_xattr_delete_inode(handle_t *handle, struct inode *inode,
if (error == -EIO)
EXT4_ERROR_INODE(inode, "block %llu read error",
EXT4_I(inode)->i_file_acl);
bh = NULL;
goto cleanup;
}
error = ext4_xattr_check_block(inode, bh);
@ -3066,7 +3061,6 @@ ext4_xattr_block_cache_find(struct inode *inode,
if (IS_ERR(bh)) {
if (PTR_ERR(bh) == -ENOMEM)
return NULL;
bh = NULL;
EXT4_ERROR_INODE(inode, "block %lu read error",
(unsigned long)ce->e_value);
} else if (ext4_xattr_cmp(header, BHDR(bh)) == 0) {

View File

@ -55,13 +55,6 @@ static void ovl_do_check_copy_up(struct dentry *dentry)
iterate_fd(current->files, 0, ovl_check_fd, dentry);
}
static bool ovl_must_copy_xattr(const char *name)
{
return !strcmp(name, XATTR_POSIX_ACL_ACCESS) ||
!strcmp(name, XATTR_POSIX_ACL_DEFAULT) ||
!strncmp(name, XATTR_SECURITY_PREFIX, XATTR_SECURITY_PREFIX_LEN);
}
int ovl_copy_xattr(struct dentry *old, struct dentry *new)
{
ssize_t list_size, size, value_size = 0;
@ -133,13 +126,8 @@ retry:
continue; /* Discard */
}
error = vfs_setxattr(new, name, value, size, 0);
if (error) {
if (error != -EOPNOTSUPP || ovl_must_copy_xattr(name))
break;
/* Ignore failure to copy unknown xattrs */
error = 0;
}
if (error)
break;
}
kfree(value);
out:

View File

@ -118,7 +118,7 @@ static int show_bt_stat(struct seq_file *p, void *v)
seq_printf(p,
"\nctxt %llu\n"
"btime %lu\n"
"processes %lu\n"
"processes %d\n"
"procs_running %lu\n"
"procs_blocked %lu\n",
nr_context_switches(),

View File

@ -169,7 +169,7 @@ static int show_stat(struct seq_file *p, void *v)
seq_printf(p,
"\nctxt %llu\n"
"btime %llu\n"
"processes %lu\n"
"processes %d\n"
"procs_running %lu\n"
"procs_blocked %lu\n",
nr_context_switches(),

View File

@ -372,9 +372,6 @@ struct pci_dev {
bool match_driver; /* Skip attaching driver */
/* These fields are used by common fixups */
unsigned int transparent:1; /* Subtractive decode PCI bridge */
unsigned int io_window:1; /* Bridge has I/O window */
unsigned int pref_window:1; /* Bridge has pref mem window */
unsigned int pref_64_window:1; /* Pref mem window is 64-bit */
unsigned int multifunction:1;/* Part of multi-function device */
/* keep track of device state */
unsigned int is_added:1;

View File

@ -15,7 +15,7 @@
DECLARE_PER_CPU(unsigned long, total_forks);
extern int nr_threads;
DECLARE_PER_CPU(unsigned long, process_counts);
extern unsigned long nr_forks(void);
extern int nr_forks(void);
extern int nr_processes(void);
extern unsigned long nr_running(void);
extern unsigned long nr_running_cpu(int cpu);

View File

@ -94,16 +94,6 @@ extern int __cpu_up(unsigned int cpunum, struct task_struct *tidle);
*/
extern void smp_cpus_done(unsigned int max_cpus);
#define smp_call_function_many_async_begin(cpumask) \
preempt_disable();
int smp_call_function_many_async(int cpu, call_single_data_t *csd, struct cpumask *mask);
#define smp_call_function_many_async_end(cpumask) \
arch_send_call_function_ipi_mask(cpumask); \
preempt_enable();
/*
* Call a function on all other processors
*/
@ -146,10 +136,6 @@ static inline int get_boot_cpu_id(void)
static inline void smp_send_stop(void) { }
#define smp_call_function_many_async_begin(cpumask)
#define smp_call_function_many_async(cpu, csd, mask) smp_call_function_single_async(cpu, csd)
#define smp_call_function_many_async_end(cpumask)
/*
* These macros fold the SMP functionality into a single CPU system
*/

View File

@ -32,7 +32,7 @@
#include <net/netfilter/nf_conntrack.h>
#endif
#include <net/net_namespace.h> /* Netw namespace */
#include <linux/netdevice.h>
#define IP_VS_HDR_INVERSE 1
#define IP_VS_HDR_ICMP 2
@ -1712,15 +1712,6 @@ struct bpf_sym_addrs {
const struct file_operations *bpf_prog_fops;
};
struct net *ip_vs_skb_net(struct sk_buff *skb);
enum {
IPVS_ORIGIN_MODE,
IPVS_BPF_MODE,
IPVS_SHARE_NS_MODE,
IPVS_MAX_MODE
};
extern unsigned int ipvs_mode;
extern struct net init_net;
extern struct bpf_sym_addrs resolve_addrs;
extern struct bpf_map *conntrack_map;
extern bool bpf_mode_on;
@ -1747,4 +1738,5 @@ extern struct cidrs __rcu *non_masq_cidrs;
#define IP_VS_SVC_TAB_BITS 8
#define IP_VS_SVC_TAB_SIZE (1 << IP_VS_SVC_TAB_BITS)
extern struct hlist_head ip_vs_svc_table[IP_VS_SVC_TAB_SIZE];
#endif /* _NET_IP_VS_H */

View File

@ -725,29 +725,24 @@ TRACE_EVENT(ext4_mb_release_group_pa,
);
TRACE_EVENT(ext4_discard_preallocations,
TP_PROTO(struct inode *inode, unsigned int len, unsigned int needed),
TP_PROTO(struct inode *inode),
TP_ARGS(inode, len, needed),
TP_ARGS(inode),
TP_STRUCT__entry(
__field( dev_t, dev )
__field( ino_t, ino )
__field( unsigned int, len )
__field( unsigned int, needed )
__field( dev_t, dev )
__field( ino_t, ino )
),
TP_fast_assign(
__entry->dev = inode->i_sb->s_dev;
__entry->ino = inode->i_ino;
__entry->len = len;
__entry->needed = needed;
),
TP_printk("dev %d,%d ino %lu len: %u needed %u",
TP_printk("dev %d,%d ino %lu",
MAJOR(__entry->dev), MINOR(__entry->dev),
(unsigned long) __entry->ino, __entry->len,
__entry->needed)
(unsigned long) __entry->ino)
);
TRACE_EVENT(ext4_mb_discard_preallocations,

View File

@ -146,5 +146,4 @@ struct virtio_scsi_inhdr {
#define VIRTIO_BLK_S_OK 0
#define VIRTIO_BLK_S_IOERR 1
#define VIRTIO_BLK_S_UNSUPP 2
#define VIRTIO_BLK_S_TIMEOUT 3
#endif /* _LINUX_VIRTIO_BLK_H */

View File

@ -133,10 +133,10 @@ int lockdep_tasklist_lock_is_held(void)
EXPORT_SYMBOL_GPL(lockdep_tasklist_lock_is_held);
#endif /* #ifdef CONFIG_PROVE_RCU */
unsigned long nr_forks(void)
int nr_forks(void)
{
int cpu;
unsigned long total = 0;
int total = 0;
for_each_possible_cpu(cpu)
total += per_cpu(total_forks, cpu);

View File

@ -2790,7 +2790,7 @@ static struct task_struct *pick_next_task_bt(struct rq *rq, struct task_struct *
se = pick_next_bt_entity(bt_rq);
set_next_bt_entity(bt_rq, se);
bt_rq = group_bt_rq(se);
}while(bt_rq);
}while(bt_rq && bt_rq->nr_running);
p = bt_task_of(se);
@ -4939,8 +4939,6 @@ static ssize_t offline_proc_write(struct file *file, const char __user *ubuf,
struct bt_rq *bt_rq;
unsigned long tmp;
int cpn;
struct rq *rq;
unsigned long flags;
cpn = min((int)OFFLINE_NUMBUF, (int)cnt);
if (copy_from_user(buffer, ubuf, cpn))
@ -4951,14 +4949,11 @@ static ssize_t offline_proc_write(struct file *file, const char __user *ubuf,
return -EINVAL;
*to = tmp;
rq = cpu_rq(cpu);
bt_rq = &cpu_rq(cpu)->bt;
bt_rq = &rq->bt;
raw_spin_lock_irqsave(&rq->lock, flags);
raw_spin_lock(&bt_rq->bt_runtime_lock);
bt_rq->bt_runtime = (u64)sysctl_sched_bt_period * NSEC_PER_USEC * tmp / 100;
raw_spin_unlock(&bt_rq->bt_runtime_lock);
raw_spin_unlock_irqrestore(&rq->lock, flags);
return cnt;
}

View File

@ -2481,17 +2481,9 @@ int sched_fork(unsigned long clone_flags, struct task_struct *p)
p->sched_class = &rt_sched_class;
#ifdef CONFIG_BT_SCHED
} else if(bt_prio(p->prio)){
p->normal_prio = p->static_prio = p->prio;
p->policy = SCHED_BT;
p->sched_class = &bt_sched_class;
set_bt_load_weight(p);
#endif
} else {
#ifdef CONFIG_BT_SCHED
p->normal_prio = p->static_prio = p->prio;
p->policy = SCHED_NORMAL;
set_load_weight(p);
#endif
p->sched_class = &fair_sched_class;
}
@ -6474,10 +6466,6 @@ struct task_group *sched_create_group(struct task_group *parent)
if (!alloc_rt_sched_group(tg, parent))
goto err;
#ifdef CONFIG_BT_GROUP_SCHED
mutex_init(&tg->offline_mutex);
#endif
return tg;
err:
@ -6550,13 +6538,9 @@ static void sched_change_group(struct task_struct *tsk, int type)
tsk->sched_task_group = tg;
#ifdef CONFIG_BT_GROUP_SCHED
/*
* No need to re-setcheduler when a task is exiting or the task
* is in an autogroup.
*/
if (offlinegroup_enabled && !rt_task(tsk)
&& !(tsk->flags & PF_EXITING)
&& !task_group_is_autogroup(tg)) {
/* No need to re-setcheduler when fork or exit a task */
if (offlinegroup_enabled && !rt_task(tsk) &&
!(tsk->flags & PF_EXITING) && (type != TASK_SET_GROUP)) {
struct rq *rq = task_rq(tsk);
struct sched_attr attr = {
.sched_priority = 0,
@ -6570,6 +6554,16 @@ static void sched_change_group(struct task_struct *tsk, int type)
attr.sched_policy = SCHED_NORMAL;
}
/*
* FIXME: __setscheduler before task_change_group would
* lead to missing prev cfs_rq/bt_rq stats updating.
* Otherwise, putting __setscheduler after task_change_group
* is not right either, which would miss next cfs_rq/bt_rq
* stats updating.
* In fact, __setscheduler should be right before set_task_rq
* in task_change_group callback, but if so, the code would be
* messed up.
*/
__setscheduler(rq, tsk, &attr, 0);
}
#endif
@ -7061,16 +7055,15 @@ static int cpu_offline_write_uint(struct cgroup_subsys_state *css,
if (!tg->se[0])
return -EINVAL;
mutex_lock(&tg->offline_mutex);
if (tg->offline == !!offline_input)
goto done;
return 0;
if (!tg->offline && offline_input) {
sched_class = SCHED_BT;
} else if (tg->offline && !offline_input) {
sched_class = SCHED_NORMAL;
} else
goto done;
return 0;
tg->offline = !!offline_input;
@ -7084,9 +7077,6 @@ static int cpu_offline_write_uint(struct cgroup_subsys_state *css,
}
}
css_task_iter_end(&it);
done:
mutex_unlock(&tg->offline_mutex);
return 0;
}

View File

@ -5449,43 +5449,28 @@ static int wake_wide(struct task_struct *p)
* soonest. For the purpose of speed we only consider the waking and previous
* CPU.
*
* wake_affine_idle() - only considers 'now', it check if the waking CPU is
* cache-affine and is (or will be) idle.
* wake_affine_idle() - only considers 'now', it check if the waking CPU is (or
* will be) idle.
*
* wake_affine_weight() - considers the weight to reflect the average
* scheduling latency of the CPUs. This seems to work
* for the overloaded case.
*/
static int
static bool
wake_affine_idle(struct sched_domain *sd, struct task_struct *p,
int this_cpu, int prev_cpu, int sync)
{
/*
* If this_cpu is idle, it implies the wakeup is from interrupt
* context. Only allow the move if cache is shared. Otherwise an
* interrupt intensive workload could force all tasks onto one
* node depending on the IO topology or IRQ affinity settings.
*
* If the prev_cpu is idle and cache affine then avoid a migration.
* There is no guarantee that the cache hot data from an interrupt
* is more important than cache hot data on the prev_cpu and from
* a cpufreq perspective, it's better to have higher utilisation
* on one CPU.
*/
if (idle_cpu(this_cpu) && cpus_share_cache(this_cpu, prev_cpu))
return idle_cpu(prev_cpu) ? prev_cpu : this_cpu;
if (idle_cpu(this_cpu))
return true;
if (sync && cpu_rq(this_cpu)->nr_running == 1)
return this_cpu;
return true;
if (idle_cpu(prev_cpu))
return prev_cpu;
return nr_cpumask_bits;
return false;
}
static int
static bool
wake_affine_weight(struct sched_domain *sd, struct task_struct *p,
int this_cpu, int prev_cpu, int sync)
{
@ -5499,7 +5484,7 @@ wake_affine_weight(struct sched_domain *sd, struct task_struct *p,
unsigned long current_load = task_h_load(current);
if (current_load > this_eff_load)
return this_cpu;
return true;
this_eff_load -= current_load;
}
@ -5516,16 +5501,7 @@ wake_affine_weight(struct sched_domain *sd, struct task_struct *p,
prev_eff_load *= 100 + (sd->imbalance_pct - 100) / 2;
prev_eff_load *= capacity_of(this_cpu);
/*
* If sync, adjust the weight of prev_eff_load such that if
* prev_eff == this_eff that select_idle_sibling() will consider
* stacking the wakee on top of the waker if no other CPU is
* idle.
*/
if (sync)
prev_eff_load += 1;
return this_eff_load < prev_eff_load ? this_cpu : nr_cpumask_bits;
return this_eff_load <= prev_eff_load;
}
static int wake_affine(struct sched_domain *sd, struct task_struct *p,
@ -5533,17 +5509,14 @@ static int wake_affine(struct sched_domain *sd, struct task_struct *p,
{
int this_cpu = smp_processor_id();
bool affine = false;
int target = nr_cpumask_bits;
if (sched_feat(WA_IDLE))
target = wake_affine_idle(sd, p, this_cpu, prev_cpu, sync);
if (sched_feat(WA_IDLE) && !affine)
affine = wake_affine_idle(sd, p, this_cpu, prev_cpu, sync);
if (sched_feat(WA_WEIGHT) && target == nr_cpumask_bits)
target = wake_affine_weight(sd, p, this_cpu, prev_cpu, sync);
if (sched_feat(WA_WEIGHT) && !affine)
affine = wake_affine_weight(sd, p, this_cpu, prev_cpu, sync);
schedstat_inc(p->se.statistics.nr_wakeups_affine_attempts);
if (target == this_cpu)
affine = true;
if (affine) {
schedstat_inc(sd->ttwu_move_affine);
schedstat_inc(p->se.statistics.nr_wakeups_affine);

View File

@ -375,10 +375,7 @@ struct task_group {
u64 cpuquota_aware;
struct cfs_bandwidth cfs_bandwidth;
#ifdef CONFIG_BT_GROUP_SCHED
unsigned long offline;
struct mutex offline_mutex;
#endif
};
#ifdef CONFIG_FAIR_GROUP_SCHED

View File

@ -139,7 +139,7 @@ static DEFINE_PER_CPU_SHARED_ALIGNED(call_single_data_t, csd_data);
* ->func, ->info, and ->flags set.
*/
static int generic_exec_single(int cpu, call_single_data_t *csd,
smp_call_func_t func, void *info, struct cpumask *mask)
smp_call_func_t func, void *info)
{
if (cpu == smp_processor_id()) {
unsigned long flags;
@ -176,8 +176,7 @@ static int generic_exec_single(int cpu, call_single_data_t *csd,
* equipped to do the right thing...
*/
if (llist_add(&csd->llist, &per_cpu(call_single_queue, cpu)))
if (!mask) arch_send_call_function_single_ipi(cpu);
else __cpumask_set_cpu(cpu, mask);
arch_send_call_function_single_ipi(cpu);
return 0;
}
@ -297,7 +296,7 @@ int smp_call_function_single(int cpu, smp_call_func_t func, void *info,
csd_lock(csd);
}
err = generic_exec_single(cpu, csd, func, info, NULL);
err = generic_exec_single(cpu, csd, func, info);
if (wait)
csd_lock_wait(csd);
@ -337,50 +336,13 @@ int smp_call_function_single_async(int cpu, call_single_data_t *csd)
csd->flags = CSD_FLAG_LOCK;
smp_wmb();
err = generic_exec_single(cpu, csd, csd->func, csd->info, NULL);
err = generic_exec_single(cpu, csd, csd->func, csd->info);
preempt_enable();
return err;
}
EXPORT_SYMBOL_GPL(smp_call_function_single_async);
/**
* smp_call_function_many_async(): Run an asynchronous function on a
* specific CPU.
* @cpu: The CPU to run on.
* @csd: Pre-allocated and setup data structure
*
* Like smp_call_function_single(), but the call is asynchonous and
* can thus be done from contexts with disabled interrupts.
*
* The caller passes his own pre-allocated data structure
* (ie: embedded in an object) and is responsible for synchronizing it
* such that the IPIs performed on the @csd are strictly serialized.
*
* NOTE: Be careful, there is unfortunately no current debugging facility to
* validate the correctness of this serialization.
*/
int smp_call_function_many_async(int cpu, call_single_data_t *csd, struct cpumask *mask)
{
int err = 0;
preempt_disable();
/* We could deadlock if we have to wait here with interrupts disabled! */
if (WARN_ON_ONCE(csd->flags & CSD_FLAG_LOCK))
csd_lock_wait(csd);
csd->flags = CSD_FLAG_LOCK;
smp_wmb();
err = generic_exec_single(cpu, csd, csd->func, csd->info, mask);
preempt_enable();
return err;
}
EXPORT_SYMBOL_GPL(smp_call_function_many_async);
/*
* smp_call_function_any - Run a function on any of the given cpus
* @mask: The mask of cpus it can run on.

View File

@ -26,8 +26,6 @@
#include <linux/smpboot.h>
#include <linux/tick.h>
#include <linux/irq.h>
#include <linux/sched/stat.h>
#include <linux/sched/clock.h>
#define CREATE_TRACE_POINTS
#include <trace/events/irq.h>
@ -79,32 +77,6 @@ static void wakeup_softirqd(void)
wake_up_process(tsk);
}
unsigned int sysctl_softirq_accel_target = 2 * 1000 * 1000; //2ms
int sysctl_softirq_accel_mask;
int min_softirq_accel_mask;
int max_softirq_accel_mask = (1 << NR_SOFTIRQS) - 1;
static bool need_softirq_accel(struct task_struct *tsk, unsigned long pending)
{
#ifdef CONFIG_SCHED_INFO
if (tsk && tsk->state == TASK_RUNNING &&
(pending & sysctl_softirq_accel_mask)) {
u64 delta = sched_clock_cpu(smp_processor_id());
if (!sched_info_on() || current == tsk ||
!tsk->sched_info.last_queued ||
delta <= tsk->sched_info.last_queued)
return false;
delta -= tsk->sched_info.last_queued;
if (delta >= sysctl_softirq_accel_target) {
tsk->sched_info.last_queued += delta;
return true;
}
}
#endif
return false;
}
/*
* If ksoftirqd is scheduled, we do not want to process pending softirqs
* right now. Let ksoftirqd handle this at its own rate, to get fairness,
@ -117,8 +89,6 @@ static bool ksoftirqd_running(unsigned long pending)
if (pending & SOFTIRQ_NOW_MASK)
return false;
if (sysctl_softirq_accel_mask && need_softirq_accel(tsk, pending))
return false;
return tsk && (tsk->state == TASK_RUNNING);
}

View File

@ -334,15 +334,7 @@ static int sysrq_use_leftctrl_sysctl_handler(struct ctl_table * table ,int write
extern int sysctl_min_epoll_wait_time;
extern int sysctl_clocksource_switch_unstable_cs;
extern int sysctl_clocksource_unstable_cnt;
extern unsigned int sysctl_softirq_accel_target;
extern int sysctl_softirq_accel_mask;
extern int min_softirq_accel_mask;
extern int max_softirq_accel_mask;
extern unsigned int sysctl_memcg_stat_show_subtree;
extern unsigned int sysctl_memcg_usage_show_sched;
#ifdef CONFIG_RPS
extern unsigned int sysctl_rps_using_pvipi;
#endif
unsigned int sysctl_cgroup_stats_isolated = 0;
@ -373,22 +365,6 @@ static struct ctl_table kern_table[] = {
.mode = 0644,
.proc_handler = proc_dointvec,
},
{
.procname = "softirq_accel_target_us",
.data = &sysctl_softirq_accel_target,
.maxlen = sizeof(unsigned int),
.mode = 0644,
.proc_handler = proc_dointvec,
},
{
.procname = "softirq_accel_mask",
.data = &sysctl_softirq_accel_mask,
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
.extra1 = &min_softirq_accel_mask,
.extra2 = &max_softirq_accel_mask,
},
{
.procname = "memcg_stat_show_subtree",
.data = &sysctl_memcg_stat_show_subtree,
@ -396,22 +372,7 @@ static struct ctl_table kern_table[] = {
.mode = 0644,
.proc_handler = proc_dointvec,
},
{
.procname = "memcg_usage_show_sched",
.data = &sysctl_memcg_usage_show_sched,
.maxlen = sizeof(unsigned int),
.mode = 0644,
.proc_handler = proc_dointvec,
},
#ifdef CONFIG_RPS
{
.procname = "rps_using_pvipi",
.data = &sysctl_rps_using_pvipi,
.maxlen = sizeof(unsigned int),
.mode = 0644,
.proc_handler = proc_dointvec,
},
#endif
#ifdef CONFIG_PID_NS
{
.procname = "watch_host_pid",

View File

@ -2775,8 +2775,7 @@ static void tree_events(struct mem_cgroup *memcg, unsigned long *events)
}
}
static unsigned long mem_cgroup_usage_atomic(struct mem_cgroup *memcg,
bool swap, bool atomic)
static unsigned long mem_cgroup_usage(struct mem_cgroup *memcg, bool swap)
{
unsigned long val = 0;
@ -2788,8 +2787,6 @@ static unsigned long mem_cgroup_usage_atomic(struct mem_cgroup *memcg,
val += memcg_page_state(iter, MEMCG_RSS);
if (swap)
val += memcg_page_state(iter, MEMCG_SWAP);
if (!atomic)
cond_resched();
}
} else {
if (!swap)
@ -2800,16 +2797,6 @@ static unsigned long mem_cgroup_usage_atomic(struct mem_cgroup *memcg,
return val;
}
__read_mostly unsigned int sysctl_memcg_usage_show_sched;
static unsigned long mem_cgroup_usage(struct mem_cgroup *memcg, bool swap)
{
if (sysctl_memcg_usage_show_sched)
return mem_cgroup_usage_atomic(memcg, swap, 0);
else
return mem_cgroup_usage_atomic(memcg, swap, 1);
}
enum {
RES_USAGE,
RES_LIMIT,
@ -3346,7 +3333,7 @@ static void __mem_cgroup_threshold(struct mem_cgroup *memcg, bool swap)
if (!t)
goto unlock;
usage = mem_cgroup_usage_atomic(memcg, swap, 1);
usage = mem_cgroup_usage(memcg, swap);
/*
* current_threshold points to threshold just below or equal to usage.

View File

@ -3791,7 +3791,7 @@ EXPORT_SYMBOL(rps_may_expire_flow);
/* Called from hardirq (IPI) context */
static void rps_trigger_softirq(void *data)
{
struct softnet_data *sd = &per_cpu(softnet_data, smp_processor_id());
struct softnet_data *sd = data;
____napi_schedule(sd, &sd->backlog);
sd->received_rps++;
@ -5131,34 +5131,16 @@ __sum16 __skb_gro_checksum_complete(struct sk_buff *skb)
}
EXPORT_SYMBOL(__skb_gro_checksum_complete);
#ifdef CONFIG_RPS
static DEFINE_PER_CPU_SHARED_ALIGNED(struct cpumask, ipi_mask);
unsigned int sysctl_rps_using_pvipi = 1;
#endif
static void net_rps_send_ipi(struct softnet_data *remsd)
{
#ifdef CONFIG_RPS
if (sysctl_rps_using_pvipi) {
cpumask_var_t tmpmask = this_cpu_ptr(&ipi_mask);
cpumask_clear(tmpmask);
smp_call_function_many_async_begin(tmpmask);
while (remsd) {
struct softnet_data *next = remsd->rps_ipi_next;
while (remsd) {
struct softnet_data *next = remsd->rps_ipi_next;
if (cpu_online(remsd->cpu))
smp_call_function_many_async(remsd->cpu, &remsd->csd, tmpmask);
remsd = next;
}
smp_call_function_many_async_end(tmpmask);
} else
while (remsd) {
struct softnet_data *next = remsd->rps_ipi_next;
if (cpu_online(remsd->cpu))
smp_call_function_single_async(remsd->cpu, &remsd->csd);
remsd = next;
}
if (cpu_online(remsd->cpu))
smp_call_function_single_async(remsd->cpu, &remsd->csd);
remsd = next;
}
#endif
}

View File

@ -1294,7 +1294,7 @@ struct request_sock_ops tcp_request_sock_ops __read_mostly = {
.syn_ack_timeout = tcp_syn_ack_timeout,
};
static struct tcp_request_sock_ops tcp_request_sock_ipv4_ops = {
static const struct tcp_request_sock_ops tcp_request_sock_ipv4_ops = {
.mss_clamp = TCP_MSS_DEFAULT,
#ifdef CONFIG_TCP_MD5SIG
.req_md5_lookup = tcp_v4_md5_lookup,

View File

@ -75,7 +75,7 @@ static void tcp_v6_reqsk_send_ack(const struct sock *sk, struct sk_buff *skb,
static int tcp_v6_do_rcv(struct sock *sk, struct sk_buff *skb);
static struct inet_connection_sock_af_ops ipv6_mapped;
static const struct inet_connection_sock_af_ops ipv6_mapped;
struct inet_connection_sock_af_ops ipv6_specific;
#ifdef CONFIG_TCP_MD5SIG
static const struct tcp_sock_af_ops tcp_sock_ipv6_specific;
@ -756,7 +756,7 @@ struct request_sock_ops tcp6_request_sock_ops __read_mostly = {
.syn_ack_timeout = tcp_syn_ack_timeout,
};
static struct tcp_request_sock_ops tcp_request_sock_ipv6_ops = {
static const struct tcp_request_sock_ops tcp_request_sock_ipv6_ops = {
.mss_clamp = IPV6_MIN_MTU - sizeof(struct tcphdr) -
sizeof(struct ipv6hdr),
#ifdef CONFIG_TCP_MD5SIG
@ -1677,7 +1677,7 @@ static const struct tcp_sock_af_ops tcp_sock_ipv6_specific = {
/*
* TCP over IPv4 via INET6 API
*/
static struct inet_connection_sock_af_ops ipv6_mapped = {
static const struct inet_connection_sock_af_ops ipv6_mapped = {
.queue_xmit = ip_queue_xmit,
.send_check = tcp_v4_send_check,
.rebuild_header = inet_sk_rebuild_header,

View File

@ -54,26 +54,15 @@ static int ip_vs_conn_tab_bits = CONFIG_IP_VS_TAB_BITS;
module_param_named(conn_tab_bits, ip_vs_conn_tab_bits, int, 0444);
MODULE_PARM_DESC(conn_tab_bits, "Set connections' hash size");
unsigned int ipvs_mode;
module_param_named(mode, ipvs_mode, uint, 0444);
MODULE_PARM_DESC(mode, "set mode in IPVS");
EXPORT_SYMBOL_GPL(ipvs_mode);
bool bpf_mode_on;
module_param_named(mode, bpf_mode_on, bool, 0444);
MODULE_PARM_DESC(mode, "set bpf mode in IPVS");
EXPORT_SYMBOL_GPL(bpf_mode_on);
/* size and mask values */
int ip_vs_conn_tab_size __read_mostly;
static int ip_vs_conn_tab_mask __read_mostly;
/* retrieve origin net in skb for xmit
* local-out: ip_queue_xmit->skb_dst_set_noref
* local-in: ip_route_input_slow set it
*/
struct net *ip_vs_skb_net(struct sk_buff *skb)
{
if (skb_dst(skb))
return dev_net(skb_dst(skb)->dev);
return NULL;
}
/*
* Connection hash table: for input and output packets lookups of IPVS
*/
@ -252,7 +241,7 @@ static void ip_vs_unlink_bpf(struct ip_vs_conn *cp)
struct bpf_map *map;
int err = 0;
if (ipvs_mode != IPVS_BPF_MODE)
if (!bpf_mode_on)
return;
k.sip = cp->caddr.ip;
@ -330,7 +319,7 @@ static inline bool ip_vs_conn_unlink(struct ip_vs_conn *cp)
hlist_del_rcu(&cp->c_list);
cp->flags &= ~IP_VS_CONN_F_HASHED;
ret = true;
if (ipvs_mode == IPVS_BPF_MODE)
if (bpf_mode_on)
ip_vs_unlink_bpf(cp);
}
} else
@ -1124,7 +1113,7 @@ static bool ip_vs_conn_new_bpf(struct ip_vs_dest *dest,
BUILD_BUG_ON(sizeof(atomic_t) != 4);
if (ipvs_mode != IPVS_BPF_MODE)
if (!bpf_mode_on)
return true;
svc = rcu_dereference(dest->svc);
@ -1262,7 +1251,7 @@ ip_vs_conn_new(const struct ip_vs_conn_param *p, int dest_af,
p->protocol);
int skip = 0;
if (ipvs_mode == IPVS_BPF_MODE) {
if (bpf_mode_on) {
if (!ip_vs_conn_new_bpf(dest, flags, p, &skip))
return NULL;
}
@ -1281,7 +1270,7 @@ ip_vs_conn_new(const struct ip_vs_conn_param *p, int dest_af,
cp->protocol = p->protocol;
ip_vs_addr_set(p->af, &cp->caddr, p->caddr);
cp->cport = p->cport;
if (ipvs_mode == IPVS_BPF_MODE)
if (bpf_mode_on)
cp->skip_bpf = skip;
/* proto should only be IPPROTO_IP if p->vaddr is a fwmark */
ip_vs_addr_set(p->protocol == IPPROTO_IP ? AF_UNSPEC : p->af,
@ -1806,7 +1795,7 @@ int __init ip_vs_conn_init(void)
spin_lock_init(&__ip_vs_conntbl_lock_array[idx].l);
}
if (ipvs_mode == IPVS_BPF_MODE) {
if (bpf_mode_on) {
for (idx = 0; idx < BPF_CONN_LOCKS; idx++)
spin_lock_init(&bpf_conntrack_locks[idx]);
}

View File

@ -747,18 +747,11 @@ static int ip_route_me_harder2(struct net *net, struct sk_buff *skb,
static int ip_vs_route_me_harder(struct netns_ipvs *ipvs, int af,
struct sk_buff *skb, unsigned int hooknum)
{
struct net *net;
if (ipvs_mode != IPVS_BPF_MODE && !sysctl_snat_reroute(ipvs))
if (!bpf_mode_on && !sysctl_snat_reroute(ipvs))
return 0;
/* Reroute replies only to remote clients (FORWARD and LOCAL_OUT) */
if (NF_INET_LOCAL_IN == hooknum)
return 0;
net = ipvs->net;
if (ipvs_mode == IPVS_SHARE_NS_MODE)
net = ip_vs_skb_net(skb);
#ifdef CONFIG_IP_VS_IPV6
if (af == AF_INET6) {
struct dst_entry *dst = skb_dst(skb);
@ -768,9 +761,9 @@ static int ip_vs_route_me_harder(struct netns_ipvs *ipvs, int af,
return 1;
} else
#endif
if (ipvs_mode != IPVS_BPF_MODE) {
if (!bpf_mode_on) {
if (!(skb_rtable(skb)->rt_flags & RTCF_LOCAL) &&
ip_route_me_harder(net, skb, RTN_LOCAL) != 0)
ip_route_me_harder(ipvs->net, skb, RTN_LOCAL) != 0)
return 1;
} else {
if (ip_route_me_harder2(ipvs->net, skb, RTN_LOCAL) != 0)
@ -937,12 +930,12 @@ static int handle_response_icmp(int af, struct sk_buff *skb,
else
ip_vs_update_conntrack(skb, cp, 0);
if (ipvs_mode == IPVS_BPF_MODE)
if (bpf_mode_on)
(*(resolve_addrs.ip_finish_output))(cp->ipvs->net, skb->sk,
skb);
ignore_cp:
if (ipvs_mode == IPVS_BPF_MODE)
if (bpf_mode_on)
verdict = NF_STOLEN;
else
verdict = NF_ACCEPT;
@ -1361,14 +1354,14 @@ handle_response(int af, struct sk_buff *skb, struct ip_vs_proto_data *pd,
else
ip_vs_update_conntrack(skb, cp, 0);
if (ipvs_mode == IPVS_BPF_MODE)
if (bpf_mode_on)
(*(resolve_addrs.ip_finish_output))(cp->ipvs->net, skb->sk,
skb);
ip_vs_conn_put(cp);
LeaveFunction(11);
if (ipvs_mode == IPVS_BPF_MODE)
if (bpf_mode_on)
return NF_STOLEN;
else
return NF_ACCEPT;
@ -1379,12 +1372,6 @@ drop:
return NF_STOLEN;
}
static void switch_netns(struct netns_ipvs **ipvs, struct sk_buff *skb)
{
if (ipvs_mode == IPVS_SHARE_NS_MODE)
*ipvs = net_ipvs(&init_net);
}
/*
* Check if outgoing packet belongs to the established ip_vs_conn.
* bpf: previously, local-in, forward, and local-out may call here!
@ -1401,8 +1388,6 @@ ip_vs_out(struct netns_ipvs *ipvs, unsigned int hooknum, struct sk_buff *skb, in
EnterFunction(11);
switch_netns(&ipvs, skb);
/* Already marked as IPVS request or reply? */
if (skb->ipvs_property)
return NF_ACCEPT;
@ -1417,7 +1402,7 @@ ip_vs_out(struct netns_ipvs *ipvs, unsigned int hooknum, struct sk_buff *skb, in
}
/* In bpf mode, this is null */
if (ipvs_mode != IPVS_BPF_MODE && unlikely(!skb_dst(skb)))
if (!bpf_mode_on && unlikely(!skb_dst(skb)))
return NF_ACCEPT;
if (!ipvs->enable)
@ -1646,8 +1631,6 @@ ip_vs_in_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, int *related,
unsigned int offset, offset2, ihl, verdict;
bool ipip, new_cp = false;
switch_netns(&ipvs, skb);
*related = 1;
/* reassemble IP fragments */
@ -1932,8 +1915,6 @@ ip_vs_in(struct netns_ipvs *ipvs, unsigned int hooknum, struct sk_buff *skb, int
int conn_reuse_mode;
struct sock *sk;
switch_netns(&ipvs, skb);
/* Already marked as IPVS request or reply? */
if (skb->ipvs_property)
return NF_ACCEPT;
@ -1945,7 +1926,7 @@ ip_vs_in(struct netns_ipvs *ipvs, unsigned int hooknum, struct sk_buff *skb, int
*/
if (unlikely((skb->pkt_type != PACKET_HOST &&
hooknum != NF_INET_LOCAL_OUT) ||
(ipvs_mode != IPVS_BPF_MODE && !skb_dst(skb)))) {
(!bpf_mode_on && !skb_dst(skb)))) {
ip_vs_fill_iph_skb(af, skb, false, &iph);
IP_VS_DBG_BUF(12, "packet type=%d proto=%d daddr=%s"
" ignored in hook %u\n",
@ -2014,9 +1995,7 @@ ip_vs_in(struct netns_ipvs *ipvs, unsigned int hooknum, struct sk_buff *skb, int
* as zero rs may be killed already.
* Only enable in bpf mode currently. Shall promote to IPVS mode later.
*/
if (no_route_to_host_fix &&
ipvs_mode == IPVS_BPF_MODE &&
cp && cp->dest &&
if (no_route_to_host_fix && bpf_mode_on && cp && cp->dest &&
unlikely(!atomic_read(&cp->dest->weight)) &&
is_new_conn(skb, &iph) && !iph.fragoffs &&
conn_reuse_mode == 0) {
@ -2139,8 +2118,7 @@ ip_vs_remote_request4(void *priv, struct sk_buff *skb,
* the defrag may impact performance greatly! Lukily, this is not the
* case for us!
*/
if (ipvs_mode == IPVS_BPF_MODE &&
unlikely(ip_is_fragment(ip_hdr(skb)))) {
if (bpf_mode_on && unlikely(ip_is_fragment(ip_hdr(skb)))) {
if (ip_vs_gather_frags(net_ipvs(state->net), skb,
IP_DEFRAG_VS_IN))
/* return 0 will call skb_free in nf_hook */
@ -2376,113 +2354,6 @@ static const struct nf_hook_ops ip_vs_bpf_ops[] = {
*/
};
static const struct nf_hook_ops ip_vs_ns_ops[] = {
/* After packet filtering, change source only for VS/NAT */
{
.hook = ip_vs_reply4,
.pf = NFPROTO_IPV4,
.hooknum = NF_INET_LOCAL_IN,
.priority = NF_IP_PRI_NAT_SRC - 2,
},
{
.hook = ip_vs_remote_request4,
.pf = NFPROTO_IPV4,
.hooknum = NF_INET_LOCAL_IN,
.priority = NF_IP_PRI_NAT_SRC - 1,
},
/* Delete IPVS nf local_out hook to handle response packet.
* Consider following steps:
* 1. curl vip:vport on a vm. packet is (nodeip:tmport->vip:vport)
* 2. Ipvs does DNAT and choose a POD on this vm.
* packet is (nodeip:tmpport->rsip:rsport)
* 3. The POD replies. packet is (rsip:rsport -> nodeip:tmpport)
* In nf local-out, ipvs ip_vs_local_reply4
* does reverse DNAT, and modifies the packet to be
* (VIP:VPORT->nodeip:tmpport)
* 4. The packet go out of the POD's ENI to the iaas switch.
* 5. Iaas switch will drop the packet as it expects the source
* to be the ENI's ip.
* Any side effect to delete the hook?
* If a client out of the cluster accesses the service on a cvm,
* and the cvm choose a process runs on default net ns as the target,
* Break!. However, It doesn't matter as we haven't such case.
*/
/* After mangle, schedule and forward local requests */
{
.hook = ip_vs_local_request4,
.pf = NFPROTO_IPV4,
.hooknum = NF_INET_LOCAL_OUT,
.priority = NF_IP_PRI_NAT_DST + 2,
},
/* After packet filtering (but before ip_vs_out_icmp), catch icmp
* destined for 0.0.0.0/0, which is for incoming IPVS connections
*/
{
.hook = ip_vs_forward_icmp,
.pf = NFPROTO_IPV4,
.hooknum = NF_INET_FORWARD,
.priority = 99,
},
/* After packet filtering, change source only for VS/NAT */
{
.hook = ip_vs_reply4,
.pf = NFPROTO_IPV4,
.hooknum = NF_INET_FORWARD,
.priority = 100,
},
#ifdef CONFIG_IP_VS_IPV6
/* After packet filtering, change source only for VS/NAT */
{
.hook = ip_vs_reply6,
.pf = NFPROTO_IPV6,
.hooknum = NF_INET_LOCAL_IN,
.priority = NF_IP6_PRI_NAT_SRC - 2,
},
/* After packet filtering, forward packet through VS/DR, VS/TUN,
* or VS/NAT(change destination), so that filtering rules can be
* applied to IPVS
*/
{
.hook = ip_vs_remote_request6,
.pf = NFPROTO_IPV6,
.hooknum = NF_INET_LOCAL_IN,
.priority = NF_IP6_PRI_NAT_SRC - 1,
},
/* Before ip_vs_in, change source only for VS/NAT */
{
.hook = ip_vs_local_reply6,
.pf = NFPROTO_IPV6,
.hooknum = NF_INET_LOCAL_OUT,
.priority = NF_IP6_PRI_NAT_DST + 1,
},
/* After mangle, schedule and forward local requests */
{
.hook = ip_vs_local_request6,
.pf = NFPROTO_IPV6,
.hooknum = NF_INET_LOCAL_OUT,
.priority = NF_IP6_PRI_NAT_DST + 2,
},
/* After packet filtering (but before ip_vs_out_icmp), catch icmp
* destined for 0.0.0.0/0, which is for incoming IPVS connections
*/
{
.hook = ip_vs_forward_icmp_v6,
.pf = NFPROTO_IPV6,
.hooknum = NF_INET_FORWARD,
.priority = 99,
},
/* After packet filtering, change source only for VS/NAT */
{
.hook = ip_vs_reply6,
.pf = NFPROTO_IPV6,
.hooknum = NF_INET_FORWARD,
.priority = 100,
},
#endif
};
/*
* Initialize IP Virtual Server netns mem.
*/
@ -2491,8 +2362,6 @@ static int __net_init __ip_vs_init(struct net *net)
struct netns_ipvs *ipvs;
int ret;
if (ipvs_mode >= IPVS_MAX_MODE)
return -EINVAL;
ipvs = net_generic(net, ip_vs_net_id);
if (ipvs == NULL)
return -ENOMEM;
@ -2523,19 +2392,12 @@ static int __net_init __ip_vs_init(struct net *net)
if (ip_vs_sync_net_init(ipvs) < 0)
goto sync_fail;
if (ipvs_mode == IPVS_ORIGIN_MODE) {
if (!bpf_mode_on) {
ret = nf_register_net_hooks(net, ip_vs_ops,
ARRAY_SIZE(ip_vs_ops));
if (ret < 0)
goto hook_fail;
}
if (ipvs_mode == IPVS_SHARE_NS_MODE) {
ret = nf_register_net_hooks(net, ip_vs_ns_ops,
ARRAY_SIZE(ip_vs_ns_ops));
if (ret < 0)
goto hook_fail;
}
if (ipvs_mode == IPVS_BPF_MODE) {
} else {
ret = nf_register_net_hooks(net, ip_vs_bpf_ops,
ARRAY_SIZE(ip_vs_bpf_ops));
if (ret < 0)
@ -2556,6 +2418,7 @@ static int __net_init __ip_vs_init(struct net *net)
goto hook_fail;
}
}
return 0;
/*
* Error handling
@ -2582,14 +2445,10 @@ static void __net_exit __ip_vs_cleanup(struct net *net)
{
struct netns_ipvs *ipvs = net_ipvs(net);
if (ipvs_mode == IPVS_ORIGIN_MODE)
nf_unregister_net_hooks(net, ip_vs_ops, ARRAY_SIZE(ip_vs_ops));
if (ipvs_mode == IPVS_SHARE_NS_MODE)
nf_unregister_net_hooks(net, ip_vs_ns_ops,
ARRAY_SIZE(ip_vs_ns_ops));
if (ipvs_mode == IPVS_BPF_MODE) {
if (!bpf_mode_on)
nf_unregister_net_hooks(net, ip_vs_ops,
ARRAY_SIZE(ip_vs_ops));
else {
nf_unregister_net_hooks(net, ip_vs_bpf_ops,
ARRAY_SIZE(ip_vs_bpf_ops));
free_percpu(ipvs->bpf_stat);
@ -2635,8 +2494,7 @@ struct bpf_sym_addrs resolve_addrs;
static int __init ip_vs_init(void)
{
int ret;
if (ipvs_mode == IPVS_BPF_MODE) {
if (bpf_mode_on) {
resolve_addrs.ip_finish_output =
(output_t)kallsyms_lookup_name("ip_finish_output");
if (!resolve_addrs.ip_finish_output) {
@ -2672,7 +2530,7 @@ static int __init ip_vs_init(void)
}
}
pr_info("ipvs_mode is %d\n", ipvs_mode);
pr_info("bpf_mode_on is %d\n", bpf_mode_on);
ret = ip_vs_control_init();
if (ret < 0) {
pr_err("can't setup control.\n");
@ -2726,7 +2584,7 @@ static void __exit ip_vs_cleanup(void)
ip_vs_conn_cleanup();
ip_vs_protocol_cleanup();
ip_vs_control_cleanup();
if (ipvs_mode == IPVS_BPF_MODE)
if (bpf_mode_on)
ip_vs_bpf_put();
pr_info("ipvs unloaded.\n");
}

View File

@ -1191,7 +1191,7 @@ ip_vs_add_service(struct netns_ipvs *ipvs, struct ip_vs_service_user_kern *u,
ip_vs_use_count_inc();
/* in bpf mode, avoid loopback traffic */
if (ipvs_mode == IPVS_BPF_MODE && strcmp(u->sched_name, "wrr") != 0 &&
if (bpf_mode_on && strcmp(u->sched_name, "wrr") != 0 &&
strcmp(u->sched_name, "rr") != 0 &&
strcmp(u->sched_name, "lc") != 0 &&
strcmp(u->sched_name, "wlc") != 0) {
@ -1885,12 +1885,6 @@ static struct ctl_table vs_vars[] = {
.mode = 0644,
.proc_handler = proc_dointvec,
},
{
.procname = "conn_reuse_old_conntrack",
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec,
},
{
.procname = "schedule_icmp",
.maxlen = sizeof(int),

View File

@ -47,8 +47,7 @@ ip_vs_lc_schedule(struct ip_vs_service *svc, const struct sk_buff *skb,
list_for_each_entry_rcu(dest, &svc->destinations, n_list) {
if ((dest->flags & IP_VS_DEST_F_OVERLOAD) ||
atomic_read(&dest->weight) == 0 ||
(ipvs_mode == IPVS_BPF_MODE &&
dest->addr.ip == iph->saddr.ip))
(bpf_mode_on && dest->addr.ip == iph->saddr.ip))
continue;
doh = ip_vs_dest_conn_overhead(dest);
if (!least || doh < loh) {

View File

@ -76,7 +76,7 @@ ip_vs_rr_schedule(struct ip_vs_service *svc, const struct sk_buff *skb,
if (!(dest->flags & IP_VS_DEST_F_OVERLOAD) &&
atomic_read(&dest->weight) > 0) {
/* HIT */
if (ipvs_mode != IPVS_BPF_MODE)
if (!bpf_mode_on)
goto out;
else if (dest->addr.ip != iph->saddr.ip)
goto out;

View File

@ -56,9 +56,8 @@ ip_vs_wlc_schedule(struct ip_vs_service *svc, const struct sk_buff *skb,
list_for_each_entry_rcu(dest, &svc->destinations, n_list) {
if (!(dest->flags & IP_VS_DEST_F_OVERLOAD) &&
atomic_read(&dest->weight) > 0) {
if (ipvs_mode != IPVS_BPF_MODE ||
(ipvs_mode == IPVS_BPF_MODE &&
dest->addr.ip != iph->saddr.ip)) {
if (!bpf_mode_on ||
(bpf_mode_on && dest->addr.ip != iph->saddr.ip)) {
least = dest;
loh = ip_vs_dest_conn_overhead(least);
goto nextstage;
@ -75,8 +74,7 @@ ip_vs_wlc_schedule(struct ip_vs_service *svc, const struct sk_buff *skb,
/* in bpf mode, avoid loopback traffic */
list_for_each_entry_continue_rcu(dest, &svc->destinations, n_list) {
if ((dest->flags & IP_VS_DEST_F_OVERLOAD) ||
(ipvs_mode == IPVS_BPF_MODE &&
dest->addr.ip == iph->saddr.ip))
(bpf_mode_on && dest->addr.ip == iph->saddr.ip))
continue;
doh = ip_vs_dest_conn_overhead(dest);
if ((__s64)loh * atomic_read(&dest->weight) >

View File

@ -185,7 +185,7 @@ ip_vs_wrr_schedule(struct ip_vs_service *svc, const struct sk_buff *skb,
n_list) {
if (!(dest->flags & IP_VS_DEST_F_OVERLOAD) &&
atomic_read(&dest->weight) >= mark->cw) {
if (ipvs_mode != IPVS_BPF_MODE)
if (!bpf_mode_on)
goto found;
else if (dest->addr.ip != iph->saddr.ip)
goto found;

View File

@ -316,13 +316,7 @@ __ip_vs_get_out_rt(struct netns_ipvs *ipvs, int skb_af, struct sk_buff *skb,
int mtu;
int local, noref = 1;
if (ipvs_mode == IPVS_SHARE_NS_MODE)
net = ip_vs_skb_net(skb);
if (!net)
return -1;
/* when share netns, the cache will error */
if (dest && ipvs_mode != IPVS_SHARE_NS_MODE) {
if (dest) {
dest_dst = __ip_vs_dst_check(dest);
if (likely(dest_dst))
rt = (struct rtable *) dest_dst->dst_cache;
@ -371,7 +365,7 @@ __ip_vs_get_out_rt(struct netns_ipvs *ipvs, int skb_af, struct sk_buff *skb,
/* In bpf mode, this check always return false.Don't call it to avoid
* access of skb->dst
*/
if (ipvs_mode != IPVS_BPF_MODE &&
if (!bpf_mode_on &&
unlikely(crosses_local_route_boundary(skb_af, skb, rt_mode,
local))) {
IP_VS_DBG_RL("We are crossing local and non-local addresses"
@ -379,10 +373,7 @@ __ip_vs_get_out_rt(struct netns_ipvs *ipvs, int skb_af, struct sk_buff *skb,
goto err_put;
}
/* traffic to local address shall route to lo dev
* so that traffic from a POD can choose itself as rs.
*/
if (ipvs_mode != IPVS_SHARE_NS_MODE && unlikely(local)) {
if (unlikely(local)) {
/* skb to local stack, preserve old route */
if (!noref)
ip_rt_put(rt);
@ -416,7 +407,7 @@ __ip_vs_get_out_rt(struct netns_ipvs *ipvs, int skb_af, struct sk_buff *skb,
skb_dst_set(skb, &rt->dst);
/* In bpf mode, like ip_output, set output dev */
if (ipvs_mode == IPVS_BPF_MODE)
if (bpf_mode_on)
skb->dev = skb_dst(skb)->dev;
return local;
@ -627,11 +618,6 @@ static inline int ip_vs_nat_send_or_cont(int pf, struct sk_buff *skb,
struct ip_vs_conn *cp, int local)
{
int ret = NF_STOLEN;
struct net *net;
net = cp->ipvs->net;
if (ipvs_mode == IPVS_SHARE_NS_MODE)
net = ip_vs_skb_net(skb);
skb->ipvs_property = 1;
if (likely(!(cp->flags & IP_VS_CONN_F_NFCT)))
@ -648,7 +634,7 @@ static inline int ip_vs_nat_send_or_cont(int pf, struct sk_buff *skb,
if (!local) {
skb_forward_csum(skb);
NF_HOOK(pf, NF_INET_LOCAL_OUT, net, NULL, skb,
NF_HOOK(pf, NF_INET_LOCAL_OUT, cp->ipvs->net, NULL, skb,
NULL, skb_dst(skb)->dev, dst_output);
} else
ret = NF_ACCEPT;
@ -661,11 +647,6 @@ static inline int ip_vs_send_or_cont(int pf, struct sk_buff *skb,
struct ip_vs_conn *cp, int local)
{
int ret = NF_STOLEN;
struct net *net;
net = cp->ipvs->net;
if (ipvs_mode == IPVS_SHARE_NS_MODE)
net = ip_vs_skb_net(skb);
skb->ipvs_property = 1;
if (likely(!(cp->flags & IP_VS_CONN_F_NFCT)))
@ -673,7 +654,7 @@ static inline int ip_vs_send_or_cont(int pf, struct sk_buff *skb,
if (!local) {
ip_vs_drop_early_demux_sk(skb);
skb_forward_csum(skb);
NF_HOOK(pf, NF_INET_LOCAL_OUT, net, NULL, skb,
NF_HOOK(pf, NF_INET_LOCAL_OUT, cp->ipvs->net, NULL, skb,
NULL, skb_dst(skb)->dev, dst_output);
} else
ret = NF_ACCEPT;
@ -779,20 +760,21 @@ ip_vs_nat_xmit(struct sk_buff *skb, struct ip_vs_conn *cp,
IP_VS_DBG(10, "filled cport=%d\n", ntohs(*p));
}
/* originally, this was set in ip_route_input_slow
/* In ipvs mode, ip_route_input_slow will set me to 1 for
* local_in pkt from nic! For pkt local-out this is not set!
* In bpf mode, this is not useful since local rs is not allowed
*/
if (ipvs_mode == IPVS_BPF_MODE)
was_input = 1;
else
if (!bpf_mode_on)
was_input = rt_is_input_route(skb_rtable(skb));
else
was_input = 1;
local = __ip_vs_get_out_rt(cp->ipvs, cp->af, skb, cp->dest, cp->daddr.ip,
IP_VS_RT_MODE_LOCAL |
IP_VS_RT_MODE_NON_LOCAL |
IP_VS_RT_MODE_RDR, NULL, ipvsh);
if (local < 0)
goto tx_error;
if (ipvs_mode == IPVS_BPF_MODE && local == 1) {
if (bpf_mode_on && local == 1) {
pr_err("shall not route to local rs in bpf mode\n");
BPF_STAT_INC(cp->ipvs, BPF_XMIT_LOCAL_RS);
goto tx_error;
@ -851,7 +833,7 @@ ip_vs_nat_xmit(struct sk_buff *skb, struct ip_vs_conn *cp,
/* Another hack: avoid icmp_send in ip_fragment */
skb->ignore_df = 1;
if (ipvs_mode != IPVS_BPF_MODE) {
if (!bpf_mode_on) {
rc = ip_vs_nat_send_or_cont(NFPROTO_IPV4, skb, cp, local);
} else {
/* used by bpf egress to construct the key!
@ -1343,7 +1325,7 @@ ip_vs_icmp_xmit(struct sk_buff *skb, struct ip_vs_conn *cp,
/*
* mangle and send the packet here (only for VS/NAT)
*/
if (ipvs_mode != IPVS_BPF_MODE)
if (!bpf_mode_on)
was_input = rt_is_input_route(skb_rtable(skb));
else
was_input = 1;
@ -1396,7 +1378,7 @@ ip_vs_icmp_xmit(struct sk_buff *skb, struct ip_vs_conn *cp,
/* Another hack: avoid icmp_send in ip_fragment */
skb->ignore_df = 1;
if (ipvs_mode != IPVS_BPF_MODE) {
if (!bpf_mode_on) {
rc = ip_vs_nat_send_or_cont(NFPROTO_IPV4, skb, cp, local);
} else {
/* used by bpf egress to construct the key!

View File

@ -2192,8 +2192,7 @@ static int tpacket_rcv(struct sk_buff *skb, struct net_device *dev,
int skb_len = skb->len;
unsigned int snaplen, res;
unsigned long status = TP_STATUS_USER;
unsigned short macoff, hdrlen;
unsigned int netoff;
unsigned short macoff, netoff, hdrlen;
struct sk_buff *copy_skb = NULL;
struct timespec ts;
__u32 ts_status;
@ -2255,10 +2254,6 @@ static int tpacket_rcv(struct sk_buff *skb, struct net_device *dev,
}
macoff = netoff - maclen;
}
if (netoff > USHRT_MAX) {
po->stats.stats1.tp_drops++;
goto drop_n_restore;
}
if (po->tp_version <= TPACKET_V2) {
if (macoff + snaplen > po->rx_ring.frame_size) {
if (po->copy_thresh &&

View File

@ -163,6 +163,7 @@ CONFIG_DEBUG_BLK_CGROUP=y
CONFIG_CGROUP_WRITEBACK=y
CONFIG_CGROUP_SCHED=y
CONFIG_FAIR_GROUP_SCHED=y
CONFIG_BT_GROUP_SCHED=y
CONFIG_CFS_BANDWIDTH=y
CONFIG_RT_GROUP_SCHED=y
CONFIG_CGROUP_PIDS=y
@ -184,7 +185,7 @@ CONFIG_IPC_NS=y
CONFIG_USER_NS=y
CONFIG_PID_NS=y
CONFIG_NET_NS=y
# CONFIG_BT_SCHED is not set
CONFIG_BT_SCHED=y
CONFIG_SCHED_AUTOGROUP=y
# CONFIG_SYSFS_DEPRECATED is not set
CONFIG_RELAY=y
@ -1258,9 +1259,7 @@ CONFIG_IP6_NF_TARGET_REJECT=m
CONFIG_IP6_NF_MANGLE=m
CONFIG_IP6_NF_RAW=m
# CONFIG_IP6_NF_SECURITY is not set
CONFIG_IP6_NF_NAT=m
CONFIG_IP6_NF_TARGET_MASQUERADE=m
CONFIG_IP6_NF_TARGET_NPT=m
# CONFIG_IP6_NF_NAT is not set
CONFIG_BRIDGE_NF_EBTABLES=m
CONFIG_BRIDGE_EBT_BROUTE=m
CONFIG_BRIDGE_EBT_T_FILTER=m
@ -1399,7 +1398,6 @@ CONFIG_DNS_RESOLVER=y
CONFIG_OPENVSWITCH=m
CONFIG_OPENVSWITCH_GRE=m
CONFIG_OPENVSWITCH_VXLAN=m
CONFIG_OPENVSWITCH_GENEVE=m
CONFIG_VSOCKETS=m
CONFIG_VMWARE_VMCI_VSOCKETS=m
CONFIG_VIRTIO_VSOCKETS=m
@ -1659,8 +1657,6 @@ CONFIG_BLK_DEV_SR=m
CONFIG_BLK_DEV_SR_VENDOR=y
CONFIG_CHR_DEV_SG=y
CONFIG_CHR_DEV_SCH=y
CONFIG_MEGARAID_LEGACY=y
CONFIG_MEGARAID_SAS=y
CONFIG_SCSI_CONSTANTS=y
CONFIG_SCSI_LOGGING=y
# CONFIG_SCSI_SCAN_ASYNC is not set
@ -1712,6 +1708,8 @@ CONFIG_SCSI_MVUMI=y
# CONFIG_SCSI_ADVANSYS is not set
CONFIG_SCSI_ARCMSR=y
# CONFIG_SCSI_ESAS2R is not set
CONFIG_MEGARAID_SAS=y
CONFIG_MEGARAID_LEGACY=y
CONFIG_SCSI_MPT3SAS=y
CONFIG_SCSI_MPT2SAS_MAX_SGE=128
CONFIG_SCSI_MPT3SAS_MAX_SGE=128
@ -1943,7 +1941,7 @@ CONFIG_MACVTAP=m
CONFIG_IPVLAN=m
CONFIG_IPVTAP=m
CONFIG_VXLAN=m
CONFIG_GENEVE=m
# CONFIG_GENEVE is not set
# CONFIG_GTP is not set
# CONFIG_MACSEC is not set
CONFIG_NETCONSOLE=m
@ -3778,13 +3776,11 @@ CONFIG_OVERLAY_FS=m
# Caches
#
CONFIG_FSCACHE=m
CONFIG_FSCACHE_STATS=y
# CONFIG_FSCACHE_STATS is not set
# CONFIG_FSCACHE_HISTOGRAM is not set
# CONFIG_FSCACHE_DEBUG is not set
# CONFIG_FSCACHE_OBJECT_LIST is not set
CONFIG_CACHEFILES=m
# CONFIG_CACHEFILES_DEBUG is not set
# CONFIG_CACHEFILES_HISTOGRAM is not set
# CONFIG_CACHEFILES is not set
#
# CD-ROM/DVD Filesystems
@ -3844,10 +3840,10 @@ CONFIG_SQUASHFS_FILE_CACHE=y
CONFIG_SQUASHFS_DECOMP_SINGLE=y
# CONFIG_SQUASHFS_DECOMP_MULTI is not set
# CONFIG_SQUASHFS_DECOMP_MULTI_PERCPU is not set
CONFIG_SQUASHFS_XATTR=y
CONFIG_SQUASHFS_ZLIB=y
CONFIG_SQUASHFS_LZ4=y
CONFIG_SQUASHFS_LZO=y
# CONFIG_SQUASHFS_XATTR is not set
# CONFIG_SQUASHFS_ZLIB is not set
# CONFIG_SQUASHFS_LZ4 is not set
# CONFIG_SQUASHFS_LZO is not set
CONFIG_SQUASHFS_XZ=y
# CONFIG_SQUASHFS_ZSTD is not set
# CONFIG_SQUASHFS_4K_DEVBLK_SIZE is not set