commit <ffa358dcaae1f2f00926484e712e06daa8953cb4>
commit <b78820937b4762b7d30b807d7156bec1d89e4dd3>
commit <c45e6a037a536530bd25781ac7c989e52deb2a63>
commit <38cfb5a45ee013bfab5d1ae4c4738815e744b440>
We have two potential issues:
1) After commit 2887e41b910b, we only wake one process at the time when
we finish an IO. We really want to wake up as many tasks as can
queue IO. Before this commit, we woke up everyone, which could cause
a thundering herd issue.
2) A task can potentially consume two wakeups, causing us to (in
practice) miss a wakeup.
Fix both by providing our own wakeup function, which stops
__wake_up_common() from waking up more tasks if we fail to get a
queueing token. With the strict ordering we have on the wait list, this
wakes the right tasks and the right amount of tasks.
Signed-off-by: Chunguang Xu <brookxu@tencent.com>
commit <2887e41b910bb14fd847cf01ab7a5993db989d88>
At present, wbt's wake-up mechanism for wait_queue can induce
thundering herd issue and intense lock competition. In the
context of ipi, this may induce hard lockup.
Signed-off-by: Chunguang Xu <brookxu@tencent.com>
commit <0aa69fd32a5f766e997ca8ab4723c5a1146efa8b>
commit <17d51b10d7773e4618bcac64648f30f12d4078fb>
bio_iov_iter_get_pages() currently only adds pages for the next non-zero
segment from the iov_iter to the bio. That's suboptimal for callers,
which typically try to pin as many pages as fit into the bio. This patch
converts the current bio_iov_iter_get_pages() into a static helper, and
introduces a new helper that allocates as many pages as
1) fit into the bio,
2) are present in the iov_iter,
3) and can be pinned by MM.
Error is returned only if zero pages could be pinned. Because of 3), a
zero return value doesn't necessarily mean all pages have been pinned.
Callers that have to pin every page in the iov_iter must still call this
function in a loop (this is currently the case).
This change matters most for __blkdev_direct_IO_simple(), which calls
bio_iov_iter_get_pages() only once. If it obtains less pages than
requested, it returns a "short write" or "short read", and
__generic_file_write_iter() falls back to buffered writes, which may
lead to data corruption.
Signed-off-by: Chunguang Xu <brookxu@tencent.com>
Kernel now supports cgroup bps and iops hard limit by io-throttling.
But that's not a scalable way, because block devices' bandwith is
always changing over time.
This patch dynamically adjusts bps limit of each blkio cgroup according
to its pre-set weight. Additionally, if a process has not issued any io
request during a time window, its weight will be shared by other processes
until its next io request.
Signed-off-by: Lei Chen <lennychen@tencent.com>
In order to facilitate each container to obtain its own IO statistics,
we implement per blkcg diskstats and expose some data from the host
into the container such as io_ticks.
Signed-off-by: Chunguang Xu <brookxu@tencent.com>
Chunguang Xu