Skip to content

Commit a9ed27a

Browse files
Ming Leiaxboe
authored andcommitted
blk-mq: fix is_flush_rq
is_flush_rq() is called from bt_iter()/bt_tags_iter(), and runs the following check: hctx->fq->flush_rq == req but the passed hctx from bt_iter()/bt_tags_iter() may be NULL because: 1) memory re-order in blk_mq_rq_ctx_init(): rq->mq_hctx = data->hctx; ... refcount_set(&rq->ref, 1); OR 2) tag re-use and ->rqs[] isn't updated with new request. Fix the issue by re-writing is_flush_rq() as: return rq->end_io == flush_end_io; which turns out simpler to follow and immune to data race since we have ordered WRITE rq->end_io and refcount_set(&rq->ref, 1). Fixes: 2e315dc ("blk-mq: grab rq->refcount before calling ->fn in blk_mq_tagset_busy_iter") Cc: "Blank-Burian, Markus, Dr." <[email protected]> Cc: Yufen Yu <[email protected]> Signed-off-by: Ming Lei <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Jens Axboe <[email protected]>
1 parent c2da19e commit a9ed27a

File tree

3 files changed

+7
-6
lines changed

3 files changed

+7
-6
lines changed

block/blk-flush.c

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -262,6 +262,11 @@ static void flush_end_io(struct request *flush_rq, blk_status_t error)
262262
spin_unlock_irqrestore(&fq->mq_flush_lock, flags);
263263
}
264264

265+
bool is_flush_rq(struct request *rq)
266+
{
267+
return rq->end_io == flush_end_io;
268+
}
269+
265270
/**
266271
* blk_kick_flush - consider issuing flush request
267272
* @q: request_queue being kicked

block/blk-mq.c

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -911,7 +911,7 @@ static bool blk_mq_req_expired(struct request *rq, unsigned long *next)
911911

912912
void blk_mq_put_rq_ref(struct request *rq)
913913
{
914-
if (is_flush_rq(rq, rq->mq_hctx))
914+
if (is_flush_rq(rq))
915915
rq->end_io(rq, 0);
916916
else if (refcount_dec_and_test(&rq->ref))
917917
__blk_mq_free_request(rq);

block/blk.h

Lines changed: 1 addition & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -44,11 +44,7 @@ static inline void __blk_get_queue(struct request_queue *q)
4444
kobject_get(&q->kobj);
4545
}
4646

47-
static inline bool
48-
is_flush_rq(struct request *req, struct blk_mq_hw_ctx *hctx)
49-
{
50-
return hctx->fq->flush_rq == req;
51-
}
47+
bool is_flush_rq(struct request *req);
5248

5349
struct blk_flush_queue *blk_alloc_flush_queue(int node, int cmd_size,
5450
gfp_t flags);

0 commit comments

Comments
 (0)