Forum | Documentation | Website | Blog

Skip to content
Snippets Groups Projects
  1. Jun 24, 2024
    • Jens Axboe's avatar
      io_uring/msg_ring: add an alloc cache for io_kiocb entries · 50cf5f38
      Jens Axboe authored
      
      With slab accounting, allocating and freeing memory has considerable
      overhead. Add a basic alloc cache for the io_kiocb allocations that
      msg_ring needs to do. Unlike other caches, this one is used by the
      sender, grabbing it from the remote ring. When the remote ring gets
      the posted completion, it'll free it locally. Hence it is separately
      locked, using ctx->msg_lock.
      
      Signed-off-by: default avatarJens Axboe <axboe@kernel.dk>
      50cf5f38
    • Jens Axboe's avatar
      io_uring/msg_ring: improve handling of target CQE posting · 0617bb50
      Jens Axboe authored
      
      Use the exported helper for queueing task_work for message passing,
      rather than rolling our own.
      
      Note that this is only done for strict data messages for now, file
      descriptor passing messages still rely on the kernel task_work. It could
      get converted at some point if it's performance critical.
      
      This improves peak performance of message passing by about 5x in some
      basic testing, with 2 threads just sending messages to each other.
      Before this change, it was capped at around 700K/sec, with the change
      it's at over 4M/sec.
      
      Signed-off-by: default avatarJens Axboe <axboe@kernel.dk>
      0617bb50
    • Jens Axboe's avatar
      io_uring: add io_add_aux_cqe() helper · f33096a3
      Jens Axboe authored
      
      This helper will post a CQE, and can be called from task_work where we
      now that the ctx is already properly locked and that deferred
      completions will get flushed later on.
      
      Signed-off-by: default avatarJens Axboe <axboe@kernel.dk>
      f33096a3
    • Jens Axboe's avatar
      io_uring: add remote task_work execution helper · c3ac76f9
      Jens Axboe authored
      
      All our task_work handling is targeted at the state in the io_kiocb
      itself, which is what it is being used for. However, MSG_RING rolls its
      own task_work handling, ignoring how that is usually done.
      
      In preparation for switching MSG_RING to be able to use the normal
      task_work handling, add io_req_task_work_add_remote() which allows the
      caller to pass in the target io_ring_ctx.
      
      Signed-off-by: default avatarJens Axboe <axboe@kernel.dk>
      c3ac76f9
    • Jens Axboe's avatar
      io_uring/msg_ring: tighten requirement for remote posting · d57afd8b
      Jens Axboe authored
      
      Currently this is gated on whether or not the target ring needs a local
      completion - and if so, whether or not we're running on the right task.
      The use case for same thread cross posting is probably a lot less
      relevant than remote posting. And since we're going to improve this
      situation anyway, just gate it on local posting and ignore what task
      we're currently running on.
      
      Signed-off-by: default avatarJens Axboe <axboe@kernel.dk>
      d57afd8b
  2. Jun 19, 2024
  3. Jun 16, 2024
  4. Jun 15, 2024