Skip to content

Commit

Permalink
utube: fix slow take on busy utubes
Browse files Browse the repository at this point in the history
If some of the utube for tasks at the top of the queue were busy
most of the time, `take` would slow down for every other task.
This problem is fixed by creating a new space `space_ready`. It
contains first task with `READY` status from each utube.

This solution shows great results for the stated problem, with the cost
of slowing the `put` method (it is ~3 times slower). Thus, this workaround is
disabled by default. To enable it, user should set the
`storage_mode = "utube_ready_space"` as an option while creating the tube.
As example:
```lua
local test_queue = queue.create_tube('test_queue', 'utube',
        {temporary = true, storage_mode = "utube_ready_space"})
```

Part of #228
  • Loading branch information
DerekBum committed May 13, 2024
1 parent aa7c092 commit eb47c49
Show file tree
Hide file tree
Showing 7 changed files with 593 additions and 123 deletions.
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,17 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.

## [Unreleased]

### Added
- `storage_mode` option for creating a `utube` tube (#228). It enables the
workaround for slow takes while working with busy tubes.

### Fixed

- Stuck in `INIT` state if an instance failed to enter the `running` mode
in time (#226). This fix works only for Tarantool versions >= 2.10.0.
- Slow takes on busy `utube` tubes (#228). The workaround could be enabled by
passing the `storage_mode = "utube_ready_space"` option while creating
the tube.

## [1.3.3] - 2023-09-13

Expand Down
5 changes: 5 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -172,6 +172,11 @@ on disk
already exists
* `on_task_change` - function name - a callback to be executed on every
operation
* `storage_mode` - string - one of
* `"default"` - default implementation of `utube`
* `"utube_ready_space"` - allows processing `take` requests faster, but
by the cost of `put` operations speed. Right now this option is enabled
only for `memtx` engine

The following options can be specified when putting a task in a `utube`
queue:
Expand Down
6 changes: 5 additions & 1 deletion queue/abstract.lua
Original file line number Diff line number Diff line change
Expand Up @@ -297,7 +297,11 @@ function tube.drop(self)

local space_name = tube[3]

box.space[space_name]:drop()
if self.raw.drop ~= nil then
self.raw:drop()
else
box.space[space_name]:drop()
end
box.space._queue:delete{tube_name}
-- drop queue
queue.tube[tube_name] = nil
Expand Down
Loading

0 comments on commit eb47c49

Please sign in to comment.