30 lines
1.1 KiB
Markdown
30 lines
1.1 KiB
Markdown
|
# gtq - the go task queue
|
||
|
|
||
|
## what?
|
||
|
a simple goroutine scheduler for longer-running goroutines, allowing you
|
||
|
to put them into priority buckets, and get roughly fair queuing of which
|
||
|
tasks will be processed next.
|
||
|
|
||
|
## why?
|
||
|
goroutines are awesome, and typically super fast. but what if you need to
|
||
|
queue up a lot of longer running work in the background? and what if you
|
||
|
want some control over which chunks of work will get executed next?
|
||
|
|
||
|
gtq provides simple task queuing and scheduling for longer running goroutines.
|
||
|
in this case, "long" means on the order of 1ms or longer, as many simple
|
||
|
goroutines can run in a matter of ns.
|
||
|
|
||
|
## how?
|
||
|
We use simple heap based queues for each priority level, and track statistics
|
||
|
in a rolling window for task execution times in each of those. Then we
|
||
|
have a scheduler which looks at those statistics, in relation to the priority
|
||
|
level for the queue, and decides which queue to pull from next and feed to task
|
||
|
runners.
|
||
|
|
||
|
Overhead for this process is typically < 1000ns, so it's not suitable for
|
||
|
typical short lived goroutines, but those type of goroutines don't really,
|
||
|
need scheduling anyway.
|
||
|
|
||
|
## usage
|
||
|
TODO
|