gtq - a Golang Task Queue
Go to file
2019-04-24 14:29:26 -07:00
go.mod initial working release 2019-04-11 16:21:27 +02:00
go.sum initial working release 2019-04-11 16:21:27 +02:00
queue_test.go initial working release 2019-04-11 16:21:27 +02:00
queue.go initial working release 2019-04-11 16:21:27 +02:00
README.md initial working release 2019-04-11 16:21:27 +02:00
scheduler.go Added a smarter base scheduler that actually does a bit of schedule time calculation. 2019-04-24 14:29:26 -07:00
stats_test.go initial working release 2019-04-11 16:21:27 +02:00
stats.go initial working release 2019-04-11 16:21:27 +02:00
taskqueue_test.go Added a smarter base scheduler that actually does a bit of schedule time calculation. 2019-04-24 14:29:26 -07:00
taskqueue.go Added a smarter base scheduler that actually does a bit of schedule time calculation. 2019-04-24 14:29:26 -07:00
uintslice.go initial working release 2019-04-11 16:21:27 +02:00

gtq - the go task queue

what?

a simple goroutine scheduler for longer-running goroutines, allowing you to put them into priority buckets, and get roughly fair queuing of which tasks will be processed next.

why?

goroutines are awesome, and typically super fast. but what if you need to queue up a lot of longer running work in the background? and what if you want some control over which chunks of work will get executed next?

gtq provides simple task queuing and scheduling for longer running goroutines. in this case, "long" means on the order of 1ms or longer, as many simple goroutines can run in a matter of ns.

how?

We use simple heap based queues for each priority level, and track statistics in a rolling window for task execution times in each of those. Then we have a scheduler which looks at those statistics, in relation to the priority level for the queue, and decides which queue to pull from next and feed to task runners.

Overhead for this process is typically < 1000ns, so it's not suitable for typical short lived goroutines, but those type of goroutines don't really, need scheduling anyway.

usage

TODO