gtq - a Golang Task Queue
Go to file
Tessa Nordgren 89ebae6daf
All checks were successful
continuous-integration/drone/push Build is passing
fixed scheduler build error and added drone config
2021-03-12 18:46:17 -08:00
.drone.yml fixed scheduler build error and added drone config 2021-03-12 18:46:17 -08:00
go.mod initial working release 2019-04-11 16:21:27 +02:00
go.sum initial working release 2019-04-11 16:21:27 +02:00
GPL.txt Added licensing info. 2019-04-24 14:38:08 -07:00
LICENSE Added licensing info. 2019-04-24 14:38:08 -07:00
queue_test.go initial working release 2019-04-11 16:21:27 +02:00
queue.go initial working release 2019-04-11 16:21:27 +02:00
README.md typo 2019-04-24 14:42:52 -07:00
scheduler.go fixed scheduler build error and added drone config 2021-03-12 18:46:17 -08:00
stats_test.go initial working release 2019-04-11 16:21:27 +02:00
stats.go initial working release 2019-04-11 16:21:27 +02:00
taskqueue_test.go Added a smarter base scheduler that actually does a bit of schedule time calculation. 2019-04-24 14:29:26 -07:00
taskqueue.go Added a smarter base scheduler that actually does a bit of schedule time calculation. 2019-04-24 14:29:26 -07:00
uintslice.go initial working release 2019-04-11 16:21:27 +02:00

gtq - the go task queue

what?

a simple goroutine scheduler for longer-running goroutines, allowing you to put them into priority buckets, and get roughly fair queuing of which tasks will be processed next.

why?

goroutines are awesome, and typically super fast. but what if you need to queue up a lot of longer running work in the background? and what if you want some control over which chunks of work will get executed next?

gtq provides simple task queuing and scheduling for longer running goroutines. in this case, "long" means on the order of 1ms or longer, as many simple goroutines can run in a matter of ns.

how?

We use simple heap based queues for each priority level, and track statistics in a rolling window for task execution times in each of those. Then we have a scheduler which looks at those statistics, in relation to the priority level for the queue, and decides which queue to pull from next and feed to task runners.

Overhead for this process is typically < 1000ns, so it's not suitable for typical short lived goroutines, but those type of goroutines don't really need scheduling anyway.

usage

TODO

legalese

© 2019 - Tessa Nordgren

Licensed under the GNU Lesser General Public License, version 3.0. You should have received a copy of this license with this source code. If not, you can find it at at the LGPL 3.0 homepage.