Lockless Queue Second Draft
Last week I wrote a first draft of a lockless queue for my event system. This week, I wrote a second draft, and I'm pretty happy with it. I'll likely write one more version but it'd be similar to this one. Since Bold will have a lot of events and asynchronous code it's important that I get this right. Some features are
- It uses cost to pick which queue to place items in, rather than the least full one, as before.
- It is easy to use. You create the object you pass in the worker count and 2 callbacks, one to process the event and another to estimate the cost.
- Allows blocking and non-blocking calls. If the thread wants to do other things it can pass in the non-blocking flag, if it's a test it may block so events are reproducible. It may also choose order if that's important to the test.
- The producer can query how many free slots there are. This allows a producer to never block or grow the queue. If I know a function will always produce 2 or 10 events I can check before I enter the function so I don't have to worry about the edge case of when it gets full.
I didn't implement work stealing. I want to implement real code so I can figure out if the best strategy is to steal 1 message (from the end,) or a group of messages, or if the message type affects the optimal way of stealing work.