python threading and accessing shared data

| No Comments | No TrackBacks
I've been using threads recently in python, and I've been reading how the GIL (Global Interpreter Lock) means that C-python can only run one thread at a time, and that only one thread can access a python object at a time.

For awhile this was confusing me. Did that mean that I didn't have to worry about thread safety? To clear that up for myself, I wrote a short test:
import unittest
import threading

iterations=1000
num_threads=10

plus_equal_counter=0
class Plus_Equal(threading.Thread):
    def run(self):
        global plus_equal_counter
        for x in xrange(iterations):
            plus_equal_counter += 1

class TestThread(unittest.TestCase):
    def setUp(self):
        threads = []
        for n in xrange(num_threads):
            t = Plus_Equal()
            threads.append(t)
            t.start()
        for t in threads:
            t.join()

    def test_count_ok(self):
        self.assertEqual(plus_equal_counter, iterations*num_threads)

if __name__ == '__main__':
    unittest.main()
If you run this, there is an excellent chance that the test will fail. As plus_equal_counter is a global python object, all the threads are accessing the same object when they are incrementing it. And while the GIL protects python from messing up it's internals, the '+=' operator is not atomic. 

What do I mean by not atomic? I mean that a python thread that is doing a += line of code can get interrupted by another thread halfway through. If a second thread starts a += operation halfway through another threads effort, then they'll both end up setting plus_equal_counter to the same value, so we miss one of the increments we were expecting.

So to fix this simple example, I'd need to modify the threads to use Locks so only one thread access the variable at a time:
plus_equal_counter=0
plus_equal_counter_lock=threading.Lock()
class Plus_Equal(threading.Thread):
    def run(self):
        global plus_equal_counter
        for x in xrange(iterations):
            plus_equal_counter_lock.acquire()
            plus_equal_counter += 1
            plus_equal_counter_lock.release

No TrackBacks

TrackBack URL: http://geoff-blog.cromp.id.au/cgi-bin/movabletype/mt-tb.cgi/196

Leave a comment