September 2012 Archives

pingboard

| 2 Comments | No TrackBacks
At my new home I've got a peg board, which is great. What I think is also great is the utility I've written called pingboard:

https://bitbucket.org/go8ose/pingboard

It's a utility written in python that pings servers. Unlike normal ping, it pings multiple servers at once, and tells you which ones are alive at the moment, and how long they've been up. Please give it a go, and tell me what you think about it.

python threading and accessing shared data

| No Comments | No TrackBacks
I've been using threads recently in python, and I've been reading how the GIL (Global Interpreter Lock) means that C-python can only run one thread at a time, and that only one thread can access a python object at a time.

For awhile this was confusing me. Did that mean that I didn't have to worry about thread safety? To clear that up for myself, I wrote a short test:
import unittest
import threading

iterations=1000
num_threads=10

plus_equal_counter=0
class Plus_Equal(threading.Thread):
    def run(self):
        global plus_equal_counter
        for x in xrange(iterations):
            plus_equal_counter += 1

class TestThread(unittest.TestCase):
    def setUp(self):
        threads = []
        for n in xrange(num_threads):
            t = Plus_Equal()
            threads.append(t)
            t.start()
        for t in threads:
            t.join()

    def test_count_ok(self):
        self.assertEqual(plus_equal_counter, iterations*num_threads)

if __name__ == '__main__':
    unittest.main()
If you run this, there is an excellent chance that the test will fail. As plus_equal_counter is a global python object, all the threads are accessing the same object when they are incrementing it. And while the GIL protects python from messing up it's internals, the '+=' operator is not atomic. 

What do I mean by not atomic? I mean that a python thread that is doing a += line of code can get interrupted by another thread halfway through. If a second thread starts a += operation halfway through another threads effort, then they'll both end up setting plus_equal_counter to the same value, so we miss one of the increments we were expecting.

So to fix this simple example, I'd need to modify the threads to use Locks so only one thread access the variable at a time:
plus_equal_counter=0
plus_equal_counter_lock=threading.Lock()
class Plus_Equal(threading.Thread):
    def run(self):
        global plus_equal_counter
        for x in xrange(iterations):
            plus_equal_counter_lock.acquire()
            plus_equal_counter += 1
            plus_equal_counter_lock.release