Friday, October 26, 2012

Python - Multithread to read one file

Today, i am working with python. I need write script to read one file, and get line by line, per line deliver one thread process ( total 10 threads). I want solution, so i chose working with thread and queue.
In python, when procsess initializate, this process will be assigned with queue, and working with this queue. We will put data ( in this case is line) to queue. Process will read from queue, so, all processes can read one file, not overlap :D

import threading
import Queue

#Number of threads
n_thread = 5
#Create queue
queue = Queue.Queue()

class ThreadClass(threading.Thread):
    def __init__(self, queue):
        threading.Thread.__init__(self)
    #Assign thread working with queue
        self.queue = queue

    def run(self):
        while True:
        #Get from queue job
            host = self.queue.get()
            print self.getName() + ":" + host
        #signals to queue job is done
            self.queue.task_done()

#Create number process
for i in range(n_thread):
    t = ThreadClass(queue)
    t.setDaemon(True)
    #Start thread
    t.start()

#Read file line by line
hostfile = open("hosts.txt","r")
for line in hostfile:
    #Put line to queue
    queue.put(line)
#wait on the queue until everything has been processed
queue.join()

------------------------------------------------------------
Thanks for reading
--------------------------------------------------------------------------
Security Research
All my Lab:
Linux Lab -- window and Cisco Lab
to be continued - I will update more.  

4 comments:

Anonymous said...

thanks!!!!

Peder Zickler said...

hello kendy, thank you for saving my ass. I have a file consisting of 18k ip address, need to read line by line and telnet and configure.

Anonymous said...

Beautiful! Thanks!

Shahid Mahmood said...

This is beautiful!! Thankyou Kendy...

Install Xposed Inspector and Frida on Genymotion

Today i had some work with android. So i need trace application. I found 2 nice tool can help me: Xposed Inspector and Frida. To setup ther...