Linux Today: Linux News On Internet Time.

Parallel Processing Shell Script Released

Jan 06, 2009, 01:34 (1 Talkback[s])

[ Thanks to louwrentius for this link. ]

"There are many tutorials about how to execute commands in parallel, often using shell scripts. However, they are often not atomic, so parallel jobs may start processing the same file, causing trouble. Also, most examples on the web are based on some polling mechanism in order to determine when to start the next job. Polling is inefficient: you waste time between de moment a script finishes and the moment that event is picked up. Therefore PPSS is event-driven. It does absolutely nothing, thus saving CPU cylcles, while being as time-efficient as possible. The moment a job is finished, it will start a new one.

"The script will take a list of items as input. The script executes a user-specified command for each item in this list. The item is supplied as an argument to this command. Items can be files within a directory or entries in a text file (one per line)."

Complete Story

Related Stories: