Pipeline processing

When a message comes to several processing steps. Let's say there is an event triggering the process which is sending a message to several process. The processes are: authentication, decryption and updating db.

In order to maintain a loosely integration, it will be difficult. The best way is to make each process independent. There is a method called pipelines processing. Yeah, you can imagine the method is having pipe connected each other. When there is a message comes to the pipe. The process will taking the message from it and process it. Once the first process finishes it process, then it just straightly put the message to another pipe to be processed. While the first process finishes its job, it can take another message without waiting for the message to be finished until the last process. Using this method, the throughput will be increased.

Note that each process are running independently. It may be better to have a look at the following image.




The above picture is taken from the book Addison wesley- Martin Fowler - Enterprise Integration pattern.

Looking on the above image, we can see a comparison between pipeline processing and sequential processing. The throughput of pipeline is more effective compare to sequential. The time needed to process 3 messages in pipeline is almost a half of sequential.