dtkComposer layer
Since the beginning of the dtkComposer layer, the Map and Foreach nodes allow the user to apply the same sub-composition to an input container (Qt container, std::vector, dtkArray, etc .). We can for example read a bunch of files from a given directory, put the result in a QStringList and apply a treatment (sub-composition) to each of the files.
Using the same technique we use for the remote execution of composition (serialization of the composition as XML, and use of a dtkComposerReader at the remote end), we are now able to add a parallel Map node.
map node
The idea is to start a qthread based dtkDistributedCommunicator in the parallel Map node, and send to each threads the XML serialization of it’s internal composition (the visual program inside the Map node). Each thread will run the composition on a given range of the input container, and will write the result in the output container. In the next toy example, we apply to a list of images a very basic set of filters (Gaussian and Normalize filters):
parallel map node implementation
One of the limitation of the initial design of the composer was that even though the visual aspect (called the dtkComposerScene) was separated from the evaluation aspect (dtkComposerEvaluator), the Scene was used to build the evaluation graph used by the Evaluator from the XML (and the scene was also needed to write the XML). This was a real problem for the multithreaded implementation of the Map node, since each thread had no scene at all. We have now refactored completely the dtkComposerReader and dtkComposerWriter (used for the XML Serialization/Deserialization), and we are now able to use a dtkComposerEvaluator without any scene.
In the next example, we now use the Parallel Map node instead of the Map. We can notice that it’s exactly the same composition, the user only needs to switch from the Map node to the ParallelMap one in order to use all the available cores of it’s computer.
