The credit of this post goes to Guillaume Cerutti from Virtual Plants project-team.

A modular platform dealing with a given scientific domain is made of some key ingredients (see dtk-introduction post for more details):

• a set of abstract classes defining the interfaces for the data, the algorithms and the views dedicated to the scientific field
• a collection of plugins implementing these interfaces using home-made code or third-party libraries

Moreover, users of such a platform wish to prototype workflows in a very flexible manner, hence, the visual programming framework turns out to be very useful.

dtk provides tools to design such a platform with all of these ingredients. Let us see how these is done in practice.

Defining a dtk Concept for an Algorithm : Watershed example

Our goals are firstly to add a concept watershed to the dtkImaging thematic layer, then to define a plugin that will be visible for any dtk-based application, and eventually to use the third-party VT library to define an implementation of this concept.

Create a new abstraction in the dtk-imaging layer

Define the abstraction for the imaging filter

Let us create a header file that contains the code defining the interface of an abstract watershed algorithm: dtkAbstractWatershedFilter.h. This abstract class defines the input and output methods of the filter. It inherits from QRunnable so that it is directly compatible with the distributed framework of dtk which provides both multithreading and parallelism tools.

#pragma once

#include <QtCore>

class dtkImage;

class dtkAbstractWatershedFilter : public QRunnable
{

public:
    virtual void setImage(dtkImage *image) = 0;
    virtual void setSeed(dtkImage *image) = 0;

public:
    virtual dtkImage *filteredImage(void) const = 0;

public:
    virtual void run(void) = 0;

};

The class methods (setters for the input and getters for the outputs) are made pure virtual: the classes implementing this abstraction have to to implement these functions. For sake of clarity, we recall the pure virtual run method of QRunnable class to make sure it will be implemented.

One can notice the forward declaration of dtkImage class. As a pointer to this class is only used (no method of the class is required in the header), this tells the compiler that it just needs to reserve the size of a pointer in memory.

Add export header and macro for symbols’ visibility

In order to make the symbols of the class visible outside the library (especially in the plugin libraries), one has to include the export header generated automatically by CMake and add the export macro to the definition of the class.

#pragma once

#include <QtCore>

#include <dtkImagingExport.h>

class DTKIMAGING_EXPORT dtkAbstractWatershedFilter : public QRunnable

This step is mandatory for windows.

Add plugin system

The dtk plugin system is made of:

• a plugin class defining an interface that is implemented into concrete plugins and which are containers for the classes implementing the abstraction
• a plugin manager in charge of loading the concrete plugins
• a plugin factory in charge of instanciating concrete plugins using literal identifiers (usually the name of the plugin)

dtkCore provides macros that provides these three classes for any abstraction of the layer. Furthermore, as we intend to use the watershed algorithm through visual programming, there is an additional macro which enables to register the class to the QMetaType system and to use it through QVariant. In practice, these four macros are added at the bottom of the header.

};

//

DTK_DECLARE_OBJECT(dtkAbstractWatershedFilter *)
DTK_DECLARE_PLUGIN(dtkAbstractWatershedFilter, DTKIMAGING_EXPORT)
DTK_DECLARE_PLUGIN_FACTORY(dtkAbstractWatershedFilter, DTKIMAGING_EXPORT)
DTK_DECLARE_PLUGIN_MANAGER(dtkAbstractWatershedFilter, DTKIMAGING_EXPORT)

Register the abstraction to the layer

In practice when using dtkImaging layer, one wants to be able to load all the plugins of all the abstractions in one single instruction. To do so, one has to add the DTK_DECLARE_CONCEPT macro in which, one has to give the name of the abstraction, the name of the export macro, the namespace that will enable to class the plugin factory of the abstraction as follows dtkImaging::watershed::pluginFactory()->create("dummyWatershedFilter");. Here is the macro:

namespace dtkImaging {
    DTK_DECLARE_CONCEPT(dtkAbstractWatershedFilter, DTKIMAGING_EXPORT, watershed);
}

This macro has its counterpart DTK_DEFINE_CONCEPT which must be inserted into a cpp file. So, the file dtkAbstractWatershedFilter.cpp is created and contains the following code that ensure the registration to the manager of the layer:

#include "dtkImaging.h"
#include "dtkAbstractWatershedFilter.h"

namespace dtkImaging {
    DTK_DEFINE_CONCEPT(dtkAbstractWatershedFilter, watershed, dtkImaging);
}

Eventually, for aesthetic considerations, a file without extension dtkAbstractWatershedFilter that contains #include "dtkAbstractWatershedFilter.h" is added so that client code can include the abstraction as follows:

#include <dtkAbstractWatershedFilter>

Edit the compilation files to include the new abstraction

Once the files defining the abstraction have been written, one has to edit the CMakeLists.txt file of the layer and add the header file to the ${PROJECT_NAME}_HEADERS section, and the source file to the ${PROJECT_NAME}_SOURCES section:

set(${PROJECT_NAME}_HEADERS
  dtkAbstractAddFilter.h
  ...
  dtkAbstractWatershedFilter
  dtkAbstractWatershedFilter.h
  ...)

set(${PROJECT_NAME}_SOURCES
  dtkAbstractAddFilter.cpp
  ...
  dtkAbstractWatershedFilter.cpp
  ...)

One can then compile to include the changes. There is no need to re-run cmake configuration as the changes in the CMakeLists.txt will be detected.

cd build
make -j4

Define the plugin implementation

The goal is to implement the abstraction defined in the previous section and the plugin defined by the macros. The plugin will use the dtkVtImageConverter to make the bridge between the image data structure used in the VT library (native) and the dtkImage class.

Prepare the file arborescence and compilation files

• Create the file arborescence in the dtk-plugins-imaging project, under src/VTPlugins

cd src/VTPlugins
mkdir dtkVtImageWatershedFilter
cd dtkVtImageWatershedFilter

• The directory will contain 2 source files (+ headers) containing respectively the implementation of the abstraction and the definition of the plugin

dtkVtWatershedFilter.h
dtkVtWatershedFilter.cpp
dtkVtWatershedFilterPlugin.h
dtkVtWatershedFilterPlugin.cpp
dtkVtWatershedFilterPlugin.json

• Create the CMakeLists.txt file (you can copy it from another plugin if you’re lazy…) that specifies the dtk layers and other libraries it has to know about.

project(dtkVtWatershedFilterPlugin)

## ###################################################################
## Build rules
## ###################################################################

add_definitions(-DQT_PLUGIN)

add_library(${PROJECT_NAME} SHARED
  dtkVtWatershedFilter.h
  dtkVtWatershedFilter.cpp
  dtkVtWatershedFilterPlugin.h
  dtkVtWatershedFilterPlugin.cpp)

## ###################################################################
## Link rules
## ###################################################################

target_link_libraries(${PROJECT_NAME} Qt5::Core)

target_link_libraries(${PROJECT_NAME} dtkCore)
target_link_libraries(${PROJECT_NAME} dtkLog)
target_link_libraries(${PROJECT_NAME} dtkImaging)

target_link_libraries(${PROJECT_NAME} dtkVtImageConverter)
target_link_libraries(${PROJECT_NAME} vt)
target_link_libraries(${PROJECT_NAME} exec)
target_link_libraries(${PROJECT_NAME} io)
target_link_libraries(${PROJECT_NAME} basic)

## #################################################################
## Install rules
## #################################################################

install(TARGETS ${PROJECT_NAME}
  RUNTIME DESTINATION plugins/${DTK_CURRENT_LAYER}
  LIBRARY DESTINATION plugins/${DTK_CURRENT_LAYER}
  ARCHIVE DESTINATION plugins/${DTK_CURRENT_LAYER})

######################################################################
### CMakeLists.txt ends here

Define the implementation of the abstraction using the VT library

• Create the implementation header file dtkVtWatershedFilter.h with creator and destructor, and a D-Pointer to store the members of the class (to avoid listing the private data in the headers and hide the details of the implementation of the class), and a forward declaration of dtkImage.

#pragma once

#include <dtkAbstractWatershedFilter>

class dtkImage;

class dtkVtWatershedFilter final : public dtkAbstractWatershedFilter
{
public:
     dtkVtWatershedFilter(void);
    ~dtkVtWatershedFilter(void);

public:
    void setImage(dtkImage *image);
    void setSeed(dtkImage *image);

public:
    dtkImage *filteredImage(void) const;

public:
    void run(void);

private:
    class dtkVtWatershedFilterPrivate *d;
};

• Add a creation function of the plugin implementation to be able to link it from outside the library

inline dtkAbstractWatershedFilter *dtkVtWatershedFilterCreator(void)
{
    return new dtkVtWatershedFilter();
}

//
// dtkVtWatershedFilter.h ends here

• Create the implementation source file dtkVtWatershedFilter.cpp with the corresponding dtk-plugins-imaging imports and the right VT library imports…

#include "dtkVtWatershedFilter.h"
#include "dtkVtImageConverter.h"

#include <dtkFilterExecutor.h>

#include <vt_image.h>
#include <api-watershed.h>

• Create the private class for the D-Pointer, defining the private data used by the filter in its members, and implement a creator and a destructor. The private class also has a run method that will be called to perform the wrapped algorithm.

class dtkVtWatershedFilterPrivate
{
public:
    dtkImage *image_in;
    dtkImage *seed;
    dtkImage *image_out;

public:
    template < typename ImgT, int dim > void exec(void);
};

dtkVtWatershedFilter::dtkVtWathershedFilter(void): dtkAbstractWatershedFilter(), d(new dtkVtWatershedFilterPrivate)
{
    d->image_in = nullptr;
    d->seed = nullptr;
    d->image_out = new dtkImage();
}

dtkVtWatershedFilter::~dtkVtWatershedFilter(void)
{
    delete d;
}

• Implement the setters and getters of the public class

void dtkVtWatershedFilter::setImage(dtkImage *image)
{
    d->image_in = image;
}

void dtkVtWatershedFilter::setImage(dtkImage *seed)
{
    d->seed = seed;
}

dtkImage *dtkVtWatershedFilter::filteredImage(void) const
{
    return d->image_out;
}

• Finally, define the run function implementing the actual functionality of the filter. Raise a warning if one of the inputs is missing, and let the Executor call the exec function of the private class that will in turn call the functions from the VT library by its exec function.

void dtkVtWatershedFilter::run(void)
{
    if (!d->image_in || !d->seed) {
        dtkWarn() << Q_FUNC_INFO << "no image input";
        return;
    }

    dtkFilterExecutor<dtkVtWatershedFilterPrivate>::run(d, d->image_in);
}

• The only thing left to do is to implement the exec function of the Executor using the VT library functions that actually perform the job.

template < typename ImgT, int dim> inline void dtkVtWatershedFilterPrivate::exec(void)
{
    vt_image image;
    vt_image seed;
    char str[100];
     sprintf( str, "-labelchoice %s","first");

    if (dtkVtImageConverter::convertToNative(this->image_in, &image) != EXIT_FAILURE) {
      if (dtkVtImageConverter::convertToNative(this->seed, &seed) != EXIT_FAILURE) {
        if ( API_watershed( &image, &seed, nullptr, str, (char*)nullptr ) != 1 ) {
            free( image.array );
              image.array = nullptr;
            free( seed.array );
            seed.array = nullptr;
            dtkError() << Q_FUNC_INFO << "Computation failed. Aborting.";
            return;
        }

        dtkVtImageConverter::convertFromNative(&image, this->image_out);

        free( seed.array );
        seed.array = nullptr;
      }
      free( image.array );
      image.array = nullptr;

    } else {
        dtkError() << Q_FUNC_INFO << "Conversion to VT format failed. Aborting.";
    }
}

Define the plugin container

The plugins will be discovered by the manager that will initialize them using their initialize function that has to be implemented. The plugin does not have to be included in the compilation (nor linked) to be loaded dynamically at runtime.

• Create the plugin header file dtkVtWatershedFilterPlugin.h to define the specific plugin class that inherits from the abstract plugin class created by the macros in the abstraction declaration.

#pragma once

#include <dtkCore>
#include <dtkAbstractWatershedFilter>

class dtkVtWatershedFilterPlugin: public dtkAbstractWatershedFilterPlugin
{
    Q_OBJECT
    Q_INTERFACES(dtkAbstractWatershedFilterPlugin)
    Q_PLUGIN_METADATA(IID "fr.inria.dtkVtWatershedFilterPlugin" FILE "dtkVtWatershedFilterPlugin.json")

public:
     dtkVtWatershedFilterPlugin(void) {}
    ~dtkVtWatershedFilterPlugin(void) {}

public:
    void initialize(void);
    void uninitialize(void);
};

• Add the plugin metadata-file dtkVtWatershedFilterPlugin.json containing information on the plugin requirements (library dependencies, data formats, cross-plugin dependencies,…)

{
            "name" : "dtkVtWatershedFilterPlugin",
         "concept" : "dtkAbstractWatershedFilter",
         "version" : "0.0.1",
    "dependencies" : []
}

• Create the plugin source file dtkVtWatershedFilterPlugin.cpp, including the header of the implementation to be able to access the creator function.

#include "dtkVtWatershedFilter.h"
#include "dtkVtWatershedFilterPlugin.h"

#include <dtkCore>
#include <dtkImaging.h>

• Implement the initialize function to register the creator function to the plugin factory using the name of the implementation.

void dtkVtWatershedFilterPlugin::initialize(void)
{
    dtkImaging::filters::watershed::pluginFactory().record("dtkVtWatershedFilter", dtkVtWatershedFilterCreator);
}

void dtkVtWatershedFilterPlugin::uninitialize(void)
{

}

This allows to create a new instance of the concept by the command :

dtkImaging::filters::watershed::pluginFactory().create("dtkVtWatershedFilter")

• Add the macro to define the plugin, that wraps the Qt macro Q_PLUGIN_METADATA to allow the plugin to be seen by the Qt plugin system.

DTK_DEFINE_PLUGIN(dtkVtWatershedFilter)

Add the plugin to the system

• Open the CMakeLists.txt file of the parent directory (here VTPlugins) to add the new plugin directory to the Inputs section.

## #################################################################
## Inputs
## #################################################################

add_subdirectory(dtkVtImageConverter)
...
add_subdirectory(dtkVtImageWatershedFilter)
...

• Complile the plugins projects, using simply the make command.

Declare a node wrapper for the concept

The idea is to wrap our concept to make it visible in the composer. Note that the it is the abstraction that will be wrapped, not the implementation itself (so that a specific implementation) can be used at runtime. Therefore the node files will be created in the dtk-imaging project, under src/composer.

cd src/composer

Create node header and source files

• Create the header file dtkWatershedFilterNode.h that contains the node class, inheriting an object node class dtkComposerNodeObject templated by the abstraction. Once again the private data is stored as a D-Pointer (pointing on a private class defined in the source file).

#pragma once

#include <dtkImagingExport.h>
#include <dtkComposer>

class dtkAbstractWatershedFilter;
class dtkWatershedFilterNodePrivate;

class DTKIMAGING_EXPORT dtkWatershedFilterNode : public dtkComposerNodeObject<dtkAbstractWatershedFilter>
{
public:
     dtkWatershedFilterNode(void);
    ~dtkWatershedFilterNode(void);

public:
    void run(void);

private:
    dtkWatershedFilterNodePrivate *d;
};

• Create the source file dtkWatershedFilterNode.cpp including the abstraction and the necessary data structures for the members of the private class.

#include "dtkWatershedFilterNode.h"

#include "dtkAbstractWatershedFilter.h"
#include "dtkImage.h"
#include <QtCore>
#include "dtkImaging.h"

#include <dtkLog>

• The first part is the definition of the private class defining the members of the node. The private class actually defines a mapping beteen inputs/outputs of the concept abstraction and receivers/emitters for dtkComposer.

class dtkWatershedFilterNodePrivate
{
public:
    dtkComposerTransmitterReceiver<dtkImage *> image_in;
    dtkComposerTransmitterReceiver<dtkImage *> seed;
    dtkComposerTransmitterReceiver<QString> control;

    dtkComposerTransmitterEmitter<dtkImage *>  image_out;
};

• Then the creator of the public node class consists in appending the receivers and emitters corresponding to the node, and to define the plugin factory that will provide the implementation of the wrapped abstraction.

dtkWatershedFilterNode::dtkWatershedFilterNode(void) : dtkComposerNodeObject<dtkAbstractWatershedFilter>(), d(new dtkWatershedFilterNodePrivate())
{
    this->setFactory(dtkImaging::filters::watershed::pluginFactory());

    this->appendReceiver(&d->image_in);
    this->appendReceiver(&d->seed);
    this->appendReceiver(&d->control);

    this->appendEmitter (&d->image_out);
}

dtkWatershedFilterNode::~dtkWatershedFilterNode(void)
{
    delete d;
}

• Finally, complete the run method of the node that consists in passing the inputs coming from the composer to the implementation coming from the factory and enabling the following nodes to access the outputs.

void dtkWatershedFilterNode::run(void)
{

• Connecters (receivers and emitters) come with a way of testing if the are correctly connected to another node in the composer, and in the case that the mandatory connections are missing the run can not be called.

    if ((d->image_in.isEmpty() || d->seed.isEmpty()) || d->control.isEmpty()) {
        dtkError() << Q_FUNC_INFO << "The input is not set. Aborting.";
        return;

• Otherwise, a plugin can be instantiated, using the plugin factory and the plugin name that has been selected in the GUI.

    } else {

        dtkAbstractWatershedFilter *filter = this->object();
        if (!filter) {
            dtkError() << Q_FUNC_INFO << "No Watershed filter found. Aborting.";
            return;
        }

• The inputs can be set using the data methods of the receivers, and the outputs of the filter are passed to the emitters through the setData. This is the only thing to implement, the specific mapping between inputs/outputs and receivers/emitters.

        filter->setImage(d->image_in.data());
        filter->setSeed(d->seed.data());
        filter->setControlParameter(d->control.data());

        filter->run();

        d->image_out.setData(filter->filteredImage());
    }
}

//
// dtkWatershedFilterNode.cpp ends here

Add the node decoration file

• Create the dtkWatershedFilterNode.json file that defines the appearance of the node in the composer. For instance it provides a kind defining the color of the node, tags for querying nodes, a textual description that will appear in the composer, and names for the inputs and outputs, in the order that their respective receivers and emitters were appended in the node.

{
          "title" : "Watershed Filter",
           "kind" : "process",
           "type" : "dtkWatershedFilter",
           "tags" : ["filter","watershed","imaging"],
    "description" : "<h3>Filter dtkWatershedFilter</h3>
                     <p>Apply the watershed filter provided by any library (e.g. VT, ITK)</p>",
         "inputs" : ["image","seed","control"],
         "outputs": ["image"]
}

• Add the decoration filename to the dtkComposer.qrc file that will store path to the files directly in the .so, making it more portable.

<RCC>
    <qresource prefix="/dtkComposer">
        <file>dtkAddFilterNode.json</file>
         ...
         <file>dtkWatershedFilterNode.json</file>
         ...
    </qresource>
</RCC>

Add the node to the system

• Once again, open the CMakeLists.txt file to add the two files we just created in the appropriate section.

...
set(${PROJECT_NAME}_COMPOSER_HEADERS_TMP
  dtkAddFilterNode.h
  ...
  dtkWatershedFilterNode.h
  ...

set(${PROJECT_NAME}_COMPOSER_SOURCES_TMP
  dtkAddFilterNode.cpp
  ...
  dtkWatershedFilterNode.cpp
  ...

• For the nodes to appear in the composer, make sure that the lib directory of the layer is added to the ~/.config/inria/dtk-composer.ini of your system. Otherwise you have to do it by hand…

[extension]
plugins=.../dtk-imaging/build/lib