CLARIN tools for workflows

1
CLARIN tools for workflows

• Overview

2
Objective of this document

• Determine which are the responsibilities of the
  different components of CLARIN workflows.
• Explain what is expected from the CLARIN Workflow
  tools. Requirements.

3
Firstly, what is a workflow?

• Workflow is a definition of a process in a
  certain language. It is done in a non-ambiguous
  way.
• A workflow is only the definition of a process.
• Since the workflow definition is made in a
  non-ambiguous way, any software can execute it
  expecting the same results. It means that
  workflows can be shared between different
  applications.
• Workflow may be a XML file that a user can write
  without any other help.

4
Workflow tools

• Since a workflow is only a process description,
  we need 2 other components.
• Workflow Engine Application that process
  (executes) workflow descriptions.
• Workflow Editors Application for creation of
  workflow descriptions.
• Both applications can be in the same piece of
  software. Workflow Workbench.

5
Workflow Engines Expectations

• Engines are applications that execute workflows
  in the way they are described.
• Many engines can be developed. Some with GUI for
  desktop computers some other hosted in servers
  with schedule features, etc...

6
Workflow Editor Expectations

• For advanced users of CLARIN, a text editor could
  be enough for a workflow creation.
• But CLARIN objective is easiness. In this way to
  create a workflow in CLARIN must be really easy.
• The editors in CLARIN must help as much as
  possible to the user. This is done, for instance,
  querying the registry and taking decisions behind
  the scenes.
• Example User connects two web services using
  firsts output with seconds input. Metadata
  stored in the registry specifies that this
  operation is not possible without a conversion.
  Also the registry contains info about converters
  available. The editor must be aware of that and
  introduce a conversion tool between the web
  services. This can be done without users
  interaction at all.
• SMART editors are possible since the Registry
  will contain all the required metadata for taking
  decisions automatically.

7
Example

• User wants to connect 2 web services.
• Web service X
• Web service Y
• Xs output needs to be Ys input.
• But Xs output is not in the proper format for
  being Ys input.
• The workflow editor must be able to find a
  converters.

8
Example. Finding a converter

• The editor needs to do the following tasks
• Get information about web service X. Which format
  has Xs output?
• Get information about web service Y. Which format
  requires Ys input?
• Find a converter. How can data in format X be
  transformed in format Y?
• All this information must be stored and
  accessible in the registry

9
Example. Get information of X

• It is required to query the registry. Still it is
  not decided how this query will be but
  intuitively it should look like
• specObject GetWebServiceSpecification(X_persiste
  nt_id)
• This call will return the specification where
  explains that Xs output format follows the
  standard A.

10
Example. Get information of Y

• Like previously with X, the editor queries the
  Registry for Ys specification.
• specObject GetWebServiceSpecification(Y_persiste
  nt_id)
• This call will return the specification where
  explains that Ys input format follows the
  standard B.

11
Example. Finding a converter

• Now the editor needs to find a converter from
  standard A to standard B.
• It will be done using the Registry as well. Also
  it is not developed yet but it will be something
  similar to
• ResourceList GetResourceList(/standards/convert
  ers)
• The editor has downloaded the list of converters
  (resources under the standards/converters
  branch). Now inspecting this list of resource
  descriptions, the editor will find the required
  converter (if any).

12
Example. Introduce the converter in the workflow

• Before the insertion of the converter in the
  workflow, depending on the users profile
• the user will be asked for confirmation or
• the user will be noticed of different converters
  if more than one is available or
• simply wont be notified at all (novice users)
• The editor introduces the converter in the
  workflow between X and Y.
• The workflow description will keep this converter
  information. Engines executing this workflow will
  need to know about this conversion.

13
Workflow editor Requirements

• Graphical representation of all workflow
  description elements. (loops, if then clauses,
  parallelization, exceptions, etc)
• SMART editor.
• Different user profiles (Advanced users will want
  to know more about automatic decisions taken by
  the workbench while novice users wont want to
  decide anything)
• Drag Drop features.
• Registry browsing

14
Workflow workbench.

• The editor can help even more to the user if it
  is aware of the workflow execution results.
  Thats why it is recommended to develop a
  Workbench (Workflow Editor Workflow Engine
  together).
• Also for basic users it is easier when
  everything is centralized in just one
  application.

15
Workflow Workbench Requirements

• All Editor requirements
• Debugging tools breakpoints, Start-pause
  execution and inspection of intermediate steps.
• Partial workflow execution and reuse of data
  obtained in previous executions. (To avoid
  repeating calls to web services while testing)
• Output console

16
Very important. SMART applications

• Different profiles in CLARIN applications. Dont
  bother novice users with advanced questions.
• Applications remember similar situations in the
  past. You connected two web services similar to
  this one using converter AtoB. Do you want to do
  it again?
• Use of community intelligence stored in the
  registry. 250 users connected these 2 web
  services in other projects using converter AtoB,
  do you want to do it as well?