The term “workflow engine” is not clearly defined in software development and therefore we would like to give a brief overview of different aspects and types of workflow engines in the following blog post.
If you take a look at the Github page awesome-workflow-engines, you will find an impressive number of open source workflow engines. But not all of these engines solve the same problem. The idea behind the term ‘workflow’ can be very different and so the functionality of a workflow engine. In general a workflow is a series of tasks or activities that are required to complete a specific process. Such a process can involve multiple people, teams, microservices or even other IT systems. Thus, it becomes clear that when comparing Workflow Engines, various aspects must be taken into account.
Types of Workflows
Workflows can be used in a variety of contexts, including business, manufacturing, software development, and many more. For example, a workflow in a business context might involve tasks such as getting an approval from a manager, creating a proposal, making a decision by a human expert or routing important information through an organization. On the other hand in a software deployment process (DevOps), a workflow might perform automated tasks such as committing code, testing the code, building an artefact and deploying the results to a server. Depending on the specific requirements and context, different types of workflow engines are better suited to solve a problem than others.
Modelling
Workflows are often visualized as flowcharts in diagrams, which can help to clearly communicate the steps involved and the dependencies between tasks. Not all workflow engines provide a modelling tool which makes it more difficulty to maintain a workflow. The Business Process Model and Notation (BPMN) is a modelling standard that provides a graphical notation how to specify a business processes in a Business Process Diagram. BPMN is easy to understand by all project members and a good starting point to talk about a business process with technician as also with management people. In addition to the graphical notation, BPMN also offers the possibility of a machine-readable format. This makes BPMN well suited for workflow engines and has become a de facto standard for business workflows. Furthermore BPMN 2.0 supports a extension concept that allows to not only extend the modelling layer but also the execution layer in various ways.
Frameworks
Workflow Engines can be based on a specific framework or technology stack. A framework is a set of libraries, tools, and conventions that provide a structure for developing software applications. For example there are frameworks for Java like the Jakarta EE specification that simplifies the development of enterprise applications, or the Kubernetes API standardizing the way how to implement a workflow into Kubernetes. The main advantage of using a framework in case of a workflow engine is, that it provides a clear way how to integrate the solution into a specify technology stack. A workflow engine based on a framework makes the development process more efficient and consistent.
Security
Data security has become one of the essential factors in modern Software Development. Due to the clearly structured way in which data is processed by a workflow engine, corresponding solutions offer a high degree of security in data processing. But also the access to data can be secured by a workflow engine. For example, human-based workflow engines offer the possibility to control access to certain data depending on previously defined roles or business rules in a more fine grade way that this is possible with common security solutions.
Open Source
Open source refers to a type of software whose source code is available for anyone to view, modify, and distribute freely. Open Source is developed and maintained by a community in a open and transparent way. Workflow engines based on open source result in a more diverse and robust product, as those engines are developed and improved by many different users.
Open Source also allows adopters to customize the workflow engine to fit their specific needs. This is particularly useful for organizations that need to modify the software to integrate with their existing systems or processes. Since the source code for an open source workflow engine is available for anyone to view, it is also easier for security vulnerabilities to be identified and fixed. This makes open source software more secure than proprietary software, which may have undisclosed vulnerabilities.
One of the main advantages of open source is that it is generally free to use. This can be a significant cost savings for individuals and organizations that need to use software on a regular basis.
The list below gives a brief overview of some workflow engines used in different application scenarios:
| Feature | Type | Language | Framework | Modelling | Security |
| Imixs-Workflow | Business Workflow | Java | Jakarta EE | BPMN | ACL based |
| Activiti | Automation | Java | Spring | – | – |
| Apache Airflow | Automation | Python | – | – | – |
| Argo Workflow | Deployment | Kubernetes CRD | Kubernetes | – | role based |
| Bonita | Business Workflow | Java | – | BPMN | – |
| Dolphin Scheduler | Orchestration | Python | – | Graphical UI | – |
| Kubeflow Pipeline | Deployment ML | YAML | Kubernetes | – | role based |
| metaflow | ML | Python | – | – | – |
There are many other aspects that can be important when choosing a workflow engine. Therefore, we recommend aligning the criteria with your own goals and thus selecting the optimal product for the next software project. If you have any comments or ideas, we look forward to your feedback.