Microservices and Business Transactions

Microservices become the new disruptive technology for software development in it’s traditional way. It is the architectural style which brings up a new way to build software systems. The Microservice Architecture evolves very fast and in deed has a lot of success.

But there are also some disadvantages in Microservices. One of the critical parts is the rising complexity within a microservice architecture. It is a problem which is often overlooked in the beginning when all this euphoria lies within a greenfield project. And the problem is often denied for a long time by its advocates. But why does this happen?

Continue reading “Microservices and Business Transactions”

BPM with Apache Kafka

The Imixs-Workflow project supports now a native integration Adapter for Apache Kafka. With this feature asynchronous messages can be handled within a complex business process.

Apache Kafka is a distributed streaming platform. This means you can publish and subscribe to streams of records, similar to a message queue or an enterprise messaging system. Apache Kafka store streams of records in a fault-tolerant durable way and process streams of records as they occur.
Kafka is generally used for two broad classes of applications:

  • Building real-time streaming data pipelines that reliably get data between systems or applications
  • Building real-time streaming applications that transform or react to the streams of data

The Imixs-Kafka Adapter integrates Kafka in a native way and allows the producing and consuming of Imixs-Workflow Messages.

Workflow Message Autowire

With Imixs-Kafka you can easily setup a scenario where Workflow Messages are generated automatically during the processing life-cycle. With the Autowire-Function new process instances are send into a Kafka Message Queue so that any consumer interested in workflow events can consume the message and react in various ways.

The Adapter filters Workflow events by the Model Version number so you can control which kind of workflows are send into a message queue.

Workflow Messages based on Business Logic

Another way to send Workflow Messages into a Kafka queue is the Imixs-Adapter Class. This implementation is based on the Imixs-Adapter concept and allows a more fine grained modeling of an asynchronous service integration. The Imixs-Kafka Adapter can be configured directly in a BPMN 2.0 Model.

This modeling approach allows you to setup complex workflow scenarios with a asynchronous communication.

Sending Workflow Events With Kafka

The other way to integrate Imixs-Workflow into a Kafka infrastructure is to send Workflow Messages to a Kafka queue to be processed by the Imixs-Workflow Instance. In this way a client sends a Process Instance with a predefined topic into a Kafka Message queue.

Imixs-Workflow will automatically consume those messages and process the workflow data. In this way messages can be used to trigger the event-based Imixs-Workflow engine. The following example shows how a client can send a workflow event for a named topic:

String json_workitem = "{\"item\":[ "
+ " {\"name\":\"type\",\"value\":{\"@type\":\"xs:string\",\"$\":\"workitem\"}}, "
+ " {\"name\":\"$modelversion\",\"value\":{\"@type\":\"xs:string\",\"$\":\"1.0.1\"}}, "
+ " {\"name\":\"$taskid\",\"value\":{\"@type\":\"xs:int\",\"$\":\"1000\"}}, "
+ " {\"name\":\"$eventid\",\"value\":{\"@type\":\"xs:int\",\"$\":\"10\"}}, "
+ " {\"name\":\"txtname\",\"value\":{\"@type\":\"xs:string\",\"$\":\"test-json\"}}"
+ " ]}";
ProducerRecord<Long, String> record = new ProducerRecord<Long, String>("IN-1.0.1", json_workitem);

Microservices with Kafka and Imixs-Workflow

The Imixs-Kafka Adapter is a powerful feature to integrate Imixs-Workflow in a Microservice Infrastructure based on Publish-Subscribe Messaging Queues. With Apache Kafka a fault-tolerant and scalable messaging platform can be adapted to Saga Transactions based on a BPMN 2.0 model.

If you have any questions or your need help to integrate Imixs-Workflow into your Microservice Architecture you can join the project on Github.

Imixs-Workflow – Version 4.5.4 released

We release a new patch version 4.5.4 of Imixs-Workflow. The new Release includes several Bug-Fixes and Enhancements:

Enhancements

  • Rest API – ReportService – support new param format #489
  • JobHandlerUpgradeWorkitem – improved performance – requires new transaction #487
  • Rest API – ModelService – provide new resource model definition #486
  • RestClient – now supports rootURI in post method #494

Bugfixes

  • WorkflowService.processWorkItem – validate uniqueid #492
  • Evaluate WorkflowResult – fixed case of empty tags #490
  • JobHandlerRebuildIndex – skip document type “workitemlob” #488

BPMN 2.0 – The Extension Mechansim

The ‘Business Process Model And Notation’ standard is a well designed notation for describing business workflows. BPMN 2.0 becomes the standard for modeling business logic and fits very well the model driven software design in agile software projects.

The BPMN language, which is based on XML, was intended for users at all levels, from the business analysts who create the initial design, to the developers who implement the technical details, and finally, to the business users responsible for managing and monitoring the processes. BPMN 2.0 has evolved to become a complete specification trying to fit the needs to all people involved in the design of a business process. But writing BPMN in XML and visualizing business processes becomes nearly impossible without the use of a graphical tool.

Continue reading “BPMN 2.0 – The Extension Mechansim”

Imixs-Workflow 4.5 Released

Today we released the new Version 4.5.0 of Imixs-Workflow. The version now has a full support of Java EE7. The release also includes also several improvements in JSF components and The Rest-API. Here is an overview about the latest changes:

New Features

  • The XMLItem now supports mixed value lists
  • New Extend FileData Object with custom attributes
  • LuceneUpdateService – provides a new ItemAdapter class to improve the extension mechanism
  • Full Java EE7 Support
  • Performance improvements for the imixs-faces module
  • RestService supports now custom reports to open spread sheets

Enhancements

  • JobHandlerRebuildIndex – allows to overwrite the Block_Size and Time_out
  • MailPlugin – Performance improvements

Bugfixes

  • WorkflowKernel – fixed computeNext Task – conditional Events computed correctly after changed conditions

A data migration is not needed. Existing applications can switch directyl to 4.5. Only JSF modules should adapt the new JSF Controllers. You can find details here.

How to Manage Model Versions

In the following short tutorial I will explain how you can manage multiple versions of a workflow model and how Imixs-Workflow supports multiple model versions in a productive business application.

The Imixs-Model-Management

The Imixs-Workflow engine supports multiple model versions from the same process model. Each time a running process instance is updated, the Imixs-Workflow engine applies the same model version as the process instance was created with. Therefor the Imixs-Workflow engine compares the internal model version with the model versions provided by the model repository. In case the current model version is no longer supported, the active process instance will be automatically upgraded to the latest version available in the repository. With this feature it is possible to manage different versions of the same workflow process. To distinguish different model versions, you need a versioning concept.

Semantic Versioning

The concept of “Semantic Versioning” from Tom Preston-Werner will be the basis for the versioning method of process models. Applying the semantic versioning, the version number is separated into three digits:

  1. 2. 3 
  |  |  |-- patch level
  |  |-------- minor version 
  |-------------- major version

Major Version

The first digit indicates the major version number. This is in our case the general main release of a model.  The major version number should only be increased if a general redesign of  whole the process model was done.

Minor Version

The minor version number is used for new functions and enhancements in the process flow. For example after adding a new task or a new event, the minor version number should be increased.

Patch Level

The last digit is used for the Patch Level of a model. It indicates bug fixes or changes of naming or wording.

The semantic versioning can be perfectly used for the BPMN model file. This helps you to manage different version within your modellng tool:

invoicing-1.0.0.bpmn

We typically prefix the filename with the process name to maintain different workflows in the same directory.  In the example above we have an Invoicing workflow in the version 1.0.0. The model file itself should also be stored in a code repository like Git. This allows you to remove deprecated models from your code base.

The Model Version

Also the Imixs-Workflow Model holds a separate version number. This version number is mandatory and can be maintained in the Workflow Profile of the Imixs-BPMN modeling tool:

We prefix the version number again with the process name followed by a two-digit version number. Applied to the “Semantic Versioning” only the Major and Minor Version number is used here!

The reason for this shorter version number is, that the model repository of a business application should not hold different patch levels of the same model. Remember, the patch level is only used to  indicates bug fixes or changes of naming or wording. Only if the model is extended functional the minor version number is increased or in case of a new process variant the major version number:

Modelversion    File Version            Comment 
----------------------------------------------------------   
invocing-1.0    invocing-1.0.0.bpmn     First draft
                invocing-1.0.1.bpmn     typo
invocing-1.1    invocing-1.1.0.bpmn     additional approval
                invocing-1.1.1.bpmn     change mail message
                invocing-1.1.2.bpmn     change history text
invocing-2.0    invocing-2.0.0.bpmn     new Invoicing process

In this example version 1.0 of the Invoicing process was first patch with a typo and next extended with an additional approval, which leads to Minor version 1.1. In this version again then some text changes were made. And finally a complete new Invoicing process was designed, with leads to the new Major version 2.0.

From the view of the running process instances only 3 different model versions exist. From the view of the code repository the model file exists in 6 different versions.

Conclusion

With the concept of “Semantic Versioning” process models can be easily handled in different versions. Even if you need to patch an already running model, you don’t need to worry about polluting the model repository of your business application if you use only 2 digits for internal model version.

Imixs-Workflow Version 4.4.4

Imixs-Workflow Version 4.4.4 is now released! The latest version includes two important improvements and shows a much better performance.

  • LuceneSearchService – Flush Event Log
    This improvement solved an issue with overrun Lucene Event Logs. Tests showed that the latest version runs stable and with better performance.
  • BPMN: Conditional Events
    BPMN models, using conditional events, can now be parsed correctly even if the model contains standard BPMN elements between an event and the condition:

The most important improvement of version 4.4 was the transaction support of Lucene event logs. With version 4.4.4 of Imixs-Workflow, this feature is now stable and shows also a very good performance in long running transactions.

Simulating Workflows

With the new project Imixs-Mock  you can now simulate your Imixs-Workflow BPMN models in an easy way. The project provides a full mock of the Imixs-Workflow engine and allows to test and simulate different workflow scenarios. Imixs-Mock simulates the full processing life-cycle including all workflow plug-ins. You can specify also a subset of plug-ins to test specific business logic in your workflow project.

To add the mock into your own workflow project just add the following dependencies:

<dependency>
  <groupId>org.imixs.workflow</groupId>
  <artifactId>imixs-mock</artifactId>
  <version>4.4.0</version>
  <scope>test</scope>
</dependency>
<!-- JUnit Tests -->
<dependency>
  <groupId>junit</groupId>
  <artifactId>junit</artifactId>
  <version>4.8.1</version>
  <scope>test</scope>
</dependency>
<dependency>
  <groupId>org.mockito</groupId>
  <artifactId>mockito-all</artifactId>
  <version>1.9.5</version>
  <scope>test</scope>
</dependency>

Now you can write your jUnit test class to verify your BPMN model. See the following example code:

@Test 
public void testSimple() {
  workitem = new ItemCollection();
  workitem.model("ticket-workflow-1.0").task(1000).event(10);
  workitem.replaceItemValue("_subject", "Hello World");
  try {
    workitem = workflowMockEnvironment.getWorkflowService().processWorkItem(workitem);
    Assert.assertNotNull(workitem);
    Assert.assertEquals(1100, workitem.getTaskID());
    Assert.assertEquals("manfred", workitem.getItemValue("namowner", String.class));
  } catch (AccessDeniedException | ProcessingErrorException | PluginException | ModelException e) {
    e.printStackTrace();
    Assert.fail();
  }
}

The Imixs-Mock allows you to simulate different users processing a process instance. In this way you can verify the Access Control List (ACL) of a process instance and the assignment of process owners in complex workflow scenarios.

You will find the full example on Github.