Future of Enterprise Application Integration(EAI) Middleware

There are various types of middleware in software industry. This blog limits the scope to EAI middleware that is widely used in enterprises. This type of  middleware adds integration capability to a functional system that end users interacts with. It offers connection to heterogeneous systems, data transformation, fields mapping, routing logic and sometimes rule engine.  Technical teams also prefer middleware to unload certain heavy operations from main functional systems. 

Typical functionalities of EAI middleware

How important is EAI middleware in enterprise eco system?

Definitely not for this reason

As enterprises heavily relies on distributed applications,  it is equally important to connect them all together. With applications being developed using different technologies that are naturally incompatible to connect together poses a big challenge. This is one of the predominant use case for integration middleware which acts like adopter to connect disparate systems.

Another scenario is when data transformation needs to be done between two systems to keep data in sync across. Neither of the systems would like to take up that responsibility and middleware comes to rescue.

Many teams use middleware to move certain business logic out of the functional system to distribute the load. The intention of this requirement is sane but there is a danger of heavy load lurking into middleware eventually.

General Issues with Middleware

• Middleware eventually becomes heavier than main system     

         Middleware are designed to serve as connector to heterogeneous applications to enable them communicate through data flow.  However, with some complex data transformation, rule engines, exception management it tend to become heavier than main systems. This calls for more resource requirements which in turn increase the maintenance cost. 

             

• Skilled personnel requirement

         Requires specific skilled personnel with steep learning curve on certain heavy weight middleware. Most of the notable middleware in market requires specific skills to maintain. Even the ones that claim zero coding adds complex configuration with UI flows, complex navigation and significant deployment effort.   

 

• Maintenance in long term 

         As a connectivity software, middleware is mostly invisible. It provides a more standard way of doing things, it ties together complex systems and allows developers to concentrate more on the functionality of their applications. If focus on middleware is limited, then over a period of time those skills are lost due to various reasons like upskilling, attrition. As a result, ultimately it becomes hard to maintain.

 

• Legacy middleware vs new technologies 

         Middleware bring advantages with its range of use in a wide array of software systems, from distributed objects and components, to mobile application support, to message-oriented communication, and more. With technology changes, and infrequent upgrades, middleware become obsolete in a short period of time.  With fast changing communication standards in software industry also adds challenges.  Lacking suitable connectors using new technology could end up adding yet another middleware to process. I have spoken to a team with decade old heavy weight, on-premises middleware. They were searching for yet another middleware to connect to Kafka which was not supported in their existing one. The gap between not updatable software and technology grows heaps and bounds with time. 

 

• Reduced Throughput 

     Introducing middleware between two homogeneous systems surely adds overhead without any advantage. Systems are capable to communicate using simple REST API work faster with direct integration. I spoke to few teams using third party middleware even for API connections. I wanted to understand their intent. I could not get compelling reasons or proper justification. Those teams thought they are following certain design pattern.  

 

 

• Performance Bottleneck

       Middleware unnecessarily used can create serious performance issues. The extra layer and the redundant execution logic introduced within the middleware endup hampering the performance of whole system. 

                        

 

• Even middleware requires sub middleware in Enterprises

        Sounds ridiculous but true! Most of the big enterprises adopts middleware to every functional system they develop. When a System A needs to connect to System B, data flows through 2 middleware as the design lists data flow allowed only through middleware in every system.   

When developers missed following design principle

 What are the alternatives?

The best thirdparty middleware comes with huge cost associated. If cost is the constraint, teams should build the middleware in incremental manner. Keeping the design simple and nimble, flexible and scalable middleware should be designed. The efficiency can be achieved if below factors are considered thoughtfully.

• Enterprise Architecture Strategy

   With a clear understanding of the IT portfolio along with data, applications, and infrastructure, solid EA strategy needs to be devised to get value out of middleware. It is important to frame middleware as a light weight system that is flexible and scalable to ever changing requirements and technologies. Having good domain understanding of the enterprise, high level view on every system in IT portfolio and their interactions helps in the building solid strategy. 

• Flexible Design 

        Using plug and play style of architecture pattern to add new plugins easily in future. Often it requires to add or update new adapters, routing rules, mapping logic, etc. in middleware. Rigid system not allowing such modification consumes lot of effort and time to redesign the system. 

• Cloud to increase scalbility and reduce cost

   On premises hosting involves significant overhead in terms of maintenance and cost. Scalability requirement often missed out during the initial design process adds huge penalty when usage and features increases in the functional system.  

• Containers & Microservices to speed up the development process

        Helps in decentralizing the data, reducing the coding complexity, achieving scalability.