With integration projects one of the biggest drains on time and resource, is the un-planned activities, caused by unforeseen issues. The root cause of many of these unforeseen issues is an incorrectly installed and configured middleware platform.
Enterprise scalable middleware platforms are inherently complex; when we consider that an enterprise ready deployment of the Oracle SOA platform requires several hundred steps, then it should not be surprising that when performed manually, it is nearly impossible to get right in one go, and especially difficult to get right consistently across all environments.
Small errors, such as misconfiguration of a middleware component, can cause issues which are difficult to diagnose and rectify. These are not one off issues, but rather a steady drip, drip, drip of issues through all stages of the project lifecycle; resulting in many months of wasted man effort to resolve and lost productivity; leading to missed milestones , project delays and the inevitable cost blow out.
"Productivity of my development team was reduced by 20% due to an unstable middleware platform, at one point we were re-starting it 4-5 times per day … out of a team of ten, it was costing me two FTEs"
Integration Manager - Utility Company
"The test team has approx. 18% down-time, waiting for platform issues to be resolved, every 8 days of delay is costing me1 FTE"
Programme Manager - Major Bank
"The start of the project was delayed by 3 months waiting for a suitable Dev Platform, the start of SIT was delayed even longer for similar reasons ..."
Project Manager – Logistics Company
So it’s hardly surprising that time wasted due to platform related issues, often caused by an incorrect installation of the Oracle FMW Platform, is one of the major causes of project delay.
For most organizations, middleware is the glue that binds it systems together; and as such is a key component of an organization IT nervous system. So needs to be performant, scalable and highly available.
Achieving this requires the removal of bottle necks so the system can scale, the removal of single points of failure to provide redundancy, as well addressing security requirements, whilst at the same time delivering a performant solution.
In turn, the middleware is under-pinned by the IT infrastructure; including the network, load balancers, storage area networks, databases, credential and policy stores; as well as the servers (virtual or bare metal) on to which it is deployed. In addition the design of the middleware platform needs to consider other variables, including:
None of these variables can be considered in isolation, but rather need to be integrated as a coherent solution to deliver a reliable platform that meets an organization requirements.
Most organisations significantly under estimate the complexity of what it takes to stand-up a performant, scalable and highly available FMW environment. We are acclimatized to install wizards that make it simple and straight forward to install software on our phones and our desktops.
Since the Oracle Fusion Middleware comes with its own set of install wizards, it gives us the deceptive appearance of all being very easy. Now whilst installing a simple developer environment is pretty straight forward, correctly setting up a production ready enterprise Fusion middleware platform is a lot more complex.
Consider the Oracle SOA Suite, one of the simpler platforms to install; even with the install wizards, it requires six hundred plus manual steps as documented in the Enterprise Deployment Guide to be performed in the prescribed sequence. Each step requiring the correct configuration information to be provided.
Performing this for one environment, can easily take between 5-10 days, depending on the overall complexity of the configuration. All this assumes:
Typically an organization will require multiple environments, for example DEV, SIT, QA, UAT, Pre-Prod and Production. So assuming we need a total of six environments, then we are looking at approx. 30 -60 man days of effort to roll-out our environments. Whilst this is a significant amount of effort, within the context of a bigger project it still often acceptable.
In theory, this all sounds fine, but in reality it’s a very different story.
As we stated one of the assumptions is that the under-pinning IT infrastructure is in place and has been set-up correctly, including the set-up and configuration of the Servers, Load balancers, SAN, Network, Database, Credential Stores, etc. Often the set-up of each of these components are performed by different groups within an organization, each trying to fit these tasks in with the 100’s of other requests they are dealing with.
So it’s pretty much never the case, when starting the process of installing the middleware that all the pre-requisites have correctly been put in place. The result is that you get part way through the provisioning process and something fails! If you’re lucky its quick to diagnose and rectify the issue and carry on; but all too often it takes a while to diagnose the underlying cause, and then you have to wait for the appropriate team to fix the issue in the underlying infrastructure – and all too often it takes several attempts before they get it right.
Many issues tend to leave the install in an unknown state, as it will have failed with an exception partway through; when this happens the only option is to re-start the install of that component from the beginning, or worse re-start the entire install from scratch.
As a result the whole process becomes very fragmented, very quickly. When you consider that installing the SOA Suite requires 600+ manual steps, and the stop start nature of this work, it becomes very easy to miss a step, or make some other mistake, often without realising it. These mistakes often only surface as issues after you have completed the installation of the Oracle SOA Suite.
These issue are often time consuming to diagnose; once found if you are lucky, then it will be the case of applying a simple configuration change to fix the problem. But far too often either the root cause of the issue cannot be established, or it can only be resolved through a clean install. Meaning you have to tear down the environment and start the process of building it from scratch – with again no guarantee that it will be completed correctly.
So far we are talking about a single environment, but most organisation require 3-7 environments, depending on their Service Delivery Lifecycle (SDLC). When we understand the probability of error in building just a single “working” environment, then the odds of building out each of our environments so that they are consistent and correct is pretty remote.
Yet, small errors or configuration differences between environments, can cause result in code that runs in one environment, but not work as expected in another environment, making these issues difficult to diagnose and rectify; sometimes requiring weeks / months of man effort to resolve.
And all too often, issues are diagnosed and fixed in one environment, only for the same issue to surface again in another environment, and the whole troubleshooting process to be repeated.
Another assumption that we have so far skated around is change to the platform design, as is the case with implementing any complex system, we rarely get the platform design right first time, but rather it will iterate through a number of refinements; including:
Each change that we introduce, introduces additional risk that we, and can result in us breaking the platform; we also have the risk that change is not applied consistently, resulting in configuration drift. Which again can result in code not behaving as expected in some environments, and more time wasted in troubleshooting these issues.
All these issues very quickly have an impact on overall project time frames; with the development or testing team left in a holding pattern waiting for a working environment to be provided or being forced to work with an unstable environment and the suffer the corresponding impact on productivity.
As a result milestones are missed, with the inevitable project delays and cost blow outs; to make up the lost time, development and testing are squeezed further, resulting in unreliable code not fit for purpose is pushed into production on an unstable platform – leading to the inevitable fallout and fire-fighting.
MyST DevOps for Oracle provides an automated process for the installation and deployment of the Fusion Middleware platform which transforms this resource intensive and highly error prone process into an automated, predictable and low risk process.
The installation is driven by simple models(or metadata), which defines the environment specific properties, which are captured during the initial platform design. It provides a number of key capabilities, including:
MyST reduces risk and eliminates waste by automating manual tasks and bringing consistency to deployment.
MyST uses a declarative, model-based approach for the install and configuration of the Oracle Fusion stack. MyST comes with pre-defined templates which comply with Oracle Enterprise Deployment Guide (EDG) which define a reusable topology of the FMW Domain you wish to create.
MyST supports the concept of inheritance, allowing a tiered approach to defining our topologies, for example in the diagram below, we have defined a standard SOA template, with three sub-templates(Dev, Non-Prod and Prod).
For each WebLogic Domain we wish to create, we define a model based on a template; in which we specify the values that are unique to that environment, such as the hostname or database URL.
This allows us to quickly define new Fusion Middleware environment templates, from which we can provision new environments on demand.
The inheritance based model ensures key configurations, such as patch levels are identical across all environments, and in the event of a change, we need update only a single template for it to be inherited across all our environments. Ensuring strong governance and consistency across all our environments.
Automating the provisioning of your Oracle FMW platform will allow you to install and configure environments within minutes all at the push of a button, allowing this activity to be performed in minutes instead of days or weeks.
As we highlighted earlier, as is the case with implementing any complex system, it is rare get the platform design right first time, but rather it will need to iterate through a number of refinements.
MyST makes it simple for organization to utilize an iterative and incremental platform design methodology aimed at mitigating risk by discovering and resolving issues early, as illustrated below.
This approach enables the platform design to be quickly modelled, and provisioned into an environment ready for testing.
Note: It is also feasible to automate the non-functional testing, using a tool such as LoadUI.
With each design iteration; with minimal effort, the MyST models can be modified, and then executed to provision an updated version of the platform in the test environment; to allow the re-running of the non-functional testing in order to validate the revised design.
The additional advantage of using MyST, in this part of the process, is that the models can be placed under version control, making it simple to roll back to a previous version of the design if changes don’t deliver the anticipated improvements.
This also allows us to as manage the process of promoting configuration changes to each environment. Ensuring strong governance and consistency across all our environments.
Once the design / model has been finalized, it can then be leveraged to rapidly roll-out the design to each platform.
MyST is a declarative based, packaged solution for the design, install, configuration and continuous deployment of Oracle Fusion Middleware and Applications; it provides the business with a strategic advantage in its ability to be more responsive in delivering new solutions faster, cheaper and more often. Key benefits include:
MyST, the market-leading continuous deliveryautomation tool for Oracle Fusion Middleware and Applications, integrates withbest-of-breed build and provisioning tools, enabling you to achieve bestpractice in the continuous delivery of Oracle SOA and BPM solutions and all ofits benefits, including cycle time reduction, higher quality software,predictable delivery costs and reduced maintenance costs.
MyST provides the business with a strategic advantage in its ability to be more responsive in delivering new solutions faster, cheaper and more often.