Updated
Myst v7.0.0 has been released! 🎉 Read the full release notes and check out the release video highlights.
The source of competitive advantage and value that an organization delivers to its end customers is increasingly defined by the software “systems” that underpin them. As a result, organizations find themselves in a digital race, where the speed at which IT can reliably deliver new features and innovations is what sets them apart from their competition.
“In an industrial company, avoid software at your own peril . . . a software company could disintermediate GE someday, and we’re better off being paranoid about that.”
‍
Jeff Immelt, CEO, General Electric‍‍
These innovations are seldom delivered by pre-packaged business applications, whether running on-premise or delivered as Software as a Service (SaaS), but by custom solutions derived in-house. Yet, most organizations have neither the time nor funds to build these systems of innovation from the ground up. Instead, they are delivered by layering new capabilities on top of existing applications, an approach defined by Gartner as “Pace-Layered”.
Oracle Middleware, such as the Oracle BPM Suite and Oracle SOA Suite provides the application glue to rapidly and continually combine these business apps, like puzzle pieces, into a custom integrated solution in order to deliver a seamless and unified experience to the customer.
Yet even with this Pace-Layered approach, many IT projects are still failing to deliver either on-time or on-budget, with development teams often held back by their own IT organization. So how can you reduce the cost of the software you develop and decrease the time it takes to get it right?
Research shows moving development to the cloud can initially reduce development time by an order of 11 to 20 percent. Organizations that fully embrace the cloud for Dev and Test are experiencing 30%+ time savings upon maturing their DevOps capabilities.
“Companies that are able to innovate quickly with software will outcompete traditional market leaders.”
Forrester Consulting‍
Moving Oracle Middleware development to the cloud, gives organizations the ability to streamline their development process, including the ability to quickly get development assets online, so application development can begin without having to wait for hardware and software to be setup in the data center.
This paper discusses the benefits of implementing Oracle Middleware projects in the cloud, how to get started, as well as additional benefits provided by cloud-based development.
In many organizations, IT projects are failing to deliver either on-time or on-budget. Tom DeMarco, a respected authority on Project Management and author of several books, including The Deadline: A Novel About Project Management, has pointed out that, "All projects that finish late have this one thing in common: they started late…".
“All projects that finish late have this one thing in common: they started late...”
Tom DeMarco‍
DeMarco provides a number of reasons why a project starts late, but a common issue, even when the project has "notionally" started, is the speed at which an organization can provide the required development tools and environments at critical points in the project lifecycle.
Why is this? And why is it such a problem for complex integration projects?
Implementing enterprise-class Oracle Middleware projects requires several pre-production environments (development and test environments); project teams often having to wait several weeks or months for these environments to be made available, causing significant delay at the very beginning of the project.
“The start of the project was delayed by 3 months waiting for a suitable Development Platform, the start of SIT was delayed even longer for similar reasons ...”
Project Manager – Manufacturing Company, USA‍
Waiting for hardware to be delivered and installed, as well as scheduling various IT Operations teams to configure the underlying infrastructure, such as Virtual Machines, Networks, Databases, Load Balancers and Shared Storage are all key contributors to the delays.
Once the infrastructure is in place, the installation and configuration of the Oracle Middleware needs to be performed, this is often a manual and error prone process, which can take several weeks to complete, contributing to further delays.
As it takes so long to obtain a Middleware Environment, there is a tendency to hold on to them for as long as possible; environments such as UAT, Performance Testing or Training are only required at specific intervals during a project, yet project teams will hold on to them for the entire duration of the project or longer.
Keeping “hold” of all these environments, ties up resources, which may be required by other projects, thus delaying those projects. In addition, and equally concerning, is the added infrastructure and software licensing costs each and every project carries, with a full complement of pre-production environments established for the entire duration of each and every project.
Apart from being inefficient, organizations can end up with many under-utilized environments, which require on-going administration, plus additional investments in hardware and software licenses. To make matters worse, organizations quickly lose track of what each environment is required for, and thus never know when these environments can be finally retired, so hang on to them anyway.
As a result, projects end up sharing environments. If there are too many developers working in the same environment, deploying and testing code, it can quickly result in a degradation of system throughput, which has a direct impact on developer productivity.
Troubleshooting platform issues, such as memory leaks, stuck threads, etc. takes longer, as it’s not easy to determine which project introduced the issue. In the meantime, this impacts the stability and performance of the platform, requiring developers to regularly down tools to accommodate server re-starts.
“Productivity of my development team was reduced by 20% due to an unstable middleware platform, at one point we were re-starting it 4-5 times per day … out of a team of ten, it was costing me two FTE’s”
Integration Manager - Utility Company, Australia‍
In addition, configuration changes, often require server restarts, the more projects there are, the more config changes are required, resulting in more server restarts, impacting platform availability and developer productivity across all projects.
Research shows cloud platforms can reduce development time by an order of 11 to 20 percent, with some projects experiencing 30%+ time savings.
The promise of cloud is that development and test environments are delivered as a service on demand and ready to consume; so in roughly the time it takes to do a server re-start, you can create a new environment in the cloud. Key to this is providing a self-service experience, removing the need to involve IT Ops –allowing them to stay focused on running production.
And because it so easy to create new environments, then there is no issue with throwing away these environments when no longer needed, so removing the overhead of managing un-used environments, as well as allowing organizations to make optimal use of software licenses, hardware, and staff.
This elastic capability of the cloud allows each project to have its own environments, removing the impedance imposed when sharing environments with other projects. One organization we worked with, experienced a 25% increase in developer productivity, when they switched from shared to dedicated project environments. For every 4 developers they were getting the equivalent of an extra free developer.
Much of the tooling used to enable development in the cloud, is the same tooling used to enable many of the best practices in Continuous Delivery and DevOps, further improving productivity (see the Whitepaper Best Practice for Implementing Continuous Delivery for Oracle Middleware for further details).
While many organizations are making some use of public cloud, many are just not ready for large scale cloud adoption and are still coming to terms with what is involved in moving enterprise applications to the cloud.
Moving Dev and Test to the cloud can help an organization build out its cloud capability, with minimal risk and provide greater insight into what would be required to move Production to the cloud.
Cloud offerings are typically divided into one of three broad tiers, at the top of the cloud stack is Software as a Service (SaaS), followed by Platform as a Service (PaaS) in the middle, and Infrastructure as a Service (IaaS), at the bottom.
IaaS provides a layer of virtualized hardware that delivers the network, compute and storage capabilities required for applications to run, serving as a foundation for SaaS and PaaS.
PaaS encompasses a broad collection of application infrastructure (middleware) services, including application platform, integration, business process management (BPM), content and database services; though most PaaS providers tend to focus on applications PaaS (aPaaS) and database PaaS (DBaaS).
In the aPaaS category, different providers support different technology stocks, such as Java, .NET, PHP and Ruby on Rails. Oracle are one of a small number of vendors with offerings across all three tiers of the cloud stack, in the PaaS category they provide a number of services, including Java, WebLogic, Database and SOA as Service.
Many IaaS providers are now also providing PaaS like capabilities, for example Amazon Web Services provides multiple Database as a Service offerings, that span Oracle, MySQL, SQL Server and Postgres; blurring the line between IaaS, DBaaS and aPaaS providers.
When it comes to running Oracle Middleware on the cloud, Oracle’s stated strategy is to offer a broad range of solutions on the Oracle Cloud, while giving customers the choice and flexibility to run the Oracle Software on other clouds. At the time of writing, approved Oracle cloud providers are Amazon Web Services and Microsoft Azure, the two leading Infrastructure as a Service providers.
A key consideration when selecting your cloud provider for Dev and Test, is that whilst your test and development environments will be in the cloud, the code being developed, will eventually be deployed to and executed in an on-premise Production Environment. So you need to ensure the cloud environment replicates your production environment, to avoid identifying issues late in the project when deploying on-premise.
This requires selecting a cloud provider that gives you sufficient control of the environment, so that you can closely mimic your Production environment. Alternatively you can build your Production environment to closely mimic the platform delivered by the cloud provider.
To run Oracle SOA Suite on the cloud, there are two options: (i) use Oracle SOA Cloud Service, which provides a pre-installed SOA Suite Domain based on a few simple selection criteria; (ii) select an Infrastructure as a Service provider, such as Amazon Web Services or Microsoft Azure, and install Oracle SOA Suite.
For Oracle BPM Suite, there are similar options, the key difference being with the Oracle Cloud. Oracle don’t provide an equivalent cloud service for Oracle BPM Suite, rather the approach is to use Oracle Java Cloud Service, which essentially provides WebLogic as a Service and then install the Oracle BPM Suite on top of this.
Note: the further up the cloud stack you go, the less control you have over lower level components, so a key criteria when choosing your cloud development environment is to assess how closely the underlying pieces mirror your production environment. Using IaaS, you have more control over how the Oracle Middleware is setup and configured, than if you adopt a PaaS provider. For example, if you choose Oracle SOA Cloud Service for Dev/Test, you need to check if the version and patch level you are using on-premise is available in the cloud? The mis-match between dev/test environments and prod can often cause project schedules to come unstuck late in development cycles as unforeseen issues arise due to version discrepancies.
If you are looking to integrate with other applications, you may also want to replicate these in the cloud for Test and Dev purposes; where feasible it makes sense to choose a cloud provider that can host both your middleware platform as well as these applications.
A key requirement for establishing Dev/Test in the cloud is to provide development teams a self-service environment so they can provision environments on demand, without the need to involve IT operations. This is a lot more involved than just provisioning a “vanilla” environment of Oracle Middleware on the cloud; a capability that is available today with both Oracle Cloud and Amazon Web Service (Oracle SOA Service Cloud provides this capability out of the box, whilst MyST provides an equivalent capability for Amazon Web Services).
The challenge is to ensure a like-for-like configuration between the cloud based environments, Production environments and other on-premise systems at all stages of the software development lifecycle (SDLC).
This means when the Dev and Test environments are provisioned, you need to ensure they are configured in alignment with the Production system.
In addition, during the lifetime of a project, as code is promoted though various staging environments on its journey to Production, you need to ensure that the corresponding configuration changes are promoted and applied consistently across all environments, on-cloud and on-premise.
With middleware, small in-consistencies between environments, such as different patches levels, disparities in the configuration of components, can result in defects, which are difficult to diagnose and rectify. This is commonly referred to as configuration drift. To avoid this pitfall, it’s important to put in place mechanisms to ensure consistency across all environments, whether on-premise or in the cloud.
Configuration discrepancies between environments, known as configuration drift, can cause code that works as expected in one environment to break in another environment. These issues start to surface during testing or worse in production and typically are reported as code defects. The standard process to try and resolve any defect is to re-produce it in an earlier environment, so that the root cause can be diagnosed, a fix can be produced and validated before promoting to upstream environments.
But often you face a catch-22 scenario, where you cannot re-produce the issue in other environments due to the configuration differences!
As a result, issues caused by configuration drift are often the most difficult to diagnose and result in many wasted months of effort to resolve; resulting in project delays, cost blow out and missed milestones.
Worse still, if these issues surface in Production or Disaster Recovery environments, it can result in all sorts of problems including security, performance and availability issues that can have a material impact on the business.
Configuration drift is as much of an issue for on-premise development; but moving Dev and Test to the cloud will highlight this issue. This is because ensuring configuration is synchronized across all environments is key to deliver on the productivity benefits of cloud.
Much of the tooling used to enable development in the cloud, is the same tooling used to enable many of the best practices in Continuous Delivery and DevOps, further improving productivity (see the Whitepaper Best Practice for Implementing Continuous Delivery for Oracle Middleware for further details)
MyST is the enabling technology which allows you to quickly provision and configure a Middleware as a Service on top of an existing IaaS or PaaS provider – as well as fully automate the build out of your on-premise environments.
Central to MyST is the Platform Blueprint, an environment agnostic specification which is used to define a standardized Oracle Middleware topology and configuration, for example an highly available deployment of the Oracle SOA Suite incorporating OSB, SOA, WSM and BAM.
The Platform Blueprint is also used to capture all additional platform specific configurations, such as Classpath configurations, deployments of 3rd party libraries, Authentication Providers, Data Sources, JMS Modules, Work Managers and Adapters.
The Platform Blueprint, provides an abstraction layer over the underlying infrastructure, giving you the flexibility to provision a consistently configured middleware platform on-premise or on cloud. An additional benefit of this approach, is that you are not locked into a particular cloud provider.
A platform model is then used to capture all the environment specific configuration information, required to provision an instance of an Oracle Middleware platform into the selected environment. When provisioning into the cloud, MyST will also spin up the required computing resources and server instances required to host the middleware platform.
This allows you to provide a middleware as a service catalogue to your development team, enabling them to select the required platform, and then at the click of a button provision an environment in minutes; and tear down when no longer required.
An additional benefit of this approach, is that you are not locked into a particular cloud provider.
As the Platform configuration’s change overtime, you simply update the Platform Blueprint. You can then use MyST to apply those updates to an existing environment or spin up a new environment.
The Platform Blueprint and Model are placed under version control, allowing you to manage the promotion of configuration changes from Dev through to Production, as well as having the ability to roll forward / backward to a different version of the platform and hence eliminate the possibility of configuration drift.
When you start your first cloud project, it’s quite likely that you will not have established network connectivity between your data center and the cloud; yet middleware projects by their very nature are required to integrate with other systems, usually (at least for now) hosted on-premise. So initially, you need to select a project which can be developed in isolation from those on-premise systems.
When implementing a BPM solution, best practice is to implement integration with external systems as a separate services layer, using either Oracle Service Bus or Oracle SOA Suite. A benefit of this approach is it allows you to implement mock services, to simulate the behavior of the services layer.
Implementing mock services may seem like an additional overhead; but in reality it is a common approach, as it allows BPM Processes to be developed and tested in complete isolation of the underlying systems, this simplifies the implementation of automated unit and regression tests, and also allows development of the BPM process in parallel to the actual services. This will provide additional productivity gains as well as improve the overall quality of the end solution.
With integration projects, best practice is to implement a layered architecture as illustrated below. With this approach, the Internal Virtual Service layer (in red) is responsible for integrating with the underlying applications. If your organization follows a similar architecture, you can implement mock services to simulate the behavior of this layer.
Another option is to mock the underlying applications that are being integrated with; many integrations either use the database, file or ftp adapter or integrate via web service. In each of these cases it is often relatively straightforward to create mock implementations of the integration points.
Of course another alternative, where feasible, is to deploy test instances of the actual application being integrated with in the cloud. The additional benefit of isolation, is that it encourages best practice, in that it forces a clear separation between layers, and encourages the use of automated testing.
At some point, there will be a requirement to integrate with the actual on-premise systems. This will certainly be the case when your project commences System Integration Testing and User Acceptance Testing, both of which demand integration with the “real” system. In addition, integration with actual on-premise systems may be required earlier in the project lifecycle. The primary reason for this is where the level of complexity to create a mock implementation of the underlying application is high, means that the effort to do this cannot be justified.
To enable the cloud-based integrations to invoke endpoints within the on-premise data center, you will need to establish network connectivity between the cloud and the data center.
There are two options, first is to establish a VPN connection between your Data Center and the cloud over the public internet, this will often be sufficient for Dev and Test purposes. Alternatively, if you have high through put requirements, either message volumes or message size, then you may want to consider a direct connection between the public cloud provider and your data center.
The advantage of a private connection, is that it can provide increased bandwidth, a more reliable network experience and reduced latency.
On the path to production, a solution will be often released dozens of times into each environment. When performed manually, the code build and deployment process is highly error prone and inconsistent, adding both additional cost and additional time to a project. This is further compounded by the fact that in most large organizations there are different individuals and teams performing these tasks in each environment.
To unlock the commercial and speed to market benefits of developing in the cloud, the process of taking code developed and tested in the cloud and deploying it on-premise needs to be frictionless. The two key prerequisites to achieving this are, environment consistency across Cloud and On-Premise and having a standardized and consistent automated process for building and deploying code across all environments.
MyST is the enabling technology, which allows you to quickly establish a standardized and repeatable automated process for the deployment of Oracle Middleware solutions on-premise and in the cloud. MyST shifts the experience from a resource intensive and highly error prone process to an automated, predictable and low risk process that can be performed in a fraction of the time.
Application blueprints are used to define the artefacts that constitute a release, the configuration requirements for these artefacts, and the configuration changes that will need to be applied to the Oracle Middleware Platform. These blueprints are version controlled, allowing us to treat configuration as code, ensuring strong governance and consistency across the release process.
Automated builds check out the required code from your source code repository, builds and packages the code. The packaged code is stored in a software repository, such as Artifactory or Nexus, allowing each release to be deployed to any environment as required.
Before deploying the packaged code, any references it has to other artefacts, such as service end point locations, database connection details and file locations need to be configured for the target environment. These configurations are defined as part of the Application Blueprint, and automatically applied by MyST during the deployment process.
In addition to deploying the package to the target environment, configuration changes may need to be made to the Oracle Middleware Platform, such as the creation of data sources, JMS Queues and other resources, as well as the application of patches. These changes are also defined as part of the Application Blueprint, and again automatically applied by MyST as part of the deployment process.
Continuous integration (CI) is the practice of automatically building and testing a piece of software each time code is committed by a developer. Continuous delivery (CD) goes a step further to automate the build, packaging, deployment and regression testing across all environments, so that it can be released at any time into production.
Key benefits of CI/CD is it allow organizations to streamline the development process, enabling organizations to rapidly, reliably and repeatedly deliver projects faster, with less risk and less cost.
To implement continuous delivery pipelines, you need to automate not only the build, deployment and testing processes, but also the provisioning and de-provisioning of environments required to support these processes. It is also critical to ensure consistency between the various environments as code moves from development through the various stages of testing, and into production.
Automating these activities, is something that has already been performed as part of the process of moving Development and Testing to the cloud, so another benefit of the move to cloud is that it provides the perfect foundation for implementing a CI/CD development model.
MyST has plugins for popular Continuous Delivery tools, including Jenkins, Hudson and Atlassian Bamboo; allowing you to fully integrate MyST with your CI/CD tool(s) of choice. This allows organizations to quickly and easily leverage the full power of MyST and the cloud as part of their Continuous Delivery solution.
Organizations are in a digital race, where the speed at which IT can reliably deliver new features and innovations is what sets them apart from their competition.
Moving development of Oracle Middleware projects to an Infrastructure as a Service (IaaS) or Application Platform as a Service (aPaaS) provider, can deliver significant reductions in development time and costs.
MyST is the enabling technology, which allows you to quickly establish a Middleware as a Service on top of an existing IaaS or aPaaS provider, as well as fully automate the software delivery process.
Much of the tooling used to enable development in the cloud, is the same tooling used to enable many of the best practices in Continuous Delivery and DevOps, further improving productivity.
In short, the benefit of moving Development to the Cloud will provide the business with a strategic advantage in its ability to be more responsive in delivering new solutions faster, cheaper and more often.
.png)
Join Community
