Our research program for next year will focus on “Big Data And The Real-Time Enterprise.” To be honest, we’re not sure what a real-time enterprise is in respect to current technology. Nevertheless we have some ideas, and we are writing a concept paper to flesh them out before we embark on the research in earnest.
The term real-time is like big data: not very helpful. It is not as if businesses operate in some imaginary kind of time – it’s real time for them. Things happen and they respond as fast as they are able. IT systems obviously help with this.
Originally the term real-time was only applied to systems that were reactive within a given time frame. They had very definite operational constraints. So, process control systems which might, for example, be controlling the speed of a production line had to make specific decisions within a given time frame. If they were too slow, the whole operation would grind to a halt, so speed of execution was critical.
It is not so different in most modern businesses.
We recently examined this topic in some depth with Dale Skeen, CTO and one of the founders of Vitria, a company specializing in what you might call real-time business systems. Vitria prefers the term operational intelligence (OI). Vitria’s technology constitutes a combination of complex event processsing (CEP), business process management (BPM), integration technology and several other components.
- Visualization. This corresponds to gathering data and presenting it to a user or possibly sending it directly to a prepared analysis capability.
- Insight. This is the stage when data is analyzed, typically in a continuous manner using emerging technology like CEP, looking at combinations of events that happen within a time window. However, any intelligent analytic activity might be done here, in respect to both newly arrived data and historical data.
- Action. If the second step determines that some action should be taken, then the third step is to automatically take that action directly, by intervening in the associated business processes in some way. The action could involve a simple action such as altering control parameters, or it could involve initiating a new “response process” orchestrating a complex series of actions.
This process now repeats at whatever frequency is required. In effect, it could constitute a continuous feedback loop that drives the business in a similar manner to the way a process control system can drive a production line.
At the moment, a real time enterprise is one that works in this way, even if it hasn’t been well automated/instrumented yet. This continuous feedback is roughly the way that living things work, whether it be potted plants or great white sharks. They work that way because they need to be as flexible as possible to the environment that they inhabit. If a business had very little variability, then OI of this kind would be unnecessary, but most businesses have considerable variability in all their processes. As such, they operate regular cyclical processes to manage that variability. Vitria refers to this as continuous analytics or continuous insight.
The Telescoping of Time
Dale Skeen suspects that there are three fundamental time pulses that a business experiences, or if you like, three fundamental time frames across which it needs to employ operational intelligence: immediate response, short-cycle response, and long-cycle response. This is partly because of his observation of the way that various companies, from Health Care to Telecomms, have used Vitria.
Dale’s experience with customers suggests that the window of data (the time window of events plus historic data) that is examined in the area of immediate response can make a big difference. Low latency is fine, but in this area it can be a combination of low latency combined with the breadth of the time window. Computer power can count. Anyway, it’s easy enough to make the argument. It is probably well worth a business looking at itself and its automated systems in this way. It’s a fertile area for innovation.
Another aspect we discussed is the speeding up of time. There are some areas–automated trading is the most obvious one–where the ability to respond fastest is everything. It almost seems quaint nowadays to think that financial markets used to run manually. Once the human being was cut out of the equation, this area of business turned into a technology arms race. The latency between determining that a profitable trade was possible and actually making the trade just kept on shrinking. The same telescoping of time now seems to be happening in the Internet Advertising market in terms of bidding for ad space and placing adverts. We ponderously slow human beings are no longer involved in such activity.
The relentless grind of Moore’s Law–a law that has not yet been repealed–governs much of this. It doesn’t just enable the instant responses of a business to be faster, it enables all three of the time cycles to speed up. At the same time, of course, it showers us with ever increasing amounts of data to slow us down a little.
Nevertheless the more I think about business computing from the perspective of operational intelligence, the more I become convinced that technology determines timing. And timing is half of the cake.