Performance Problems in Well Operations System

The Situation:
An Oil Company invested significant time and effort to bring large volumes of realtime data from wellsites into a central database system and to automatically process the data to make both realtime and historical well information available to well operations engineers via a website. The continuous automatic processing of the realtime data took too long and the website was very sluggish for users.

The Response:
YR20 deployed Network Reliability Probes on an extended basis to capture all the network traffic at a number of key points in the system; the realtime data coming from the wellsites, the processing of the realtime data and the presentation of the data to users via the website. A joint systems and network team used the analysis from the network probes and investigated the end-to-end systems in detail. A number of bottlenecks and mis-configurations were identified.

The Result:
A number of the bottlenecks and mis-configurations were corrected and resulted in service improvements. Most importantly, the client and the developers learned in detail how the applications were working at the network level and have started work on a series of application improvements.

General Lessons from this Case Study:
This Oil Company was one of a number of Oil Companies making very farsighted and strategic investments to use realtime data in order to improve oilfield decision-making.

The overall system was, is and will be ambitious and remarkable — but it’s massively complex and it was a considerable challenge, even to the systems and application engineers, to adequately understand the system.

The overall system had been very thoroughly tested for functionality but not for scaling to very large volumes of data and large numbers of assets and wells.

The first lesson here is to add sophistication — and its associated complexity — slowly and to carefully measure, understand and document system behaviour at every stage of adding complexity. This lesson also applies to scaling the system in a careful and managed way.

The second lesson is to ensure that the design is fully understood at a software and dataflow level, not just at a hardware level. The OilCo in question had actually made a very good job of this critical requirement and, if they had not, then remedial work would have been much more difficult.

The third lesson is to carefully commission systems at each stage including the profiling of the network-level data flows so that any future departure from normal operations can quickly be diagnosed via abnormalities in data flows and a root cause identified.