The subsurface is a key player in both extracting oil and gas and for storing captured carbon. Superman isn’t currently an Equinor employee, so we don’t have anyone with x-ray vision and superpowers to tell us what secrets lie beneath the Earth’s surface. What we do have, however, is a whole lot of data to help us know more. 

We do this by running advanced calculations in a system called Fast Model Update, FMU for short. These calculations produce vast amounts of data in the shape of model results. By one press of a button, hundreds of thousands of files can be generated by our FMU users.

Some of the model results are used in further work, but most of them end up being stored on temporary disk space in folder structures, until they're written over by new data. The large amount and its cumbersome structure have made it challenging to make more use of the data. Now, that’s about to change – and a key part in the work is Sumo:

Splitting to increase traffic

In the early days of Sumo, many different technologies and ways of solving problems were tried, but none were a perfect fit – until software developers came on board with an idea.

The first leg of the journey is enabling FMU to push data into Sumo automatically. As there are hundreds of thousands of data outputs from an FMU workflow, including manual steps is simply not on the table.

“When I first heard of what Sumo wanted to do, a search engine approach seemed like the obvious way of doing it. While they used to require massive resources before, now they’re packaged technology you can implement almost anywhere,” Raymond explains. 

Together with Øistein Haaland, Raymond is one of the original Sumo software developers. If you’re going to search through petabytes of data, you need metadata to help the search engine find the results you're after.

Creating new possibilities

While data is uploaded through Sumo together with corresponding metadata, such as what field the data is for, they don’t end up in the same spot. The data heads to an Azure Blob Storage, and the metadata head off into Elastic Search - their search engine of choice. Sumo has a dedicated, private link to Elastic Search, which helps in many aspects.

“This direct connection means we don’t have to route anything across nodes on the Internet and makes it a lot faster. It also means it’s more secure and not accessible for every random Internet user,” Raymond explains.

Designing a dashboard

Ådne Aarthun Jacobsen joined the Sumo team in August of 2021 and has been working on the frontend where improvements have been made with more on the way. The frontend mainly runs on React components and consists of elements from the Equinor Design System (EDS).

The challenge of designing a frontend for Sumo is that it must be easy to use and understand - preferably without any training. Consulting with UX designers, and a lot of trial and error leads the way to finding the best possible solution.

Calculating a distribution of surfaces 

Having made all this data accessible through Sumo, the possibility of building services to make more use of them opens. One example is the aggregation service. 

This uses the data that Sumo uploaded to do a wide range of calculations on reservoir model predictions in the form of surfaces, one of many data types generated by FMU (Fast Model Update). 

"We have to think of this as a distribution of surfaces, similar to how we think of distributions of numbers,” Daniel Garip says.

This means describing that distribution in terms of statistical representations such as the mean, max/min, P10, P90 etc.  

“Previously, these statistics have been calculated for all output as part of the model run. With model results available in Sumo, however, we can create a service that produces what we need, when we need it,” Daniel says.

Daniel has been working on the surface aggregation service together with Dafferianto Trinugroho, and building the aggregation service required a whole lot of testing, Daniel explains:

“I’ve tested the service in different environments like Radix and Azure, and it was quite the experiment to find the right balance between performance and costs when deciding where to host it,” Daniel says and adds:

Increasing speed by more than 50%

While Golang is a relatively new programming language, it’s proven to be a solid choice for the aggregation service due to its ability to do things simultaneously without sacrificing speed - thanks to Goroutines.

“We’ve really put an emphasis on maintaining a high speed for the aggregation service. Adding a step of data validation could add milliseconds, which could really slow down the result. It’s been a really interesting focus to have,” Daniel says.

During development, Daniel and Dafferianto would each work on different tasks, but they did code review as pair programming. 

Understanding the bigger picture

Getting an understanding of how reservoir models work, what metadata it needs or spotting what kind of possibilities the data provide us with is no easy task. Luckily, the Sumo team has got experts ready to answer their questions and help when needed. One of them is the product owner, Per Olav Eide Svendsen. 

“Per Olav has done a great job in many areas. He created a specification for what metadata FMU needed to apply to the data, and he knows what the users need and want – and if he doesn’t know, he’ll find out. This helps ensure we won’t work on something that the users don’t want or need,” Raymond explains.

Developing Sumo has also meant learning a great deal about the world of reservoirs and the subsurface.

Now, Sumo can enable data from FMU to be integrated into Equinor services like the Reservoir Experience Platform and Webviz. They both use Sumo to visualize different reservoir model results, but the team keeps close contact with several other services to create other use cases.

Rebuilding roads while driving

Work on Sumo also means that the FMU community is facing quite a change story. Not only do they have to deal with the sheer amount of data, but they also need to include metadata and refer to master and reference data for every single data output. A significant undertaking, especially when considering that FMU is a complex system and that it is in active use, every day, on more than 40 assets in Equinor.

"We are reconstructing the roads while a lot of people are driving on them, which is quite challenging,” says Jan C. Rivenæs. 

Jan is heading the implementation of metadata functionality for FMU results and together with his colleagues, he is key to success for Sumo

"The FMU community is both our primary customers, and our most important collaborators. This is not about building and delivering a component. It’s about collaboration and continuously improving so that we can utilize advances in technology and so that we remain competitive," Per Olav says. 

Creating value for FMU is key and ensuring that it keeps its flexibility has been a key requirement for the development of Sumo.