Speeding things up with Segyio

[cmake] [GCC] [Python2] [Python3] [pip]

Seismic surveying generates a massive amount of data, which isn’t easily read without the proper software. When Research & Technology placed an order to Software Innovation in Bergen for a tool to make it easier to read and write seismic data, they got a lot more in return.

“The order was a standalone product that could write onto the data and to do this we would need a library that could make the data readable. But as we did our research, we couldn’t find any existing solutions that were good enough and we soon realized we had to create our own,” Jørgen Kvalsvik explains.

Because seismic surveys generate staggering amounts of data, you need the right software to be able to read and make sense of it. And to make sure they had the right tools, segyio was born.

While segyio might sound like a droid’s name, the explanation behind it is pretty simple: SEG-Y is the most common format for files containing seismic data and IO simply means “input output”. Simply put, it lets the user access massive amounts of data - and then read or write onto it.

“R&T had a tool available already, which was based on routines more than a decade old. Additionally, it had been coded in a language that made it very difficult to put into actual production and make it available to others,” he says.

Fancy a trip down memory lane? Then check out this 1950's video of the library component of EDSAC (Electronic Delay Storage Automatic Computer), the world’s first stored-program computer to operate a regular computing service:

Started from scratch

To put it simply a library, in terms of software, is written functionality or routines that can be easily reused for new data. Imagine you need to apply a certain function onto your seg-y file. Instead of writing it all by yourself you can head into segyio, find the function you need and have it written onto your data. Easy as 1-2-3!

To get a full grasp of how it all came to be we need to head back in time. The original order was placed during the summer of 2016, when Kvalsvik was asked if he could take it on as a smaller project.

“I was the only one at work when the order came in and the task was primarily to create a small library component. I said we could do it two ways; the quick and easy way, where they got exactly what they wanted, or the longer route, where we would take a step back and create a better and faster library component.”

Two men sitting by a desk and working on something on a computer — Instead of just improving the already existing product, Jørgen Kvalsvik (left) and colleague Lars Petter Øren Hauge decided that designing a new and faster library was a better solution.

(Photo: Torstein Lund Eik/Equinor)

R&D were on board with the idea of a faster component right from the start and work could then begin. Kvalsvik tells us that while the latest revision of the SEG-Y format uses more modern data representations, most data still complies with the older format and modernization is still slow-moving. Naturally, speed became a criteria.

“One of our criteria right from the start was that the component had to have a fast core that could be accessed in two different programming languages. So we took the already existing library part, switched it’s core with faster routines and kept it as a drop-in component."

Jørgen Kvalsvik

Now, the tool has been both available and in use by Research & Technology for well over a year.

“We created versions in both MATLAB and Python, where the latter has had significantly more users. The positive feedback has been that it’s a easy to use and a lightweight standalone tool that’s not part of some enormous suite,” Kvalsvik explains.

The team has seen that segyio has been used for more than what the order was, and it’s even got more functionalities than what was originally planned.

"We’re even getting feedback from users we’ve never spoken with previously, which is great fun," Jørgen says.

The design of both the interface and code turned out to be the biggest challenge on segyio.

A changing attitude

Even though SEG-Y is a well-established format, having been in existence since 1973, Kvalsvik was surprised to see how few public libraries of this kind were available.

“But I guess that it’s because most companies that have these tools are very proprietary and want to protect their own intellectual property. That’s a common attitude in the field,” Kvalsvik says.

But for Kvalsvik and the merry band of programmers it was actually the other way around: their client didn’t mind them working open source and neither did IT SI management. They have it in clear writing that open source should always be considered as the first option.

“I’ve noticed that the attitude towards software has changed from viewing it as an asset to a cost or a burden. This often results in leaning towards free software when you can, which has also become Equinor's strategy. For us it was simply the natural choice.”

A group of people gathered in front of a display during a video conference — The team at IT SI in Bergen are all eager to work open source, and believe it leads to better quality work.

(Photo: Torstein Lund Eik/Equinor)

segyio efficiency

faster than the previous tool

Contributing more than code

Kvalsvik emphasizes the opportunities that working open source brings. It’s more than just being a simpler way of distributing software or collaborating across offices, companies or even borders:

“People often forget that contributions are more than just code. Testing, user feedback and functionality requests are all incredibly valuable to the project. Qualified users going through the code and providing feedback means we get a better result in the end.”

Jørgen Kvalsvik

One example was a user that requested functionality with xarray, an open source project and Python package. Unfortunately, the team behind segyio didn’t have the funding available to complete the task.

“Then, the user wrote a NetCDF conversion that we didn’t know of, which meant that we could use our library as something we hadn’t thought of. This is one of the great aspects of working in open source; our software can be used in entirely new ways.” Kvalsvik says.

Jørgen explains the biggest advantages to working open source.

Full steam ahead

The amount of users and contributors to the project isn’t the only thing that makes segyio a project still worth keeping an eye out for in the future. If you want, you can even have a go at the tool from within the Debian operating system - a stamp of approval in itself. Yet the work is far from over and they have a long list of things they would like to do but would require a sponsor.

“There is usability needs we don’t know of yet, that we know will appear, as well as new products we can create here that will mean segyio has to grow. Other than that it’s about maintaining the speed, usability and staying on top of technology in order to provide a product that people will want to use,” Kvalsvik says.