Fo.tone - a journey of sound
Thanks to engineering, data engineering and software development, we can use fiber optic cables to livestream sounds from the subsurface, hundreds of kilometers out at sea. Let's meet Fo.tone!
Listening to these sounds means we can reduce downtime on our wells and run them more efficiently. It’s been a dream for quite some time, and when the Fo.tone-team started in 2019 it had been seemingly out of reach.
“We had to create a minimum viable product in a matter of months at a time when no one had succeeded in doing this before. We knew what we had to do, but not how to do it."
Kjetil Tonstad, Fo.tone team lead
Fast forward to 2021 and here’s Fo.tone: a web app that handles live stream data from offshore fiber optic cables, presents it in a display with other wellbore data and enables a whole new world of information.
“Although we were an experienced team, everyone’s had a steep learning curve along the way, and I’m very proud of how far we have come as an agile team. We have gone all the way from just being tasked with creating an MVP to now having a functional application that can bring a lot of value,” Kjetil smiles.
Plugged into the source
We know you’re eager to hear about how they did it and what world of possibilities this can open for Equinor, but first there are a couple of things that quickly need some explanations and acknowledgments.
While this story focuses on Fo.tone and the team behind it, this is only one part of the Fiber Optic Data Pipeline that spans across different areas of Equinor: GBS IT, Drilling & Well, Research & Technology and Petroleum Technology.
The aim is to gather more data on how the wells flow and operate, for example how much oil flows into different parts of the well.
“Reservoir- and production engineers having access to this kind of data means they can optimize production in the wells. When every well produces for tens of millions every day, one can just imagine the potential impact. It is really exciting to lead the development of a solution with such significance.
Stay in the Loop
Breaking new ground
With the basics of what the Fiber Optic project tries to achieve out of the way, we can focus on the work done by the Fo.tone team and the first station is back-end.
In the beginning, the team had two main issues on their agenda. The first was being able to stream the data in near real-time, while the other was to make sure you could search and interact with the data in an efficient way.
“No one had done anything like this previously in Equinor, so we didn’t have any previous work to lean on. There was a lot of research, trial and error in the beginning.”
John Harald Brattåker, back-end tech lead
Profile types, feature types and features
Keeping latency down
One of the main driving forces for the back-end architecture was the front-end. The front-end team’s vision was to create a tool that worked a little like Google Maps: allowing users to view an overview of data and then zoom into the details.
“Making all of this data available for immediate queries and processing “on the fly” has been the biggest challenge. Keeping the latency down was crucial, since no one would want to use a laggy or slow software,” John Harald explains.
And the latency has been kept down, providing the 85 users with a quick and responsive front-end for them to dive into data.
“Now that we have the data available and visualized, we can continue working on finding areas of use for them. We’ve already created an app that “listens” to the data and lets us know when a valve opens or closes in near real-time,” John Harald says.
“Previously, this is something we wouldn’t have been able to directly measure until weeks after it happened. Now, we can know right away and help reduce the downtime for our wells."
John Harald Brattåker
Reducing downtime doesn’t just mean an increase in production - it also means more efficient wells with less emissions.
We have installed 150 kilometers of fiber optic strands at Johan Sverdrup alone.
Turning data-dense into responsive
When using the actual web app, you’ll be looking at the heatmap - something that looks more like 70’s wallpaper than data.
“This map is a key part of the application, allowing users to see - and listen to - what the fiber optic cables are picking up on, but that isn’t the only data you have access to,” front-end tech lead Frits Talbot tells us.
“Fo.tone also integrates production data, as well as apps that run on top of all these data, so it’s fair to say it's a very data-dense app. Finding a way to display all these different types of data in a way that made sense as well as being responsive is a real challenge,” Frits says.
“We wanted to make it responsive so that you could freely look up the data you need and personalize the app to your needs. Different types of users will look at different types of data, so it was a challenge to piece it all together.”
Frits Talbot, front-end tech lead
The amount of data sent from offshore to our processing plant equals 10,000 Netflix movies - each second.
There’s little room for a one-trick pony when working on Fo.tone, and the team had to juggle a wide variety of skills; from data engineering and data science to signal processing and computer graphics knowledge.
“And within all these disciplines, we also had to figure out how to do these things in a web application, and how to best balance the work between backend and frontend,” Frits says.
“We quickly learned to process and deliver the raw data into audio for playback using the WebAudio API. Building on that experience, we used Web Workers to speed up computations (e.g. median filtering, STFT display) with parallel and asynchronous processing."
Fo.tone does have certain use cases clearly defined, but there’s still a lot of undiscovered potential in the data it provides and as such the team always focus on sharing knowledge and learning.
One time this approach proved useful was when they got a use case saying that users needed to be able to see different frequency bands at the same time.
“The heat map shows a single frequency band of sound at a time and the case was that they wanted to look at several of them at the same time, because different physical phenomena or events respond differently across the frequency bands,” Frits explains.
“My initial thought was that this would draw too much processing power from the browser, as well as create significant challenges with layout. Then I remembered coming across a concept named broadcast channels while studying the Mozilla Developer Network documentation,” he adds.
He investigated further and then suggested that taking this approach would be the way to go. Users would open the application in multiple tabs or windows, choosing different frequencies (or temperature) in each, finally linking their time and depth ranges.
The applications would now in effect be running as separate processes, allowing more efficient scheduling. Broadcast channels would then automatically be initiated between the application processes and messages would start flowing between them so that they could behave in concert.
“Then, as they zoomed or made changes to their selection, each of the maps would update themselves accordingly even if they were on separate screens,” Frits says.
“One of our users told us he has four separate screens connected and thanks to this approach he can use them with the browser still remaining responsive."
The fiber optic cables in the North Sea are so sensitive they could "hear" an earthquake in Turkey in 2020.
Dedicating time for competence sharing
Starting out, the team focused on learning and sharing competence within the team, but it didn’t stop there. This way of working has followed them since the start and is still a major part of work, developer Aisha Akram tells us.
“We always spend time on pull request reviews and have since the start. Not just as a formal thing that we must do, but we at times spend an entire day just on improving the code and working on pull request comments. There’s a lot of learning to be found by looking into the code."
Work is divided into two-week long sprints, but whenever they start on a new user story the team makes sure there is time specifically for “moral support” and pair programming.
“It’s in case you’re working on something you’re not familiar with, and means you have a given time to ask someone with more experience if you have any questions. This way of doing it has been efficient and helped us solve user stories much quicker than usual,” Aisha tells us.
There’s a lot to learn about all aspects of development and being a full stack team means they can even “switch” between the front and back-end if they want. This means that they’re always learning even if you’ve been working on the team since the start - or as a graduate.
“It’s a very complex application and the back-end part is almost like magic to me but as a graduate it’s an excellent team to learn from and be part of.”
She joined the team mere two months ago, and while she’s been working little on her own in order to get a feel for the project first - she’s now working on code by herself.
“It would have been really hard if I was left all by myself, but Frits, Aisha and everyone else will always help if needed. When I’m stuck, they’ll come up with a suggestion and it’s almost like a light bulb appears above my head,” Zuzanna laughs.
“I’m always learning something new when working on Fo.tone. In the future, there’s talk of implementing more machine learning tools on top of the data - which is something I’ve worked on during my studies. As a graduate it’s a great learning experience."
Hit the subscribe button below to stay in the Loop on all thing's software development in Equinor. That way you’ll have new stories flying into your inbox as soon as they’re published.
Until next time, stay safe and take care!
Kjell Erik Reed Anda
John Harald Brattåker
Zuzanna Paula Materny
Ådne Elias Moldesæter