Augmented reality and data visualization (in aviation)

Present-day technology is so powerful that the perception of reality can be easily and realistically modified with IT tools, providing users withan experience beyond “simple” reality. This is achievable by mixing real-world environment elements supplemented and/or augmented by computer-generated inputs. The current post unpacks this topic, focusing specifically on the data visualization aspects. In brief, augmented reality can take two approaches:

• First, inventing totally new scenarios, in which the user becomes part of a “parallel universe”. Supplementing the real-world environment with an unreal one; either a virtual place (video game) or a different location (i.e. another real location). This is the case of futuristic 90’s and early 2000’s alike head-mounted displays with users’ eyes looking at full screens recreating other places. The ergonomics aspects are usually modest for most of the applications due to the head-mounted displays weight and size.

• The second, and closer to “data visualization” area is the so called “mediated reality”. The real-world environment enhanced by virtual elements displayed in glasses, windscreens etc. In them, additional information/data is provided. The real challenges is to decide what, how and when to display the information, without requiring users to look away from their usual viewpoints, while providing extra value. The integration and user experience is much more natural and enjoyable than the fully immersive systems.

In this context, one of the very early examples of head-up displays can be found precisely in aviation, almost 80 years ago. In 1937, the German ReviC12/A fighter aircraft included a basic reflector sight indicating some basic aircraft magnitudes such as speed and turn rate, to reduce the (visual) workload of pilots in case of extreme maneuvering

Nowadays virtually all modern fighters (F18, F16, Eurofighter) use head-up displays. The most modern versions (F35) do not have head-up displays, and instead include helmet mounted displays, ensuring the proper orientation of the user’s head, for all circumstances.


One of the trending topics of augmented reality within aviation is its usage in air traffic control (ATC), particularly in Tower environments. Below are two common approaches:

• Visual information is enhanced to ease identification and tracking of aircraft. This includes tools similar to head-up displays and/or helmets-displays that enhance the information (providing for instance, aircraft ID, scheduled times, etc). This approach could be extremely useful in low visibility conditions by facilitating the tower ATCOs tasks. It also avoids dividing attention between the primary visual field (the window) and the auxiliary tools (surface radar, strips etc).

• The extreme version is a complete virtual control tower, the so called “remote tower”. ATC would have remote control rooms with video-sensors on-site, including augmented reality enhancements. The synthetic augmentation of vision increases the situational awareness at the airport, especially during poor visibility conditions, or blocked line-of-sight areas due to airport geometry. It additionally provides benefits in terms of cost saving (no need to build and maintain control tower facilities) and a more efficient use of human resources (potentially serving multiple airports with low traffic events from a centralised location). Research in this field started in FP6 project “ART” and is now being progressed by SESAR WP6. In fact, Örnsköldsvik/Gideå airport is the first on the world deployment of remote tower, in late 2015, by the Swedish LFV. In US, Fort Collins-Loveland Municipal Airport was the first approved and tested airport with a remote tower in 2016.

For the air passenger and mobility context, augmented reality and the wide range of solutions providing additional real-time information to passengers is taking off as well. (No pun intended.)
These technological innovations include indoor location tracking, real-time information on boarding gates, real-time updates on flight delays, and information on airport facilities and shops. This is also being expanded to knowing the number and location of available parking spaces to facilitate the passenger experience in the (sometimes not so easy) airport processes. For example, Copenhagen airport, in collaboration with SITA, created the very first augmented reality indoor app in 2012. Now there is an endless list of both airlines and airports with similar apps.

Do you think augmented reality together with innovative data visualization can have a significant impact in future aviation?
What are its challenges and potential benefits?
We’re interested in hearing your thoughts and ideas.