In order to assist the pilot to safely operate helicopters in degraded visual environment (DVE), an integrated system of
various technologies that will provide improved situation awareness with minimal interpretation whilst controlling the
aircraft appears to be essential.
To determine the most effective and affordable solution set to enhance helicopter operations in DVE at low altitudes and
to help scope the potential range of solutions, the critical tasks of helicopters with the inherent requirements are to be
This paper will provide an overview on the operational environment, today procedures and the resulting general
requirements for operating helicopters low level in degraded visual environment.
Helicopter operations at low altitude are to this day only performed under VFR conditions in which safe piloting of the
aircraft relies on the pilot’s visual perception of the outside environment. However, there are situations in which a
deterioration of visibility conditions may cause the pilot to lose important visual cues thereby increasing workload and
compromising flight safety and mission effectiveness. This paper reports on a pilot assistance system for all phases of
flight which is intended to:
• Provide navigational support and mission management
• Support landings/take-offs in unknown environment and in DVE
• Enhance situational awareness in DVE
• Provide obstacle and terrain surface detection and warning
• Provide upload, sensor based update and download of database information for debriefing and later missions.
The system comprises a digital terrain and obstacle database, tactical information, flight plan management combined
with an active 3D sensor enabling the above mentioned functionalities. To support pilots during operations in DVE, an
intuitive 3D/2D cueing through both head-up and head-down means is proposed to retain situational awareness. This
paper further describes the system concept and will elaborate on results of simulator trials in which the functionality was
evaluated by operational pilots in realistic and demanding scenarios such as a SAR mission to be performed in
mountainous area under different visual conditions. The objective of the simulator trials was to evaluate the functional
integration and HMI definition for the NH90 Tactical Transport Helicopter.
Surface operation is currently one of the least technologically equipped phases of aircraft operation. The increased air
traffic congestion necessitates more aircraft operations in degraded weather and at night. The traditional surface
procedures worked well in most cases as airport surfaces have not been congested and airport layouts were less complex.
Despite the best efforts of FAA and other safety agencies, runway incursions continue to occur frequently due to
incorrect surface operation. Several studies conducted by FAA suggest that pilot induced error contributes significantly
to runway incursions. Further, the report attributes pilot’s lack of situational awareness - local (e.g., minimizing lateral
deviation), global (e.g., traffic in the vicinity) and route (e.g., distance to next turn) - to the problem. An Enhanced
Vision System (EVS) is one concept that is being considered to resolve these issues. These systems use on-board
sensors to provide situational awareness under poor visibility conditions. In this paper, we propose the use of an Image
processing based system to estimate the aircraft position and orientation relative to taxiway markings to use as lateral
guidance aid. We estimate aircraft yaw angle and lateral offset from slope of the taxiway centerline and horizontal
position of vanishing line. Unlike automotive applications, several cues such as aircraft maneuvers along assigned route
with minimal deviations, clear ground markings, even taxiway surface, limited aircraft speed are available and enable us
to implement significant algorithm optimizations. We present experimental results to show high precision navigation
accuracy with sensitivity analysis with respect to camera mount, optics, and image processing error.
Helicopters experience nearly 10 times the accident rate of fixed wing platforms, due largely to the nature of their
mission, frequently requiring operations in close proximity to terrain and obstacles. Degraded visual environments
(DVE), including brownout or whiteout conditions generated by rotor downwash, result in loss of situational awareness
during the most critical phase of flight, and contribute significantly to this accident rate. Considerable research into
sensor and system solutions to address DVE has been conducted in recent years; however, the promise of a Synthetic
Vision Avionics Backbone (SVAB) extends far beyond DVE, enabling improved situational awareness and mission
effectiveness during all phases of flight and in all visibility conditions. The SVAB fuses sensor information with high
resolution terrain databases and renders it in synthetic vision format for display to the crew. Honeywell was awarded the
DARPA MFRF Technical Area 2 contract in 2011 to develop an SVAB1. This work includes creation of a common
sensor interface, development of SVAB hardware and software, and flight demonstration on a Black Hawk helicopter. A
“sensor agnostic” SVAB allows platform and mission diversity with efficient upgrade path, even while research
continues into new and improved sensors for use in DVE conditions. Through careful integration of multiple sources of
information such as sensors, terrain and obstacle databases, mission planning information, and aircraft state information,
operations in all conditions and phases of flight can be enhanced. This paper describes the SVAB and its functionality
resulting from the DARPA contract as well as Honeywell RD investment.
NASA Langley Research Center and the FAA collaborated in an effort to evaluate the effect of Enhanced Vision (EV) technology display in a commercial flight deck during low visibility surface operations. Surface operations were simulated at the Memphis, TN (FAA identifier: KMEM) airfield during nighttime with 500 Runway Visual Range (RVR) in a high-fidelity, full-motion simulator. Ten commercial airline flight crews evaluated the efficacy of various EV display locations and parallax and minification effects. The research paper discusses qualitative and quantitative results of the simulation experiment, including the effect of EV display placement on visual attention, as measured by the use of non-obtrusive oculometry and pilot mental workload. The results demonstrated the potential of EV technology to enhance situation awareness which is dependent on the ease of access and location of the displays. Implications and future directions are discussed.
The paper describes the approach of a sensor based landing aid for helicopters in degraded visual conditions. The system
concept presented employs a long range high resolution ladar sensor allowing for identifying obstacles in the flight and
in the approach path as well as measuring landing site conditions like slope, roughness and precise position relative to
the helicopter during long final approach. All these measurements are visualized to the pilot. Cueing is done by 3D
conformal symbology displayed in a head-tracked HMD enhanced by 2D symbols for data which is perceived easier by
2D symbols than by 3D cueing. All 3D conformal symbology is placed on the measured landing site surface which is
further visualized by a grid structure for displaying landing site slope, roughness and small obstacles. Due to the limited
resolution of the employed HMD a specific scheme of blending in the information during the approach is employed. The
interplay between in flight and in approach obstacle warning and CFIT warning symbology with this landing aid
symbology is also investigated and exemplarily evaluated for the NH90 helicopter which has already today implemented
a long range high resolution ladar sensor based obstacle warning and CFIT symbology. The paper further describes the
results of simulator and flight tests performed with this system employing a ladar sensor and a head-tracked head
mounted display system. In the simulator trials a full model of the ladar sensor producing 3D measurement points was
used working with the same algorithms used in flight tests.
Piloting a rotorcraft in a Degraded Visual Environment (DVE) is a very complex task, and the evolution of the rotorcraft
missions tend to augment the probability of such degraded flight conditions (increase of night flights, all-weather flights,
with brownout or whiteout phenomena…). When the direct view of the external situation is degraded, the avionic system
can be of great help for the crew to recover the lost visual references. TopOwl® Head Mounted Sight and Display
(HMSD) system is particularly adapted to such situations, allowing the pilot to remain "eyes-out" while visualizing on a
large field of view different information: a conformal enhanced image (EVS) coming from an on-board sensor, various
2D and 3D symbologies (flight, navigation, mission specific symbols), a conformal synthetic representation of the terrain
(SVS), a night vision image coming from the integrated Image Intensifier Tubes, or a combination of these data,
depending on the external conditions and the phase of flight, according to the pilot’s choice.
One of the major causes for hazardous situations in aviation is the lack of a pilot’s situational awareness. Common
causes for degraded situational awareness are Brownout and Whiteout situations, low level flights, and flights in DVE.
In this paper, we propose Advanced Synthetic Vision (ASV), a modern situational awareness solution. ASV combines
both Synthetic Vision and Enhanced Vision in order to provide the pilot most timeliness information without being
restricted in the spatial coverage of the synthetic representation. The advantages to a common Enhanced Synthetic
Vision System are the following: (1) ASV uses 3D ladar data instead of a 2D sensor. The 3D point cloud is classified in
real-time to distinguish between ground, wires, poles and buildings; (2) the classified sensor data is fused with onboard
data base contents like elevation or obstacles. The entire data fusion is performed in 3D, i.e. output is a merged 3D
scenario instead of a blended 2D image. Once the sensor stopped recording due to occlusion, ASV switches to pure data
base mode; (3) the merged data is passed to a 3D visualization module, which is fully configurable in order to support
synthetic views on head down displays as well as more abstract augmented representations on helmet mounted displays;
(4) the extendable design of ASV supports the graphical linking of functions like 3D landing aid, TAWS, or navigation
The OPAL obscurant penetrating LiDAR was developed by Neptec and characterized in various degraded visual
environments (DVE) over the past five years. Quantitative evaluations of obscurant penetration were performed using
the Defence RD Canada – Valcartier (DRDC Valcartier) instrumented aerosol chamber for obscurants such as dust and
fog. Experiments were done with the sensor both at a standoff distance and totally engulfed in the obscurants. Field
trials were also done to characterize the sensor in snow conditions and in smoke. Finally, the OPAL was also mounted
on a Bell 412 helicopter to characterize its dust penetration capabilities, in environment such as Yuma Proving Ground.
The paper provides a summary of the results of the OPAL evaluations demonstrating it to be a true “see through”
obscurant penetrating LiDAR and explores commercial applications of the technology.
Proc. SPIE 8737, MMW radar enhanced vision systems: the Helicopter Autonomous Landing System (HALS) and Radar-Enhanced Vision System (REVS) are rotary and fixed wing enhanced flight vision systems that enable safe flight operations in degraded visual environments, 87370G (16 May 2013); doi: 10.1117/12.2016302
Sierra Nevada Corporation (SNC) has developed rotary and fixed wing millimeter wave radar enhanced vision systems.
The Helicopter Autonomous Landing System (HALS) is a rotary-wing enhanced vision system that enables multi-ship
landing, takeoff, and enroute flight in Degraded Visual Environments (DVE). HALS has been successfully flight tested
in a variety of scenarios, from brown-out DVE landings, to enroute flight over mountainous terrain, to wire/cable
detection during low-level flight. The Radar Enhanced Vision Systems (REVS) is a fixed-wing Enhanced Flight Vision
System (EFVS) undergoing prototype development testing. Both systems are based on a fast-scanning, threedimensional
94 GHz radar that produces real-time terrain and obstacle imagery. The radar imagery is fused with
synthetic imagery of the surrounding terrain to form a long-range, wide field-of-view display. A symbology overlay is
added to provide aircraft state information and, for HALS, approach and landing command guidance cuing. The
combination of see-through imagery and symbology provides the key information a pilot needs to perform safe flight
operations in DVE conditions. This paper discusses the HALS and REVS systems and technology, presents imagery, and
summarizes the recent flight test results.
Areté Associates recently developed and flight tested a next-generation low-latency near real-time dust-penetrating
(DUSPEN) imaging lidar system. These tests were accomplished for Naval Air Warfare Center (NAWC) Aircraft
Division (AD) 4.5.6 (EO/IR Sensor Division) under the Office of Naval Research (ONR) Future Naval Capability (FNC)
Helicopter Low-Level Operations (HELO) Product 2 program. Areté’s DUSPEN system captures full lidar waveforms
and uses sophisticated real-time detection and filtering algorithms to discriminate hard target returns from dust and other
obscurants. Down-stream 3D image processing methods are used to enhance pilot visualization of threat objects and
ground features during severe DVE conditions. This paper presents results from these recent flight tests in full brown-out
conditions at Yuma Proving Grounds (YPG) from a CH-53E Super Stallion helicopter platform.
Image registration is a process of transforming a data set from one coordinate system into another. There are two
typical approaches for image registration: Feature point match based and Area similarity comparison based. The
feature point match based approach, using points to establish the correspondence between two images, is relatively
fast, but it involves feature extractions and parameter selection to create feature points. Feature extractions involve
derivatives which are ill-posed problems and may lead to robustness issues. The area similarity comparison based
approach compares intensity patterns using a correlation metric such as normalized cross correlation (NCC). Since
it does not require feature extraction, is simple and not sensitive to noise. However its computational cost is high.
Even when some fast techniques like FFT are used to reduce the computational cost, the implementation is still time
In this paper, we propose a diffusion equation and normalized cross correlation (NCC) combined method to perform
robust image registration with low computational cost. We first apply the diffusion equation to two images received
from two sensors (or the same sensor) and allow these two images to evolve by this diffusion equation. Based on
the characteristics of evolutions, we select a very small percentage of stable points in the first image and perform the
normalized cross correlation to the second image at each transformation point. The highest NCC point provides the
transformation parameters for registering these two images. This new method is resistant to noise since the
evolution of the diffusion equation reduces noise and it chooses only stable points for the NCC computation.
Furthermore, the new method is computationally efficient since only a small percentage of pixels involve in the
transformation estimation. Finally, the experiments for video motion estimation and image registration are provided
to demonstrate that the new method is able to estimate the registration transformation reliably in real time.
In this paper, we present an algorithm for obtaining an original image which becomes blurred due to the size of
the aperture of the CMOS camera module in capturing images of the objects near the camera. We introduce the
mathematical property of a circulant matrix which can be used to describe the PSF and propose a new algorithm
based on this matrix. We suggest new algorithms for both one-dimensional and two-dimensional signal processing
case. These proposed algorithms were validated by the results of computer simulation for two-dimensional images
synthesized by a CMOS camera model which was based on a pinhole camera model previously proposed by our
A novel spatial domain image enhancement algorithm, in which dynamic range of the scene illumination is compressed from the human visual perspective to improve the visual quality and visibility in digital images captured under degraded visual conditions, is proposed. The proposed algorithm employs an adaptive approach so that local image statics, namely the local standard deviation and the local mean in the image are modified simultaneously utilizing an intensity transform based on a “S” shape curve, the curvature parameter of which is determined adaptively followed by a local contrast enhancement process. The performance of the algorithm is evaluated by a statistical visual measure, along with visual comparisons of the proposed method with state-of-the-art enhancement algorithms are given.
Supporting a helicopter pilot during landing and takeoff in degraded visual environment (DVE) is one of the challenges
within DLR's project ALLFlight (Assisted Low Level Flight and Landing on Unprepared Landing Sites). Different types
of sensors (TV, Infrared, mmW radar and laser radar) are mounted onto DLR’s research helicopter FHS (flying
helicopter simulator) for gathering different sensor data of the surrounding world. A high performance computer cluster
architecture acquires and fuses all the information to get one single comprehensive description of the outside situation.
While both TV and IR cameras deliver images with frame rates of 25 Hz or 30 Hz, Ladar and mmW radar provide georeferenced
sensor data with only 2 Hz or even less. Therefore, it takes several seconds to detect or even track potential
moving obstacle candidates in mmW or Ladar sequences. Especially if the helicopter is flying with higher speed, it is
very important to minimize the detection time of obstacles in order to initiate a re-planning of the helicopter’s mission
timely. Applying feature extraction algorithms on IR images in combination with data fusion algorithms of extracted
features and Ladar data can decrease the detection time appreciably. Based on real data from flight tests, the paper
describes applied feature extraction methods for moving object detection, as well as data fusion techniques for
combining features from TV/IR and Ladar data.