FIELD OF THE INVENTION
The present invention generally relates to cockpit displays and more particularly, relates
to a method and a system for retrieving and storing cockpit display data of an aircraft.
BACKGROUND
It is commonly known that during any flight accident or unexpected tragic incident
involving an aircraft, a flight recorder is the primary source of information, which is used for
analyzing the reasons for such accidents. The flight recorder is an electronic device placed in an
aircraft to acquire and record important information about the operation and status of the aircraft
during a flight. Currently, flight recorder include a Flight Data Recorder (FDR) and a Cockpit
Voice Recorder (CVR). The FDR is used to record various flight/aircraft parameters such as
engine status, fuel status, airspeed, altitude, attitude, control settings etc., and the CVR is used to
record audio environment in a flight deck/cockpit of the aircraft. Thus, the flight recorder stores
the voice recordings of the cockpit area and flight deck which provides an insight to the
conversation being exchanged in the minutes leading to and just prior to the incident. Further, the
flight recorder stores various other parameters relating to flight information and parameters from
data aggregated through various sensors.
In the event of an accident or an unexpected incident involving the aircraft, the
information recorded in the flight recorder is used to aid an investigation by the investigating
into the possible cause of the accident or the unexpected event and can also be used for general
monitoring. The analyzed data also assists in preparing for situations similar to the accident
and/or event in the future. However, existing flight recorders have a major limitation since they
record only aircraft parameters and audio communications taking place in the cockpit of the
aircraft because the data obtained from these flight recorders may not be sufficient to analyze the
accident scenario with the specificity required to understand the cause of the incident.
Particularly, there is no provision in the flight recorders to record the vital information displayed
on one or more displays in the cockpit of the aircraft.
2
One solution to overcome the aforementioned problem would be to use video cameras in
the cockpit of the aircraft to capture images/video of the display(s) in the cockpit and to record
the data/readings displayed on the displays at any given time during the flight. For example, U.S.
Patent No. US 5742336 A describes a method for recording images of instrument panels using a
camera. However, such a solution/system suffers from various limitations/drawbacks such as the
video frames/images recorded might be blurred or of low quality, or the video camera view
might not be able to capture all the displays due to blocked view caused by the movement of the
flight crew/pilots in the cockpit area. Further, storing the video data recorded for the whole flight
duration requires a large memory storage that may not be readily available in storage that is
present in the flight recorder system. Additionally, from a personnel standpoint, there have been
concerns raised by the pilot unions that placing video cameras in the aircraft/cockpit may lead to
invasion of privacy of the pilots and other crew members. Therefore, the video camera solution
is infeasible.
Hence, there is a need to reconstruct the display data in addition to data from the FDR
and CVR so that investigative agencies can reproduce information as close as possible to what
happened irj the cockpit just before an incident. Further, there is also a need to collect such
display data and FDR/CVR data and to use the same for pilot training or to, for example, conduct
random audit of the cockpit during a flight.
Accordingly, in order to overcome the aforementioned limitations, there is a need of a
method and a system, wherein the cockpit display data is efficiently recorded and easily
accessible, particularly in the event of an accident or an unexpected incident involving the
aircraft. Also, there is a need of a system, wherein the cockpit display data is recorded in a way
such that the recorded data does not occupy large memory making storage and retrieval very
efficient.
SUMMARY OF THE INVENTION
It is an objective of the present invention to provide a system for storing cockpit display
data, said system comprising at least one sensor to measure at least one flight parameter, a data
3
acquisition unit (DAU) comprising at least one data acquisition module (DAM) associated with
said sensor for generating graphics commands based on the at least one measured flight
parameter received from the at least one sensor, a graphics server (GS) coupled to said data
acquisition unit for receiving and processing said graphics commands, a Graphics Engine (GE)
coupled to said graphics server (GS) for receiving the processed commands and generating pixel
data that would be displayed on cockpit display units in the form of graphics, and a port
operatively connected to said graphics engine (GE) and an external memory coupled to said
graphics engine (GE) via said port for retrieving and storing the pixel data being generated by
said graphics engine (GE).
It is yet another objective of the present invention to provide a processing unit coupled
between said graphics engine (GE) and said external memory, said processing unit being
configured to sample, select and compress the pixel data retrieved from said graphics engine
(GE), in order to process and store only non-redundant pixel frames thereby occupying less
memory space in the external memory.
I
It is yet another objective of the present invention to provide a method for storing the
cockpit display information, comprising at least the steps of measuring at least one aircraft
parameter by at least one sensor, generating graphics commands by the at least one data
acquisition module (DAM) associated with the at least one sensor based on the at least one
measured parameter received from the at least one sensor, receiving and processing said graphics
commands by a graphics server (GS) coupled to said data acquisition module (DAM), receiving
the processed graphics commands by a graphics engine (GE) coupled to said graphics server
(GS) to generate pixel data that would be displayed on cockpit display units in the form of
graphics, retrieving the pixel data from the graphics engine (GE) via a port operatively connected
to said graphics engine and storing the retrieved pixel data in an external memory coupled to said
graphics engine (GE) via said port.
It is yet another objective of the present invention to provide a method for processing the
pixel data retrieved from said graphics engine (GE), said process comprising at least the steps of
sampling, selecting and compressing the pixel data, to store only non-redundant pixel frames
4
thereby occupying less memory space in the external memory.
To foregoing and other objects, features and advantages of the invention will be apparent
from the following detailed description in conjunction with the drawings described hereinafter. It
is to be appreciated that these drawings depict only typical embodiments of the invention and are
therefore not to be considered limiting in its scope.
BRIEF DESCRIPTION OF THE DRAWINGS
Figure 1 illustrates a flight data acquisition and display system of an aircraft;
Figure 2 illustrates a cockpit display retrieving and storing system according to an
embodiment of the present invention;
Figure 3 illustrates the method of transferring pixel data from a graphics engine to an
external memory according to an embodiment of the present invention;
Figure 4 is a flowchart illustrating the process of retrieving and storing the cockpit
display data according to the embodiments of the present invention;
Figure 5 is a flow chart illustrating the process of retrieving and processing pixel data
according to an embodiment of the present invention.
A more complete understanding of the present invention and its embodiments thereof
may be acquired by referring to the following description and the accompanying drawings.
DETAILED DESCRIPTION OF THE PRESENT INVENTION
Figure 1 illustrates a flight data acquisition and display system 100 of an aircraft. The
flight data acquisition and display system 100 comprises plurality of sensors (101-1, 101-
2...101-n) which are configured to measure one or more parameters of aircraft flight between
5
two destinations, such as airspeed, altitude, temperature, etc. According to the avionics
industry's norms, the flight data acquisition system gathers and records approximately, 240
different aircraft parameters. The flight data acquisition and display system 100 further comprise
a flight data acquisition unit (FDAU) 102 which is coupled to one or more sensors (101-1, 101-
2 101-n) and is configured to retrieve the data sensed by the said plurality of sensors (101-1,
101-2 101-n).
A flight recorder 103 is coupled with said flight data acquisition unit 102 and is
configured to process and store the data obtained through one or more sensors. The flight
recorder 103 comprises a flight data recorder (FDR) 103-a which is in communication with the
FDAU 102 and is configured to store all the flight parameter retrieved from the sensors (101-1,
101-2 101-n). The FDR 103-a includes a solid state digital memory device for storing the
recorded flight parameters. The FDR 103-a may be additionally equipped with various transfer
media such as an optical disc or floppy disc for downloading data stored in a digital memory
device of the FDR 103-a. The digital memory device of the FDR 103-a may also be directly
accessed via a port which can connect to a portable reader device or personal computer. The
flight recorder (103) further comprises a cockpit voice recorder (CVR)) 103-b which is designed
and configured to store audio recording inside the flight deck and cockpit area via microphones
placed at strategic/predetermined locations throughout the cockpit. The CVR 103-b is operative
to convert audio signals received from the microphones and converted into a format suitable for
storage. The FDR 103-a and CVR 103-b together constitute the fl ight recorder 103 of an aircraft.
Inside the cockpit area, one or more cockpit display units 104 are provided which are
configured to retrieve the data from the FDAU 102 and display the same to pilots inside the
cockpit. The cockpit display units 104 primarily comprise a primary flight display (PFD) unit
and a multi-function display (MFD) unit (not shown in Fig.l). The PFD unit displays all
information critical to a flight, including calibrated airspeed, altitude, heading, attitude, vertical
speed and yaw. The MFD unit displays navigational and weather information from multiple
systems and is most frequently designed as "chart-centric", where aircrew can overlay different
information over a map or chart. The MFD unit can also be used to view other non-overlay type
6
of data (e.g., current route plan) and calculated overlay-type data. The MFD unit can also display
other information about aircraft systems, such as fuel and electrical systems.
Figure 2 illustrates a cockpit display retrieving and storing system 200 according to an
embodiment of the present invention. The cockpit display retrieving and storing system 200
comprises a graphics server (GS) 202 that is a processing apparatus and preferably implemented
as an integrated circuit and, in particular, as an FPGA (field programmable gate array). In an
exemplary embodiment, the graphics server (GS) is based on an AR1NC661 standard, an
industry standard for normalizing/standardizing the definition of the cockpit display system 200.
The cockpit display system 200 further comprises one or more data acquisition modules (201-1,
201-2), comprised in a data acquisition unit, associated with the sensors 101 as depicted in figure
1 as well as to input devices (not shown in fig.) of the cockpit display system 200 and are
configured to receive data/aircraft parameters from the associated sensors 101 and input devices.
In an embodiment, the data acquisition module (DAM) 201 may be implemented as
hardware, software, or a combination that may be embedded on one or more circuit boards
and/or circuit card assemblies. The DAM 201 may be located alone in hardware (e.g. jcircuit
board) or may share hardware with the associated sensors 101 or the cockpit display system 200.
In an exemplary embodiment, the DAM 201 processes the data/parameters provided by the
sensors 101 and/or input devices; and provides the processed data in the form of various graphics
commands and/or control parameters to the graphics server (GS) 202.
The GS 202 processes the commands and/or control parameters from the DAM 201 for
displaying graphics on the cockpit display units 204. The GS 202 is further coupled to a graphics
engine (GE) 203 that receives the processed graphics commands and/or control parameters. The
GE 203 includes a graphics generator/generating module that transforms the received graphics
commands into pixel coordinates and intensity values i.e. pixel data for displaying graphics on
the cockpit display units 204. Subsequent to generation of pixel data, the same is displayed to
pilots via the cockpit display units 204.
As explained above, the graphics engine 203 based on the graphics commands generated
7
by the data acquisition modules 201, generates the pixel data/frames which are to be displayed
on the display units 204 in the cockpit in the form of graphics. The pixel data generated by the
graphics engine (GE) 203 is the representation of the display being displayed on the cockpit
display units 204 and therefore the present invention proposes a method and system for
retrieving the pixel data from the graphics engine 203 and storing the same in an external
memory 209 via a port, so as to be enable the investigation agency to recreate the cockpit display
based on retrieved pixel data from the graphics engine 203. In an embodiment, the port can be
one of universal serial bus (USB) connector, a network interface card (NIC) connector, a serial
port, an IEEE 1394 interface, and other suitable types of connectors. The pixel data is recorded
with time stamp information and can be used for cross-relation to other internal/external events
or the sensor data as stored in the flight recorder 103.
In an embodiment, the pixel data that has been stored in the external memory 209 for a
time period more than a predetermined time period, say two hours, gets automatically erased
from the external memory 209 thus making way for a new pixel data. For example, in a flight of
a comparatively longer duration, say twelve hours, the pixel data that has been stored for more
than two hours gets automatically erased to make way for the new pixel data. Thus, at any point
in time, only the pixel data pertaining to the last two hours is being present in the external
memory 209. Hence, the external memory 209 is operative to store and retain pixel data
pertaining to a last few hours of flight duration that is always very vital, in case of any untowards
accident or incident, to determine posssible cause of the accident or incident.
In an embodiment, the external memory is comprised in the flight recorder 103.
In a further embodiment, it is also necessary to capture and store the pilot's input
provided via interactive screen of cockpit display units 204. The pilot when provides an input via
the interactive screen is provided to the data acquisition module ( DAM) 201 which executes the
input provided by the pilots and the same is provided to the GS 202 in the form of graphics
commands. Further, the GS 202 processes the graphics commands and/or control parameters
received from the DAM 201 for displaying graphics on the cockpit display units 204 and the
pixel data corresponding to the display is generated by the GE 203 based on the processed
8
graphics commands. The pixel data being generated is displayed on the cockpit display units
204. The port as coupled to the graphics engine 203 retrieves the graphics engine data
corresponding to the pixel data being generated therein and the same is stored in the external
memory 209. In this way, the pilot's input which has triggered the change of display in the
cockpit display units 204 is also captured and stored in the external memory 209.
Further, the present invention proposes a method and system for processing the pixel data
retrieved from the graphics engine 203, so as to reduce the memory space being occupied by the
pixel data, wherein the pixel data is processed by a processing unit 205 coupled to the graphics
engine (GE) 203. The processing unit 205 is further coupled to an external memory 209.
The processing unit 205 is coupled to the graphics engine 203 via the port. As illustrated
above, in an embodiment, the port can be one of universal serial bus (USB) connector, a network
interface card (NIC) connector, a serial port, an IEEE 1394 interface, and other suitable types of
connectors. Further, an external memory 209 is coupled to the processing unit 205 (via said port)
and is configured to store the pixel data generated by the graphics engine (203) and processed by
the processing unit 205.
In an illustrative example, where the flight duration between two destinations is of a
longer duration, for example 12 to 15 hours, it is required to process the pixel data retrieved from
the graphics engine 203, so that minimum memory space is occupied in the external memory
209.
In a preferred embodiment of the present invention, the processing unit 205 comprises a
sampling module 206, a selection module 207 and a compression module 208. When the
graphics engine's pixel data is retrieved through the port, said pixel data is received by the
sampling module 206. Said sampling module 206 samples the received pixel data/pixel frames
based on the predetermined sampling rate and transfers the sampled pixel data/pixel frames to a
selection module 207 of the processing unit 205. The selection module 207 receives the sampled
pixel data and compares the received pixel data against the previously stored sampled data to
determine if there is a change in value of the pixels. The selection module 207 selects the pixel
9
data/pixel frames whose value has changed beyond a certain threshold and transfers the selected
pixel data/pixel to a compression module 208 of the processing unit 205.
Thus, the selection module 207 discards the duplicate frames based on the comparison
and transfers only the non-redundant data to a compression module 208 of the processing unit
205. The compression module 208 compresses the received data from the selection module 207
to reduce the size of the data and further transfers the compressed data to the external memory
209 coupled to the processing unit 205. The external memory 209 receives the compressed pixel
data and stores the compressed data in a format suitable for storage. In an embodiment, the pixel
data as retrieved from the graphics engine via a port is stored in a binary format. The sampling
module 206, selection module 207, and compression module 208 may be implemented as
hardware, software, or a combination of hardware and software.
In an example, where the flight duration between destinations is of a shorter duration, for
example 30 minutes, then the pixel data generated by the graphical engine 203 can be retrieved
using the port coupled to said graphics engine 203 and can be stored directly into the external
memory 209 without processing unit 205 being implemented as discussed with respect to
embodiment above.
One of the important advantage of the retrieving and storing the pixel data from the
graphics engine is that said pixel data at graphics engine represents the actual
graphics/parameters/infomiation/images which are shown to pilots of the aircraft at any time of
the flight and the pixel data being retrieved and stored can be processed to simulate the display
being stored in the cockpit display.
In another advantage, retrieving and storing the pixel data from the graphics engine is
that the pixel data requires much lesser memory storage than the digital data being generated by
the GS 202.
In further advantage, the processing of the pixel data for sampling/compression is much
easier than the conventional binary data thereby resulting in optimized use of memory.
10
Figure 3 illustrates the method of transferring pixel data from a graphics engine 303 to an
external memory 306 according to an embodiment of the present invention. The GE 303 as
illustrated in fig. 2 generates the pixel data for displaying the graphics on the cockpit display
units 304. The GE 303 employs dual frame buffers 301, 302 for storing the pixel data that is to
be transferred to the cockpit display units 304. A current frame buffer 301 is a memory device
that stores pixel data that is currently being transferred to the cockpit display units 304. A future
frame buffer 302 is a memory device that receives and stores pixel data that is currently being
generated by the GE 303 and is to be displayed on the cockpit display units 304 in future. The
pixel data stored in the future frame buffer 302 is to be transferred/presented on the cockpit
display units 304 following the switch of the frame buffers 301, 302 functions. In an exemplary
embodiment, during a first cycle, the frame buffer 301 operates as a current frame buffer and the
frame buffer 302 operates as a future frame buffer. During the next cycle, the role/operation of
the frame buffer changes i.e. the frame buffer 302 operates as a future frame buffer and the frame
buffer 301 operates as a current frame buffer.
The frame buffers 301, 302 are further operatively coupled to the external memory 306.
The pixel data as generated by the GE 303 and stored in the frame buffers 301, 302 for the
purpose of displaying on the cockpit display units 304 is simultaneously transferred to the
external memory 306 that receives the data and stores the data in a suitable format. In another
embodiment of the present invention, the frame buffers 301, 302 are operatively coupled to the
processing unit 305 that is further coupled to the external memory 306. The processing unit 305
receives the pixel data from the frame buffers 301, 302 and processes the data as illustrated in
Fig. 2 to reduce memory requirements for the pixel data before transferring the data to the
external memory 306 that receives the data and stores the data in a suitable format.
Figure 4 illustrates the process of retrieving and storing the cockpit display data
according to the embodiments of the present invention. The process begins at step 401, wherein
the sensors 101 measures one or more aircraft parameters. Next, in step 402, one or more data
acquisition modules (DAM) 201, associated with the sensors 101, receive the measured
parameters. In step 403, the DAM 201 receives input data via input devices/interactive screen of
11
the cockpit display system. In step 404, the DAM 201 processes the received parameters and
input data to generate graphics commands and in step 405, the DAM 201 transmits the generated
graphics commands to the graphics server (GS) 202. In step 406, the GS 202 processes the
graphics commands and/or control parameters received from the UA 201 for displaying graphics
on the cockpit display units 204 and the graphics engine (GE) 203 processes the processed
graphics commands to generate pixel data for displaying graphics on the cockpit display units
204. In step 407, the GE 203 transmits the pixel data to the cockpit display units 204 for
displaying measured parameters and/or input data in the form of graphics. Next in step 408, the
GE 203 simultaneously transmits the pixel data to the processing unit 205 for processing as
illustrated in Fig. 5. In step 409, the processing unit 205 processes the received pixel data to
reduce size of the data and lastly in step 410, the compressed pixel data is transmitted to the
external memory 209 for storage, wherein the pixel data is storage in a suitable format. As
illustrated in different embodiments of Fig. 2, the steps 408 and 409 may not be carried out when
the pixel data pertains to small flight duration and processing of the pixel data is not required.
Figure 5 illustrates the process of retrieving and processing pixel data by the processing
unit 205 according to an embodiment of the present invention. In step 501, the pixel data is
received at the processing unit 205 from the graphics engine 203. In step 502, the pixel data is
received by a sampling module 206 of the processing unit 205 and the sampling module 206
samples the received pixel data as per the predetermined sampling rate. In step 503, the sampling
module 206 transmits the sampled pixel data to the selection module 207 of the processing unit
205. In step 504, the selection module 207 of the processing unit 205 compares the sampled
display data with the previously sampled data to determine the change in value of the pixel data.
In step 505, the selection module 207 selects the pixel data whose value has changed beyond a
certain threshold and in step 506, the selection module 207 transmits the selected pixel data to
the compression module 208 of the processing unit 205. At step 507, the compression module
208 compresses the received pixel data to reduce the size of the data. Lastly, at step 508, the
compression module 208 transmits the compressed data to the external memory 209, wherein the
pixel data is stored in a suitable format.
The pixel data retrieved and stored in accordance with the present invention can be of
12
immense help to the investigation agencies in case of an accident of any untowards incident with
the aircraft. In case of such accidents/incidents, the recorded pixel data can be easily retrieved
from the flight recorders and can be used for reproducing the data/information displayed to the
pilots at any particular point of time before the accident. Such reproduced displays can be quite
helpful in determining the exact cause of the accident. The pixel data as recorded the flight
recorders complements the sensor data that is typically stored in the flight recorder to determine
if there is any discrepancy in the data/parameters recorded by the sensors and the data displayed
on the cockpit display units.
The invention has been described above with reference to numerous embodiments and
specific examples. Many variations will suggest themselves to those skilled in this art in light of
the above detailed description. All such obvious variations are within the full intended scope of
the appended claims.

We Claim:
1. A system (200) for storing cockpit display data, comprising:
at least one sensor to measure at least one flight parameter;
a data acquisition unit comprising at least one data acquisition module (201a) associated
with said sensor for generating graphics commands based on the at least one measured flight
parameter;
a graphics server (202) coupled to said data acquisition unit for receiving and processing
said graphics commands;
a graphics engine (203) coupled to said graphics server (202) for receiving the processed
graphics commands and configured to generate pixel data based on the processed graphics
commands;
a cockpit display unit (204) coupled to said graphics engine (203) and configured to
display the pixel data; characterized in that
the graphics engine (203) is coupled to a processing unit (205) configured to sample and
compress the pixel data received from the graphics engine (203); and
said processing unit (20fj) coupled to an external memory (209) configured to receive and
store the compressed pixel data.
2. The system as claimed in claim 1, wherein said processing unit (205) is configured to:
sample the received pixel data from said graphics engine (203) based on
predetermined sampling rate;
select the pixel data whose value has changed beyond a certain threshold; and
compress the selected pixel data.
3. The system as claimed in claim 1, wherein said processing unit (205) is operatively
coupled to said graphics engine (203) via a port.
4. The system as claimed in claim 1, wherein said external memory (209) is comprised in a
flight recorder.
5. The system as claimed in claim 1, wherein said data acquisition unit is coupled to an
14
interactive screen of the cockpit display unit (204).
6. The system as claimed in claim 1, wherein the pixel data is stored with time stamp
information.
7. The system as claimed in claim 1, wherein said cockpit display unit (204) is at least one of:
a primary flight display unit (PFD) or a multi-function display unit (MFD).
8. A method for storing cockpit display information, comprising the steps of:
measuring (401) at least one flight parameter by at least one sensor;
receiving (402) the measured flight parameter by at least one data acquisition module
associated with said at least one sensor;
generating (404) graphics commands by the at least one data acquisition module based on
the measured at least one parameter;
processing (405) said graphics commands by a graphics server coupled to said data
acquisition module;
generating (406) pixel data based on the processed graphics commands by a graphics
engine (GE) coupled to said graphics server (AGS); characterized by
processing the generated pixel data (409) to sample and compress said pixel data; and
storing the compressed pixel data in an external memory.
9. The method as claimed in claim 8, wherein processing the generated pixel data comprises:
receiving, by a sampling module, the pixel data from the graphics engine;
sampling, by the sampling module, the received pixel data based on
predetermined sampling rate;
selecting, by a selection module, the sampled pixel data whose value has changed
beyond a certain threshold; and
compressing, by a compression module, the selected pixel data.
10. The method as claimed in claim 8, wherein the graphics commands are generated by the
said data acquisition module 201 based on user input.
15
11. The method as claimed in claim 8, wherein the pixel data comprises time stamp
information.
12. The method as claimed in claim 8, wherem the pixel data is stored in the external
memory in a binary format.