The present invention relates to an inertial / video hybridization device and method.

STATE OF THE ART

It is known to embed in a carrier such as an aircraft an inertial unit producing inertial data relating to this carrier (its position, its attitude, and its speed typically). The inertial unit produces such data from inertial sensors such as accelerometers and gyrometers.

However, the data produced by an inertial unit drifts. This is why it has been proposed to use other types of sensors than inertial sensors, producing data making it possible to correct the inertial data produced by the inertial unit (one also speaks of resetting inertial data in the literature). The processing combining inertial data and data of another type is commonly referred to as hybridization.

Different types of hybridization have been proposed. The best known is inertial / satellite hybridization (generally abbreviated as GPS / INS), which consists of combining inertial data and satellite positioning data captured by a receiver communicating with a constellation of satellites. The satellite positioning data drift little, and are therefore able to correct the inertial data.

This type of hybridization has in particular been proven to aid in the navigation of an aircraft during its flight.

However, it should be remembered that the GPS signals received by the receiver are very sensitive to the presence of jammers. Such jammers are now used by a large number of motorists so as not to be able to be geo-located and in particular around airport areas. Another disadvantage of GPS signals is that their propagation is hampered by the presence of buildings. For all these reasons, an aircraft in the taxiing or landing phase receives GPS signals noisy to the point of rendering the GPS / INS hybridization ineffective.

To overcome this drawback, alternative hybridizations have been proposed. One of them is inertial / video hybridization. Video inertial hybridization consists in determining the speed of the wearer using images acquired by a camera which scans the environment of the wearer during his movement. It is this speed that is used to correct the inertial data.

For an inertial / video hybridization to work effectively, the inertial unit and the camera which acquires the images must be harmonized, i.e. the relative positioning of the frame in which the inertial data is expressed and the frame in which is expressed the estimated speed from the camera images is known.

A mismatch between the camera and the inertial unit can be caused by excessive movement of the camera relative to the aircraft. This jeopardizes the precision of the navigation solution resulting from the inertial / video hybridization.

Now, in the field of aeronautics, it is crucial that errors be kept within a limited tolerance interval in order to comply with safety certifications (“safety”), and to detect when an error leaves its tolerance interval.

DISCLOSURE OF THE INVENTION

An aim of the invention is to detect an excessive level of mismatch between an inertial unit and a camera of an inertial / video hybridization device.

To this end, there is proposed, according to a first aspect of the invention, an inertial / video hybridization device intended to be carried on board a carrier, the device comprising:

- a camera configured to acquire a first image showing a predetermined landmark fixed on the wearer,

a processing unit configured to estimate a speed of the carrier from the first image acquired, with a view to hybridizing the estimated speed with inertial data relating to the carrier produced by an inertial unit,

the device being characterized in that the processing unit is further configured for:

- locate a position of the landmark in the first image acquired,

- calculate a difference between the localized position and a reference position of the landmark,

- compare the calculated deviation with a predetermined threshold,

- signaling an alert when the calculated deviation is greater than the predetermined threshold.

The device according to the first aspect of the invention can be supplemented with the aid of the following characteristics taken alone or else in combination when this is technically possible.

The reference position of the landmarks can be a position of the landmarks in a reference image acquired by the camera while the inertial unit and the camera are harmonized.

Alternatively, the hybridization device comprises a second camera configured to acquire a second image also showing the bitter, and the processing unit is further configured to:

- locate a second position of the predetermined landmarks in the second acquired image,

determining the reference position of the landmark by applying to the second position a predetermined transformation representative of a change of frame from the second camera to the first camera.

Preferably, the processing unit is configured so as not to hybridize the inertial data based on the estimated speed, when the deviation is greater than the predetermined threshold.

Preferably, the processing unit is configured to estimate the speed of the carrier with the help of the calculated deviation, so as to correct a mismatch of the camera with respect to the inertial unit, when the calculated deviation is not not greater than the predetermined threshold, and to hybridize the speed with the inertial data so as to produce a carrier navigation solution.

Preferably, the processing unit is configured for:

- determine a level of contrast in an area showing the bitterness in the acquired image,

- calculate a contrast difference between the contrast level determined in the area and a reference contrast level,

- compare the difference in contrast with a predetermined contrast threshold,

- signaling an alert when the contrast difference is greater than the predetermined contrast threshold.

There is also proposed a carrier, such as an aircraft, comprising an inertial / video hybridization device according to the first aspect of the invention, and a landmark attached to the carrier in view of the camera.

There is also proposed, according to a third aspect of the invention, an inertial / video hybridization method, the method comprising steps of:

- acquisition, by a camera on board a carrier, of a first image showing a predetermined landmark fixed on the carrier,

- estimation of a speed of the carrier from the first image acquired, with a view to hybridizing the estimated speed with inertial data relating to the carrier produced by an inertial unit,

the method being characterized in that it comprises steps of:

- localization of a position of the landmark in the first acquired image,

- calculation of a difference between the localized position and a reference position of the landmark,

- comparison of the calculated difference with a predetermined threshold,

- signaling of an alert when the calculated deviation is greater than the predetermined threshold.

The carrier is for example an aircraft. The steps of the method can advantageously be implemented when the aircraft is in the taxiing or landing phase.

DESCRIPTION OF FIGURES

Other characteristics, aims and advantages of the invention will emerge from the following description, which is purely illustrative and non-limiting, and which should be read with reference to the appended drawings in which:

[Fig. 1] FIG. 1 schematically illustrates an inertial / video hybridization device according to a first embodiment of the invention, on board a carrier.

[Fig. 2] FIG. 2 is a flowchart of steps of a method according to an embodiment of the invention.

[Fig. 3] FIG. 3 schematically illustrates an inertial / video hybridization device according to a second embodiment of the invention, on board a carrier.

In all of the figures, similar elements bear identical references.

DETAILED DESCRIPTION OF THE INVENTION

With reference to FIG. 1, a carrier 1 such as an aircraft comprises an inertial / video hybridization device 2, more simply called hybridization device 2 below.

The hybridization device 2 comprises an inertial unit 4, a camera 6, a processing unit 8, and a memory 10.

The conventional inertial unit 4 comprises a plurality of inertial sensors, such as accelerometers and gyrometers. The unit is configured to produce inertial data relating to the carrier 1. These inertial data typically include data for the position, speed and attitude of the carrier 1). The inertial data are expressed in a first frame of reference.

The inertial unit 4 is fixed to a wall of the carrier 1.

Camera 6 is attached to an exterior surface of wearer 1. Camera 6 is positioned to view the environment of wearer 1. Camera 6 is further positioned so that part of wearer 1 is in the field of view of. camera 6. In particular, it is ensured that at least one predetermined landmark 12, fixed on the carrier 1, is in view of the camera 6.

A landmark 12 is an element serving as a reference point for the camera 6. A landmark 12 can for example be native, that is to say be formed by an existing element of the aircraft, such as an antenna, a device. pattern, a pitot probe, etc. Alternatively, a landmark 12 is artificial in nature, i.e. it is added to the aircraft to serve only as a landmark. For example, the artificial bitter 12 can take the form of a test pattern applied to an exterior surface of the aircraft.

The processing unit 8 is configured to implement a conventional inertial / video hybridization using inertial data supplied by the inertial unit 4 and data calculated from images acquired by the camera 6. We will see in the following section. further that the processing unit 8 is configured to carry out other processing operations upstream of such hybridization, aimed at detecting excessive disharmonization between the camera 6 and the inertial unit 4.

The processing unit 8 comprises for example at least one processor configured to execute a computer program carrying out these processing operations.

The hybridization device 2 further comprises a memory 10 configured to store data. The memory 10 comprises in particular data allowing the processing unit 8 to recognize each landmark 12 in an image, using a pattern recognition process known per se.

With reference to FIG. 2, an inertial / video hybridization method comprises the following steps.

In a preliminary calibration phase 100, the camera 6 is positioned in a reference position. In this position, the camera 6 and the inertial unit 4 are harmonized. The camera 6 acquires a reference image showing the or each landmark 12 attached to the carrier 1. For at least one landmark 12, the processing unit 8 applies a shape recognition processing to recognize the landmark 12 in the image. reference. The processing unit 8 locates in this image a position of the landmark 12, called reference. This position is typically a pixel of the image defined by a row number and a column number (i, j). The reference position of each landmark 12 is stored in the memory

10.

This preliminary calibration phase 100 is typically implemented while the carrier 1 is stationary, for example during a maintenance period for the carrier 1.

In a subsequent use phase, the carrier 1 sets in motion, and the hybridization device 2 implements the following steps.

The inertial unit 4 produces inertial data using its inertial sensors, in a conventional manner.

In parallel, the camera 6 acquires a plurality of images, including a first image showing the environment of the wearer 1 (step 102). As indicated previously, due to the position in which the camera 6 has been placed on the carrier 1, the first image acquired by the camera 6 also show the or each landmark 12 used.

The processing unit 8 applies the shape recognition processing available to it to recognize a landmark 12 in the first acquired image. The processing unit 8 locates the position of the recognized landmark 12 (step 104). This position is a pixel of the image defined by a row number and a column number (i ', j').

Furthermore, the processing unit 8 determines the reference position (i, j) (step 106). In the embodiment illustrated in FIG. 1, this determination step comprises a simple reading of this datum previously stored in the memory 10 of the hybridization device 2.

The processing unit 8 calculates the difference between the position of the landmark 12 in the first image and the reference position (step 108).

The processing unit 8 then compares this difference with a predetermined threshold (step 110).

If the difference is greater than the predetermined threshold, then the processing unit 8 signals an alert (step 112). Indeed, an excessively high difference between the reference position and the position of the pin 12 in the first image is very probably indicative of a displacement of the camera 6 with respect to its reference position, and therefore of a mismatch. potential between camera 6 and inertial unit 4.

The alert is for example signaled during step 112 to a cockpit of the carrier 1, so that a pilot of the carrier 1 can be informed of the failure of the hybridization device 2.

Furthermore, in the event of an alert, the processing unit 8 ensures that no inertial / video hybridization is implemented from a speed coming from the camera 6, in particular a speed which would be calculated from of the image in which a deviation greater than the threshold has been detected.

If, on the contrary, the difference is not greater than the predetermined threshold, the processing unit 8 calculates an estimated speed of the carrier 1 using the first image and the difference (step 114).

More precisely, during step 114, the processing unit 8 locates at least one external landmark not attached to the wearer in two images acquired at different times by the camera 6: the first image and another image acquired before or after the first image by the camera 6. This external landmark occupies different positions in these two images when the carrier 1 is moving. The processing unit 8 first estimates the displacement of this external landmark between the two images. Then, the processing unit corrects this displacement using the position difference calculated previously in step 108. The corrected displacement is then divided by the period of time which separates the respective acquisitions of the two images, so to get the estimated speed.

Then, the processing unit 8 implements a hybridization between the inertial data acquired by the inertial unit 4 and the calculated speed, so as to produce a navigation solution for the carrier 1 (step 116).

The processing unit 8 can also, optionally, determine a level of contrast in an area showing the bitterness 12 in the first acquired image. This level of contrast can be a contrast between different parts of the bitter 12 displayed (for example, in the case of a bitter 12 of the artificial test pattern type, the contrast between black areas and white areas of the test target) or a contrast between bitter 12 itself and its surroundings). In this case, the processing unit 8 then calculates a contrast difference between the contrast level determined in the zone considered, and a reference contrast level, and compare the contrast difference with a predetermined contrast threshold. An alert is selectively signaled when the contrast difference is greater than the predetermined contrast threshold. These additional steps make it possible to detect conditions of visibility of the camera 6 leading to obtaining a speed estimate that is insufficiently reliable to be injected into inertial / video hybridization. These steps can be implemented before, during or after steps 104, 106, 108, 110.

The steps described above are repeated for several images acquired by the camera 6.

It will be noted that the use of a single bitter is sufficient to allow the detection of a disharmonization, and to correct to a certain extent the speed estimated by the processing unit. However, using several landmarks allows a more efficient correction of such a speed.

During the method described above, a memorized position was used as a reference position which had been determined in a preliminary phase during which the harmonized character of the camera 6 and of the inertial unit 4 could be guaranteed.

It should however be noted that a carrier 1 can be deformable, and therefore certain parts of such a carrier 1 are liable to move relative to each other. This is the case in particular with the wing tips of an aircraft, which may deform relative to the fuselage of this aircraft. Thus, it is possible that a landmark 12 can move relative to the camera 6, without there being any substantial disharmonization between the camera 6 and the inertial unit 4. One can for example cite the case a camera 6 mounted on the fuselage, and turned towards the side of the aircraft, in the direction of a wing, and a landmark 12 located at the end of this wing. During the movement of the aircraft, the wing can flap slightly from bottom to top so that the position of the bit 12 which will then be determined by the processing unit 8 will be variable. In such a situation, the above method could conclude that there is a disharmonization between the camera 6 and the inertial unit 2 on the basis of an excessive position deviation, when this is not the case.

FIG. 3 illustrates a second embodiment of a hybridization device 2 making it possible to overcome this problem of detecting false positives. The hybridization device 2 according to this second embodiment comprises a second camera 7 oriented towards at least one landmark 12 already seen by the camera 6. This second camera 7 used as a reference position source.

Each of the two cameras 6, 7 is harmonized with the inertial unit 4 during the preliminary calibration phase. A transformation is stored in memory 10 making it possible to go from a mark attached to the second camera 7 to the mark attached to the first camera 6.

The method implemented by the hybridization device 2 according to the second embodiment differs from the method described above by the step of determining 106 of the reference position of the lander 12, which is implemented as follows.

The second camera 7 acquires at least a second image showing the landmark 12 also shown in the first image acquired by the first camera 6. However, in the second image, the landmark 12 is seen from a different angle, and at a distance different.

The processing unit 8 applies the step 104 to the second image, namely recognizes the landmark 12 in the second image and locates its position in the second image, which is conventionally called the second position of the landmark 12.

The processing unit 8 then applies a change of frame of reference to the second position, so as to project this position in the plane seen by the first camera 6. The result of this change of frame is the reference position as described previously, whose deviation from the position of the landmark 12 in the first image is then compared to a threshold.

If by chance the two cameras 6, 7 remain harmonized with the inertial unit 4, but the lander 12 moves relative to these two cameras 6, 7 because of a deformation of the carrier 1, this deformation is compensated by the calculation of the difference during step 108, which will theoretically be zero. On the other hand, if one of the two cameras 6, 7 is out of harmony with respect to the inertial unit 4, this disharmonization will be reflected in the calculated position difference, which will then be of a higher value. Exceeding the threshold by this difference will reveal the disharmonization of one of the cameras 6, 7 present. Ultimately, this second embodiment has the advantage of avoiding reporting a false positive caused by the deformation of part of the wearer 1.

The methods described above advantageously find application during a taxiing or landing phase of an aircraft for the reasons given in the introduction.

CLAIMS

1. Inertial / video hybridization device (2) intended to be carried on board a carrier (2), the device comprising:

- a camera (6) configured to acquire a first image showing a predetermined landmark (12) fixed on the carrier (2),

- a processing unit (8) configured to estimate a speed of the carrier (2) from the first image acquired, with a view to hybridizing the estimated speed with inertial data relating to the carrier (2) produced by an inertial unit ( 4),

the device being characterized in that the processing unit (8) is further configured for:

- locate a position of the landmark (12) in the first image acquired,

- calculate a difference between the localized position and a reference position of the landmark (12),

- compare the calculated deviation with a predetermined threshold,

- signaling an alert when the calculated deviation is greater than the predetermined threshold.

2. Device according to claim 1, wherein the reference position of the landmark (12) is a position of the landmark (12) in a reference image acquired by the camera (6) while the inertial unit (4) ) and the camera (6) are harmonized.

3. Device according to claim 1, comprising a second camera (7) configured to acquire a second image also showing the bitter (12), and in which the processing unit (8) is further configured to:

- locate a second position of the predetermined landmark (12) in the second acquired image,

- determining the reference position of the landmark (12) by applying to the second position a predetermined transformation representative of a change of frame from the second camera (7) to the first camera (6).

4. Device according to one of the preceding claims, wherein the processing unit (8) is configured not to hybridize the inertial data based on the estimated speed, when the difference is greater than the predetermined threshold.

5. Device according to one of the preceding claims, wherein the processing unit (8) is configured to

- estimate the speed of the carrier using the calculated deviation, so as to correct a mismatch of the camera (6) with respect to the inertial unit (4), when the calculated deviation is not greater than the threshold predetermined, and for

- hybridize speed with inertial data so as to produce a carrier navigation solution (2).

6. Device according to one of the preceding claims, wherein the processing unit (8) is configured for:

- determine a level of contrast in an area showing the bitterness (12) in the acquired image,

- calculate a contrast difference between the contrast level determined in the area and a reference contrast level,

- compare the difference in contrast with a predetermined contrast threshold,

- signaling an alert when the contrast difference is greater than the predetermined contrast threshold.

7. Carrier (2), such as an aircraft, comprising an inertial / video hybridization device (1) according to one of the preceding claims, the bit (12) being fixed to the carrier (2).

8. A method of inertial / video hybridization, the method comprising the steps of:

- acquisition (102), by a camera (6) on board a carrier (2), of a first image showing a predetermined landmark (12) fixed on the carrier (2),

- estimation (114) of a speed of the carrier (2) from the first image acquired, with a view to hybridizing the estimated speed with inertial data relating to the carrier (2) produced by an inertial unit (4),

the method being characterized in that it comprises steps of:

- localization (104) of a position of the landmark (12) in the first acquired image,

- calculation (108) of a difference between the localized position and a reference position of the landmark

(12),

- comparison (110) of the calculated deviation with a predetermined threshold,

- signaling (112) of an alert when the calculated deviation is greater than the predetermined threshold.

9. Method according to the preceding claim, wherein the carrier (2) is an aircraft.

10. Method according to the preceding claim, the steps of which are implemented when the aircraft (2) is in the taxiing or landing phase.