Sign In to Follow Application
View All Documents & Correspondence

Method For Acquiring Images Of A Scene From A Sensor On Board A Moving Carrier With Servocontrol Of Its Line Of Sight

Abstract: The invention relates to a method for acquiring images of a scene from a moving carrier (2) equipped with a sensor which comprises a step of acquiring images (1) of the scene during the movement of the carrier and a step of servocontrolling the angular direction of the line of sight of the sensor. The acquisition is carried out: for a first position of the carrier with a servocontrol allowing an outward scan of a strip of terrain (4a) of the scene combined with a scan in a "step and stare" mode with a biaxial "stare" micro movement to be carried out a first band of images thus being acquired and in that at least one other band of images is acquired in a given zone of terrain by reiterating these scanning steps for at least one other position (P2) of the carrier each image of another band being acquired with a preset degree of overlap with the images of the first band.

Get Free WhatsApp Updates!
Notices, Deadlines & Correspondence

Patent Information

Application #
Filing Date
14 May 2018
Publication Number
36/2018
Publication Type
INA
Invention Field
PHYSICS
Status
Email
Parent Application
Patent Number
Legal Status
Grant Date
2023-08-09
Renewal Date

Applicants

THALES
Tour Carpe Diem Place des Corolles Esplanade Nord 92400 Courbevoie

Inventors

1. HORAK Rapha??l
Thales Optronique SA 2 avenue Gay Lussac 78995 Elancourt

Specification

The field of the invention is that the acquisition of images by slaving the line of sight of a sensor coupled to image processing, for example for surveillance applications or recognition.

FIGS scan the line of sight (LOS) optical sensors or radar slave implemented in PODs, aircraft or land vehicles are conventional figures of the type:

- side scan constant angle, also called strip map mode in the radar field,

- scanning "spotlight" on a fixed point (= on fixed point tracking scanning),

- tracking scanning moving target,

- circular scan for terrestrial applications.

The first scan allows to scan a ground strip but with a constant point of view between the aircraft and the ground point that is scanned.

The second allows you to see a next ground object different views but covers an area very limited ground.

The third is a variant of the second and does not guarantee the multiplicity of viewpoints.

The fourth is similar to the first and has a constant azimuth angle.

These four scanning classes do not allow you to view with a good angular resolution a large area of ​​different soil following viewpoints.

The object of the invention is to overcome these disadvantages.

The solution provided is to apply a user specific servo to the sensor boresight by plural scans sets the ground go, each outward scan being generally followed by a scanning ground rewind, to which combine

micro-movements of "step-and-stare" (translation and score in a summary translation) that achieve sufficiently stable and enlightened image despite rapid movements of scans. An image processing to control this control to increase accuracy.

More specifically the invention relates to a method of acquiring images of a scene at predetermined soil, from a carrier moving along a path and equipped with an optical sensor having a line of sight that comprises a step of acquiring by the successive image sensor of the stage during the movement of the wearer, and a control step of the angular direction of the line of sight by a processing unit connected to the sensor, the acquisition being carried out:

- for a first position of the carrier along its path, with a slaving of the angular direction of the line of sight for performing a scan of a predetermined pitch band of the scene, said outward scan, combined with a manual scanning "step-and-stare"

a first image strip being acquired,

- and in that at least another image band is acquired on the same ground area as the first strip by repeating these steps of scanning for at least one other position of the carrier along its path, each image of another headband being acquired for a strip portion with a recovery rate with the (or) frame (s) of the first strip to the same portion of the strip of land, above a safe predetermined recovery rate, the overlapping images one strip to the other from the same plot of land being thus acquired respectively during these iterations in different directions of the line of sight.

It is mainly characterized in that the mode "step-and-stare", which includes micro-movements of said translation "steps" having an amplitude controlled by the processing unit, comprises:

o at least one "step" in main component perpendicular to the outward scan and said side combined with o at least one "step" in main component parallel to said longitudinal and outward scan,

o a micro-movement "stare" bi-axis to compensate for a translational movement of the line of sight during acquisition of each image,

and in that the slaving of the angular direction of the line of sight is performed by image processing on the acquired images.

This solution with sweeping movements adapted to the line of sight, allows both to cover a wide area optionally in the form of different strips of land adjacent or not and to visualize all of those scanned points at different angles. It allows in particular to provide image banners covering a large area of ​​ground at times and different view angles.

It complies with a specific movement "step-and-stare" bidirectional, synchronous with the main slaving the line of sight, a sufficiently long illumination time for each image and thus ensures images of sufficient quality.

The solution also allows, after image processing headbands, these containing the points on the ground that are seen at different angles of sight, build consistent image headbands geometrically, that is to say superimposed on an ortho -Photography or a map, which does not allow other methods when the movement of the carrier is important relative to the terrain in the area.

The proposed solution uses the following new concepts:

- Scan and against repeated front-scanning the line of sight, followed by acquisitions, on one or more bands on the ground, the carrier equipped with the sensor being in motion, so as to return several times (typically between 5 and 20 times) on the same point on the ground with different viewpoints and this for all the points of a strip of wide longitudinal extent (or, if appropriate for all points of different bands of ground covered in parallel),

Step-and-stare combining two axes in addition to continuous micro-movements of step-and-stare classic, side steps and stare potentially significant amplitude, combined with image processing to correctly assemble the images of each band. While in the state of the art micro-movement is the single axis, another feature of this step-and-stare is that it introduces a micromotion stare biaxial to eliminate blur and improve the quality of image during its integration time.

It also leaves the possibility of adding a rotary component of the image at the stare, thus ensuring a perfect assembly as possible to the images in the image band generated.

simultaneous production of various bands of images, each strip corresponds to a strip having a controllable shaped form in both width and length.

Production for the same strip of land selected in the scene of different superimposed bands at different moments and regular taken along different points of view.

- Ability to generalize the bands for non-limited scenes in the lengthwise (bands occurring following the advance of the carrier).

The process is feasible in real-time as and when the progress of the wearer without constraint imposed on the trajectory thereof: the servo compensates the wearer's movements during scanning of the bands to the ground.

The first position and the other successive positions of the scanning base for carrier can either be unique when scanning or multiple and decomposed into as many positions as there are images when scanning when the wearer moves during the scanning.

Each image of another band is advantageously obtained for a portion of strip of land, with a positional accuracy with the (or) frame (s) of the first strip in the same plot of land band higher than a precision predetermined alignment.

The slaving of the angular direction of the line of sight is advantageously carried out by image processing on the acquired images so that within a single strip, adjacent frames are aligned with an upper alignment quality to quality predefined.

The slaving of the angular direction of the line of sight is advantageously carried out by image processing on the acquired images so that within a single strip, adjacent images have a recovery rate greater than a low recovery rate predetermined so as to obtain a continuous band.

The slaving of the angular direction of the line of sight is preferably carried out by image processing on the acquired images so that the images from two bands on the same stage portion are aligned with an upper alignment quality with predefined quality.

The micro-movement "stare" biaxial may further be associated with a movement rotating against-the line of sight determined by the processing unit, to compensate for a rotational movement of the line of sight during acquisition of image or to align the best images together.

According to one characteristic of the invention, at least one other predetermined ground strip is associated with the stage and is traversed by the scanning performed from said wearer's positions and mode "step-and-stare" further comprises at least one lateral moving web change to switch from a strip of land to another strip of land.

This band changeover lateral movement can be combined with a band changeover frontal movement.

The forward scan is usually performed in the direction of the trajectory.

Generally, a reverse-sweep against the forward scan, said against-scanning or scan back is performed by the scan and go before the reiteration.

This against-scan may be direct or itself be combined with a scanning step-and-stare mode. In fact, images are optionally acquired during the scan-against which is combined with a scanning mode "step-and-stare" with micro-movements of said translation "steps" which have a controlled amplitude by the processing unit for the successively acquired images overlap, which comprises:

1. at least one "step" perpendicular to the scanning-and against said lateral, laterally to scan the area combined with

2. at least one "step" parallel to and against said longitudinal-scanning step to scan the longitudinally zone 3. A micro-motion "stare" bi-axis to compensate for a translational movement of the line of sight during acquisition of each image.

Thus, a process that allows:

- to slave the boresight of a sensor carrier in motion, so that it scans a large stage floor while ensuring that each point on the ground is seen several times with view angles different covering the widest range of possible angles; - simultaneously reproducing one or several bands of contiguous images (or adjacent) and partially overlapping, and corresponding to scene freely chosen by the operator;

- that any point within these bands is reconstituted following temporally different perspectives covering a wide angular range, making possible such as 3D rendering processing of objects in these bands;

- that the images for reconstitution headbands are corrected in terms of compliance by exploiting

3D information resulting from multiple points of view in an extended angle range.

It also complies with the constraints of servo related sensor: rotational speed and acceleration of the line of sight.

Other advantages of the method can be cited as:

allow the operator to see in real time, from different points of view of the scene objects designated by himself,

make possible a 3D mapping on all bands,

make possible modes of scrolling bands continuously (not restricted spatially in the direction of movement of the wearer) make possible the concatenation of the bands in a wide band at any point of the path (concept of programmable surveillance zone real time),

allow a path of freely chosen carrier, and rolling movements, pitch, yaw bearing, within the limits imposed by the system (line of sight that must continue to be capable of sweeping the bands).

Other features and advantages of the invention will become apparent from reading the following detailed description, given by way of example and with reference to the accompanying drawings in which:

Figures 1 diagrammatically illustrate from above a frame grabber example from a sensor installed on board an aircraft and for a scene with three strips of land disjoint not closed on themselves (Fig 1 a) with three strips of land which 2 closed on themselves (Figure 1 b), Figure 1 c illustrate in more detail consecutive scans carried out in parallel on two strips of separate fields,

Figures 2 schematically illustrate in top view an example image acquisition mode "step-and-stare" according to the prior art (Figure 2a) and according to the invention (fig 2b)

3 schematically illustrates an example of movement of a sensor line of sight according to the method of the invention, the elevation and azimuth being expressed in degrees.

From one figure to another, the same elements are identified by the same references.

We will describe the invention by considering that the scene which is desired to acquire images is predetermined and divided into several strips of land possibly disjointed and possibly parallel, but not necessarily. Designates band, an area of ​​terrain of average longitudinal dimension greater than its mean width. These bands correspond to the ground space under the carrier, which may have infrastructure, but may be generalized to any other type of surface of the scene visible from the wearer, for example building facades seen from a vehicle earthly.

The shape of the scene to be scanned may be one, the corresponding strips may also be any, as well as their number and orientation. The shape / width of the covered tape can be coated e.g. elliptical or S, the width of the strip may also widen or otherwise decrease during scanning. More complex shapes can be imagined as a Y-shape (road junction), star shaped (crossroads and roads) form comprising holes or any decomposable shaped elementary strips. The orientation of the ground strip is freely chosen.

Examples will show scanning strips of land in relation to FIGS 1.

Figure 1a illustrates a first initial scan three strips of land 4a, 4b and 4c from a first position P1 of the carrier 2 (which itself moves during this first scan), the lines of sight being shown in solid lines, and a second scan of the three bands from a second distinct position P2 of the wearer, the sight lines being indicated in dotted lines. It is understood that several situations may occur:

If the inital scanning is done strictly on the same position with a second scan (and possibly other scans) is (s) from one (or) point (s) one (s) different (s) of the first up / down (that would be the case for a carrier moving time making jumps to change position as a helicopter mixing stationary positions for scanning the field movement from one point to another to switch point of view).

If the initial scan and the following are in a continuous movement of the carrier (the points (P1) corresponding to each image of the scan are then different successive points near each other). The proximity of these points depends on the supposedly large enough scanning speed for a second scan (or other scans) can do before starting another scan to a point (Pi) far enough from the previous, but not too much away either for different scans (at least two) may be made before the strips of land covered by the sensor line of sight outside the visual field of the sensor in the process of scanning.

- The third case is the case mixing the two previous modes: in the case of a carrier scanning the scene with its all sensor during its movement over a portion of its trajectory, and performing scans for different positions different spatial stationary on the rest of the track (if the holder permitting).

The sweep of the invention covers these cases, as long as it is considered in all cases early next scanning points (P1), (P2), (P) different. One can have i = 3 but typically it has from 5 to 20 different positions of the carrier or more. Figures 1 and 2 only show the case 1 or case 2 with a very fast scanning movement (lines of sight depart each point (P1) almost the same). The case 2 with a somewhat slower sweeping motion (but still fast enough at the end of the scan, the holder does not have time to reach the next position at which begins next scan) is not shown in the figures.

In any case, an image corresponding to an area of ​​ground (this is the footprint of the projected image in the field along the line of sight and sensor field) and at one point of the wearer's trajectory unique. We can also say that a point of the field swept two times or more, is the starting point trajectories (Pi) necessarily different.

The various scans are made so that most of the images from one of the first scan field strips are covered with images from the second scan (or subsequent scans if they exist).

Figure 1 b illustrates different directions of the line of sight during a first scan of two strips of lands 4a and 4b it closed on itself and surrounds in this case the holder 2, and a third strip of land 4c. One direction of the visual line 3 P i,. . . 3 P5 , is indicated for each of the five positions P1, ..., P5 of the carrier shown in the figure, and from the 5 th steering 3 P5images are acquired which are superimposed on some of the images acquired from the position P1 of the wearer. The presented case is a textbook that would be a very fast movement of the wearer and slowest of the line of sight, because we did not want to overload the figure. In fact we can interpret the 5 boresight directions as an example of lines of sight corresponding to 5 complete scans of the line of sight to 5 positions of the wearer.

The scanning direction before is generally adapted to the direction of the trajectory of the carrier, however, for a closed band on itself (as shown in Figure 1b with the strips 4a and 4b) or a strongly inclined belt with respect to the trajectory the carrier (see strip 4c of Figure 1 b), the scanning direction "forward" is determined by the processing unit so as to minimize deflection stresses the line of sight during the wearer's movement and increase ultimately the number of return on each band.

According to the invention described in connection with Figures 1 and 2, images 1 are acquired by a sensor on board a carrier 2 moving along a trajectory 20.

We initially considered the acquisition of a first image strip designated as original band. It is performed for a first position P1 of the carrier 2 on its path, with a feedback control of the line of sight 3 for performing:

- scanning a first predetermined dividing strip 4 of the stage, from an initial position to the ground line of sight, said outward scan, combined with a scan "step-and-stare" mode with micro-movements of said translation "steps", which comprises:

o at least one "step" perpendicular to the outward scan and said side 5, to scan the ground strip 4 laterally (or rather laterally), combined with

o at least one "step" 7 parallel to said longitudinal and outward scan step to scan the tape longitudinally field (or rather longitudinally)

o a micro-movement "stare" bi-axis to compensate for a translational movement of the line of sight during acquisition of each image one.

The combination of these micro-movements of the "steps", their length and their number before each micro-movement "stare" are for example selected by the processing unit in order to optimize the travel time of the overall scanning and / or time acquisition of images on different areas of the strip of land. Some longer steps between each "stare" as possible not to acquire images of portions of the strip of land.

A first image band is acquired on a first strip of land. More generally considering at least one other strip of land, the two strips of land (or more) are scanned in parallel simultaneously, through movements of "steps" additional side or front to move from a tape the other during scanning, providing a set of initial image bands covering these strips of land. Preferably, for this

image strip is continuous, the adjacent images are partially overlapping each other. A small overlap adjacent images is sufficient; this recovery rate is greater than a predetermined low recovery rates, typically 20% or even 10%. It opts mostly for minimum collection rates to maximize the number of bands stored in the processing unit.

Once this initial strip (possibly supplemented by other initial bands if multiple strips of land have been predetermined) was acquired, other image strips are acquired for the same strip of ground (the same strips of land in the general case), repeating these steps of scanning for one (and preferably several) other (s) P2 positions of the carrier along its path 20. Each image of another band is obtained for a strip portion with a strong with the recovery rate (or the) picture (s) of the original strip in the same plot of land strip. Thus, the overlapping images of a band to another, derived from

Generally, at least one other predetermined pitch band is associated with the scene. In Figures 1 a and 1 b, one can see three bands of distinct field 4a, 4b and 4c. So, the "step-and-stare" mode further comprises at least one lateral movement 6 strip of land change, to go from a strip to another strip of land. The images of bands corresponding to different strips of land can be acquired in parallel. It is shown four image bands 2b: 40a, 40b, 40c, 40d.

We will now detail these steps. The line of sight of the sensor 3 is controlled so as to:

1) make it sweep forward these predetermined field strips 4. This lead scan of the line of sight (the ground path can reach and even exceed a speed of 1500m / s in the case of an on-board sensor on board an aircraft) is further combined with

o a succession of local micro-movements of the "step-and-stare" bidirectional. Indeed, because of the very fast scanning speed before the line of sight (and very fast speed of its implementation on the ground), it is necessary to introduce into the general movement of the scanning front of microbalayages " step-and-stare "micro-scan" stare "to ensure during the image capture, image stability and an integration time sufficient and micro-scan" step "to move from one image to the other. This is to make the scan "step" by leaps 5 or 7 very fast between each image acquisition (= to make the transition from one image to another on the same lane 4) or lateral jumps 6 larger (to go from one band to another) and then to

These micro-movements 5, 6, 7 of translation are not all identical from one image to the next image; they are controlled by the servo unit. This should make checks to ensure controlled partial overlap between successive images on the same tape but also between adjacent images of the same band separated in time by a lateral flyback applied during the passage of the first strip to bands land and back to the first strip. These checks consist in measuring by inertial means, whether or not combined with image processing, the recovery rate between adjacent images of the same band and verifying that a minimum overlap (respecting a predetermined criterion by the system) are therebetween.

Generally, the micro-movements of "step" decompose in a component 5 or 6 perpendicular to the direction of forward sweep (5 is a "step" of small amplitude corresponding to the passage from one image to the next in the within a band, 6 is a "step" of greater

amplitude corresponding to the passage from one band to another), and a component parallel to the direction 7 of the prior scan. The "steps" whose main component is parallel to the scanning direction before are called "steps" front-end; the "steps" of which the main component is perpendicular to the scanning direction before are called "steps" side. Different scanning figures of these steps can be performed. When only one band is to be scanned, one can for example produce a scanning winding with a succession of steps 5 exclusively lateral (side to principal components) to traverse the strip 40 laterally as shown in Figure 2a, followed by a "step "7 exclusively frontal (front main component) to provide a frontal advance of the strip, then a series of "steps" exclusively side in the direction opposite to the previous to traverse the strip sideways in the opposite direction, etc. We can of course provide less simple scanning figures using the same "steps" basic but combined differently.

The micro-movements of "stare" to compensate for movement of the LOS during the image taking are also bi-directional translatory movements; they are not shown in the figures. For increased precision of assembly images, a scanning micro-against-rotary is advantageously applied to rotate around the line of sight between each movement of "step" and each image taken at the "stare" . In addition to assembling precision, it can offer an integration time (or illumination in the case of a lidar) longer.

these micro-movements of "steps" are accompanied by additional side 6, controlled movements of the line of sight to pass from one strip to the other, these movements can be short if the two bands are close, or further if they are remote. Similarly movements to move between

band to another can be divided into side and end components components; but these additional movements 6 have a lateral component usually larger than their front component, unless the two ground strips are offset along the front axis. Figure 2b shows four bands 40 (band 40a, 40b strip, strip 40c, 40d strip) respectively corresponding to four strips of land 4a, 4b, 4c and 4d (not shown in this figure). If the scene has only one band, these movements 6 to move from one band to the other does not exist.

Figure 1 c illustrates a scanning strategy where the emphasis is on lateral jumps (in azimuth) to first browse all the strips in parallel in some way, then the frontal jumps on the outermost bands of the carrier before to go on to the nearest bands, in order to minimize yaw jumps that may take longer to realize that jumps round. In this way the separate ground bands are scanned almost in parallel. Specifically, Figure 1 c shows in more detail two consecutive scans respectively made from two positions P1 and P2, on two lots of strips near 4a and 4b; on images referenced 1 are shown the temporal order of acquisition of the images during the scan of two bands: t1, t2, t6.

For every other panel, the servo processing of the line of sight scans on the same strip of land, must also ensure that the current image (= being acquired) projected on the ground is a high overlap rate with corresponding image generated from the first band. In other words, each image is acquired another band for a portion of this strip of land with a recovery rate with the (or) frame (s) of the first strip in the same plot of land strip. The recovery rate is greater than a predetermined safe rate of recovery, for example 80%. Thus, the overlapping image (depending on rate) of a strip to another, from of the same portion of land are acquired respectively during these iterations in different directions of the line of sight (seen in the 3D landmark stage) that is to say different angles of view. Indeed, for a recovery rate fixed for example at 80%, at least 80% points from the initial strip of land are "modified" during the acquisition of images of each iteration; in other words, at least 80% of the points of the image acquired headband initially find themselves in each band respectively acquired during iterations. For example in Figure 1a, the points of the scene at the bottom left of the band 4a are present in two images 1: one taken when the carrier 2 is in a first position P1 on the path 20, the ' another outlet when the wearer is in a different position P2. We find the same principle in Figure 1c, except that this time the numbers indicate an example of successive positions of the footprint of the images acquired by the holder in position 2 when scanning two precursor webs, these fingerprints successive corresponding to successive moments t1, t2, t6 of acquiring these images. Note that the images acquired during the second scan may well not match the images acquired in the first scan and overlap while ensuring maximum coverage of all the images on the two initial strips. A rotation of the sensor line of sight may however also be applied to ensure a perfect alignment of the images produced. This rotation, if performed, is performed by servo itself can be controlled by image processing. The processing of acquired images to generate expanded image banners seen along different angles and each

strip images may be temporally regenerated so that each point in the scene imaged in this band can be seen temporally different directions. for example is repeated until the scanned scene can no longer be crossed by the line of sight (sensor out of range or boresight angle exceeding an acceptable threshold).

It may be noted that a "step-and-stare" classic that is shown in Figure 2a, is known and applied to quickly scan the lateral continuous ground (with small front jumps) and it generates a longitudinal image headband extended, but the band is generated once by the conventional means and is necessarily viewed in a single view (the sweep mode is called "strip-map"). It therefore differs from the scanning according to the invention which generates multiple bands seen along the different perspectives on the same area of ​​land, and excludes the possibility of covering several strips of land. Another known method is the "spot-light" mode that consists of controlling the line of sight on the same ground location. It operates the appropriate treatment of Images for controlling and stabilizing the line of sight on the same soil scene (the image processing correlates the different successive images of the sensor to maintain a fixed point on the ground). It follows from this mode a series of images whose ground footprint does not exceed the size of each image projected on the ground. The intersection of the projected image then defines the following field area view different angles by various successive images. Again, this embodiment differs from many of the scanner according to the invention since the area of ​​land is rendered highly localized (restricted to the intersection of the projected images on the ground) and does not correspond to a strip extended as in application we offer. According to the invention, generates one or more bands of image browsing one or more strips of land high longitudinal extension, which are repeated at least twice, typically between 5 and 20 times or even more, by browsing the or strips of land by successive scans as shown in Figure 2b, which means that each point of the scene present in successive bands is seen temporally different directions of angles of the line of sight of the sensor. The angular differences of direction angles of lines of sight of the sensor on the same piece of land area are typically limited to a maximum angle of 30 ° by browsing or the field strips by successive scans as shown in Figure 2b, which means that each point of the scene present in successive bands is seen temporally different directions of angles of the line of sight of the sensor. The angular differences of direction angles of lines of sight of the sensor on the same piece of land area are typically limited to a maximum angle of 30 ° by browsing or the field strips by successive scans as shown in Figure 2b, which means that each point of the scene present in successive bands is seen temporally different directions of angles of the line of sight of the sensor. The angular differences of direction angles of lines of sight of the sensor on the same piece of land area are typically limited to a maximum angle of 30 °

so they can be used more easily in processing mapping images to extract features or make 3D reconstruction. The processing unit promotes the establishment of a minimum number of different points of view (typically between 5 to 20 various views of a portion of acquired image) with respect to the extent of the acquisition on the ground.

For every other panel, the servo processing of the line of sight scans on the same strip of land, must also ensure that within each other band, the adjacent images are overlapping the preferably with a low recovery rate predetermined minimum, so that every other image headband also be a continuous strip, as already indicated for the initial strip.

This treatment of the line of sight ensures that images taken on the same in any portion of a strip of land at times and different lines of sight are all overlapping with a high recovery rate (80% in our example). It is thus possible to extract a common field region containing all the images and which is large enough and operable to obtain a long stretch of the field which corresponds to the intersection of the longitudinal bands returned (closed or not themselves, see Figures 1 a or 1 b) to any area on which it can view images with different perspectives of the wearer viewing the area. For a given area of ​​land covered by different banners returned, it is very easy, knowing the geographical coordinates of the selected area to extract in different bands corresponding images which visualize the area according to the different views of the carrier having scanned the area. Thus the same ground point is seen several times at different times and with different perspectives.

The servo processing applied to control the movement of the line of sight can be achieved according to different embodiments.

According to a first embodiment based on the use of inertial means, this treatment consists in:

- projecting the images to the ground using the inertial means and calculating the overlap between two adjacent images projected on the ground when these images are part of a single image strip (the technique calculates e.g. the projection on the ground of the current image and the adjacent image to measure the existing geometrical transformation between the two projected images and measure their recovery rates and their actual alignment)

- do the same between the image commonly acquired in the current band image (assuming the current strip from a scanning line of sight which is other than the initial scan) and that (or those) of headband original image resulting from the first scan on the same land area as that of the current image,

- verify that the cover is less than the setpoint (= predetermined rate) referred to the low overlap between two images of the same strip (less than 10% for example) and it is greater than the setpoint relative to the strong recovery before exist between the images of another strip with those of the initial strip (greater than 80% for example),

- check the same to the desired alignment between images,

- if the cover is less than the target setpoint, slowing the relative movement of the line of sight and the speed in the opposite case (or vice versa depending on whether the cover is on the left in the direction of movement or right, or according to whether the overlap between images on the same area of ​​land taken at different times and for different scan, or collection of adjacent images within the same band and in the course of the same scan),

- same for image alignment, correcting if necessary by acting on the rotary motion of the line of sight

(When this movement is possible).

This type of control is sufficient when the angular position information of boresight are sufficiently accurate in relation to movement of the carrier (absolute position boresight having a

drift low vis-à-vis the movement of the carrier) and that has a sufficiently accurate knowledge of the environment to calculate a projection of the image on the ground, from a terrain model provided by the system navigation example. These conditions exist in many applications, for example for a land vehicle with a GPS and MEMS IMU, with a mean-field camera (typically 40 °) quickly scanning the scene near; but this is not the case for a carrier moving very rapidly and with a very small field camera scanning a scene with a very great distance inertial system of which measurement errors or angular drift are much greater than the resolution of the pixel the camera.

In the case where the accuracy of information on the angular position of the line of sight and / or field, obtained by the inertial means is insufficient to allow a projection of images sufficiently precise ground to obtain the required recoveries and alignments , preferably employing a servo of a second type based on an image processing:

o Mapping images by techniques known image processing (pixel correlation sub-images, mapping primitives), calculation of geometric transformations existing between these images from the preceding correspondence set, to measure directly recovery and alignment thereof when projected on the ground. This mapping is applied to adjacent images from a first scanning of a strip, but also on the overlapping images from successive scans over the same area of ​​ground.

o Measures correlations and geometric transformations between projected images and those of the initial strip to accurately measure the position of the projected images in the coordinate system of the initial bands, return from these measurements of the angular position of movement of the line

sighting (in relative and absolute if one has a precise angular reference for the first headband)

o Operation of the measured angular position of the sighting line for applying accordingly the angular corrections with a possible temporal matched filter. These angular corrections take into account the measured angular position of the line of sight and that it should have in imposing the recovery criterion "low" on the adjacent images and the recovery criterion "strong" to exist between overlapping images obtained during successive scans over the same area of ​​land underlying different times.

The result is a fine positioning of the under ground line that would not have been possible without the contribution of the image processing despite the fact that the system is not accurate enough with direct measurement of angular position the line of sight. Indeed, when the inertial means are insufficient and that part of the application can not simply technical servo presented above, image processing ensures precisely, a minimal percentage points each strip of land chosen is revisited by several successive scans angles of view and different lines of sight and the successive images of each band will be properly aligned and spliced. As a reminder : the line of sight is the direction connecting the carrier of the image sensor to a point of land covered by the latter and the view angles are the angles of the line of sight in the terrestrial reference frame, in other words the absolute angles of the line of sight. By definition, these angles are also the following angles which a point on the ground is seen by the sensor (and vice versa the following angles which a point on the ground "sees" the sensor).

2) Once the strip of land 4 has been scanned by the scanning front handset, or the ground strips 4a, 4b, 4c have been scanned in parallel, to scan the line of sight directly rearward as shown in Figure 2b by the downward arrow, dotted, or the arrow 8 in Figure 1a, so as to bring the line of sight (or almost to) the

starting position selected in the scene, but from a different point of view due to the advance of the carrier. This scan back or against-scanning can be direct is to say, be done in one "step" (or jump) then it is faster than before scanning. This against-scanning also may not be direct. Indeed during the back scan operations can eventually achieve "step-and-stare" as described above, depending on the speed and the time allotted to it. The ground position of the LOS, reached at the end of the backward scan (= return position) may not coincide exactly with the position of initial ground LOS initially obtained on the first strip: a shift, previously set by the system and typically 20% of the size of the first image is accepted especially when strip chart as seen below. This back position is determined by the image processing unit acquired up to that moment. The expected return position, closer to the position of the initial line of sight on the first band takes into account the constraints prohibited the enslavement (called constraints "deadlock") and the maximum deflection allowed by the system to the line of sight. A carrier drift can cause servo corrections by the system in order to bring the line of sight toward the forward dot. An image processing, if it is applied, also can accurately measure the

The foregoing description is based on a prior scan, that is to say in the direction of the trajectory of the carrier and against-scan in the opposite direction. More generally we consider a forward scan and against-scan back that are not necessarily related to the trajectory of the carrier, but the direction is chosen so as to minimize travel constraints of the line of sight and to optimize the number round trip made by the line of sight. It is thus possible to adapt the choice of the scanning direction to strips closed on themselves or steeply inclined with respect to the path. One can also have as outward scan a scanning in the direction opposite to the trajectory (= a backward scan), then with as against-scanning direction, a forward scan.

The front and side movements of "step-and-stare" imposed on the line of sight permit to sweep the desired number of bands, and determine the number of times they are searched and the desired shape and orientation of each gang.

According to a particular case illustrated in Figure 1 b, the stage floor form one or more bands 4 which can be opened or closed for coating for example the shape of a circle or a rectangle. A plurality of concentric shapes can be established simultaneously around an original shape, joining the idea of ​​distinct bands. Scanning a closed strip can be done by a forward scan single loop repeated all along the closed bands (in the case where they surround the carrier as is the case in Figure 1b, the scan loop of the line of sight is made all around the carrier by successively scanning the different bands as more particularly illustrated in Figure 1 c) or return scans repeated between 2 points of the strip, depending on the choice made on the servo.

The method provides two scanning modes go back next we want to or not to maximize the number of points of view on the ground.

A first mode of repeated scanning and scan-cons is suitable to maximize the number of points of view on the ground for a given band size. 3 illustrates the movements of the line of sight in the case of scans back and forth in elevation / deposit made for a scene ground 10km long and 500m wide, from a POD embedded sub scanning plane the scene 6 km in height with a lateral distance of the strip relative to the plumb of the POD 8 km, and a maximum angular displacement of +/- 45 °. In the example, the different points of the ground can be seen up to 1 6 times at different times and different viewpoints.

According to a second embodiment, the stage floor form one or more strips which change during the advance of the carrier. This is for example drop bands which take place following the advance of the carrier, and which are therefore not limited in the longitudinal direction. Banners images that are generated are constantly updated according to different points of view angle throughout the movement of the seamless carrier. The length of the strip corresponds to almost exactly to the distance traveled by the carrier (tens or hundreds of kilometers in the case of an aircraft).

In the case of the second mode where you want to view several different bands following continuous views, we show that it is enough to have a constant movement in the figures sweeps back and forth, and that synchronizing properly, it leads to the production of bands corresponding to bands shifted to the ground (in the direction of the trajectory of the carrier) whose points are reviewed successively in time the following various points of view angle. The number of times that all points of the band are seen depends directly on the maximum scanning speed of the beam to the ground, but will still be lower than the first mode of scanning and scan-cons described above. In our example, there will be 5 different angular views from each point of the terrain being flown continuously.

Prior to the implementation of this enslavement of

LOS of the invention, optimizing the control parameters according to:

- the average distance and the spatial extent of the scene to be scanned,

- the constraints of shapes of bands associated with the scene,

- the desired average rate of recovery of images from several bands and from the same portion of strip of ground to be scanned, and the average recovery rate of two adjacent images of the same band,

- the maximum speed and maximum acceleration of the scanning,

- the inclination of the field of view of resolution of the sensor and,

- the integration time required to correctly image the scene,

to determine :

image size on the ground scanned by the sensor at a time t, the minimum displacement (lateral and frontal step) to be performed by scanning for partially covering the different images with each other to produce a continuous image strip. This displacement is calculated at each time taking into account the available ground current 2 adjacent frames, but can also be adjusted generally complete after a first angular scan;

the instantaneous scanning speed or the average scanning speed required to achieve the above condition and to comply with the integration time and the time required to perform the micro-scanning pattern "step-and-stare" associated;

the maximum angular displacement (= maximum angle that crosses LOS) can be realized which ensures that the scanning speed constraints and maximum acceleration of the sensor will not be exceeded;

the maximum number of scans return to cover the stage floor and the maximum lateral deflection can be realized;

the maximum average recovery rate that can actually be achieved in the recovery of successive bands and the minimum average rate of overlap between adjacent images of the same bandage to ask for.

Scans are going at a speed taking into account as indicated above the various constraints of the system trajectory of the carrier, size ground photographs integration time, maximum speed and acceleration of the servo according to different axes (e.g. Site / deposit that may not have the same constraints).

The against-scans (Full Scan reverse the previous) Direct are at maximum speed, regardless of integration time or imaging-related system data (unless the system allows), and are linked only the maximum acceleration and maximum speed of the servo.

Scans back and forth must be adjusted angularly to ensure a sweeping or against-scanning on the same portion of the soil.

The purpose of this control is to create contiguous or overlapping image bands, regenerated at regular time intervals according to different angular views. This allows the operator:

To see the different elements of the scene according to different viewpoints. This allows particularly unmasking of objects that may be obscured by the elements of the scene, but also to see the various facets of an object in the scene to better recognize.

Rebuild with algorithms and appropriate software and known 3D structures of the scene and dress up this reconstruction with different images headbands covering these structures.

To allow other algorithms and software to restore an ortho-photography conform to different parts of the stage and following different plans. This aspect is not obtained or only a very small extent by other scanning techniques.

The method can be generalized to the servo other types of sensors (radar, lidar, ...).

The method is applicable to an aircraft as in the examples, but is fully applicable to any moving carrier (land vehicle, boat, short or long range drone, ...).

From the images acquired by this method can be considered as industrial applications:

- SAR or specific lidar applications demanding passages different inclinations,

generation mosaicked cards conforming very determined over a wide area, including very large distance by using an optical low-field and high resolution,

the generation of these same cards at different times with different presentation angles, which facilitates the detection of objects with low contrast or partially masked,

the possibility of 3D reconstruction on a large spatial extent, high resolution and greater distance.

The present invention can be implemented from hardware and / or software elements. The slaving the line of sight may especially implement from a computer program product, the computer program comprising code instructions for performing the steps of the control method. It is recorded on a computer readable medium. The support may be electronic, magnetic, optical, electromagnetic, or be an infrared type diffusion medium. Such carriers are for example, semiconductor memories (Random Access Memory RAM, read-only memory ROM), tapes, diskettes or magnetic or optical disks (Compact Disk - Read Only Memory (CD-ROM), Compact disk - read / write (CD-R / W) and DVD).

CLAIMS

A method of acquiring images of a scene at predetermined soil, from a carrier (2) travel along a trajectory (20) and equipped with an optical sensor having a line of sight (3), which comprises a step of acquiring by the image sensor (1) of the successive stage during movement of the wearer, and a control step of the angular direction of the line of sight by a processing unit connected to the sensor, the acquisition being performed:

- for a first position (P1) of the carrier along its path, with a slaving of the angular direction of the aiming line for performing scanning of a strip (4, 4a) of the predetermined stage, said scanning go, combined with a scanning mode "step-and-stare" first band image (40, 40A1) being acquired,

- and in that at least one other strip of images (40a2) is acquired on the same ground area as the first strip by repeating these steps of scanning for at least one other position (P2) of the carrier along its path, each image being acquired another strip for a strip portion with a recovery rate with the (or) frame (s) of the first strip in the same plot of land strip exceeds a predetermined safe rate overlap, the overlapping images of a strip to the other from the same plot of land being thus acquired respectively during these iterations in different directions of the line of sight,

characterized in that the mode "step-and-stare", which includes micro-movements of said translation "steps" having an amplitude controlled by the processing unit, comprises:

o at least one "step" in main component perpendicular to the outward scan and said side (5), combined with

o at least one "step" (7) with main component parallel to said longitudinal and outward scan,

o a micro-movement "stare" bi-axis to compensate for a translational movement of the line of sight during acquisition of each image,

and in that the slaving of the angular direction of the line of sight is performed by image processing on the acquired images.

A method for acquiring images according to the preceding claim, characterized in that the first position (P1) and the position (P2) of successive scanning the base carrier used are multiple and decomposed into as many positions as there are d images when scanning when the wearer moves during scanning.

A method of acquiring images according to one of the preceding claims, characterized in that the acquisition is carried out for K different carrier positions, where K is between 5 and 20.

A method of acquiring images according to one of the preceding claims, characterized in that the slaving of the angular direction of the line of sight is performed by image processing on the acquired images so that within a same strip, adjacent frames are aligned with an upper alignment quality to a predetermined quality.

A method of acquiring images according to one of the preceding claims, characterized in that the slaving of the angular direction of the line of sight is performed by image processing on the acquired images so that within a same strip, adjacent images have a recovery rate greater than a predetermined low recovery rate, so as to obtain a continuous band.

A method of acquiring images according to one of the preceding claims, characterized in that the slaving of the angular direction of the line of sight is performed by image processing on the acquired images so that the images from two

bands on the same stage portion are aligned with an upper alignment quality to a predetermined quality.

7. A method for acquiring images according to one of the preceding claims, characterized in that the micro-movement "stare" bi- axis is further associated with a-against rotational movement of the line of sight (3) determined by the processing unit, to compensate for a rotational movement of the sight line during the image acquisition.

A method of acquiring images according to one of the preceding claims, characterized in that at least one further strip (4b, 4c) of predetermined field is associated with the stage and is traversed by the scanning performed from said positions carrier and in that the "step-and-stare" mode further comprises at least one lateral movement (6) of strip of land change, to move from a strip of land to another strip of land.

A method of image acquisition according to the preceding claim, characterized in that the lateral movement (6) of strip of land change is combined to a band changeover frontal movement.

10. A method for acquiring images according to one of the preceding claims, characterized in that the forward scan is performed in the direction of the path (20).

January 1. A method of acquiring images according to one of the preceding claims, characterized in that a reverse-scan against the outward scan, against said-scanning direction, is carried out following the forward scan and before retries.

12. Image acquisition method according to the preceding claim, characterized in that the against-scan is against a direct-scanning (8).

13. Image acquisition method according to claim 1 1, characterized in that the images (1) are acquired during the scan-against which is combined with a scanning mode "step-and-stare" with micro- said translational movements "steps" which have an amplitude controlled by the processing unit so that the images (1) successively acquired overlap, which comprises: a. at least one "step" perpendicular to the scanning-and against said lateral, laterally to scan the area combined with b. at least one "step" parallel to and against said longitudinal-scanning step to scan the longitudinally area, c. a micro-movement "stare" bi-axis to compensate for a translational movement of the line of sight during acquisition of each image.

14. image acquisition method according to one of the preceding claims, characterized in that at least a dividing strip takes place with the progress of the wearer.

15. image acquisition method according to one of the preceding claims, characterized in that at least a dividing strip (4a,

4b) is closed on itself.

16. A method for acquiring images according to one of the preceding claims, characterized in that at least two ground strips (4) traversed by the line of sight intersect or touch each other in places.

17. A method for acquiring images according to one of the preceding claims, characterized in that the carrier (2) is an aircraft.

18. A computer program product, said computer program comprising code instructions for performing the steps of the image acquisition method according to any one of claims 1 to 17 when said program is run on a computer.

Documents

Application Documents

# Name Date
1 201817017998-IntimationOfGrant09-08-2023.pdf 2023-08-09
1 201817017998-TRANSLATIOIN OF PRIOIRTY DOCUMENTS ETC. [14-05-2018(online)].pdf 2018-05-14
2 201817017998-PatentCertificate09-08-2023.pdf 2023-08-09
2 201817017998-STATEMENT OF UNDERTAKING (FORM 3) [14-05-2018(online)].pdf 2018-05-14
3 201817017998-PRIORITY DOCUMENTS [14-05-2018(online)].pdf 2018-05-14
3 201817017998-FORM 3 [19-09-2022(online)].pdf 2022-09-19
4 201817017998-FORM 3 [10-12-2021(online)].pdf 2021-12-10
4 201817017998-FORM 1 [14-05-2018(online)].pdf 2018-05-14
5 201817017998-FER.pdf 2021-10-18
5 201817017998-DRAWINGS [14-05-2018(online)].pdf 2018-05-14
6 201817017998-DECLARATION OF INVENTORSHIP (FORM 5) [14-05-2018(online)].pdf 2018-05-14
6 201817017998-2. Marked Copy under Rule 14(2) [09-09-2021(online)].pdf 2021-09-09
7 201817017998-COMPLETE SPECIFICATION [14-05-2018(online)].pdf 2018-05-14
7 201817017998-COMPLETE SPECIFICATION [09-09-2021(online)].pdf 2021-09-09
8 201817017998.pdf 2018-05-15
8 201817017998-DRAWING [09-09-2021(online)].pdf 2021-09-09
9 201817017998-FER_SER_REPLY [09-09-2021(online)].pdf 2021-09-09
9 abstract.jpg 2018-06-28
10 201817017998-Information under section 8(2) [09-09-2021(online)].pdf 2021-09-09
10 201817017998-Verified English translation (MANDATORY) [10-09-2018(online)].pdf 2018-09-10
11 201817017998-OTHERS [09-09-2021(online)].pdf 2021-09-09
11 201817017998-Proof of Right (MANDATORY) [10-09-2018(online)].pdf 2018-09-10
12 201817017998-FORM-26 [10-09-2018(online)].pdf 2018-09-10
12 201817017998-PETITION UNDER RULE 137 [09-09-2021(online)].pdf 2021-09-09
13 201817017998-FORM 3 [10-09-2018(online)].pdf 2018-09-10
13 201817017998-Retyped Pages under Rule 14(1) [09-09-2021(online)].pdf 2021-09-09
14 201817017998-Certified Copy of Priority Document (MANDATORY) [10-09-2018(online)].pdf 2018-09-10
14 201817017998-FORM 4(ii) [07-06-2021(online)].pdf 2021-06-07
15 201817017998-FORM 3 [03-03-2021(online)].pdf 2021-03-03
15 201817017998-Power of Attorney-250918.pdf 2018-10-01
16 201817017998-FORM 3 [03-05-2020(online)].pdf 2020-05-03
16 201817017998-OTHERS-250918.pdf 2018-10-01
17 201817017998-OTHERS-250918-.pdf 2018-10-01
17 201817017998-FORM 18 [28-08-2019(online)].pdf 2019-08-28
18 201817017998-Correspondence-250918.pdf 2018-10-01
18 201817017998-FORM 3 [19-12-2018(online)].pdf 2018-12-19
19 201817017998-Correspondence-250918.pdf 2018-10-01
19 201817017998-FORM 3 [19-12-2018(online)].pdf 2018-12-19
20 201817017998-FORM 18 [28-08-2019(online)].pdf 2019-08-28
20 201817017998-OTHERS-250918-.pdf 2018-10-01
21 201817017998-FORM 3 [03-05-2020(online)].pdf 2020-05-03
21 201817017998-OTHERS-250918.pdf 2018-10-01
22 201817017998-FORM 3 [03-03-2021(online)].pdf 2021-03-03
22 201817017998-Power of Attorney-250918.pdf 2018-10-01
23 201817017998-FORM 4(ii) [07-06-2021(online)].pdf 2021-06-07
23 201817017998-Certified Copy of Priority Document (MANDATORY) [10-09-2018(online)].pdf 2018-09-10
24 201817017998-FORM 3 [10-09-2018(online)].pdf 2018-09-10
24 201817017998-Retyped Pages under Rule 14(1) [09-09-2021(online)].pdf 2021-09-09
25 201817017998-FORM-26 [10-09-2018(online)].pdf 2018-09-10
25 201817017998-PETITION UNDER RULE 137 [09-09-2021(online)].pdf 2021-09-09
26 201817017998-OTHERS [09-09-2021(online)].pdf 2021-09-09
26 201817017998-Proof of Right (MANDATORY) [10-09-2018(online)].pdf 2018-09-10
27 201817017998-Information under section 8(2) [09-09-2021(online)].pdf 2021-09-09
27 201817017998-Verified English translation (MANDATORY) [10-09-2018(online)].pdf 2018-09-10
28 201817017998-FER_SER_REPLY [09-09-2021(online)].pdf 2021-09-09
28 abstract.jpg 2018-06-28
29 201817017998-DRAWING [09-09-2021(online)].pdf 2021-09-09
29 201817017998.pdf 2018-05-15
30 201817017998-COMPLETE SPECIFICATION [14-05-2018(online)].pdf 2018-05-14
30 201817017998-COMPLETE SPECIFICATION [09-09-2021(online)].pdf 2021-09-09
31 201817017998-DECLARATION OF INVENTORSHIP (FORM 5) [14-05-2018(online)].pdf 2018-05-14
31 201817017998-2. Marked Copy under Rule 14(2) [09-09-2021(online)].pdf 2021-09-09
32 201817017998-FER.pdf 2021-10-18
32 201817017998-DRAWINGS [14-05-2018(online)].pdf 2018-05-14
33 201817017998-FORM 3 [10-12-2021(online)].pdf 2021-12-10
33 201817017998-FORM 1 [14-05-2018(online)].pdf 2018-05-14
34 201817017998-PRIORITY DOCUMENTS [14-05-2018(online)].pdf 2018-05-14
34 201817017998-FORM 3 [19-09-2022(online)].pdf 2022-09-19
35 201817017998-STATEMENT OF UNDERTAKING (FORM 3) [14-05-2018(online)].pdf 2018-05-14
35 201817017998-PatentCertificate09-08-2023.pdf 2023-08-09
36 201817017998-IntimationOfGrant09-08-2023.pdf 2023-08-09
36 201817017998-TRANSLATIOIN OF PRIOIRTY DOCUMENTS ETC. [14-05-2018(online)].pdf 2018-05-14

Search Strategy

1 201817017998table1E_20-11-2020.pdf

ERegister / Renewals

3rd: 19 Oct 2023

From 18/11/2018 - To 18/11/2019

4th: 19 Oct 2023

From 18/11/2019 - To 18/11/2020

5th: 19 Oct 2023

From 18/11/2020 - To 18/11/2021

6th: 19 Oct 2023

From 18/11/2021 - To 18/11/2022

7th: 19 Oct 2023

From 18/11/2022 - To 18/11/2023

8th: 19 Oct 2023

From 18/11/2023 - To 18/11/2024

9th: 21 Oct 2024

From 18/11/2024 - To 18/11/2025

10th: 15 Oct 2025

From 18/11/2025 - To 18/11/2026