This technique is an image processing apparatus, image processing method, and a program, for example, an image processing apparatus which allow to simplify the stitching process, image processing method, and a program.
BACKGROUND
[0002]
　Empty Tosa image by using a Cessna and drones or stitching being performed or to map.
[0003]
　Further, it has also been proposed to sense the growth condition of the plant. Plant photosynthesis is not a light energy, being affected by the number of photons is a light particles are known. By measuring the effective photon flux density photosynthetic plants, it has been proposed to sense the growth state of the plant (e.g., see Patent Document 1).
CITATION
Patent Document
[0004]
Patent Document 1: JP 2012-163482 JP
Summary of the Invention
Problems that the Invention is to Solve
[0005]
　Images were transferred empty location city like buildings is large, easy to extract a feature quantity, relative ease or stitching, or can be mapped. However, small fields of buildings, such as in forests, difficult to extract the feature amount of the image, or stitching, it is difficult or mapping.
[0006]
　For example, in order to sense the growth of plants, and transferred empty field, or stitching empty Transfer images, if or mapping, difficult to extract the feature quantity from the sky Tosa image, Ya stitching it was difficult to perform the mapping.
[0007]
　This technology has been made in view of such circumstances, it easy to extract the feature amount, and is to enable easy stitching or mapping.
Means for Solving the Problems
[0008]
　An image processing apparatus according to an embodiment of the present technology, based on the plurality of first image for the first imaging region to generate a first reference image relating to the first imaging region, at least said first imaging region an image generator part generates the second reference image relating to the second imaging area based on the second plurality of second related to the imaging area of ​​the image that overlaps, with the first reference image and the second comprising of, based on the reference image, and a processing unit that generates positioning information indicating the correspondence relationship between the first imaging region and the second imaging region.
[0009]
　According to another embodiment of the present technology, based on the plurality of first image for the first imaging region to generate a first reference image relating to the first imaging region, at least said first imaging region some generates a second reference image relating to the second imaging area based on the second image of the plurality relating to the second imaging area overlapping, with the first reference image and the second reference image based on, and generates positioning information indicating the correspondence relationship between the first imaging region and the second imaging region, comprising the steps.
[0010]
　A program according to an embodiment of the present technology, the computer generates a first reference image for the first image capturing area based on the plurality of first image for the first imaging region, and the first imaging region generating a second reference image relating to the second imaging area based on at least part of the plurality for a second imaging area overlapping the second image, the first reference image and the second reference image based on the bets, and generates positioning information indicating the correspondence relationship between the first imaging region and the second imaging region, to execute processing including steps.
[0011]
　According to an embodiment of the present technology, an image processing method, and program, the first reference image for the first imaging region based on a plurality of first image for the first imaging region is generated, the at least partially the second of the second reference image relating to the second imaging region based on a plurality of the second image to an imaging region overlapping generating one of the imaging region, the first reference image and the second based on the reference image, positioning information indicating a correspondence relationship between the first imaging region and the second imaging region is generated.
[0012]
　The image processing apparatus may be an independent apparatus or may be an internal block constituting one apparatus.
[0013]
　The program can be transmitted via a transmission medium or by being recorded on a recording medium, it can be provided.
The invention's effect
[0014]
　According to one aspect of the present technology, and easy to extract the feature amount, easily stitching or mapping.
[0015]
　Here, the advantages described in the present invention is not necessarily limited, it may be any of the effects described in the present disclosure.
BRIEF DESCRIPTION OF THE DRAWINGS
[0016]
FIG. 1 is a diagram showing the configuration of an embodiment of an image processing system to which the present technique is applied.
It is a diagram showing a configuration of another embodiment of an image processing system [2] The present technology is applied.
It is a diagram illustrating a configuration example of FIG. 3 the image processing system.
It is a diagram illustrating a configuration example of FIG. 4 the image processing system.
It is a diagram for FIG. 5 the unit pixel will be described.
6 is a diagram for explaining the characteristics of the filter.
7 is a diagram for explaining the characteristics of the filter.
8 is a diagram illustrating a configuration of an example of a multi-camera.
9 is a diagram illustrating a configuration of an example of a multi-camera.
It is a diagram for explaining FIG. 10 characteristic of the filter.
11 is a diagram for explaining the process when aerial.
It is a diagram for explaining an image obtained by [12] aerial.
13 is a diagram for describing the multispectral image.
14 is a diagram for explaining generation of the reference image.
15 is a diagram for explaining stitch process of the reference image.
16 is a diagram for explaining stitch process of the reference image.
17 is a diagram for explaining stitch process of the inspection image.
18 is a diagram for explaining the operation of the image processing system.
19 is a diagram for describing the light source information.
[Figure 20] object by reflectance is a diagram for describing different.
21 is a diagram for explaining the spectral characteristics of the MS sensor.
22 is a flowchart for explaining the first generation process.
FIG. 23 is a flowchart for explaining a first feature quantity arithmetic.
It is a flow chart for explaining FIG. 24 the reference image generation process.
[FIG 25 is a flowchart for explaining a second feature quantity calculation process.
[FIG. 26] is a diagram for explaining the creation of the reduced image.
FIG. 27 is a diagram for explaining generation of the reference image.
FIG. 28 is a flowchart illustrating a third feature calculation processing.
Is a diagram for dividing process will be described in the FIG. 29 block.
It is a diagram for explaining generation of [30] Reference image.
FIG. 31 is a flowchart for explaining a third reference image generation process.
[FIG. 32] is a diagram for describing re-configuration of the feature image.
[33] is a diagram for describing re-configuration of the feature image.
FIG. 34 is a flowchart for explaining a fourth process of generating.
[FIG. 35] is a flowchart for explaining a fifth generation processing.
[FIG. 36] is a diagram for explaining another configuration of the image processing system.
[FIG. 37] is a flowchart for illustrating processing for generating a sixth.
[FIG. 38] is a diagram for explaining a recording medium.
DESCRIPTION OF THE INVENTION
[0017]
　Hereinafter, embodiments of the present technology (hereinafter, referred to as embodiments) will be described.
[0018]
　
　FIG 1 is a diagram showing the configuration of an embodiment of an image processing system including an image processing apparatus to which the present technology is applied.
[0019]
　The image processing system 10 images a subject, performs stitch processing, a system for creating an image that is desired. For example, in the image processing system 10, plants (vegetation) targeted as a subject map representing the state of growth of the plant (image) is created.
[0020]
　1, the image processing system 10 is comprised of an imaging device 11, the illumination sensor 12, hub 13, and a computing unit 14. The imaging device 11, the illumination sensor 12, and the arithmetic unit 14 are connected to each other through the hub 13.
[0021]
　The imaging device 11 takes an image of an object to be measured. Illuminance sensor 12 includes a light source, a device for measuring the example illuminance of sunlight, and supplies the measured illuminance information (intensity value), the imaging device 11.
[0022]
　For example, imaging device 11 is mounted on the remotely piloted or autonomous lower unmanned aircraft called like a drone (ground side), the illuminance sensor 12 is mounted on the upper side of the unmanned aerial vehicle (empty side).
[0023]
　Note here, the imaging device 11 and the illuminance sensor 12 is continuously described as being separate, may be provided in the same housing. The imaging device 11 and the illuminance sensor 12, it is also possible to adopt a configuration that is included in an unmanned aircraft.
[0024]
　Here, although the description will be continued by taking as an example the case of processing an image obtained by aerial using unmanned aircraft, such as an image obtained by panoramic photographing, processing the image obtained by photographing a non Aerial case also, applying the present technology may.
[0025]
　The imaging device 11 takes an image of a subject, and outputs the data obtained by the imaging, the arithmetic unit 14 via the hub 13. Illuminance sensor 12 is a sensor for measuring the illuminance, and outputs an illuminance value is a measurement result, the arithmetic unit 14 via the hub 13.
[0026]
　Arithmetic unit 14 is a device having an arithmetic function by the circuit such as a CPU (Central Processing Unit) and FPGA (Field Programmable Gate Array). For example, computing device 14 is configured as a personal computer or a dedicated terminal.
[0027]
　In the image processing system 10 shown in FIG. 1, the computing device 14 such as a personal computer, a local environment via the hub 13, had the image processing is performed, a cloud environment via a network, the image processing line it may be cracked.
[0028]
　Figure 2 is, as another configuration example of an image processing system, shows a configuration example of an image processing system 30 corresponding to a cloud environment.
[0029]
　In the image processing system 30 of FIG. 2, the imaging device 11 and the illuminance sensor 12, like the imaging device 11 and the illumination sensor 12 in FIG. 1, and outputs the image data and intensity values, the client device 31 via the hub 13 .
[0030]
　The client device 31 is constituted by a personal computer or the like, via the hub 13, the image data and the illuminance value input from the imaging device 11 and the illumination sensor 12, and outputs the router 32. That is, the client device 31 is corresponds to the arithmetic unit 14 of FIG. 1, the image processing is not performed, or is provided as a device that does not only partially.
[0031]
　Router 32 may, for example, a router for mobile, via the base station 33 may be connected to a network 34 such as the Internet. Router 32, the image data and the illuminance value input from the client device 31 via the network 34, to the server 35.
[0032]
　Server 35 via the network 34, receives the image data and the illuminance value transmitted from the client device 31. Here, the server 35 includes a computing unit 14 function of the equivalent functions or at least some of the functions shown in FIG.
[0033]
　The server 35, the storage 36 is connected, the supplied image data and intensity values ​​to the server 35, or stored in the storage 36 as needed, the data needed when performing such stitching process or appropriately read.
[0034]
　The image processing system 10 (or the image processing system 30), (including functional) having the structure shown in FIG. Here, for applying the image processing system 10, an image processing system 30 to distinguish between the image processing system shown in FIG. 3 describes an image processing system 50.
[0035]
　The image processing system 50 includes a lens 101, an exposure unit 102, MS sensor 103, the designated wavelength processor 104, reference image generation processing unit 105, the inspection wavelength image extracting unit 106, the reference image stitching processing unit 107, and the inspection image stitching processing It is configured to include a section 108.
[0036]
　The image processing system 50 shown in FIG. 3, when configured by the image processing system 10 shown in FIG. 1, or, for the case of a configuration using an image processing system 30 will be described with reference to FIG.
[0037]
　Top view of FIG. 4 is an image processing system 50 shown in FIG. 3, below shows an example configuration, here showing a configuration example A to G. First, as shown in the upper diagram of FIG. 4, the image processing system 50, divided into four. Lens 101, an exposure unit 102, MS sensor 103, an imaging unit 61 for imaging a subject.
[0038]
　Designated wavelength processor 104, reference image generation processing unit 105, the inspection wavelength image extracting unit 106, and the reference image, and the image generating unit 62 for generating an image, such as inspection wavelength images. Reference image stitching processing unit 107, a first stitch processing unit 63 for performing stitch process. Inspection image stitching processing unit 108, a second stitch processing unit 64 for performing stitch process.
[0039]
　Configuration Example A, the image processing system 50, a case that is configured to complete the imaging device within 11, the imaging unit 61, the image generating unit 62, the first stitch processing unit 63, and the second stitch processing unit 64 is an example of the configuration included in all the imaging device 11. Although not shown in FIGS. 1 and 2, only the imaging device 11, it can be configured to perform up stitch process.
[0040]
　Configuration example B, of the image processing system 50, the imaging unit 61, an image generation unit 62 provided in the image pickup apparatus 11, the arithmetic unit and the first stitch processing unit 63 and the second stitch processing unit 64 14 (FIG. 1 ) or is a configuration example in which the server 35 (FIG. 2).
[0041]
　Configuration Example C, in the image processing system 50 is provided with an imaging unit 61 to the imaging device 11, the image generating unit 62, the first stitch processing unit 63, and the second stitch processing unit 64 an arithmetic unit 14 (FIG. 1) or a structural example in which the server 35 (FIG. 2).
[0042]
　Configuration example D, of the image processing system 50 is provided with an imaging unit 61 to the imaging device 11, an image generation unit 62, the client device 31 is provided (FIG. 2) within the first stitch processing unit 63 second the stitch processing unit 64 is a configuration example in which the server 35 (FIG. 2).
[0043]
　Configuration Example E, in the image processing system 50 is provided with an imaging unit 61 to the imaging device 11, the image generating unit 62, the first stitch processing unit 63, and the second stitch processing unit 64 to the server 35 (FIG. 2 ) in a configuration example of providing.
[0044]
　Configuration Example F, of the image processing system 50 is provided with an imaging unit 61 to the imaging device 11, the image generation unit 62 and provided with the first stitch processing unit 63 to the client device 31 (FIG. 2), the second stitch the processing unit 64 is a configuration example in which the server 35 (FIG. 2).
[0045]
　Configuration Example G, of the image processing system 50 is provided with an imaging unit 61 to the imaging device 11, an image generation unit 62, the client device 31 is provided (FIG. 2) within the first stitch processing unit 63 the server 35 provided (FIG. 2), a configuration example in which a second stitch processing unit 64 to the client device 31 (FIG. 2).
[0046]
　The image processing system 50 is configured Examples A to may be any arrangement of G, it may be configured as a single device, or may be composed of a plurality of devices. According to this technique, to apparent hereinafter, also have any configuration, it is possible to reduce the processing, and shorten the time required for the processing, it is possible to reduce the processing load.
[0047]
　Returning to the description of the image processing system 50 shown in FIG. Lens 101, an exposure unit 102, MS sensor 103, in any of the exemplary configurations A to G described with reference to FIG. 4, is configured to be included in the imaging device 11.
[0048]
　In the image pickup apparatus 11, light (reflected light) from the object, such as the measuring object is incident on MS sensor 103 via the lens 101 and the exposure unit 102. The MS of MS sensor 103, means multispectral. As described later, the imaging device 11, from one unit pixel, and is configured so as to obtain a plurality of different wavelength signals of.
[0049]
　Exposure unit 102, the MS sensor 103, as in the state in which the signal charge is in the dynamic range without saturation sensing is performed, an optical system such as a lens 101 and aperture amount of the iris (aperture), MS sensor 103 exposure time, by adjusting the shutter speed, performs the exposure control.
[0050]
　MS sensor 103, and a MS filter 103-1 and the sensor 103-2 Metropolitan as shown in FIG. MS filter 103-1 can be an optical filter corresponding to the index to be measured, a plurality of different wavelengths, there is a filter that transmits respectively. MS filter 103-1, the light incident through the lens 101, transmits the sensor 103-2 MS sensor 103.
[0051]
　Sensor 103-2 is provided at its sensor surface, an image sensor consists of a sensing element in which a plurality of pixels are two-dimensionally arranged in a repeating pattern. MS sensor 103, MS light that has passed through the filter 103-1, by detecting the sensing element (sensor 103-2), the measurement signal corresponding to the amount of light (measurement data), the designated wavelength processor 104 and outputs it to.
[0052]
　Here, for example, if the image processing system 50 is used as a device for sensing the vegetation, photosynthetic photon flux density (PPFD: Photosynthetic Photon Flux Density) is to be sensed. Plant photosynthesis is governed by the number of photons is a light particles (photons), photons at a wavelength of 400 nm ~ 700 nm is the absorption wavelength of chlorophyll is in unit time, the number of incident per unit area shown in becomes the PPFD values.
[0053]
　When calculating the PPFD values, RGB signal is required, as MS filters 103-1, R (red), G (green), B (blue), respectively transmit the wavelengths, such as IR (infrared light) is intended to filter are combined. For example, as shown in FIG. 3, one unit pixel is composed of vertical × horizontal = 2 × 4 = 8 pixels, each pixel receives light of wavelength A through H. MS filter 103-1 is a filter that transmits each light of wavelength A through H, the pixels of the MS sensor 103 is a sensor for receiving the transmitted light.
[0054]
　In the following description, one unit pixel is composed of 8 pixels as shown in FIG. 5, each pixel is continuously described as a pixel receiving light of different wavelengths. In the following description, simply when the description of the pixel, described as a representative of one pixel in one unit pixel, when the description of a unit pixel, a pixel (s) composed of 8 pixels continue.
[0055]
　The one unit pixel is composed of a sensor for receiving the light of the wavelength A to H, for example, a sensor for receiving light 400 nm ~ 750 nm, 400 nm wavelength A, 450 nm wavelength B, 500 nm wavelength C, the wavelength D the 550 nm, 600 nm wavelength E, 650 nm wavelengths F, the wavelength G 700 nm, it is possible to wavelength H and 750 nm.
[0056]
　Incidentally, the number of pixels included in one unit pixel in this case is an example, rather than the description indicating limitation, for example, four pixels and 2 × 2, be constituted by 16 pixels 4 × 4 good. Further, one unit or range of wavelengths of the pixel the light received by the wavelength of light received by each pixel, but is not limited to the example described above, the wavelength can be properly sense the measurement object being a sensing target it can be set. In addition, the wavelength is not limited to visible light, it may be an infrared light or ultraviolet light.
[0057]
　The MS filter 103-1 that transmits light of a plurality of wavelengths as is an optical filter which transmits narrow band light of a predetermined narrow wavelength band (narrow-band).
[0058]
　Such a camera is using narrow band filters, sometimes called like a multispectral camera, even in the art, can Le used multispectral camera. Here, the multispectral camera Add description.
[0059]
　As shown in FIG. 6, when the wavelength of the light passing through the MS filters 103-1 overlap, by solving the inverse matrix will be described later, it is possible to retrieve the information of an arbitrary wavelength of the subject.
[0060]
　As shown in FIG. 7, if with a reduced range of wavelengths of light passed through the MS filters 103-1 overlap can be a narrow band filter that independently. In this case, it is difficult to retrieve any information of the wavelength of the subject (interpolation from a particular wavelength, can be predicted by extrapolation), without solving the inverse matrix, there is a merit that can predict the sensor output.
[0061]
　As shown in FIG. 6, as the optical filter when the wavelength of light passing through the MS filters 103-1 overlap, is a kind of the metal thin film filter using the metal thin film of aluminum or the like, the surface plasmon resonance it can be applied to those using the principle.
[0062]
　As shown in FIG. 7, the optical filter in the case of reducing the range of wavelengths of light passed through the MS filters 103-1 overlap, applying the one utilizing the principle of the formed Fabry-Perot resonator a thin film on the sensor surface be able to.
[0063]
　It is also possible to configure a multi-spectral sensors in units of pixels of the MS sensor 103-2.
[0064]
　Further, as shown in FIG. 8, by combining multi-filter 151 and the multi-camera 152, it is also possible to constitute the multi-spectral camera.
[0065]
　Multi filter 151, rather than in units of pixels, each pixel group, there is a filter that transmits different light. For example, regions A to H are respectively set to the size of a × b pixels groups, the corresponding region of the multi-camera 152 is constituted by the size of the same size of the corresponding areas A to H (a × b pixels groups are disposed).
[0066]
　Furthermore, as shown in FIG. 9, (corresponding to the lens 101 in FIG. 3) multi-lens array 161, by combining the multi-filter 162, it is also possible to constitute the multi-spectral camera.
[0067]
　The multi-lens array 161, the convex lens are arranged in two dimensions, a light beam incident on the multi-lens array 161, forming a light source image in two dimensions in a multi-filter 162 (for each lens constituting the multi-lens array 161 the light source image formed) to.
[0068]
　Multi filter 162, as well as the multi-filter 151 in FIG. 8, a filter that is divided into a plurality of regions, in this case, is a filter which is divided for each region corresponding to one lens of the multi-lens array 161 there. Or, multi filter 162, as well as the MS filter 103-1 in FIG. 5, there is a filter that passes a predetermined wavelength for each pixel.
[0069]
　For multi-camera shown in FIG. 8 or FIG. 9, as the characteristics of the multi-filter 151 (162), may be employed a filter having the characteristics shown in FIG. 6, characteristics as shown in FIG. 7 It may be a filter with. Furthermore, the multi-filter 151 (162) of the ultra-narrow band with a characteristic as shown in FIG. 10 may be used.
[0070]
　Filter having a characteristic as shown in FIG. 10 may be referred to as dichroic filter, constituted by multilayer films of different dielectric refractive index.
[0071]
　This technique can be applied to any of these multi-camera (used). Here, an example in which using a filter having the characteristics shown in FIG. 6, as shown in FIG. 5, use a multi-camera composed of MS filters 103-1 and MS sensor 103-2 continuing with the description as an example the case you were.
[0072]
　Returning to the description of the configuration of an image processing system 50 shown in FIG. 3, the signal from MS sensor 103 (image signal) is supplied to the specified wavelength processor 104. Designated wavelength processor 104, using the supplied image signal, executes a process of generating an image of a wavelength of light that is desired. MS sensor 103, as described above, a sensor for receiving a plurality of different wavelengths of light, respectively, images obtained from the respective sensors are supplied to the designated wavelength processor 104.
[0073]
　For example, as described with reference to FIG. 5, the sensor for receiving the wavelength A through H respectively, when the unit pixel of the MS sensor 103 is configured, constituted by the output from the pixels for receiving light of a wavelength A is (a picture a) an image is, (the image B) image and an output from the pixels for receiving light of a wavelength B, image and an output from the pixels for receiving light of a wavelength C ( and image C), the image (image D and an output from the pixels for receiving light of a wavelength D), the image (image E and an output from the pixels for receiving light of a wavelength E ), and images (image F and an output from the pixels for receiving light of a wavelength F), the image (image G and an output from the pixels for receiving light of a wavelength G) wavelength H It is an image and an output from the pixels receiving light (image H ) Is supplied to the specified wavelength processor 104.
[0074]
　Thus, the image capturing apparatus 11 according to the present technology, a plurality of images by having received a plurality of different wavelengths are obtained at a time. The image obtained in this temporary, described as appropriate with multispectral image. Multispectral image is a plurality of spectral images extracted according to the characteristics of the multi-spectral filter (MS filter 103-1 as described above).
[0075]
　Designated wavelength processor 104, used as a multi-spectral image supplied from the MS sensor 103 performs inverse matrix calculation using the multispectral image, extracts an image of arbitrary wavelength (generated) is used, or, using any image wavelength which is generated by calculating a multispectral image. Used in the specified wavelength processor 104 image is supplied to the reference image generation processing unit 105 or the inspection wavelength image extracting unit 106.
[0076]
　Reference image generation processing unit 105 generates a reference image. The reference image is an image used in generating the underlying information (Stitch reference information 303 to be described later) when performing stitch process. The reference image is a most characteristic image that is matched to characteristics of the object. The generation of the reference image, with respect to stitch processing a subsequent processing will be described later. Reference image generated by the reference image generation processing unit 105 is supplied to the reference image stitching processing unit 107.
[0077]
　Inspection wavelength image extracting unit 106 generates an inspection image. For example, if you want to generate an image to examine the growth state of a plant, an image of the wavelength suitable for sensing the vegetation (test wavelength) is generated (extracted). Inspection image extracted in the inspection wavelength image extracting unit 106 is supplied to the inspection image stitching processing unit 108.
[0078]
　Reference image stitching processing unit 107 performs a stitch processing on the reference image. Image reference image is generated by being stitched processed (information such images) is used when stitching processing inspection image. Here, an image (information) generated by the reference image is stitched processed describe stitching reference map or stitch reference information, and.
[0079]
　Stitch reference information generated is supplied to the inspection image stitching processing unit 108. The inspection image stitching processing unit 108, the inspection image is also supplied from the inspection wavelength image extracting unit 106. Inspection image stitching processing unit 108, based on the stitch reference information, an inspection image by stitching, to produce a test image after stitching.
[0080]
　
　FIG. 11, with reference to FIG. 12, the operation of the image processing system shown in FIGS.
[0081]
　At time T1 (FIG. 11), the flight plan is set. For example, by moving on a predetermined field 211 in the mobile measuring apparatus 201, when imaging the plant and the like of the field 211 as measured, is flight plan as shown at the time T1 in FIG. 11 is set.
[0082]
　Mobile measuring apparatus 201, for example, unmanned aerial vehicles (UAV: Unmanned Aerial Vehicle) and a, and flight by a propeller-like rotor blades 202 rotates, from the air, sensing the measured object plants etc. Field 211 (empty Taking) to. When aerial, on the surface capable of receiving sunlight of the mobile measuring apparatus 201, the illumination sensor 12 has (Fig. 1) is attached, and is configured to sense the illuminance value. Further, when aerial, on the surface facing the field 211 side of the mobile measuring apparatus 201, the imaging device 11 is attached.
[0083]
　Mobile measuring apparatus 201, or Transfer empty radio control based on the flight plan, previously stored flight plan as coordinate data, by autonomous flight using the position information such as GPS (Global Positioning System), empty or Transfer. Note here, the mobile measuring apparatus 201 has been described as a rotary wing aircraft having a rotary wing 202, the mobile measuring apparatus 201 may be a fixed-wing aircraft.
[0084]
　At time T2, empty real field 211 based on the flight plan shooting is performed. The aerial is imaged so that there is overlap captured image. At time T3 (FIG. 12), Aerial results. As shown in FIG. 12, the captured image is overlap exists. This overlapping portion deletes the other leaving one, trimming and stitching such as synthesis is carried out. As a result, as shown at time T4 (FIG. 12), a plurality of images, images placed so as not overlap are generated.
[0085]
　For example, the time T2, if the 500 images are captured, using 500 images, cropping and, stitching is performed, the final mapping to map is performed.
[0086]
　The imaging device 11 according to the present technique, as described with reference to FIG. 5, have a MS sensor 103 receives light having a wavelength A to H, respectively, to generate the image A to the image H. In this MS sensor 103, as described above, the case where 500 sheets of imaging the image will be described with reference to FIG. 13.
[0087]
　By aerial is performed, (the pixel A, hereinafter, the other also pixels perform the same stated) pixels for receiving light of a wavelength A of MS sensor 103 from the image 301A-1 to image 301A-500 500 images of the resulting aerial.
[0088]
　Similarly, from the pixel B for receiving light of a wavelength of MS sensors 103 B, 500 images of the image 301B-1 to image 301B-500 is obtained as a result Aerial, pixel C that receives light wavelength C from the 500 images an image 301C-1 to image 301C-500 is obtained as a result Aerial, from the pixel D that receives light wavelength D, image 301D-1 to image 301D-500 500 sheets of image is obtained as a Aerial result.
[0089]
　More Similarly, from the pixel E for receiving light of a wavelength E, 500 images of the image 301E-1 to image 301E-500 is obtained as a result Aerial, from pixel F for receiving light of a wavelength F, 500 images of the image 301F-1 to image 301F-500 is obtained as a result Aerial, from the pixels G for receiving light of a wavelength G, 500 images of the image 301G-1 to image 301G-500 is empty obtained as shooting result, the pixels H for receiving light of a wavelength H, 500 images of the image 301H-1 to image 301H-500 is obtained as a result aerial.
[0090]
　In this case, from each of the pixels A to H, it is obtained 500 images. As Aerial result at time T3 (FIG. 12), for example, images or field 211 consisting of image 301A-1 to image 301A-500, an image of the field 211 consisting of image 301B-1 to image 301B-500 is produced It is. If the image to be finally obtained, when an image based on the wavelength A, using the image 301A-1 to image 301A-500, trimming and stitching is performed.
[0091]
　Therefore, if an image to be finally obtained, when an image based on the respective wavelength A through H, with an image 301A-1 to image 301A-500, trimming and stitching is performed, the image 301B-1 or by using an image 301B-500, as such is carried out trimming and stitching, treatment with each of the images 301A through image 301H is performed.
[0092]
　Such processing becomes enormous processing, it becomes greater amount of data to be handled. If, with the structure not in the image processing system 50 shown in FIG. 3, when constructing an image processing system 10 shown in FIG. 1, the number amount of data transferred from the image pickup device 11 to the processing unit 14 , there is a possibility that the time of this transfer is long. Further, the number of calculation amount by the arithmetic unit 14, there is a possibility that the processing time of the arithmetic unit 14 becomes longer. In other words, according to the configuration it is not in the image processing system 50 shown in FIG. 3, the possibility that the transfer and processing time of the data becomes longer was high.
[0093]
　However, in this embodiment, since it has a configuration of an image processing system 50 shown in FIG. 3, it is possible to shorten the transfer and processing time of the data.
[0094]
　Sequentially described for detail, an outline. As shown in FIG. 14, pixels A to 4000 obtained from H (= 8 × 500) Like the image 301 are used of the reference image 302 is generated. First, from the image 301A-1 to 301H-1 is imaged at the same time (the time t1), the reference image 302-1 is generated.
[0095]
　It should be noted that in this case, images captured at the same time, since the image captured by the MS sensor 103 at the same time, the image taken of the same portion. That is, the image 301A-1 to 301H-1 that are captured, for example, time t1 is an image obtained by imaging the same portion, the image of different wavelengths. Thus, a plurality of images 301 obtained by imaging the same portion, the reference image 302 is generated.
[0096]
　From the image 301A-2 through 301H-2 were imaged at time t2, the reference image 302-2 is generated from the image 301A-3 to 301H-3 captured in time t3, the reference image 302-3 is generated . By the same process is repeated, until the reference image 302-500 is generated. That is, the reference image 302-1 to 302-500 are generated.
[0097]
　Reference image 302 is keyed to the characteristics of the object, an image that is the most characteristic images. For example, in the image building as a subject is captured, the building has become a characteristic image, for example, part of the building has become a picture such as is extracted as an edge. Therefore, it is possible to perform stitching by superimposing the image is the characteristic portion coincide.
[0098]
　For example, referring to FIG. 12 again, as shown at time T3, there is a overlap between images, the image, when a reference image 302, for example, a matching between the adjacent reference image 302 characteristic portion (e.g., a region where an edge is detected, hereinafter referred to as appropriate with feature region) overlapping, for example, by superimposing a portion of a building, perform stitching.
[0099]
　For example, the reference image 302 (with some overlap) stitching as shown in FIG. 15 has been to. Figure 15 illustrates the same state as aerial results shown in time T3 in FIG. 12.
[0100]
　Referring to FIG. 15, the right side of the reference image 302-1, in the presence of overlap, the reference image 302-100 is located. On the right side of the reference image 302-100, in the presence of overlap, the reference image 302-150 is located. On the right side of the reference image 302-150, in the presence of overlap, the reference image 302-200 is located.
[0101]
　Below the reference image 302-1, in the presence of overlap, the reference image 302-300 is located. On the right side of the reference image 302-300, in the presence of overlap, the reference image 302-400 is located. On the right side of the reference image 302-400, in the presence of overlap, the reference image 302-450 is located.
[0102]
　When placed by overlapping feature region matching between the reference image 302 adjacent, when the image of the state was obtained as shown in FIG. 15, by removing the overlapping portion, as shown in FIG. 16 stitching reference map 303 can be obtained.
[0103]
　Referring to FIG. 16, the upper right of the stitching reference map 303, a portion of the reference image 302-1 is disposed on the right side of the reference image 302-1, a part of the reference image 302-100 It is placed. Similarly, the right side of the reference image 302-100, a portion of the reference image 302-150 is disposed on the right side of the reference image 302-150, a portion of the reference image 302-200 is located.
[0104]
　More Similarly, on the lower side of the reference image 302-100, a portion of the reference image 302-300 is disposed on the right side of the reference image 302-300, it is disposed a portion of the reference image 302-400, the right of the reference image 301-400, a portion of the reference image 302-450 is located.
[0105]
　Such stitching reference map 303 (Stitch reference information 303) is generated. Stitching reference map 303 (stitch Reference information 303) is information of the positioning between the reference image 302. The reference image 302 is, for example, as described with reference to FIG. 14, is an image generated from a plurality of images 301.
[0106]
　For example, in FIG. 14, the image 301A-1 to 301H-1, the reference image 302-1 is generated from the image 301A-2 through 301H-2, an example was described in which the reference image 302-2 is generated .
[0107]
　The image 301A-1 to 301H-1 is, for example, an image obtained by capturing an image of a predetermined imaging area A, the image 301A-2 through 301H-2 is, for example, an image obtained by capturing an image of a predetermined imaging area B. The imaging region A and the imaging region B is at least partially overlap region.
[0108]
　That is, To summarize the processing up to this point, the reference image 302-1 is generated about the imaging area A on the basis of a plurality of image 301A-1 to 301H-1 to an imaging area A, at least partially overlaps the imaging area A reference image 302-2 are respectively generated an imaging region B on the basis of a plurality of images 301A-2 through 301H-2 relates to an imaging region B.
[0109]
　Then, based on the reference image 302-1 and the reference image 302-2, the stitching reference map 303 (Stitch reference information 303) is generated a positioning information indicating a correspondence relationship between the imaging area A and the imaging region B .
[0110]
　Here, although described as an example of generating the stitching reference map as the positioning information, positioning information, information indicating a relative positional deviation in superimposing the reference image 302-1 and the reference image 302-2 it may be.
[0111]
　After such stitching reference map 303 (Stitch reference information 303) is generated, on the basis of the stitching reference map 303, the inspection image of a wavelength which is desired is mapped.
[0112]
　For example, the test image of the wavelength A is based on the stitching reference map 303, for the case to be mapped, is described with reference to FIG. 17. The image of a wavelength A, as described with reference to FIG. 13, the image 301A-1 to 301A-500 are acquired.
[0113]
　The portion of the reference image 302-1 in the stitching reference map 303 is arranged, a part of the image 301A-1 are disposed. Reference image 302-1, the image 301A-1 is a reference image 302 generated from image 301A-1 to 301H-1 captured at time t1 taken.
[0114]
　Thus, an image 301 used when generating the reference image 302 is mapped. Also when mapped, image 301 is the same shape and the reference image 302 in the stitching reference map 303, the corresponding region is disposed excised with.
[0115]
　The stitching reference map 303 (stitch Reference information 303), an image 301 captured at any time, where to place or also any shape or cut at (size) reveals information is written.
[0116]
　Referring to FIG. 17, in a portion reference image 302-100 at the stitching reference map 303 is arranged, a part of the image 301A-100 are arranged. The portion of the reference image 302-150 at the stitching reference map 303 is arranged, a part of the image 301A-0.99 is disposed on a portion where the reference image 302-200 at the stitching reference map 303 is disposed , part of the image 301A-200 are arranged.
[0117]
　Further, in the portion where the reference image 302-300 at the stitching reference map 303 is arranged, a part of the image 301A-300 are arranged, part reference image 302-300 at the stitching reference map 303 is disposed , the part of the image 301A-400 is disposed in a portion where the reference image 302-450 at the stitching reference map 303 is arranged, a part of the image 301A-450 are arranged.
[0118]
　Thus, to create stitching the reference map 303 (stitch reference information 303), on the basis thereof, by mapping the test image of a desired wavelength, the inspection images of the final desired wavelength (final inspection image and appropriately it can generate according to).
[0119]
　Although cited case of generating the final inspection image of wavelength A in, for example, even when generating a final inspection image of the wavelength B, and using the same stitching reference map 303 (stitch reference information 303) , it can be performed in the manner described above. Furthermore, even when generating a final inspection image of the wavelength C to H, using the same stitching reference map 303 (stitch reference information 303), performed as described above.
[0120]
　That is, by generating the stitching reference map 303 (Stitch reference information 303), the stitching reference map 303 using (Stitch reference information 303), it is possible to generate a final inspection image of a desired wavelength.
[0121]
　Further, other than the acquired image as a multi-spectral image, generated as test image may be from the inspection image to generate a final inspection image. In other words, here, and the description of the wavelength A to H as using MS sensor 103 having a pixel which receives each image A to B corresponds to the wavelength A to H (the image 301A to 301H) is obtained are, but the MS sensor 103 generates an image X of wavelengths that are not generated directly (the wavelength X), can also be a test image the image X, to produce the final inspection image.
[0122]
　
　Here, the generation of the image X of wavelengths other than the multispectral image generated from the signal (the wavelength X) from MS sensor 103 is added description. Desired and is the wavelength X of the image can be generated by performing an inverse matrix calculation using the multispectral images obtained from MS sensor 103. To illustrate this, first with reference to FIG. 18, again outline of operation of the imaging apparatus 11.
[0123]
　Sunlight 402 from the sun 401, plants 403, while being irradiated on the road 404, is also irradiated onto the illumination sensor 12. Illuminance sensor 12 measures the illuminance of the irradiated sunlight 402.
[0124]
　Illuminance sensor 12 measures the illuminance of sunlight 402 irradiated, obtains the illuminance value. Illumination values ​​that are retrieved, for example, the spectral characteristics of the solar 402 as shown in FIG. 19. Graph shown in FIG. 19 shows the spectral characteristics of solar 402, the horizontal axis represents the wavelength of light, the vertical axis indicates the intensity of light.
[0125]
　Note here, the description will be continued with an example of using the illumination sensor 12, in the inverse matrix calculation using the multispectral image, which will be described later, since necessarily be calculated without using the illuminance value obtained from the illuminance sensor 12 , in the configuration without the illuminance sensor 12, by applying the present technique can be.
[0126]
　Some of the sunlight 402 irradiated to the plant 403 is reflected by the plant 403, the reflected light 405 is received by the imaging device 11. A part of the sunlight 402 irradiated on the road 404 is reflected by the road 404, the reflected light 406 is received by the imaging device 11.
[0127]
　Reference is now made to Figure 20. Figure 20 shows the vegetation and concrete, when the respectively measured, a graph showing the relationship between the wavelength and reflectance. In Figure 20, the horizontal axis represents the wavelength and the vertical axis represents reflectance. Further, in FIG. 20, the graph shown by a solid line, represents the results obtained when the object to be measured and vegetation, the graph shown by a dotted line, the measured concrete (here, and the structure of the roads 404 It represents the results obtained when to).
[0128]
　As shown in FIG. 20, than the plant 403 and the road 404 (concrete), reflectance when light is irradiated in the same wavelength are different. For example, in FIG. 20, the reflectance of the concrete as shown by the dotted line is higher than the reflectance of the vegetation indicated by a solid line.
[0129]
　Further, in FIG. 20, the reflectance of the concrete as shown in dotted lines, regardless of the wavelength of light, has a substantially constant reflectance, the reflectance of the vegetation indicated by the solid line, steeply at the wavelength of the particular light Change. From Figure 20, when the plant measured, If it is light having a wavelength of above 700 nm, the reflectance is can be read high.
[0130]
　Thus, even when irradiated with same sunlight 402, and the reflected light 405 from the plant 403, unlike in the reflected light 406 from the road 404, when measuring the reflectance results.
[0131]
　Further, even when the same plant as measured by the growth condition, the wavelength of the highest reflectance is high light varies.
[0132]
　For example, generates an image wave X that is appropriate for a measurement target object, obtained when imaged with MS not in the wavelength A to H is set to the sensor 103, the wavelength X that is appropriate for the measured object there is a case where you want to. In such a case, the wavelength X of the image that is desired, is produced by performing an inverse matrix calculation using the multispectral images obtained from MS sensor 103.
[0133]
　Desired and the image of the wavelength X to be, can be represented by the spectral reflection characteristics of the object of measurement object. Generally, the spectral characteristics of the light source (the L (lambda)), (and P (lambda)) spectral characteristics of the subject, (and S (lambda)) spectral characteristic of the imaging system, the captured image (O (lambda) between the to), the equation (1) below is established.
　(Spectral characteristics of the light source L (lambda)) × (spectral characteristic of the subject P (lambda)) × (spectral characteristic of the imaging system S (λ)) = (Image (O (λ)) ··· ( 1)
[0134]
　Spectral characteristics of the light source is a spectral characteristic obtained from the illuminance sensor 12, for example, the spectral characteristics of the solar 402 as shown in FIG. 19. Incidentally, the spectral characteristics of the light source may be used an arbitrary value. In other words, rather than the spectral characteristics obtained from the illuminance sensor 12 may be a spectral characteristic which is set in advance is used.
[0135]
　Spectral characteristic of the subject is a spectral characteristic obtained from the reflected light from the object, for example, sunlight is irradiated on the plants and concrete as shown in Figure 20, when it is reflected, the spectral characteristics of the reflected light it is.
[0136]
　The spectral characteristic of the imaging system, a spectral characteristic of MS sensor 103, for example, the spectral characteristics as shown in FIG. 21. MS sensor 103, as described above, for example, in a unit pixel, a sensor for receiving a signal of 8 types of wavelengths.
[0137]
　MS sensor 103 are the sensor comprising a combination of MS filter 103-1 and the sensor 103-2, the spectral characteristics of MS sensor 103, and the transmittance of the MS filter 103-1, the influence of the sensitivity of the sensor 103-2 the receiving characteristics. Specifically, a value obtained by multiplying the sensitivity of the transmission and sensors 103-2 MS filter 103-1. Also, the sensitivity of the sensor 103-2 is a sensitivity to set sensitivity as a standard, sensitivity is normalized, it is used.
[0138]
　Spectral characteristics of MS sensor 103, as shown in FIG. 21, the spectral characteristics of each wavelength of the wavelength A to H.
[0139]
　By modifying Equation (1), equation (2) is obtained to determine the spectral characteristics of the subject.
[0140]
[Number 1]

[0141]
　Thus equation (2), i.e., the inverse matrix calculation, it is possible to obtain the reflectance spectral characteristics of the subject. In other words, the λ the wavelength X, by performing an inverse matrix operation according to equation (2), using the multispectral image obtained from MS sensor 103, it is possible to generate an image of the wavelength X, which is the desired it can.
[0142]
　Thus, to generate an image of a desired wavelength, the image, the inspection image by mapping based on the stitching reference map 303, it is possible to produce the final inspection image.
[0143]
　
　with reference to the flowchart of FIG. 22, the operation of the image processing system 50 (FIG. 3) will be described. Incidentally, in the following, but added sequentially described generation processing of the first to sixth, in each generation processing by the imaging unit 61 (lens 101, an exposure unit 102 and the MS sensor 103,), multispectral image is acquired and it is assumed (wavelength each image 301 of a to H are obtained) can be omitted the description of the processing at the time of aerial.
[0144]
　In the following description, as described with reference to FIGS. 13 to 21, as a multi-spectral image, the image 301 of 8 wavelengths, the description will be continued with an example to be imaged 500 sheets for each wavelength. Regarding the parts already described, description thereof is omitted as appropriate.
[0145]
　In step S11, the reference image generation processing unit 105 executes the feature quantity arithmetic. The reference image generation processing unit 105, from the designated wavelength processor 104, the multispectral image is supplied. In the first generation process, a multi-spectral image acquired by the MS sensor 103 (image 301), treatment with intact is performed.
[0146]
　The image processing system 50, in the first generation process, to produce a final test image, the designated wavelength processor 104, the reference image generation processing unit 105, multispectral images from the MS sensor 103, for example, FIG. 13 with reference to the supplied image 301 described.
[0147]
　In this case, rather than through a specified wavelength processor 104, directly from MS sensor 103, the multispectral image to the reference image generation processing unit 105 may be configured to be supplied.
[0148]
　Feature value calculation process in step S11 for the (first feature quantity calculation process) will be described with reference to the flowchart of FIG. 23. In step S31, the reference image generation processing unit 105 extracts the feature amount of the multispectral image.
[0149]
　Feature amount, as described below, is an index representing the magnitude of the characteristic of the image. For example, feature quantity (as described above, characteristic features of the image) characteristic region in the image is an index indicating the degree to which there is. Feature value is used to select the reference image, feature regions are those used during stitching process described below.
[0150]
　For example, the feature quantity (index indicating the size of features of the image), may be a spatial variation of the pixel values. The spatial variation of the pixel value can be the difference of pixel values ​​between adjacent pixels. In this case, by extracting the area of ​​the difference value is more than the threshold value of the pixel values ​​between pixels, for example, it is possible to extract an edge region.
[0151]
　Using a spatial variation of the pixel value as an index indicating the size of features of an image, the change amount as the characteristic amount, it is possible to be calculated in step S31.
[0152]
　Further, as a method for extracting the edge region from the image, a method of using differential operation, there is a method of using a high-pass filter. Edge region can change the color or the like is defined as region is steeper, the edge region in other words this means that it can be said that the gradient of the graph is steep. This obtains the gradient by differentiating by using, its slope is greater than a predetermined threshold value, to regard the edge, there is a method for extracting an edge region.
[0153]
　The region where there is edge because it contains high frequency components, using a high-pass filter, by extracting high frequency components, there is a method of extracting a region in which there is an edge.
[0154]
　Subsequent processing (e.g., step S51 in FIG. 24) at, when generating the reference image, comparing the feature quantity for each image. In step S31, if the spatial variation of the pixel value has to be calculated as a feature amount, the feature amount used in step S51 is, for example, in pixel value between adjacent pixels calculated from the first image it can be the sum of the difference.
[0155]
　Further, in step S31, by extracting high frequency components, if you choose to extract the area (= the characteristic region) where there is an edge, the size of the extracted region as an edge in the first image (ratio) is, characterized It is calculated as the amount.
[0156]
　Further, for example, as an index representing the magnitude of the characteristic of the image, extracts feature points, it may be used that feature point. Feature point divides the image into a plurality of regions, the difference between the average value and the pixel value in the region of the pixel values ​​in the area, for example, calculated in such called calculation methods mean square error, the value ( the difference), it can also be determined so as to differ pixels compared to the pixel value of the surrounding pixels.
[0157]
　As an index indicating the size of the features of the image, the case of using the feature points, the feature amounts can be used, for example, the total number of feature points extracted from the one image, the ratio and the like.
[0158]
　Further, for example, as an index representing the magnitude of the characteristic of the image, it may be used a statistical value (e.g., variance). Dispersion value, divides the image into a plurality of regions, by obtaining the variance of the region, to calculate a representative statistical values ​​for each area, it is also possible to use the value (statistic).
[0159]
　When calculating the variance of the region, the variance value represents the complexity of the pixel distribution in the region, the dispersion value increases in a region including the image pixel values ​​such as an edge is rapidly changed. Thus, obtains the variance of the region as an index indicating the magnitude of the characteristic of the image, it is also possible to use the variance value as the characteristic amount.
[0160]
　As an index indicating the size of the features of the image, the case of using the statistics, as the feature amount, for example, the sum of the statistical value calculated from the first image, or the like can be used an average value.
[0161]
　Here, feature quantity, an edge is extracted from the image, the extracted edge region will be described an example in a proportion within one image. The edge extraction, the description will be continued by taking as an example a case where performed by extracting a high frequency component as an example.
[0162]
　Returning to the flowchart shown in FIG. 23, in step S31, to calculate a feature quantity extraction of high frequency components (i.e. edge extraction) is carried out for each multispectral image. For example, as described with reference to FIG. 13, the MS sensor 103, as a multi-spectral image, the image 301A-1 to 301A-500, the image 301B-1 to 301B-500, an image 301C-1 to 301C-500, image 301D-1 to 301D-500, an image 301E-1 to 301E-500, the image 301F-1 to 301F-500, the image 301G-1 to 301G-500, for each image 301 of the image 301H-1 to 301H-500 Te, processing of extracting high-frequency components is performed.
[0163]
　As the feature amount, the ratio of the first image of the extracted high-frequency component (edge ​​area) is used, such a ratio is calculated in step S31. Also, when the stitching process is executed for the reference image as described later, since the comparison of the characteristic region is performed, while calculating the feature amount, information on the characteristics regions extracted as the edge regions are appropriately stored that.
[0164]
　In step S31, by the high-frequency component is extracted from each image 301, the arithmetic processing of the feature amount is performed, the process proceeds to step S12 (FIG. 22).
[0165]
　In step S12, the reference image generation process is executed. Reference image generation processing executed in step S12 for the (first reference image generation processing) will be described with reference to the flowchart of FIG. 24.
[0166]
　In step S51, the reference image generation processing unit 105 compares the feature amount of each image 301 calculated by the processing at step S11. Using the result of the comparison, in step S52, the reference image is generated.
[0167]
　Comparison of the feature amount in step S51, as described with reference to FIG. 14, performed in the image between captured at the same time (between images obtained by imaging the same portion). Then, the largest feature amount (characteristic of the object being imaged is most out image) image 301 is selected preferentially, is the reference image.
[0168]
　For example, referring again to FIG. 14, the image 301A-1 is imaged at time t1, image 301B-1, the image 301C-1, an image 301D-1, the image 301E-1, the image 301F-1, the image 301G-1 , each feature amount of the image 301H-1 (ratio of extracted as the edge region) are compared, the image 301 is the most characteristic amount is large, it is the reference image 302-1. For example, the feature quantity of the image 301A-1 is, if greater than the respective feature amounts of the other image 301B-1 to 301H-1, the image 301A-1 is the reference image 302-1.
[0169]
　Similarly, the image 301-2 to 301-500, a comparison of the feature quantity, the result of the comparison, the process of setting the image 301 most characteristic amount is large in the reference image 302 is performed. By such processing is performed, the reference image 302-1 to 302-500 are generated, the processing to step S13 (FIG. 22) is advanced.
[0170]
　Incidentally, if the feature amount other than the percentage of the edge region in the first image is used, for example, and the total number of feature points, even if the sum of the statistical value is used, similarly, the image 301 is large characteristic quantity, the reference It is set in the image 302.
[0171]
　In step S13, stitch process of the reference image is performed. Reference images 302-1 to 302-500 generated by the reference image generation processing unit 105 is supplied to the reference image stitching processing unit 107.
[0172]
　Reference image stitching processing unit 107 performs a stitch process using the supplied reference image 302-1 to 302-500 was to generate a stitching reference map 303 (Stitch reference information 303).
[0173]
　The stitch processing using the reference image 302-1 to 302-500 is 15, is performed as described with reference to FIG. 16. That detects a portion where the characteristic region of the reference image 302-1 to 302-500 are identical (similar), and in the superposition and trimming no overlapping partial images (stitching reference map 303) is generated.
[0174]
　Incidentally, as described above, the stitching process, the feature quantity, that is, the extracted feature region is used when calculating an index indicating the magnitude of the characteristic of the image, there was the feature area used processing line divide.
[0175]
　For example, when used for the calculation of the feature amount the results of extraction of the edge region, the stitching process, a process such as matching the extracted edge regions are performed.
[0176]
　Also, when calculating the feature amount extracting pixel different than the surrounding pixels as the feature point, the matched feature points (sequence of feature points match, match the locations similar) such process, stitching It is executed as a process.
[0177]
　Further, for example, case of calculating the feature quantity by calculating the variance value in the region as an index indicating the magnitude of the characteristic of the image, matching the dispersion value (to list the same or similar region having a dispersion value and adjust the location and) such stitching process is performed.
[0178]
　In step S14, the inspection image is generated. Test image, for example, to inspect the vegetation (sensing) when a vegetation was captured at a wavelength suitable for sensing image (image generated by the inverse matrix operation from the captured image).
[0179]
　Inspection image is generated with the specified wavelength processor 104 by the inspection wavelength image extracting unit 106. As a method of generating, for example, specify the wavelength processor 104, a multispectral image (each image of a plurality of wavelengths), and supplies the inspection wavelength image extracting unit 106, the inspection wavelength image extracting unit 106, is supplied from a plurality of images and extracting an image corresponding to a wavelength that is specified as the inspection wavelength, to produce a test image.
[0180]
　As another way of generating the test image, for example, specify the wavelength processor 104, using the multispectral image, an image of a predetermined wavelength, based on the equation (2) (see FIGS. 18 to 21 to as described) generates and supplies to the inspection wavelength image extracting unit 106, the inspection wavelength image extracting unit 106, from the supplied plurality of images, an image corresponding to a wavelength that is specified as the inspection wavelength by extracting and generates a test image.
[0181]
　Designated wavelength processing unit 104 when configured to generate an image corresponding to a plurality of wavelengths are set in advance, the inspection wavelength image extracting unit 106, is created with the specified wavelength processor 104, which is supplied from a plurality of images and extracting an image corresponding to a wavelength that is specified as the inspection wavelength, to produce a test image.
[0182]
　The designated wavelength processing unit 104 can also be configured to generate an image of a wavelength to be extracted by the inspection wavelength image extracting unit 106, when configured in this manner, the inspection wavelength image extracting unit 106, is supplied and the image corresponding handled as an image on the wavelength that is specified as the inspection wavelength (without processing such as extraction and generating, using an image that is directly supplied).
[0183]
　As another way of generating the inspection images, rather than the specified wavelength processor 104, at the inspection wavelength image extracting unit 106, it may be the image of the specified wavelength is generated. For example, the specified wavelength processor 104, the inspection wavelength image extracting unit 106, a multi-spectral image is supplied, the inspection wavelength image extracting unit 106, using the multispectral image supplied, based on Equation (2) calculation by performing the inspection image of the designated wavelength (desired wavelength) may also be generated.
[0184]
　In step S14, when the inspection image is generated, the processing to step S15 advances. Inspection image extracted by the inspection wavelength image extracting unit 106 is supplied to the inspection image stitching processing unit 108. The inspection image stitching processing unit 108, the reference image stitching processing unit 107, the stitching reference map 303 (Stitch reference information 303) is also supplied.
[0185]
　Inspection image stitching processing unit 108, as described with reference to FIG. 17, by using the stitch reference information 303, the inspection image, and stitching process to generate a final inspection image.
[0186]
　Stitch reference information 303, cut out which position of which image is information such how was attached. In other words, the stitch reference information 303 is information including at least information indicating a positional relationship between the image of the captured region of overlap with the image of the captured predetermined region and the region, when performing the stitching process is information that can be used as metadata accompanying the image.
[0187]
　Inspection image stitching processing unit 108 refers to the stitch reference information 303, among the plurality of test images, using any test image, to determine cut out which position of the inspection image and use it (which area) , the determined area, also determines a region cut out from another inspection image, how the bonding, by repeating such a process, performs a stitch process of the inspection image, the final inspection image to generate.
[0188]
　Final inspection image is used in such mapped onto a map that has already been generated.
[0189]
　Thus, to generate the stitch reference information 303, using the stitching reference information 303, by stitching process the inspection image, the inspection image of a plurality of wavelengths, even when each such that stitch process, the stitch processing the time shortened, it is possible to reduce the processing load.
[0190]
　
　for other operations of the image processing system 50 (FIG. 3) will be described. As the processing up to produce a final test image, in the first generation process described above, at step S11, when performing feature quantity arithmetic, the multispectral image acquired by MS sensor 103, the feature amount used as it was an example of calculating.
[0191]
　A second generation process up to produce a final test image, when calculating the feature amount will be described as an example the case of calculating by processing multispectral image.
[0192]
　The second generation process, as in the first generation process, the image processing system 50 performs processing based on the flowchart shown in FIG. 22.
[0193]
　In step S11, when the feature amount calculation processing is performed, since except that processing based on the flowchart of the second feature amount calculation processing shown in FIG. 25 is performed is performed in the same manner as in the flowchart shown in FIG. 22, wherein in addition the description of the different processes, will not be described processing to be similarly performed.
[0194]
　Referring to the flow chart shown in FIG. 25, when the feature quantity is calculated, first, in step S101, the reduced image is generated. As shown in FIG. 26, the image 301 as a reference image, the vertical image 301 1 / A multiplied by sideways by multiplying 1 / B, the reduced image 301ab is generated. Further, the vertical image 301 1 / M multiplied by sideways by multiplying 1 / N, the reduced image 301mn is generated.
[0195]
　For example, when the A = B = 2, the vertical image 301 1/2 multiplied by 1/2 horizontal reduced image 301ab is generated. Similarly, for example, when the M = N = 5, the vertical image 301 1/5 multiplied by the horizontal and 1/5 reduced image 301mn is generated.
[0196]
　Here, the image 301 as a reference (original image), has been described as an example a case where two reduced images are generated, and if one of the reduced image is generated, reduced more than two even if an image is generated, a scope of the present technology. In the following description, and one of the original image, the description will be continued by taking as an example a case where processing is performed using a total of three images of two reduced images.
[0197]
　Further, here, it has been given the case of a and M = N = 5 case of the A = B = 2 as an example, A and B may not be the same value, also, the same also M and N it may not be of value.
[0198]
　In step S101, the reduced image is generated, the process proceeds to step S102. In step S102, the high-frequency component is extracted (edge ​​extraction) (feature amount of calculation is performed). This process, because it can be performed in the same manner as the processing in step S31 of the flowchart shown in FIG. 23, a description thereof will be omitted.
[0199]
　However, in step S102, the image 301 is a target of the high-frequency component is extracted is the reduced image and the original image, for example, the image 301 shown in FIG. 26, the image 301Ab, and each image 301mn processing been the subject from each of the high-frequency component is extracted based on the extraction result, the feature (such as the percentage of the edge region) is calculated.
[0200]
　Incidentally, when comparing the feature amount, the feature amount of the reduced image, so as to have the same conditions as the original image, processing such as multiplying a predetermined coefficient may be performed as appropriate.
[0201]
　For example, as the feature amount, the sum and the feature point, the case of using the sum of the statistics, in the original image and the reduced image, the number of pixels that have been targeted for (number of areas) are different, the simple original images comparing the feature quantity of the feature amount and the reduced image, which can be issued erroneous comparison result.

WE claims

[Requested item 1]
　Based on the plurality of first image for the first imaging region to generate a first reference image relating to the first imaging region, for the second imaging region at least partially overlaps with the first imaging region an image generator for generating a second reference image relating to the second imaging area based on the plurality of the second image,
　based on the first reference image and the second reference image, the first a processing unit for generating positioning information indicating a correspondence relationship between the imaging area and the second imaging region of
　the image processing apparatus comprising a.
[Requested item 2]
　Wherein the processing unit, the first reference picture and the second reference image by stitching process to generate a reference synthetic image containing the positioning information,
　the image processing apparatus according to claim 1.
[Requested item 3]
　Wherein the processing unit, based on the positioning information and acquires any one of the images of the plurality of first images, the composite image by stitching process and any image of the plurality of second image ,
　the image processing apparatus according to claim 1.
[Requested item 4]
　Wherein the processing part, based on the positioning information, said first area corresponding to the imaging area of the set image based on the plurality of first images, the areas corresponding to the second imaging area more obtains the composite image by setting the second image based on the image,
　the image processing apparatus according to claim 1.
[Requested item 5]
　Said plurality of first image picture is based on different wavelengths or the plurality of second image is an image based on the different wavelengths,
　the image processing apparatus according to claim 1.
[Requested item 6]
　The image generation unit from the plurality of first image, in favor of a large image of the feature amount obtained as the first reference image, from the plurality of second images, prioritize the large picture feature amount and obtaining as the second reference image,
　the image processing apparatus according to claim 1.
[Requested item 7]
　The image generating unit acquires a large image of the most characteristic amount among the plurality of first image as the first reference image, the large image of the most characteristic amount among the plurality of second image first obtaining a second reference image,
　the image processing apparatus according to claim 1.
[Requested item 8]
　Wherein the image generator generates the first reduced image obtained by reducing the first image, the first reduced image also generates the first reference image as a target, by reducing the second image generating a second reduced image, the second reduced image also generates the second reference image as the target
　image processing apparatus according to claim 1.
[Requested item 9]
　Wherein the first reduced image, if it is set to the first image, the original first image of the first reduced image is set to the first reference image, the second reduced image but wherein when it is set in the second image, the original second image of the second reduced image is set to the second reference image
　an image processing apparatus according to claim 8.
[Requested item 10]
　The image generation unit, the first image is divided into blocks, and calculates the feature quantity for each block, from the feature quantity is larger block, and generates the first reference image, the second image is divided into blocks, and calculates the feature quantity for each block, from the feature quantity is large blocks, and generates the second reference image
　an image processing apparatus according to claim 1.
[Requested item 11]
　The image generation unit, first generates a reduced image obtained by reducing the first image, as the first reduced image is also divided into blocks, the target also block the first reduced image, the first generates the reference image, the second image to generate a second reduced image obtained by reducing the of the second reduced image is also divided into blocks, as the second also target block of the reduced image, the first generating a second reference image
　an image processing apparatus according to claim 10.
[Requested item 12]
　Block of the first reduced image, when the feature quantity is larger block, corresponding block of the first reduced image of the original first image, setting a portion of the first reference image is the block of the second reduced image, if the feature amount is large blocks, the corresponding block of the original second image of the second reduced image, a portion of the second reference image It is set to
　the image processing apparatus according to claim 11.
[Requested item 13]
　Using the first image of the plurality to generate a third image of a predetermined wavelength,
　the image generation unit generates the first reference image from said third image,
　the plurality of second using the image to generate a fourth image of a predetermined wavelength,
　the image generation unit generates the second reference image from the fourth image
　the image processing apparatus according to claim 1.
[Requested item 14]
　The first reference image and the second reference image and processing each orthorectified,
　wherein the processing unit, based on the second reference image and the orthorectified been the first reference image, the positioning information generating a
　image processing apparatus according to claim 1.
[Requested item 15]
　Based on the plurality of first image for the first imaging region to generate a first reference image relating to the first imaging region, for the second imaging region at least partially overlaps with the first imaging region based on the plurality of second image to generate a second reference image relating to the second imaging area,
　based on the first reference image and the second reference image, and the first imaging region generating positioning information indicating the correspondence relationship between the second imaging area,
　an image processing method comprising the steps.
[Requested item 16]
　The computer,
　on the basis of a plurality of first image for the first imaging region to generate a first reference image for the first imaging region and the second at least partially overlaps with the first imaging region based on the plurality of second image about the imaging region to generate a second reference image relating to the second imaging area,
　based on the first reference image and the second reference image, the first generating positioning information indicating a correspondence relationship between the imaging area and the second imaging region,
　a program for executing the processing including step.