The present invention relates to an information processing apparatus, an information processing method, and a program for performing image recognition processing.
Background technology
[0002]
In recent years, a technique for detecting a person who meets a desired condition using an image has been developed. For example, Patent Document 1 describes a system for determining whether or not a predetermined person such as an inquirer or a wanted criminal is included in a moving image taken in a city or the like. This system generates the feature data of the face image included in each frame of the captured image, and by comparing the generated feature data, the feature data is collected for each person so that the feature data of the same person becomes one group. Classify into. Then, the above system determines the characteristic data representing each person based on the classified characteristic data, and uses at least one of the determined characteristic data and the face image corresponding to the characteristic data as the representative face data as a face recognition device. Send to. The face recognition device registers the transmitted data in the face data dictionary and collates it with the face data of a predetermined person.
[0003]
Patent Document 2 describes an image processing device that automatically tracks a person by identifying the same person from images captured by different cameras. Different cameras capture the person from different directions.
[0004]
In Patent Document 3, the walking state of a person is detected from a person image sequence obtained by a distant frame or different cameras, and it is determined whether or not the people included in the different image sequences are the same based on the walking state. It is stated that it should be done.
Prior art literature
Patent documents
[0005]
Patent Document 1: Japanese Unexamined Patent Publication No. 2017-182210
Patent Document 2: Japanese Unexamined Patent Publication No. 2015-2547
Patent Document 3: International Publication No. 2006/013765
Outline of the invention
Problems to be solved by the invention
[0006]
The present inventor examined a new method for identifying a person who cannot be identified by the original image by using image processing for other images. That is, an object of the present invention is to provide a new method for identifying a person who cannot be identified by an original image by using image processing on another image.
Means to solve problems
[0007]
In each aspect of the present invention, the following configurations are adopted in order to solve the above-mentioned problems.
[0008]
The first aspect is related to information processing equipment.
The first information processing device related to the first aspect is
The first image extracted from the human area feature information related to the area including the area other than the face of the person to be searched,
The second image in which the collation result with the human area feature information is in agreement and the face area is detected, and
Result information showing the collation result between the face information stored in the storage unit and the face information extracted from the face area,
And the acquisition method to acquire each of
A display processing means for displaying at least two of the first image, the second image, and the result information on the same screen.
Have.
The second information processing device related to the first aspect is
The second image in which the collation result with the human area feature information regarding the area including the area other than the face of the person to be searched matches and the face area was detected and
Acquisition means for acquiring the result information indicating the collation result between the face information stored in the storage unit and the face information extracted from the face area, and
A display processing means for displaying the acquired second image and the result information on the same screen,
Have.
The third information processing device related to the first aspect is
The first image extracted the human area feature information related to the area including the area other than the face of the person to be searched, and
Acquisition means for acquiring the face information extracted from the second image in which the collation result with the human area feature information matches and the result information showing the collation result with the face information stored in the storage unit.
A display processing means for displaying the acquired first image and the result information on the same screen,
Have.
[0009]
The second aspect relates to the information processing method executed by at least one computer.
The first information processing method related to the second aspect is
Information processing device
The first image used to generate the human area characteristic information showing the characteristics of the person to be searched,
The second image in which the collation result with the human area feature information is in agreement and the face area is detected, and
Result information showing the collation result between the face information stored in the storage unit and the face information extracted from the face area,
Get each of
Includes displaying at least two of the first image, the second image, and the result information on the same screen.
The second information processing method related to the second aspect is
Information processing device
The second image in which the collation result with the human area feature information regarding the area including the area other than the face of the person to be searched matches and the face area was detected and
Acquire the result information indicating the collation result between the face information stored in the storage unit and the face information extracted from the face area, and obtain the result information.
Includes displaying the acquired second image and the result information on the same screen.
The third information processing method related to the second aspect is
Information processing device
The first image extracted the human area feature information related to the area including the area other than the face of the person to be searched, and
Acquires the result information showing the collation result between the face information extracted from the second image in which the collation result with the human area feature information matches and the face information stored in the storage unit.
Includes displaying the acquired first image and the result information on the same screen.
[0010]
As another aspect of the present invention, it may be a program that causes at least one computer to execute the method of the second aspect, or it is a recording medium that can be read by a computer that records such a program. You may. This recording medium includes a non-temporary tangible medium.
This computer program contains a computer program code that causes the computer to execute its control method on the information processing device when executed by the computer.
[0011]
It should be noted that any combination of the above components and the conversion of the expression of the present invention between methods, devices, systems, recording media, computer programs, etc. are also effective as aspects of the present invention.
[0012]
Further, the various components of the present invention do not necessarily have to be individually independent, and a plurality of components are formed as one member, and one component is formed of a plurality of members. It may be that a certain component is a part of another component, a part of a certain component overlaps with a part of another component, and the like.
[0013]
Further, although the method and the computer program of the present invention describe a plurality of procedures in order, the order of description does not limit the order in which the plurality of procedures are executed. Therefore, when implementing the method and computer program of the present invention, the order of the plurality of procedures can be changed within a range that does not hinder the contents.
[0014]
Furthermore, the method of the present invention and the plurality of procedures of the computer program are not limited to being executed at different timings. Therefore, another procedure may occur during the execution of a certain procedure, a part or all of the execution timing of the certain procedure and the execution timing of the other procedure may overlap, and the like.
The invention's effect
[0015]
According to each of the above aspects, it is possible to provide a new method for identifying a person who cannot be identified by the original image by using image processing for other images.
A brief description of the drawing
[0016]
The above-mentioned objectives and other objectives, features and advantages will be further clarified by the preferred embodiments described below and the accompanying drawings below.
[0017]
FIG. 1 is a diagram conceptually showing a configuration of an image processing system according to an embodiment of the present invention.
[Fig. 2] Fig. 2 is a diagram showing an example of a data structure of video data and image data.
FIG. 3 is a functional block diagram showing a logical configuration of an information processing apparatus according to the first embodiment.
FIG. 4 is a diagram for explaining a first image and a second image.
FIG. 5 is a flowchart showing an example of a procedure of a process in which an information processing apparatus generates first feature information.
FIG. 6 is a diagram showing an example of data structures of a first feature information storage unit and a second feature information storage unit.
[Fig. 7] Fig. 7 is a flowchart showing an example of the operation of the information processing apparatus.
8 is a flowchart showing details of the search process in step S110 of FIG. 7. FIG.
FIG. 9 is a functional block diagram showing a logical configuration of an information processing apparatus according to a second embodiment.
[Fig. 10] Fig. 10 is a diagram showing an example of a data structure of an authentication information database.
FIG. 11 is a flowchart showing an example of a procedure for authentication processing of an information processing device.
FIG. 12 is a flowchart showing another example of the operation of the information processing apparatus.
FIG. 13 is a diagram showing an example of a data structure of a second feature information storage unit.
FIG. 14 is a functional block diagram showing a logical configuration of an information processing apparatus according to a third embodiment.
[Fig. 15] Fig. 15 is a diagram showing an example of a result list screen displayed by an information processing apparatus.
[Fig. 16] Fig. 16 is a diagram showing an example of a data structure of an authentication result storage unit.
FIG. 17 is a flowchart showing an example of the operation of the information processing apparatus.
FIG. 18 is a flowchart showing an example of a detailed procedure of the display process of step S130 of FIG.
19 is a diagram showing details of a display example of the candidate display unit of FIG. 15. FIG.
20 is a diagram showing details of a display example of the candidate display unit of FIG. 15. FIG.
[Fig. 21] Fig. 21 is a diagram showing an example of a detailed information window.
[Fig. 22] Fig. 22 is a diagram showing an example of a procedure of display processing of an information processing apparatus.
[Fig. 23] Fig. 23 is a diagram showing an example of a procedure of display processing of an information processing apparatus.
[Fig. 24] Fig. 24 is a diagram showing an example of a procedure for display processing of an information processing apparatus.
[Fig. 25] It is a figure for demonstrating the transition of a display screen.
FIG. 26 is a diagram showing an example of a configuration of a computer that realizes the information processing apparatus of each embodiment.
Embodiment for carrying out the invention
[0018]
Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all drawings, similar components are designated by the same reference numerals, and the description thereof will be omitted as appropriate.
Further, in each figure of the present specification, the structure of the portion not related to the essence of the present invention is omitted and is not shown.
[0019]
(First Embodiment)
FIG. 1 is a diagram conceptually showing the configuration of an image processing system according to an embodiment of the present invention.
The image processing system 1 includes an information processing device 100. The information processing apparatus 100 searches for a certain person from images captured by a plurality of cameras 5a, 5b, cameras 5c ... (referred to as "camera 5" when it is not particularly necessary to distinguish them).
[0020]
The person to be searched is shown in the original image (second image 22) in the back, for example, and therefore face recognition may not be possible. In such a case, the information processing apparatus 100 resembles the person from another image (first image 10) captured by the camera 5 using information such as body shape (hereinafter referred to as first feature information). A candidate is searched for, and a person is identified by face authentication using an image showing a face in the candidate.
[0021]
In the embodiment, "acquisition" means that the own device obtains data or information stored in another device or storage medium (active acquisition), and outputs the data to the own device from the other device. Includes at least one of the input of data or information to be performed (passive acquisition). Examples of active acquisition include making a request or inquiry to another device and receiving the reply, and accessing and reading another device or storage medium. Further, an example of passive acquisition may be receiving information to be delivered (or transmitted, push notification, etc.). Further, "acquisition" may be to select and acquire from received data or information, or to select and receive delivered data or information.
[0022]
The camera 5 includes an image sensor such as a lens and a CCD (Charge Coupled Device) image sensor. For example, it is a network camera such as an IP (Internet Protocol) camera. The network camera is, for example, a wireless LAN (Loca).l Area Network) It has a communication function and is connected to the information processing device 100 via a communication network, that is, a relay device (not shown) such as a router. These cameras 5 may be so-called surveillance cameras installed in the city or inside or outside the building. The camera 5 may be provided with a mechanism that follows the movement of a specific person to control the orientation of the camera body or men, zoom control, focusing, and the like.
[0023]
The camera 5 and the information processing device 100 may not be connected, may be directly connected, or may be indirectly connected via a communication network 3 or the like. When the camera 5 and the information processing device 100 are not connected, for example, the media 7a and the media 7b in which the images captured by the cameras 5a and the camera 5b are recorded, respectively (hereinafter, when it is not necessary to distinguish between the media 7 and the media 7). The image data is read and acquired from).
[0024]
The camera 5c in the example connected to the information processing device 100 via the communication network 3 or the like has, for example, a wireless LAN (Local Area Network) communication function, and is a communication network 3, that is, a relay device such as a router (not shown). ) Is connected to the information processing apparatus 100. The camera 5c is preferably a network camera such as an IP (Internet Protocol) camera.
[0025]
Further, the timing of image transmission from the camera 5 to the information processing apparatus 100 may be real-time distribution, for example, streaming distribution, or images for a predetermined period may be transmitted at predetermined intervals. This transmission timing may be appropriately selected based on the memory capacity, communication capacity, image processing capacity of the camera 5 and the information processing device 100, the communication status between the camera 5 and the information processing device 100, and the like. , May be changed according to changing circumstances.
[0026]
For example, the image data captured by the camera 5 may be directly transmitted to the information processing device 100, and the information processing device 100 may sequentially receive the image data. Further, a storage device (not shown) accessible to both the camera 5 and the information processing device 100 may be provided. In this case, the image data captured by the camera 5 is stored in the storage device. Then, the information processing device 100 reads out the image data from this storage device.
[0027]
Here, the image data may be at least one of a still image and a moving image. The data format, file format, file size, image size, image resolution, video frame rate, etc. are not particularly limited, and the specifications, standards, performance, etc. of the camera 5 and the information processing device 100, or the processing capacity of image analysis, etc. Various formats can be adopted depending on the accuracy and the like. Then, at least one frame of this image data may be at least one of the first image 10 and the second image 22 described later.
[0028]
FIG. 2A is a diagram showing an example of the data structure of the video data 40. The video data 40 is taken by a certain camera 5, is received from the camera 5, or is recorded on the media 7. The video data 40 includes a video ID that identifies the video data 40, a media ID that identifies the media 7 on which the video data 40 is recorded, a camera ID that identifies the camera 5 in which the video data 40 is captured, and a shooting of the camera 5. Includes information indicating conditions (for example, shooting location, shooting time, etc.), and the main body of video data.
[0029]
FIG. 2B is a diagram showing an example of the data structure of the first image 10. The first image 10 is a part of the video data 40 taken by a certain camera 5, and may be a frame with the video data 40. The first image 10 includes an image ID that identifies the image, information indicating an image ID that identifies the image data 40 that is the source of the image, and a main body of the image data. When the main body of the image data is a frame, the first image 10 further includes information indicating a frame number that can identify the frame. Further, the image ID may be a file name. Note that, unlike the second image 22, the first image 10 is an image to be searched for the person to be searched. Further, the first image 10 may be captured by a camera 5 different from the second image 22.
[0030]
FIG. 3 is a functional block diagram showing a logical configuration of the information processing apparatus 100 of the present embodiment. The information processing apparatus 100 includes a collation unit 102, an extraction unit 104, and a registration unit 106. In the example shown in this figure, the information processing apparatus 100 further includes a second feature information storage unit 110. The second feature information storage unit 110 may be integrated with the main body of the information processing device 100 or may be a separate body. The second feature information storage unit 110 may have a database structure or may have another data structure.
[0031]
The collation unit 102 performs collation processing for collating the first feature information indicating the characteristics of the search target person with the feature information of the human area extracted from the person included in the first image 10. Here, the first feature information extracted from the second image 22 is the feature information indicating the feature of the human area of ​​the person to be searched, and is also referred to as the first feature information 14a hereafter. Further, the characteristic information of the human area extracted from the person included in the first image 10 to be searched is also referred to as the first characteristic information 14b. When the collation results by the collation unit 102 match, the extraction unit 104 extracts the second feature information from the person included in the first image 10. The registration unit 106 stores the second feature information extracted from the person included in the first image 10 in the second feature information storage unit 110. In other words, the registration unit 106 registers the second feature information extracted from the person included in the first image 10 in the second feature information storage unit 110.
[0032]
Specifically, the collation unit 102 acquires the first image 10 by reading it from the media 7 or receiving it from the camera 5. Then, the collation unit 102 first identifies a human area from the acquired first image 10, and performs collation processing with the first feature information 14a for the specified human area. Here, there may be a plurality of first feature information 14a.
[0033]
FIG. 4A shows an example of the second image 22. The second image 22 is an image in which a person is shown but the person cannot be identified by face recognition or the like. The second image 22 is, for example, an image in which a face is not shown, such as a person facing backward, an image in which a face is not shown to the extent that matching accuracy equal to or higher than a reference value can be obtained in face recognition, or a low image quality to which face recognition cannot be performed. It is an image of. The image in which the person cannot be identified is not limited to the case where face recognition cannot be performed. The image may be an image in which the person cannot be identified by another authentication method. Another authentication method is, for example, biometric authentication, which is at least one of an iris, auricle, vein, and fingerprint. Further, with respect to the person shown in the second image 22, the first feature information 14a may be extracted from the person area of ​​the person without specifying the person by face recognition processing or other authentication processing. That is, it is not always necessary to perform face recognition processing or other authentication processing on the second image 22. In other words, the second image 22 may be an image in which the person in the second image 22 is not specified.
[0034]
In the collation process, the collation unit 102 determines that the human area whose degree of coincidence with the first feature information 14a is equal to or higher than the reference is the area corresponding to the first feature information 14a. The extraction unit 104 specifies a human region including a region in which the degree of coincidence with the first feature information 14a from the first image 10 is equal to or higher than the reference, and further identifies a face region of the human region.
[0035]
The degree of matching is a value indicating the degree of matching, and in the present embodiment, the degree of matching is a numerical value of 0 to 1, 1 for perfect match and 0 for perfect mismatch.
[0036]
The extraction unit 104 extracts the feature information of the specified face region to generate the second feature information, and the registration unit 106 stores the feature information in the second feature information storage unit 110. As shown in FIGS. 4 (b) and 4 (c), the human region 16 and the face region 20 are specified from the first image 10. People including these areas are search target candidates.
[0037]
The extraction unit 104 extracts the first feature information 14a of the plurality of detection target persons from the human area of ​​the person included in the different second images 22, and the registration unit 106 identifies the plurality of first feature information 14a. The information of the person may be associated with each other and stored in the first feature information storage unit 112. The collation unit 102 may perform a collation process using a plurality of first feature information 14a.
[0038]
Further, the information processing apparatus 100 does not necessarily have to include the first feature information storage unit 112. The collation unit 102 may read the first feature information 14a from the first feature information storage unit 112 and acquire it, or may acquire it by input from an external device. Further, the first feature information storage unit 112 may also serve as the second feature information storage unit 110. In this case, the information processing apparatus 100 uses the generated second feature information as the first feature information.
[0039]
In the present embodiment, the human area includes an area corresponding to the first part and the second part. The second part is, for example, at least one area of ​​the face, iris, pinna, veins, and fingerprints. Alternatively, the second part may be a person's gait rather than an area. The first part may or may not include the second part. In the present embodiment, the first part is the area of ​​the whole person, and the second part is the face area. The first feature information (first feature information 14a and first feature information 14b) of the first part is feature information indicating a feature of appearance such as a person's back appearance or clothes. In the present specification, the first feature information is also referred to as "human domain feature information", and the second feature information is also referred to as "face information".
[0040]
The first feature information (first feature information 14a and first feature information 14b) includes height, shoulder width, ratio of body parts, clothes (shape, color, material, etc.), hairstyle (including hair color), and ornaments (hat). , Eyeglasses, accessories, etc.), personal belongings (bags, umbrellas, canes, etc.), etc. Further, the human domain characteristic information may include information such as the likelihood of the characteristic information.
[0041]
In the present embodiment, the first feature information 14a of the search target person is generated by using the second image 22 in which the search target person is shown. The second image 22 may be an image captured by the camera 5, an image captured by another imaging means, or an image read by a scanner.
[0042]
The information processing device 100 further includes a first feature information extraction unit (not shown). The first feature information generation unit generates the first feature information 14a using the second image 22 in which the person to be searched is captured, and stores it in the first feature information storage unit 112.
[0043]
FIG. 5 is a flowchart showing an example of a processing procedure in which the information processing apparatus 100 generates the first feature information 14a. The first feature information generation unit acquires the image (second image 22) of the search target person (step S10), generates the second feature information (face information) from the acquired image, and generates the second feature information (face information) to the first feature information storage unit 112. (Step S20).
[0044]
FIG. 6A is a diagram showing an example of the data structure of the first feature information storage unit 112. The first feature information storage unit 112 stores the search target ID that identifies the search target person, the first feature information 14a of the person, and the image data of the original image from which the first feature information 14a is acquired in association with each other. There is. The first feature information storage unit 112 may store the main body of the image data, or may store the location where the image data is stored and the path indicating the file name.
[0045]
FIG. 6B is a diagram showing an example of the data structure of the second feature information storage unit 110. The second feature information storage unit 110 stores the candidate ID for identifying the candidate of the search target person in association with the second feature information of the candidate. The second feature information storage unit 110 stores information on at least one candidate for the search target person. When there are a plurality of search target persons, as shown in FIG. 6C, the second feature information storage unit 110 includes the candidate ID and the second feature information of the candidate in the search target ID that is the source of the candidate. May be associated with each other.
[0046]
FIG. 7 is a flowchart showing an example of the operation of the information processing apparatus 100. FIG. 8 is a flowchart showing the details of the search process in step S110 of FIG.
[0047]
First, the collation unit 102, The first feature information 14a (human area feature information) of the search target person is acquired (step S101). Further, the collation unit 102 acquires the first image 10 (step S103). Then, the process proceeds to the search process of step S110 in FIG.
[0048]
As shown in FIG. 8, in the search process of step S110, the collation unit 102 first first features the first feature information 14a of the search target person and the first feature information of the person region extracted from the person included in the first image 10. Collation processing with 14b is performed (step S111). When the collation results of the first feature information 14a and the first feature information 14b match, in other words, when a region having a degree of matching with the first feature information 14a equal to or higher than the reference is found (YES in step S113), the extraction unit. 104 generates the second feature information (face information) from the person included in the first image 10 (step S115). For example, the extraction unit 104 identifies a human area including the area, identifies a face area among the specified human areas, extracts feature information of the specified face area, and extracts second feature information (face information). May be generated. Then, the registration unit 106 stores the generated face information in the second feature information storage unit 110 (step S117).
[0049]
In the present embodiment, when the collating unit 102 finds a person who matches the human area characteristic information (first feature information 14a) of the person to be searched from the first image 10, the extraction unit 104 finds the feature information of the face area of ​​the person. Is generated, and the registration unit 106 stores it in the second feature information storage unit 110. Then, the feature information of the face region stored in the second feature information storage unit 110 is used for a person's identification process in the image processing. As described above, according to the present embodiment, in the first image 10 captured by the camera 5 using the human area feature information (first feature information 14a) obtained from the image in which the face of the search target person is not captured. It is possible to search for the person to be searched from and extract and store the feature information of the face of the found person.
[0050]
(Second embodiment)
FIG. 9 is a functional block diagram showing a logical configuration of the information processing apparatus 100 according to the second embodiment. The information processing apparatus 100 of the present embodiment is different from the point that the second feature information stored in the second feature information storage unit 110 is collated with the face recognition information registered in advance to acquire the candidate information of the person to be searched. Is the same as the above embodiment.
[0051]
The information processing apparatus 100 of FIG. 9 further includes an authentication unit 120 and an acquisition unit 122 in addition to the configuration shown in FIG. The authentication unit 120 executes a collation process with respect to the second feature information stored in the second feature information storage unit 110 using the authentication information database 114. The second characteristic information of a specific person is registered in the authentication information database 114 in association with the information of the person. The acquisition unit 122 acquires the information of the person associated with the second feature information determined to have the matching degree equal to or higher than the standard by the collation process from the authentication information database 114.
[0052]
FIG. 10 is a diagram showing an example of the data structure of the authentication information database 114. In the authentication information database 114, an authentication ID for identifying a specific person (hereinafter referred to as a registrant) in which the authentication information is registered, and facial feature information (second feature information) indicating the facial features of the registrant. , The registrant's face image data, the registrant's name, and the registrant's attribute information are registered in association with each other.
[0053]
The entity of the face image data may be registered in the authentication information database 114, or the place where the face image data is saved and the path indicating the file name may be registered. This face image data is used for screen display as described later, and the operator can browse the face photo displayed on the screen together with the name and attribute information.
[0054]
In the example of FIG. 9, the authentication information database 114 is a part of the information processing apparatus 100. The authentication information database 114 may be integrated with or separate from the main body of the information processing apparatus 100. The authentication information database 114 may be included in the same device as the second feature information storage unit 110, or may be included in different devices. The authentication information database 114 has a database structure, but may have other data structures.
[0055]
The function of the authentication unit 120 of the present embodiment may be possessed by the collation unit 104.
Further, when the authentication results by the authentication unit 120 match, the information processing apparatus 100 reads out the identification information (authentication ID and name) corresponding to the face feature information stored in the authentication information database 114 and outputs the unit. An output processing unit (not shown) that performs output processing may be provided. The output processing will be described in detail later.
[0056]
FIG. 11 is a flowchart showing an example of the procedure of the authentication process of the information processing apparatus 100 of the present embodiment.
First, the authentication unit 120 executes a collation process for the second feature information stored in the second feature information storage unit 110 using the authentication information database 114 (step S121). Then, when there is a person who is determined to have a matching degree equal to or higher than the standard by the collation process (YES in step S123), the authentication unit 120 acquires the information (name, etc.) of the person from the authentication information database 114 (step). S125).
[0057]
Further, in the present embodiment, the registration unit 106 of the information processing apparatus 100 associates at least the area corresponding to the first feature information 14a in the first image 10 with the second feature information and stores it in the second feature information storage unit 110. Let me.
[0058]
For example, as shown in FIG. 12, after step S117 of the flowchart of FIG. 8, the registration unit 106 associates the image data of the human area of ​​the first image 10 with the second feature information (face information) and the second feature information. It may be stored in the storage unit 110 (step S119).
[0059]
FIG. 13 is a diagram showing an example of the data structure of the second feature information storage unit 110 of the present embodiment. In the present embodiment, the second feature information storage unit 110 adds the information described in FIG. 6 (b), the matching degree information calculated in the collation process of step S121 of FIG. 11, and the image data of the person area. , And an image ID (indicated as the original image ID in the figure) that identifies the first image 10 that has acquired the person area may be associated and stored.
[0060]
The information processing apparatus 100 acquires a video ID from the first image 10 exemplified in FIG. 4B by associating the original image ID, and further, a shooting location is taken from the second image 22 exemplified in FIG. 4A. Etc. can be acquired in association with the second feature information. Further, the information processing apparatus 100 may store the second feature information in the second feature information storage unit 110 in association with the shooting conditions such as the shooting location, the shooting direction, and the shooting time of the camera 5.
[0061]
As described above, in the present embodiment, the authentication unit 120 performs collation processing with the face information stored in the second feature information storage unit 110 with the face information registered in the authentication information database 114. , The information of a similar person can be acquired by the acquisition unit 122. As described above, according to the present embodiment, the same effect as that of the above embodiment is obtained, and further, a face similar to the search target person is obtained from the first image 10 captured by the camera 5 by using the human area feature information. It is possible to perform authentication processing and acquire information on similar persons.
[0062]
(Third embodiment)
FIG. 14 is a functional block diagram showing a logical configuration of the information processing apparatus 100 according to the third embodiment. The information processing apparatus 100 of the present embodiment is the same as that of the second embodiment except that it has a configuration for displaying the information of the detected candidate. The information processing apparatus 100 further includes a display processing unit 130 in addition to the configuration shown in FIG. The display processing unit 130 causes various information to be displayed on a display device 132 such as a liquid crystal display or an organic EL (ElectroLuminescence) display connected to the information processing device 100.
[0063]
In the present embodiment, the acquisition unit 122 acquires a plurality of candidate person area images and information (names and attributes) from the authentication information database 114. Then, the display processing unit 130 displays each candidate person area image on the image display unit 214, and for each candidate, the information of at least one person detected as a candidate as a result of the face recognition processing is a candidate. It is displayed on the person display unit 222. Specifically, the display processing unit 130 displays at least two of the search target display unit 202, the search result display unit 210, and the authentication result display unit 220 on the same screen.
[0064]
FIG. 15 is a diagram showing an example of a result list screen 200 displayed by the information processing apparatus 100 of the present embodiment. In the example of this figure, the search target display unit 202, the search result display unit 210, and the authentication result display unit 220 are displayed on the same result list screen 200. An example of displaying at least two of the search target display unit 202, the search result display unit 210, and the authentication result display unit 220 on the same screen will be described later.
[0065]
In the present embodiment, the acquisition unit 122 acquires at least a part of each information to be displayed on the display processing unit 130. Some of these information are, for example, the second image 22 from which the first feature information 14a (human area feature information) showing the features related to the area other than the face of the person to be searched is extracted, and the first feature information 14a (human area feature). The first image 10 (hereinafter, also referred to as a candidate person area image) in which the collation result with the information) is in agreement and the face area is detected, and the result information are included. The result information shows the result of collating the face information stored in the authentication information database 114 with the second feature information extracted from the face area, and is generated by the authentication unit 120. Specifically, the result information is data stored in the authentication result storage unit 116.
The matching result with the first feature information 14a (human area feature information) means, for example, including a region where the degree of matching with the first feature information 14a (human area feature information) is equal to or higher than the standard.
[0066]
The search target display unit 202 displays the second image 22 in which the search target person is shown. The second image 22 displayed on the search target display unit 202 is a part of the second image 22 including at least the human area of ​​the search target person, and is generated by cutting out a rectangle including the human area of ​​the search target person, for example. It may be an image. The search result display unit 210 includes at least one candidate display unit 212. Each candidate display unit 212 includes an image display unit 214. The candidate area image of the candidate is displayed on the image display unit 214 together with the imaging conditions (for example, the imaging location) of the image and the degree of matching with the second image 22 displayed on the search target display unit 202. NS. The candidate person area image is at least a part of the first image 10, and may be, for example, an image generated by cutting out a rectangle including a person area matching by matching. The authentication result display unit 220 displays the result of face recognition, and the candidate display unit 222 is displayed for each candidate of the candidate display unit 212. The candidate display unit 222 includes an image display unit 224. The image display unit 224 displays at least a face image of a person having a high degree of matching with the candidate person area image. The information displayed on the image display unit 224 is included in the above-mentioned result information.
[0067]
In the example of FIG. 15, the information of the plurality of candidates is displayed in descending order of the degree of coincidence with the first feature information 14a. Here, in the candidate display unit 222, the information of the plurality of candidates may be displayed in descending order of the degree of coincidence with the second feature information. Further, the number of candidates displayed on the search result display unit 210 and the authentication result display unit 220 may be set by a setting menu (not shown) or the like. Further, all the candidates whose matching degree is equal to or higher than the predetermined reference value may be displayed by the list screen or scrolling. In this case, a predetermined reference value may be set in the setting menu.
[0068]
Further, as shown in FIG. 14, the information processing apparatus 100 further includes an authentication result storage unit 116. The authentication result storage unit 116 may be integrated with the main body of the information processing apparatus 100 or may be a separate body. The authentication result storage unit 116 may be included in the same device as the second feature information storage unit 110 and the authentication information database 114.It may be included in different devices. The authentication result storage unit 116 may have a database structure.
[0069]
FIG. 16 is a diagram showing an example of the data structure of the authentication result storage unit 116 of the present embodiment. For each candidate, the authentication result storage unit 116 includes a candidate ID, an authentication result ID for identifying the result information, a matching degree of face recognition, and an authentication information ID indicating the original authentication information from which the result information was obtained. Is associated and registered.
[0070]
FIG. 17 is a flowchart showing an example of the operation of the information processing apparatus 100 of the present embodiment. First, after executing the authentication process (step S120) described with reference to FIG. 11, the acquisition unit 122 stores the information about the person acquired by the authentication process in the authentication result storage unit 116 as the result information (step S127). Then, the display processing unit 130 causes the display device 132 to display the acquired information (result list screen 200 in FIG. 15) (step S130).
[0071]
FIG. 18 is a flowchart showing an example of a detailed procedure of the display process of step S130 of FIG. First, the acquisition unit 122 acquires the image of the search target person (second image 22) from the first feature information storage unit 112 (step S131). Further, the acquisition unit 122 acquires a candidate image (first image 10) including a region having a degree of coincidence equal to or higher than the reference degree and a face region from the second feature information storage unit 110 (step S133). Then, the acquisition unit 122 acquires the result information from the authentication result storage unit 116 (step S135).
[0072]
Then, the display processing unit 130 displays the second image 22 acquired in step S131 on the search target display unit 202 of the result list screen 200, and displays the candidate image acquired in step S133 in the image display unit 214 of the search result display unit 210. And the result information acquired in step S135 is displayed on the authentication result display unit 220 (step S137).
[0073]
FIG. 19 is a diagram showing details of a display example of the candidate display unit 212 of FIG. The candidate display unit 212 includes an image display unit 214, a candidate rank display unit 215, a matching degree display unit 216, and a shooting location display unit 218. An image including the candidate's human area 16 is displayed on the image display unit 214, and a frame surrounding the human area 16 and a frame surrounding the face area 20 may be displayed, respectively.
[0074]
FIG. 20 is a diagram showing details of a display example of the candidate display unit 222 of FIG. The candidate display unit 222 includes an image display unit 224, a candidate rank display unit 225, a matching degree display unit 226, a name display unit 228, a candidate attribute information display unit 229, and a detail display button 230. .. The image display unit 224 displays a face image of a person detected by the face recognition process. The name of the person is displayed on the name display unit 228. At least one of the face image and the identification information of the person detected by the face authentication process is displayed on the candidate display unit 222. The identification information of the person may be the identification information (authentication ID in FIG. 10) of the authentication information database 114, or may be at least one of the name and the attribute.
[0075]
The candidate attribute information display unit 229 displays the attribute information of the person detected by the face recognition process, for example, gender and age. The detail display button 230 is an operation button for displaying further attribute information of the person. When the detail display button 230 is pressed, the detailed information window 240 of FIG. 21 is displayed. The detailed information displayed in the detailed information window 240 may be registered in the authentication information database 114, or may be stored in another storage device in association with the authentication ID. The authentication result storage unit 116 stores information for arriving at this detailed information as a part of the result information.
[0076]
As described above, in the present embodiment, the image of the search target person (second image 22) by the display processing unit 130 and the first image 10 to the first feature information 14a (human area feature) captured by the camera 5. The image of the candidate's human area detected by the collation process with the information) and the image and information of the candidate detected by the collation process using the authentication information database 114 for the face area included in the person area are the same result. It is displayed on the list screen 200.
[0077]
As described above, according to the present embodiment, the image of the candidate is detected from the first image 10 captured by the camera 5 based on the image of the detection target person (second image 22) in which the face is not captured. Further, from the images of the candidate, the face image and the attribute information of the person registered in the authentication information database 114 for the face area can be viewed on the same result list screen 200.
[0078]
(Fourth Embodiment)
In the above embodiment, the configuration in which the search target display unit 202, the search result display unit 210, and the authentication result display unit 220 are simultaneously displayed on the result list screen 200 has been described. In the information processing apparatus 100 of the present embodiment, the search target display unit 202 and the candidate display unit 212 can display images of at least one candidate, or the candidates can be sequentially switched and displayed in order of rank. Further, the candidate display unit 222 showing the face recognition result for the selected candidate is displayed.
[0079]
22 to 24 are diagrams showing an example of a procedure for display processing of the information processing apparatus 100 of the present embodiment. Further, FIG. 25 is a diagram for explaining the transition of the display screen. Hereinafter, the display process of the information processing apparatus 100 will be described with reference to these figures.
[0080] [0080]
First, the display processing unit 130 receives the designation of the search target person from the second image 22 and displays it on the search target display unit 202 of the screen 300 of FIG. 25 (a) (step S201). For example, when a plurality of people are included in the second image 22, an operator may operate and display a frame for designating the person area 16 of the target person on the second image 22. The screen 300 further includes a search button 302 and a cancel button 304.
[0081]
Further, in the example of this figure, the number of search target persons is one, but when the collation process is performed using the first feature information 14a of the plurality of search target persons shown in the plurality of second images 22, a plurality of persons are searched. A plurality of images of the search target person may be displayed on the search target display unit 202. At least one candidate display unit 212 may be displayed for each search candidate, or information on a plurality of candidates obtained as a result of collation processing using the first feature information 14a of a plurality of search target persons may be displayed as candidates. A plurality of persons may be displayed on the person display unit 212.
[0082]
When the press of the search button 302 is accepted (YES in step S203), the information processing apparatus 100 executes the search process described with reference to FIG. 8 (step S110). During the search process, a message 306 indicating that the search is in progress is displayed (step S205). When the information processing apparatus 100 receives the press of the cancel button 304 during the search process, the information processing apparatus 100 cancels the detection process and returns to step S201 (not shown).
[0083]
If no candidate is found as a result of the search process (NO in step S207), the screen 310 of FIG. 25 (e) is displayed. The screen 310 includes a message 312 indicating that no candidate was found. Further, the screen 310 may include a search button 314. When the re-search button 314 is pressed, the process returns to step S201, a message prompting the search target person to reselect the second image 22 is displayed, and the re-selection may be accepted (not shown).
[0084]
In this embodiment, the display method of the candidate can be set in advance by the setting menu (not shown) or the like. In the setting menu, it is possible to select whether to display the detected candidates one by one or all the detected candidates (for a predetermined number of people (for example, for the top three people)). When at least one candidate is found (YES in step S207), the information processing apparatus 100 switches the display method of the candidate according to the setting (step S211). For example, when all (in this example, the top three persons) are selected, the process proceeds to the authentication process of step S120, and the face authentication process is performed on the face area of ​​each candidate. Then, the result list screen 200 (FIG. 15) showing the result of the face recognition process is displayed (step S213).
[0085]
When the setting for sequentially displaying the candidates one by one is selected (sequentially in step S211), the information processing apparatus 100 proceeds to step S221 (FIG. 23). First, 1 is set in the counter i (step S221). When i is 1 or less, as shown in FIG. 25B, the next button 332 for displaying the next candidate is displayed on the screen 330, and the front button for displaying the previous candidate is displayed. Is not displayed (step S225). The search target display unit 202 and the candidate display unit 212 of the first candidate are displayed on the screen 330 (step S233). Further, the screen 330 displays a search button 334 that accepts the face recognition process of the candidate.
[0086]
Processing can be switched according to the type of operation button pressed on the screen 330 (step S235). When the press of the next button 332 is accepted, i is incremented to 2 (step S243), and the process returns to step S223. Since i is 2, it becomes NO in step S223, and the front button 342 is further displayed on the screen 340 (step S227). Here, assuming that the maximum value of the number of display candidates is set to 3, since i> the maximum value imax-1 (2) is not (NO in step S229), the process proceeds to step S233, and FIG. The candidate display unit 212 of the second candidate is displayed on the screen 340 of c) (step S233).
[0087]
When the front button 342 is pressed (before step S235), i is decremented to 1 (step S241), and the process returns to step S223. Then, since i ≦ 1, the process proceeds to step S225, the next button 332 is displayed, and the front button 342 is hidden. Then, the candidate display unit 212 of the first candidate is displayed on the screen 330 (FIG. 25 (b)) (step S235).
[0088]
When the search button 334 is pressed on the screen 330 of FIG. 25 (b) (search in step S235), face recognition processing is performed on the face area of ​​the first candidate (step S120). Then, when the result information is acquired, the result screen 350 of FIG. 25 (d) is displayed (step S237). The search target display unit 202, the candidate display unit 212, and the candidate display unit 222 are displayed on the result screen 350, and a change button 352 for executing the face recognition process of another candidate is further displayed. May be done.
[0089]
When the change button 352 is pressed (YES in step S239), the process returns to step S223. Since i is 1 here, the screen 330 of FIG. 25 (b) is displayed. Then, when the next button 332 is selected, the second candidate is displayed (step S233), and when the search button 334 is selected (search in step S235), face recognition processing is performed for the second candidate (step). S120).
[0090]
Further, as another example, instead of displaying one candidate at a time, a predetermined number of candidates (for example, the top three) may be displayed, and a person who performs face recognition processing may be selected from the candidates. The flowchart of FIG. 24 shows an example of the display processing procedure. When a candidate is detected in step S207 of FIG. 22 (YES in step S207), the screen 360 of FIG. 25 (f) is displayed (step S251). The screen 360 displays an authentication result display unit 220 and a search result display unit 210 including a plurality of candidate candidate display units 212 (three in the example of this figure). The candidate display unit 212 displays a search button 362 that accepts selection of a candidate for face recognition processing.
[0091]
When the search button 362 is pressed (YES in step S253), the face recognition process is executed by the authentication unit 120 for the selected candidate (step S120). Then, the result of the face recognition process is displayed on the screen 350 of FIG. 25 (d). On the screen 350, the image of the candidate selected as the target of the face recognition process is displayed on the candidate display unit 212.
[0092]
Explanation above As described above, in the present embodiment, the display processing unit 130 can display the screen 320 or the screen 360 including the search target display unit 202 and the search result display unit 210. Then, on the screen 320 or the screen 360, the operator can select a candidate to perform the face authentication process and perform the face authentication process, so that the face authentication process can be omitted for a clearly different person, which is efficient. Is improved. In addition, since the display content can be changed according to the size of the display and the like, an easy-to-see screen display becomes possible.
[0093]
FIG. 26 is a diagram showing an example of the configuration of a computer 80 that realizes the information processing apparatus 100 of each of the above-described embodiments.
The computer 80 stores a program 90 that realizes the components of the information processing apparatus 100 of FIGS. 2, 9 and 11 loaded in the CPU (Central Processing Unit) 82, the memory 84, and the memory 84, and the program 90 thereof. It includes a storage 85, an I / O (Input / Output) 86, and a network connection interface (communication I / F87).
[0094]
The CPU 82, the memory 84, the storage 85, the I / O 86, and the communication I / F 87 are connected to each other via the bus 89, and the entire information processing apparatus is controlled by the CPU 82. However, the method of connecting the CPU 82 and the like to each other is not limited to the bus connection.
[0095]
The memory 84 is a memory such as a RAM (Random Access Memory) or a ROM (Read Only Memory). The storage 85 is a storage device such as a hard disk, SSD (Solid State Drive), or a memory card.
[0096]
The storage 85 may be a memory such as RAM or ROM. The storage 85 may be provided inside the computer 80, or may be provided outside the computer 80 and may be connected to the computer 80 by wire or wirelessly if the computer 80 is accessible. Alternatively, it may be detachably provided on the computer 80.
[0097]
The CPU 82 can realize each function of each unit of the information processing apparatus 100 of each embodiment by reading the program 90 stored in the storage 85 into the memory 84 and executing the program 90.
[0098]
The I / O 86 controls input / output of data and control signals between the computer 80 and other input / output devices. The other input / output devices include, for example, an input device (not shown) such as a keyboard, touch panel, mouse, and microphone connected to the computer 80, and an output device (not shown) such as a display, a printer, and a speaker. These input / output devices and the interface of the computer 80 are included. Further, the I / O 86 may perform data input / output control with a reading or writing device (not shown) of another recording medium.
[0099]
The communication I / F 87 is a network connection interface for communicating between the computer 80 and an external device. The communication I / F 87 may be a network interface for connecting to a wired line or a network interface for connecting to a wireless line. For example, the computer 80 that realizes the information processing apparatus 100 is connected to at least one camera 5 via a network by communication I / F 87.
[0100]
Each component of the information processing apparatus 100 of each embodiment is realized by any combination of the hardware and software of the computer 80 in this figure. And, it is understood by those skilled in the art that there are various variations in the method of realizing the device and the device. The functional block diagram showing the information processing apparatus 100 of each of the above-described embodiments shows a block of logical functional units, not a configuration of hardware units.
[0101]
The information processing device 100 may be configured by a plurality of computers 80 or may be realized by a virtual server.
[0102]
The present invention can be realized as an information processing system exemplified below as an example composed of a plurality of computers 80, but the present invention is not limited thereto.
(1) The extraction unit 104 and the collation unit 102 are carried out by different devices (computer 80). For example, the information processing system may include a terminal device having an extraction unit 104 and a server device having a collation unit 102.
(2) The storage means for realizing the second feature information storage unit 110 and the first feature information storage unit 112 may be realized by a plurality of devices. For example, a storage device that stores the second feature information extracted from the person included in the first image and a storage device that stores the identification information and the second feature information of the specific person may separately exist. ..
(3) Among the extraction processes of the extraction unit 104, the extraction process of the feature information from the first image 10 and the extraction process of the feature information from the second image 20 are performed by another device (computer 80). For example, the information processing system may include a plurality of terminal devices having the extraction unit 104 for analyzing images, and a server device that integrates information from them and performs collation processing. A plurality of terminal devices may be arranged separately for each area, and the extraction unit 102 may be realized by a plurality of physical devices.
(4) The device (computer 80) may be divided according to the type of feature information (face feature information and human area feature information) that is the target of video analysis (extraction unit 102 and collation unit 104).
[0103]
Although the embodiments of the present invention have been described above with reference to the drawings, these are examples of the present invention, and various configurations other than the above can be adopted.
For example, in the above embodiment, the feature information extracted from the human area of ​​the person included in the first image 10 and the first feature information 14a of the search target person are matched in the first collation process and the collation process. The second collation process was performed between the second feature information extracted from the face area of ​​the person in the first image 10 and the face feature information of the registrant. In another embodiment, using the first feature information 14a and the second feature information, the collation unit 102 uses the first feature information 14a to make a first image with respect to a third image different from the first image 10 and the second image 22. At least one of the collation process and the second collation process may be performed.
[0104]
In this configuration, if the face region can be detected from the third image, the second collation process with the second feature information can be performed instead of the first collation process with the first feature information 14a. If the face region cannot be detected from the third image, the first collation process with the first feature information 14a may be performed. Then, the new second feature information generated by the first collation process may be stored in the second feature information storage unit 110.
[0105]
According to this configuration, for example, a collation process using the first feature information 14a or the second feature information that has already been acquired is performed on the third image captured at a time and place different from that of the first image 10. By doing so, it is possible to search for the person to be searched for the third image as well.
[0106]
Further, in the above embodiment, the first feature information is the feature information extracted from the first part of the human region, and the second feature information is the feature information extracted from the second part of the human region, particularly the face region. However, other combinations of the first feature information and the second feature information can be considered.
[0107]
As a first example, the first characteristic information is information indicating at least one of the color, age group, gender, and height of a person's clothing, and the second characteristic information is facial feature information of a person. May be.
[0108]
In this example, for example, when face recognition cannot be performed because the face of the search target person is not shown in the second image 22, instead of performing the process of extracting the first feature information 14a from the human area of ​​the second image 22. , At least one of the color, age group, gender, and height of the clothes of the search target person may be designated by an operator operation and stored in the first feature information storage unit 112 as the first feature information 14a. .. In this example, the operation reception unit (not shown) that accepts the input of the first feature information of the search target person may be provided on the screen together with the search target display unit 202 that displays the second image 22. The operator can input the first feature information of the person while looking at the image of the search target person displayed on the search target display unit 202.
[0109]
As a second example, the first feature information is the feature information of the entire human domain, and the second feature information is a plurality of biometric authentication information for performing a multimodal biometric authentication process in which a plurality of biometric informations are combined. May be.
[0110]
In this example, for example, it can be applied to a case where face authentication is difficult because the face of the search target person is not shown in the second image 22, and gait authentication is difficult because the lower half of the body is not shown. The first feature information 14a is extracted from the human area of ​​the person shown in the second image 22, a candidate is detected using the first feature information 14a for the first image 10, and the detected candidate is described. A person may be identified by a multimodal biometric authentication process using a plurality of biometric information as the second feature information.
[0111]
Although the invention of the present application has been described above with reference to the embodiments and examples, the invention of the present application is not limited to the above embodiments and examples. Various changes that can be understood by those skilled in the art can be made within the scope of the present invention in terms of the configuration and details of the present invention.
In the present invention, when information about a user is acquired and used, this shall be done legally.
[0112]
A part or all of the above embodiment may be described as in the following appendix, but is not limited to the following.
1. 1. The first image extracted from the human area feature information related to the area including the area other than the face of the person to be searched,
The second image in which the collation result with the human area feature information is in agreement and the face area is detected, and
Result information showing the collation result between the face information stored in the storage unit and the face information extracted from the face area,
And the acquisition method to acquire each of
A display processing means for displaying at least two of the first image, the second image, and the result information on the same screen.
Information processing device equipped with.
2. 2. The second image in which the collation result with the human area feature information regarding the area including the area other than the face of the person to be searched matches and the face area was detected and
Acquisition means for acquiring the result information indicating the collation result between the face information stored in the storage unit and the face information extracted from the face area, and
A display processing means for displaying the acquired second image and the result information on the same screen,
Information processing device equipped with.
3. 3. The first image extracted the human area feature information related to the area including the area other than the face of the person to be searched, and
Acquisition means for acquiring the face information extracted from the second image in which the collation result with the human area feature information matches and the result information showing the collation result with the face information stored in the storage unit.
A display processing means for displaying the acquired first image and the result information on the same screen,
Information processing device equipped with.
4. 1. From 3. In the information processing apparatus described in any one of
The acquisition means acquires a plurality of the second images and obtains them.
The display processing means is an information processing device that displays the result information for each of the plurality of acquired second images.
5. 1. From 4. In the information processing apparatus described in any one of
The display processing means is
An information processing device that displays the first image, the second image, and the result information on the same screen.
6. 1. From 5. In the information processing apparatus described in any one of
The display processing means is an information processing device that displays the first image and the second image on the same screen.
7. 6. In the information processing apparatus described in
The acquisition means acquires a plurality of the second images and obtains them.
The display processing means is an information processing device that displays the result information for the selected second image when it receives a selection operation for selecting at least one image from the plurality of the second images.
8. 1. From 7. In the information processing apparatus described in any one of
The facial feature information of a specific person and the information of the specific person are linked to each other and stored in the storage unit.
The result information has a degree of coincidence with the facial feature information stored in the storage unit. , And an information processing apparatus containing at least one of the information of the specific person.
9. 8. In the information processing apparatus described in
The information of the specific person is
An information processing device containing at least one of the facial photograph and identification information of the specific person.
[0113]
10. Information processing device
The first image used to generate the human area characteristic information showing the characteristics of the person to be searched,
The second image in which the collation result with the human area feature information is in agreement and the face area is detected, and
Result information showing the collation result between the face information stored in the storage unit and the face information extracted from the face area,
Get each of
An information processing method for displaying at least two of the first image, the second image, and the result information on the same screen.
11. Information processing device
The second image in which the collation result with the human area feature information regarding the area including the area other than the face of the person to be searched matches and the face area was detected and
Acquire the result information indicating the collation result between the face information stored in the storage unit and the face information extracted from the face area, and obtain the result information.
An information processing method for displaying the acquired second image and the result information on the same screen.
12. Information processing device
The first image extracted the human area feature information related to the area including the area other than the face of the person to be searched, and
Acquires the result information showing the collation result between the face information extracted from the second image in which the collation result with the human area feature information matches and the face information stored in the storage unit.
An information processing method for displaying the acquired first image and the result information on the same screen.
13. 10. From 12. In the information processing method described in any one of
The information processing device
Acquired multiple second images,
An information processing method for displaying the result information for each of the plurality of acquired second images.
14. 10. From 13. In the information processing method described in any one of
The information processing device
An information processing method for displaying the first image, the second image, and the result information on the same screen.
15. 10. From 14. In the information processing method described in any one of
The information processing device
An information processing method for displaying the first image and the second image on the same screen.
16. 15. In the information processing method described in
The information processing device
Acquired multiple second images,
An information processing method for displaying the result information for the selected second image when a selection operation for selecting at least one image from the plurality of the second images is accepted.
17. 10. From 16. In the information processing method described in any one of
The information processing device
The facial feature information of a specific person and the information of the specific person are linked to each other and stored in the storage unit.
The result information is an information processing method including at least one of the degree of matching with the facial feature information stored in the storage unit and the information of the specific person.
18. 17. In the information processing method described in
The information of the specific person is
Information processing method including at least one of the facial photograph and identification information of the specific person.
[0114]
19. On the computer
The first image used to generate the human area characteristic information showing the characteristics of the person to be searched,
The second image in which the collation result with the human area feature information is in agreement and the face area is detected, and
Result information showing the collation result between the face information stored in the storage unit and the face information extracted from the face area,
Procedure to get each of
A program for executing a procedure for displaying at least two of the first image, the second image, and the result information on the same screen.
20. On the computer
The second image in which the collation result with the human area feature information regarding the area including the area other than the face of the person to be searched matches and the face area was detected and
Procedure for acquiring the result information indicating the collation result between the face information stored in the storage unit and the face information extracted from the face area.
A program for executing the procedure for displaying the acquired second image and the result information on the same screen.
21. On the computer
The first image extracted the human area feature information related to the area including the area other than the face of the person to be searched, and
The procedure for acquiring the result information indicating the collation result between the face information extracted from the second image in which the collation result with the human area feature information matches and the face information stored in the storage unit,
A program for executing the acquired first image and the procedure for displaying the result information on the same screen.
22. 19. From 21. In the program described in any one of
The acquisition means is a procedure for acquiring a plurality of the second images,
The display processing means is a program for causing a computer to execute a procedure for displaying the result information for each of the plurality of acquired second images.
23. 19. From 22. In the program described in any one of
The display processing means is
A program for causing a computer to execute the procedure for displaying the first image, the second image, and the result information on the same screen.
24. 19. From 23. In the program described in any one of
The display processing means is a program for causing a computer to execute a procedure for displaying the first image and the second image on the same screen.
25. 24. In the program described in
The acquisition means is a procedure for acquiring a plurality of the second images,
When the display processing means receives a selection operation for selecting at least one image from the plurality of the second images, the display processing means causes the computer to execute a procedure for displaying the result information for the selected second image. Program.
26. 19. From 25. In the program described in any one of
The facial feature information of a specific person and the information of the specific person are linked to each other and stored in the storage unit.
The result information is a program including at least one of the degree of matching with the facial feature information stored in the storage unit and the information of the specific person.
27. 26. In the program described in
The information of the specific person is
A program that includes at least one of the specific person's face photo and identification information.
The scope of the claims
[Claim 1]
The first image extracted from the human area feature information related to the area including the area other than the face of the person to be searched,
The second image in which the collation result with the human area feature information is in agreement and the face area is detected, and
Result information showing the collation result between the face information stored in the storage unit and the face information extracted from the face area,
And the acquisition method to acquire each of
A display processing means for displaying at least two of the first image, the second image, and the result information on the same screen.
Information processing device equipped with.
[Claim 2]
The second image in which the collation result with the human area feature information regarding the area including the area other than the face of the person to be searched matches and the face area was detected and
Acquisition means for acquiring the result information indicating the collation result between the face information stored in the storage unit and the face information extracted from the face area, and
A display processing means for displaying the acquired second image and the result information on the same screen,
Information processing device equipped with.
[Claim 3]
The first image extracted the human area feature information related to the area including the area other than the face of the person to be searched, and
Acquisition means for acquiring the face information extracted from the second image in which the collation result with the human area feature information matches and the result information showing the collation result with the face information stored in the storage unit.
A display processing means for displaying the acquired first image and the result information on the same screen,
Information processing device equipped with.
[Claim 4]
In the information processing apparatus according to any one of claims 1 to 3,
The acquisition means acquires a plurality of the second images and obtains them.
The display processing means is an information processing device that displays the result information for each of the plurality of acquired second images.
[Claim 5]
In the information processing apparatus according to any one of claims 1 to 4,
The display processing means is
An information processing device that displays the first image, the second image, and the result information on the same screen.
[Claim 6]
In the information processing apparatus according to any one of claims 1 to 5,
The display processing means is an information processing device that displays the first image and the second image on the same screen.
[Claim 7]
In the information processing apparatus according to claim 6,
The acquisition means acquires a plurality of the second images and obtains them.
The display processing means is an information processing device that displays the result information for the selected second image when it receives a selection operation for selecting at least one image from the plurality of the second images.
[Claim 8]
In the information processing apparatus according to any one of claims 1 to 7.
The facial feature information of a specific person and the information of the specific person are linked to each other and stored in the storage unit.
The result information is an information processing device including at least one of the degree of matching with the facial feature information stored in the storage unit and the information of the specific person.
[Claim 9]
In the information processing apparatus according to claim 8,
The information of the specific person is
An information processing device containing at least one of the facial photograph and identification information of the specific person.
[Claim 10]
Information processing device
The first image used to generate the human area characteristic information showing the characteristics of the person to be searched,
The second image in which the collation result with the human area feature information is in agreement and the face area is detected, and
Result information showing the collation result between the face information stored in the storage unit and the face information extracted from the face area,
Get each of
An information processing method for displaying at least two of the first image, the second image, and the result information on the same screen.
[Claim 11]
Information processing device
The second image in which the collation result with the human area feature information regarding the area including the area other than the face of the person to be searched matches and the face area was detected and
Acquire the result information indicating the collation result between the face information stored in the storage unit and the face information extracted from the face area, and obtain the result information.
An information processing method for displaying the acquired second image and the result information on the same screen.
[Claim 12]
Information processing device
The first image extracted the human area feature information related to the area including the area other than the face of the person to be searched, and
Acquires the result information showing the collation result between the face information extracted from the second image in which the collation result with the human area feature information matches and the face information stored in the storage unit.
An information processing method for displaying the acquired first image and the result information on the same screen.
[Claim 13]
On the computer
The first image used to generate the human area characteristic information showing the characteristics of the person to be searched,
The second image in which the collation result with the human area feature information is in agreement and the face area is detected, and
Result information showing the collation result between the face information stored in the storage unit and the face information extracted from the face area,
Procedure to get each of
A program for executing a procedure for displaying at least two of the first image, the second image, and the result information on the same screen.
[Claim 14]
On the computer
The second image in which the collation result with the human area feature information regarding the area including the area other than the face of the person to be searched matches and the face area was detected and
Procedure for acquiring the result information indicating the collation result between the face information stored in the storage unit and the face information extracted from the face area.
A program for executing the acquired second image and the procedure for displaying the result information on the same screen.
[Claim 15]
On the computer
The first image extracted the human area feature information related to the area including the area other than the face of the person to be searched, and
The procedure for acquiring the result information indicating the collation result between the face information extracted from the second image in which the collation result with the human area feature information matches and the face information stored in the storage unit,
A program for executing the acquired first image and the procedure for displaying the result information on the same screen.