Title of invention: Information processing system, information processing device, information processing method, and program
Technical field
[0001]
　The present invention relates to an information processing system, an information processing device, an information processing method, and a program, and more particularly to an information processing system, an information processing device, an information processing method, and a program that perform image recognition processing.
Background technology
[0002]
　In recent years, a technique for detecting a person who meets a desired condition using an image has been developed. For example, in Patent Document 1, person characteristic information is accumulated for each image included in a video, a person corresponding to a preset condition is searched for the accumulated information, and information corresponding to the search result is provided. It is described to be extracted. Further, it is also described that the above-mentioned state of the person is used when determining the state of the person and identifying the person using various feature quantities.
[0003]
　Further, Patent Document 2 describes a system for identifying all persons who meet the search conditions from a group of images taken by a plurality of cameras. Further, in Patent Document 3, whether or not a person included in a different image sequence is the same based on the walking state by detecting the walking state of the person from a distant frame or a person image sequence obtained by a different camera. Is described to determine. In Patent Document 4, even when there are multiple face images in the video, the unique information of the matching face images is acquired for each face image by matching with the database, and the unique information is provided in the vicinity of the person in the video. It is stated to be displayed.
Prior art literature
Patent documents
[0004]
Patent Document 1: Japanese Patent Application Laid-Open No. 2010-257451
Patent Document 2: Japanese Patent Application Laid-Open No. 2006-301995
Patent Document 3: International Publication No. 2006/013765
Patent Document 4: Japanese Patent Application Laid-Open No. 2006-293912
Outline of the invention
Problems to be solved by the invention
[0005]
　The present inventor has investigated a new method for continuously tracking a specific person with a plurality of cameras using image processing. That is, an object of the present invention is to provide a new method for continuously tracking a specific person with a plurality of cameras by using image processing.
Means to solve problems
[0006]
　In each aspect of the present invention, the following configurations are adopted in order to solve the above-mentioned problems.
[0007]
　The first aspect relates to an information processing device.
　The information processing device of the first aspect
　extracts the first facial feature information from the face region of the person included in the first image, and extracts the first human region feature information from the region including other than the face of the person. When the extraction means has an extraction means and
　a collation means for collating two or more feature information, and the
　extraction means cannot extract the second face feature information from the face area of ​​the person included in the second image, the above-mentioned The second human area feature information is extracted from the area including the area other than the face of the person included in the second image, and the
　collating means collates the first human area feature information with the second human area feature information. do.
[0008]
　The second aspect relates to information processing methods performed by at least one computer.
　In the first information processing method according to the second aspect, the
　information processing apparatus
　extracts the first facial feature information from the face region of the person included in the first image, and the first is from the region including the region other than the face of the person. When one person area feature information is extracted,
　two or more feature informations are collated, and
　the second face feature information cannot be extracted from the face area of ​​the person included in the second image, it is included in the second image. This includes extracting the second human area characteristic information from the area including the area other than the face of the person,
　and collating the first human area characteristic information with the second human area characteristic information.
[0009]
　As another aspect of the present invention, it may be a program that causes at least one computer to execute the method of the second aspect, or a recording medium that can be read by a computer that records such a program. You may. This recording medium includes a non-temporary tangible medium.
　This computer program includes computer program code that causes the computer to implement its control method on an information processing device when executed by the computer.
[0010]
　It should be noted that any combination of the above components and the conversion of the expression of the present invention between methods, devices, systems, recording media, computer programs and the like are also effective as aspects of the present invention.
[0011]
　Further, the various components of the present invention do not necessarily have to be independent of each other, and a plurality of components are formed as one member, and one component is formed of a plurality of members. It may be that a certain component is a part of another component, a part of a certain component overlaps with a part of another component, and the like.
[0012]
　Further, although the method and the computer program of the present invention describe a plurality of procedures in order, the order of description does not limit the order in which the plurality of procedures are executed. Therefore, when implementing the method and computer program of the present invention, the order of the plurality of procedures can be changed within a range that does not hinder the contents.
[0013]
　Furthermore, the methods of the present invention and the plurality of procedures of a computer program are not limited to being executed at different timings. Therefore, another procedure may occur during the execution of a certain procedure, a part or all of the execution timing of the certain procedure and the execution timing of the other procedure may overlap, and the like.
The invention's effect
[0014]
　According to each of the above aspects, it is possible to provide a new method for continuously tracking a specific person with a plurality of cameras by using image processing.
A brief description of the drawing
[0015]
　The above-mentioned objectives and other objectives, features and advantages will be further clarified by the preferred embodiments described below and the accompanying drawings below.
[0016]
FIG. 1 is a diagram conceptually showing a configuration of an image processing system according to an embodiment of the present invention.
FIG. 2 is a functional block diagram showing a logical configuration of the information processing apparatus of the present embodiment.
FIG. 3 is a diagram showing an example of a data structure of a first image and a second image.
[Fig. 4] Fig. 4 is a flowchart showing an example of operation of an information processing device.
FIG. 5 is a functional block diagram showing a logical configuration of the information processing apparatus of the present embodiment.
[Fig. 6] Fig. 6 is a diagram showing an example of a data structure of a facial feature information database.
[Fig. 7] Fig. 7 is a flowchart showing an example of the operation of the information processing device.
FIG. 8 is a functional block diagram showing a logical configuration of the information processing apparatus of the present embodiment.
[Fig. 9] Fig. 9 is a diagram showing variations in the data structure of the human area feature information storage unit.
[Fig. 10] Fig. 10 is a diagram showing an example of a screen displayed on a display device by a display processing unit.
FIG. 11 is a functional block diagram showing a logical configuration of the information processing apparatus of the present embodiment.
[Fig. 12] Fig. 12 is a diagram showing an example of a data structure of an unregistered person information storage unit.
[Fig. 13] Fig. 13 is a flowchart showing an example of the operation of the information processing device.
FIG. 14 is a diagram showing an example of a configuration of a computer that realizes the information processing apparatus of each embodiment.
[Fig. 15] Fig. 15 is a diagram showing an example of a data structure of a human area feature information storage unit.
[Fig. 16] Fig. 16 is a schematic diagram for explaining an example of a data structure of a human area feature information storage unit.
Mode for carrying out the invention
[0017]
　Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all drawings, similar components are designated by the same reference numerals, and description thereof will be omitted as appropriate.
　Further, in each figure of the present specification, the configuration of a portion not related to the essence of the present invention is omitted and is not shown.
[0018]
(First Embodiment)
　FIG. 1 is a diagram conceptually showing a configuration of an image processing system according to an embodiment of the present invention. The image processing system 1 includes an information processing device 100. The information processing device 100 is connected to a plurality of cameras 5a, 5b, ... (referred to as "camera 5" when it is not particularly necessary to distinguish them) via a communication network 3. As will be described in detail later, the information processing apparatus 100 can continue tracking the person even if the face of the person to be tracked (hereinafter referred to as the person to be tracked) does not appear in the image.
[0019]
　The camera 5 includes an image sensor such as a lens and a CCD (Charge Coupled Device) image sensor, and is a network camera such as an IP (Internet Protocol) camera, for example. The network camera has, for example, a wireless LAN (Local Area Network) communication function, and is connected to the information processing device 100 via a communication network, that is, a relay device (not shown) such as a router. These cameras 5 may be so-called surveillance cameras installed in the city or inside or outside the building. The camera 5 may be provided with a mechanism that follows the movement of a specific person to control the orientation of the camera body or men, zoom control, focus, and the like.
[0020]
　The camera 5 and the information processing device 100 may be directly connected, or may be connected via a communication network 3 or the like as described above. For example, the image data captured by the camera 5 may be directly transmitted to the information processing device 100, and the information processing device 100 may sequentially receive the image data. Further, a storage device (not shown) that can be accessed by both the camera 5 and the information processing device 100 may be provided. In this case, the image data captured by the camera 5 is stored in the storage device. Then, the information processing device 100 reads out the image data from this storage device.
[0021]
　Here, the image data may be at least one of a still image and a moving image. The data format, file format, file size, image size, image resolution, video frame rate, etc. are not particularly limited, and the specifications, standards, performance, etc. of the camera 5 and the information processing device 100, or the processing capacity of image analysis, etc. Various types can be adopted depending on the accuracy and the like. Then, at least one frame of this image data may be at least one of the first image and the second image described later.
[0022]
　Further, in the embodiment, "acquisition" means that the own device fetches data or information stored in another device or storage medium (active acquisition), and outputs the data to the own device from the other device. Includes at least one of entering data or information to be performed (passive acquisition). Examples of active acquisition include making a request or inquiry to another device and receiving the reply, and accessing and reading another device or storage medium. In addition, an example of passive acquisition may be receiving information to be delivered (or transmitted, push notification, etc.). Further, the “acquisition” may be to select and acquire the received data or information, or to select and receive the delivered data or information.
[0023]
　Further, the timing of transmitting the image from the camera 5 to the information processing device 100 may be real-time distribution, for example, streaming distribution, or images for a predetermined period may be transmitted at predetermined intervals. This transmission timing may be appropriately selected based on the memory capacity, communication capacity, image processing capacity of the camera 5 and the information processing device 100, the communication status between the camera 5 and the information processing device 100, and the like. , May be changed according to changing circumstances.
[0024]
　FIG. 2 is a functional block diagram showing a logical configuration of the information processing device 100 of the present embodiment. The information processing device 100 includes an extraction unit 102 and a collation unit 104.
[0025]
　The extraction unit 102 extracts the first face feature information from the face region of the person included in the first image 10, and extracts the first person region feature information from the region including other than the face of the person. The collation unit 104 collates two or more feature information. Then, when the extraction unit 102 cannot extract the second face feature information from the face area of ​​the person included in the second image 20, the second person area is extracted from the area including the face other than the face of the person included in the second image 20. Extract feature information. Then, the collation unit 104 collates the first person area feature information with the second person area feature information.
[0026]
　Specifically, the first image 10 is a reference image showing a person to be tracked. The second image 20 is an image to be searched for the tracking target person. The first image 10 and the second image 20 are imaged at different timings. The first image 10 and the second image 20 may be captured by the same camera 5, or may be captured by different cameras 5.
[0027]
　Then, the extraction unit 102 receives the first image 10 from the camera 5 or reads it from another recording medium (not shown) to acquire the first image 10. The extraction unit 102 first identifies at least one person as a tracking target person from the acquired first image 10 as a tracking target person, identifies the face area, and extracts at least one first facial feature information. The tracking target person may be at least one predetermined person included in the first image 10, or all the persons included in the first image 10, or as in the embodiment described later, the face feature information database. It may be at least one specific person registered in. Further, as the tracking target person, at least one person may be selected by the operator from the person included in the first image 10 or the person registered in the face feature information database. These settings may be predetermined, or may have a configuration in which the settings can be changed in response to the operator's designation.
[0028]
　As described above, the extraction unit 102 identifies the human region connected to the first face region from the first image 10. The human area is an area that shows a person's body and belongings. Then, the extraction unit 102 extracts the first person area feature information from the person area.
[0029]
　The extracted first person area feature information may be stored in a storage device (not shown).
　The first person area feature information is information generated from the person area connected to the face area of ​​the person to be tracked extracted by the extraction unit 102, and indicates the appearance characteristics such as the person's back appearance or clothes. It is characteristic information.
[0030]
　The first person area characteristic information includes, for example, height, shoulder width, ratio of body parts, clothes (shape, color, material, etc.), hairstyle (including hair color), ornaments (hat, glasses, accessories, etc.), belongings. Includes information indicating features such as (bags, umbrellas, canes, etc.). Further, the human domain characteristic information may include information such as the likelihood of the characteristic information.
[0031]
　Further, when the second face feature information cannot be extracted from the face area of ​​the person included in the second image 20, the extraction unit 102 further extracts the second person area from the area including the face other than the face of the person included in the second image 20. Extract feature information. Then, the collation unit 104 collates the extracted first person area feature information with the second person area feature information. The person area of ​​the second image 20 from which the second person area feature information matching the first person area feature information is extracted is the person area of ​​the person to be tracked. This makes it possible to continue tracking even if the face of the person to be tracked in the second image 20 is not shown.
[0032]
　Here, the case where the second face feature information cannot be extracted includes the case where the degree of matching with the first face feature information is less than the threshold value. Further, in the second image 20, at least a part of the face of the person to be tracked is shielded or the face is turned sideways, and a feature amount suitable for matching cannot be extracted.
[0033]
　The information processing device 100 may be capable of displaying the face image data of the person on the display in order to show the operator the face of the person to be tracked.
[0034]
　FIG. 3 is a diagram showing an example of the data structure of the first image 10 and the second image 20. The first image 10 and the second image 20 have an image ID that identifies the image, a camera ID that identifies the camera 5 that captured the image, and an image capture date and time (or may be a storage date and time or an update date and time). Includes the information to be shown and the body of the image data. The image ID may be a file name. When the image is a frame, the first image 10 and the second image 20 may include information indicating a frame number that can identify the frame.
[0035]
　In the present embodiment, the second image 20 is an image captured after the first image 10, and an image received in real time from each camera 5 can be used. However, in other embodiments, the second image 20 may be recorded data captured prior to the first image 10, that is, data stored in a non-volatile storage means. As will be described later, in the present embodiment, even if the face of the person is not shown, the person is tracked by features such as clothes, so that the tracking cannot be performed if the clothes are changed. Therefore, the second image 20 is a period during which it is assumed that the clothes cannot be changed or a period during which the same clothes as those of the first image 10 are worn from the imaging time of the first image 10. It will be the one imaged in.
[0036]
　The operation of the information processing apparatus 100 of the present embodiment configured in this way will be described below.
　FIG. 4 is a flowchart showing an example of the operation of the information processing apparatus 100.
　First, the extraction unit 102 acquires the first image 10, sets at least one person included in the first image 10 as a tracking target person, and extracts at least one first face feature information from the face area (step). S101).
[0037]
　Further, the extraction unit 102 extracts the first person area feature information from the area including the person other than the face of the person from the first image 10 (step S103). The extraction unit 102 identifies a human region from, for example, a portion of the region connected to the first face region that is located below the first face region. Further, the extraction unit 102 may also include a portion of the region connected to the first face region above the first face region in the human region.
[0038]
　Next, the extraction unit 102 acquires the second image 20 from the camera 5 and determines whether or not the second face feature information can be extracted from the face region of the person included in the second image 20 (step S105). When the second face feature information cannot be extracted (NO in step S105), the extraction unit 102 extracts the second person area feature information from the area including the face other than the person's face included in the second image 20 (step S107). ).
[0039]
　Next, the collation unit 104 collates the first person area feature information extracted in step S103 with the second person area feature information extracted in step S107 (step S109). When the second person area feature information matches the first person area feature information, the person area from which the second person area feature information is extracted is the person to be tracked in the second image 20. Further, when the second face feature information is extracted in step S105 (YES in step S105), the process of extracting the second person area feature information from the second image 20 and collating it (steps S107 and S109) is performed. End the process without. That is, the face area from which the second face feature information is extracted in the second image 20 from the matching result of the face feature information becomes the tracking target person.
[0040]
　As described above, in the present embodiment, the extraction unit 102 extracts the first face feature information and the first person area feature information of the person to be tracked from the first image 10. Then, even if the second face feature information is not extracted from the second image 20 and the tracking by the face feature information cannot be performed, the second person area feature information and the first person area feature information extracted from the second image 20 The tracked person can be tracked by the second image 20 by collating.
[0041]
　As described above, according to the present embodiment, even if the face of the person to be tracked is not captured in the second image 20 received from the camera 5, the tracking of the person can be continued.
[0042]
(Second Embodiment)
　FIG. 5 is a functional block diagram showing a logical configuration of the information processing apparatus 100 of the present embodiment. The information processing device 100 of the present embodiment is the same as that of the first embodiment except that the facial feature information of the person to be tracked is registered in the blacklist or the like in advance.
[0043]
　The information processing device 100 further includes a face feature information database 110 (referred to as "face feature information DB" in the figure). The facial feature information database 110 may be included in a plurality of devices. Further, the face feature information database 110 may be integrated with the main body of the information processing device 100 or may be a separate body. The face feature information database 110 has a database structure, but may have other data structures.
[0044]
　In the present embodiment, the facial feature information of the person to be tracked is registered in the facial feature information database 110.
　The collation unit 104 collates the face feature information extracted from the face region of the person included in the first image 10 with the first face information stored in the face feature information database 110. When the collation results are in agreement, the first person area feature information is extracted from the person included in the first image 10.
[0045]
　FIG. 6 is a diagram showing an example of the data structure of the face feature information database 110. In the face feature information database 110, first face information, which is face feature information of at least one person to be tracked, is registered. The face feature information database 110 is associated with a registrant ID that identifies a registered person, face feature information (corresponding to the first face information) of the person, a name, and face image data. The face feature information does not necessarily have to be stored in the face feature information database 110. In this case, the information processing device 100 extracts face feature information from the face image data when performing the collation process.
[0046]
　The file format of the face image data is not particularly limited, and examples thereof include JPEG (Joint Photographic Experts Group), BMP (Bit MaP), PNG (Portable Network Graphics), and TIFF (Tagged Image File Format). Further, the entity of the image data may be registered in the face feature information database 110, or a path indicating a place where the image data is saved and a file name may be registered.
[0047]
　Returning to FIG. 5, the extraction unit 102 identifies a face area from the first image 10 and collates it with the first face information registered in the face feature information database 110 (hereinafter, also referred to as a face recognition process). I do. The target person to be collated may be all a plurality of persons registered in the face feature information database 110, or at least one person may be selected by the operator. These settings may be predetermined, or may have a configuration in which the settings can be changed in response to the operator's designation.
[0048]
　The information processing apparatus 100 of the present embodiment is the same except that step S111 (not shown) is included instead of step S101 in the flowchart of FIG. In step S111, first, the extraction unit 102 acquires the first image 10 and extracts the face feature information from the face region of the person included in the first image 10. Then, the collation unit 104 collates the extracted face feature information with the first face information of the face feature information database 110. Then, the extraction unit 102 uses the face feature information that matches the first face information stored in the face feature information database 110 as the first face feature information.
[0049]
　Then, the extraction unit 102 extracts the first person area feature information from the area including the person other than the person's face in the first image 10 (step S103), and cannot extract the face feature information from the second image 20 (step). NO) of S105, the second person region feature information is extracted from the second image 20 (step S107). The collation unit 104 collates the second person area feature information extracted in step S107 with the first person area feature information extracted in step S103 (step S109).
[0050]
　As described above, in the present embodiment, the extraction unit 102 sets the first face feature information that matches the first face information registered in the face feature information database 110 from the first image 10 and the person in the face region. The first person area feature information of the above is extracted. Then, as in the above embodiment, even if the second image that matches the first face feature information of the person to be tracked is not extracted from the second image 20 and the tracking with the face feature information is not possible, the second image is displayed. By collating the second person area feature information extracted from 20 with the first person area feature information, the person to be tracked can be tracked by the second image 20.
[0051]
　That is, in the present embodiment, the same effect as that of the above embodiment can be obtained, and a person registered in advance in the face feature information database 110 can be tracked as a tracking target person.
[0052]
(Third Embodiment)
　The information processing apparatus 100 of the third embodiment also performs face recognition processing on the second image 20, and then refers to a human area connected to a face area where face recognition could not be performed. It is the same as the first embodiment except that the collation process with the first person area feature information is performed. Although the operation of the third embodiment is different from that of the first embodiment, the configuration is the same as that of the information processing apparatus 100 of FIG. 2, so the functional block diagram of FIG. 2 is used in the following description. The present embodiment may have the same configuration as the information processing apparatus 100 of the other embodiments, and can be combined within a consistent range.
[0053]
　FIG. 7 is a flowchart showing an example of the operation of the information processing apparatus 100 of the present embodiment. The present embodiment has the same steps S101 to S109 as in FIG. 4, and further includes step S120.
[0054]
　In the present embodiment, when the second face feature information can be extracted from the person included in the collation unit 104 and the second image 20 (YES in step S105), the first face feature information and the second face feature information are collated. (Step S120). If the second facial feature information cannot be extracted from the person included in the second image 20 (NO in step S105), the first person area feature information extracted in step S103 and the second person extracted in step S107 The area feature information is collated (step S109).
[0055]
　In the present embodiment, when all the persons included in the first image 10 are set as the tracking target persons, among the persons included in the second image 20 in step S105, the person whose second facial feature information can be extracted is ( YES in step S105), the person is tracked by collating with the face feature information, and the person whose second face feature information could not be extracted (NO in step S105) is collated with the human area feature information. You may track the person with. In other words, when the face is visible, it can be tracked by the face, and when the face is not visible, it can be tracked by the clothes (human area).
[0056]
　In the present embodiment, the information processing apparatus 100 performs tracking processing using the human area feature information when a face matching the first face information is not found in the second image 20. Further, when the face is found, the information processing apparatus 100 does not perform the tracking process using the human area feature information, and can continue the tracking with the face of the person. As described above, according to the present embodiment, the same effect as that of the first and second embodiments is obtained, and further, when face recognition cannot be performed such as when a face is not shown, collation processing using human area feature information is performed. It is efficient because it does.
[0057]
(Fourth Embodiment)
　FIG. 8 is a functional block diagram showing a logical configuration of the information processing apparatus 100 of the present embodiment. The information processing apparatus 100 of the present embodiment has a configuration of outputting the extraction result by the extraction unit 102, the human area feature information extracted by the extraction unit 102, the tracking information of the person by the collation unit 104, and the like. Is the same as.
[0058]
　In addition to the configuration shown in FIG. 2, the information processing device 100 further includes a first registration unit 120 and an output processing unit 130. The present embodiment may have the same configuration as the information processing apparatus 100 of the other embodiments, and can be combined within a consistent range. The first registration unit 120 stores the first face feature information extracted from the person included in the first image 10 and the first person area feature information in the person area information storage unit 112 in association with the person identification information. .. In other words, the first registration unit 120 associates the first face feature information and the first person area feature information extracted from the person included in the first image 10 with the person identification information, and the person area information storage unit 112. Register with. When the collation results by the collation unit 104 match, the output processing unit 130 causes the output means to output the identification information of the person stored in the person area information storage unit 112. Hereinafter, the first face feature information and the first person area feature information registered in the person area information storage unit 112 are also referred to as registered face feature information and registered person area feature information.
[0059]
　The output means is, for example, a display device for displaying a screen, a printing device for printing and outputting, a communication device for transmitting an e-mail, and the like. In the present embodiment, the output processing unit 130 displays various information on a display device 132 such as a liquid crystal display or an organic EL (ElectroLuminescence) display connected to the information processing device 100.
[0060]
　FIG. 9 is a diagram showing an example of the data structure of the human area feature information storage unit 112.
　In the example of FIG. 9A, the human area feature information storage unit 112 has a human area information ID that identifies the human area feature information and the human area feature information extracted by the extraction unit 102 (in the first person area feature information). Equivalent) is linked.
[0061]
　Further, in the examples of FIGS. 9 (b) and 9 (c), the face of a person from which the first face feature information matching the first face information of the face feature information database 110 is extracted in the second embodiment. The first person area feature information extracted from the person area other than the area is stored in the person area feature information storage unit 112 as the registered person area feature information.
　In the example of FIG. 9B, the human area information ID, the first person area feature information, and the registrant ID of the first face information that matches the first face feature information extracted from the first image 10 are It is associated. By associating with the registrant ID, the name and face image of the person corresponding to the registrant ID can be read out from the face feature information database 110 of FIG.
[0062]
　In the example of FIG. 9C, instead of the registrant ID, at least one of the registrant's name and face image data of the first face information that matches the first face feature information extracted from the first image 10 is used. It is tied.
[0063]
　In the present embodiment, the extraction unit 102 acquires the second image 20 from the plurality of cameras 5, and performs an extraction process on the acquired second image 20.
[0064]
　FIG. 10 is a diagram showing an example of a screen displayed on the display device 132 by the output processing unit 130. FIG. 10A is a multi-screen 200 that displays images captured by a plurality of cameras 5 side by side on one screen. FIG. 10B is a view screen 210 for viewing an operation image of one camera 5 in FIG. 10A. In the example of the view screen 210 of FIG. 10B, the human area of ​​the tracking target person extracted by the extraction unit 102 is surrounded by a rectangular frame 212. Further, the name 214 of the person specified by the collating unit 104 is displayed.
[0065]
　The multi-screen 200 and the view screen 210 may be a moving image captured by the camera 5 in real time, or may be at least one frame in which the moving image is stopped at a certain point in time. Further, also on the multi-screen 200, a frame 212 or a name 214 indicating the detected tracked person may be displayed as in the view screen 210. The multi-screen 200 and the view screen 210 may be displayed in separate windows, or may be switched and displayed. Further, a plurality of view screens 210 may be displayed at the same time.
[0066]
　Further, information on the imaging date and time of the displayed image, information on the shooting location, and the like may be displayed on the multi-screen 200 and the view screen 210. Alternatively, the property information of the image displayed on the multi-screen 200 and the view screen 210 may be displayed in another window according to the operation of the operator.
[0067]
　Further, it may be possible to receive an operator's operation on the area of ​​the frame 212 and display information on the person concerned, for example, information such as a face photograph and a profile.
[0068]
　According to the present embodiment, since the output processing unit 130 can display the multi-screen 200 or the view screen 210, the same effect as that of the above-described embodiment can be obtained, and the tracking status can be grasped in real time.
[0069]
(Fifth Embodiment)
　FIG. 11 is a functional block diagram showing a logical configuration of the information processing apparatus 100 of the present embodiment. The information processing device 100 of the present embodiment is the same as the above-described embodiment except that it has a configuration in which information of a person who is not registered in the face feature information database 110 is registered as unregistered person information.
[0070]
　The information processing device 100 further includes a second registration unit 140 in addition to the configuration shown in FIG. The present embodiment may have the same configuration as the information processing apparatus 100 of the other embodiments, and can be combined within a consistent range. When the matching result of the first face feature information extracted from the face area of ​​the person included in the first image 10 and the first face information stored in the face feature information database 110 does not match, the extraction unit 102 does not match. The third person area feature information corresponding to the person area connected to the face area is extracted. The second registration unit 140 stores the extracted third person area feature information and the first face feature information in the unregistered person information storage unit 142 in association with each other. In other words, the second registration unit 140 registers the extracted third person area feature information and the first face feature information in the unregistered person information storage unit 142 in association with each other. The unregistered person information storage unit 142 may have a database structure.
[0071]
　The unregistered person information storage unit 142 may be included in the same device as at least one of the face feature information database 110 and the human area feature information storage unit 112, or may be included in different devices. Further, the unregistered person information storage unit 142 may be integrated with the main body of the information processing device 100 or may be a separate body together with the face feature information database 110 and the human area feature information storage unit 112.
[0072]
　FIG. 12 is a diagram showing an example of the data structure of the unregistered person information storage unit 142. In this example, the unregistered person ID that identifies the unregistered person is included in the first face feature information extracted from the face area of ​​the person included in the first image 10 whose collation result with the first face information does not match. Has been granted. Then, the first face feature information and the third person area feature information are associated with the unregistered person ID and registered in the unregistered person information storage unit 142. Further, at least one of the image data of the face region of the person included in the first image 10 and the image data of the human region of the third person region feature information may be registered in the unregistered person information storage unit 142. ..
[0073]
　Also in the present embodiment, in the example of the view screen 210 shown in FIG. 10B in the above embodiment, the extracted unregistered person is surrounded by a rectangular frame 212 and the person area or face area of ​​the person is surrounded by a rectangular frame 212. Information indicating that the person is an unregistered person may be superimposed and displayed on the image in the vicinity of the frame 212. Further, the unregistered person ID may be displayed. The registrant and unregistered frame 212 may be displayed separately by changing the line color and line type (for example, solid line, broken line, etc.) so that they can be distinguished. Further, the frame 212 is not always necessary.
[0074]
　FIG. 13 is a flowchart showing an example of the operation of the information processing apparatus 100 of the present embodiment. When the same process as in step S103 of FIG. 4 is performed and the face area cannot be detected (NO in step S103), the process proceeds to this process. The second registration unit 140 identifies a person area connected to the second face area determined by the extraction unit 102 to not match the first face information (step S141), and obtains the second person area feature information corresponding to the person area. Generate (step S143). Then, the second registration unit 140 registers the generated second person area feature information and the second face area in the unregistered person information storage unit 142 in association with each other (step S145).
[0075]
　As described above, in the present embodiment, the person whose face cannot be authenticated in the second image 20 can be registered as an unregistered person, and the face image and the human area feature information can be registered in the unregistered person information storage unit 142. Thereby, for example, when a non-member enters a facility used by a pre-registered member, the non-member information can be acquired and registered.
[0076]
　FIG. 14 is a diagram showing an example of the configuration of a computer 80 that realizes the information processing apparatus 100 of each of the above-described embodiments.
　The computer 80 stores a program 90 that realizes the components of the information processing devices 100 of FIGS. 2, 9 and 11 loaded in the CPU (Central Processing Unit) 82, the memory 84, and the memory 84, and the program 90. It includes a storage 85, an I / O (Input / Output) 86, and a network connection interface (communication I / F87).
[0077]
　The CPU 82, the memory 84, the storage 85, the I / O 86, and the communication I / F 87 are connected to each other via the bus 89, and the entire information processing apparatus is controlled by the CPU 82. However, the method of connecting the CPU 82 and the like to each other is not limited to the bus connection.
[0078]
　The memory 84 is a memory such as a RAM (Random Access Memory) or a ROM (Read Only Memory). The storage 85 is a storage device such as a hard disk, an SSD (Solid State Drive), or a memory card.
[0079]
　The storage 85 may be a memory such as a RAM or a ROM. The storage 85 may be provided inside the computer 80, or may be provided outside the computer 80 and connected to the computer 80 by wire or wirelessly if the computer 80 is accessible. Alternatively, it may be detachably provided on the computer 80.
[0080]
　When the CPU 82 reads the program 90 stored in the storage 85 into the memory 84 and executes it, each function of each unit of the information processing apparatus 100 of each embodiment can be realized.
[0081]
　The I / O 86 controls input / output of data and control signals between the computer 80 and other input / output devices. Other input / output devices include, for example, an input device (not shown) such as a keyboard, touch panel, mouse, and microphone connected to the computer 80, and an output device (not shown) such as a display, a printer, and a speaker. These input / output devices and the interface of the computer 80 are included. Further, the I / O 86 may perform data input / output control with a reading or writing device (not shown) of another recording medium.
[0082]
　The communication I / F 87 is a network connection interface for communicating between the computer 80 and an external device. The communication I / F 87 may be a network interface for connecting to a wired line or a network interface for connecting to a wireless line. For example, the computer 80 that realizes the information processing device 100 is connected to at least one camera 5 via a network by communication I / F 87.
[0083]
　Each component of the information processing apparatus 100 of each embodiment is realized by an arbitrary combination of hardware and software of the computer 80 of the present figure. And, it is understood by those skilled in the art that there are various modifications of the realization method and the device. The functional block diagram showing the information processing apparatus 100 of each of the above-described embodiments shows blocks of logical functional units, not configurations of hardware units.
[0084]
　The information processing device 100 may be configured by a plurality of computers 80, or may be realized by a virtual server.
[0085]
　The present invention can be realized as an information processing system illustrated below as an example composed of a plurality of computers 80, but the present invention is not limited thereto.
(1) The extraction unit 102 and the collation unit 104 are carried out by different devices (computer 80). For example, the information processing system may include a terminal device having an extraction unit 102 and a server device having a collation unit 104.
(2) Among the extraction processes of the extraction unit 102, the extraction process of the feature information from the first image 10 and the extraction process of the feature information from the second image 20 are performed by another device (computer 80). For example, the information processing system may include a plurality of terminal devices having the extraction unit 102 for analyzing the video, and a server device for integrating the information from them and performing the collation process. A plurality of terminal devices may be arranged separately for each area, and the extraction unit 102 may be realized by a plurality of physical devices.
(3) The device (computer 80) may be divided according to the type of feature information (face feature information and human area feature information) that is the target of video analysis (extraction unit 102 and collation unit 104).
[0086]
　Although the embodiments of the present invention have been described above with reference to the drawings, these are examples of the present invention, and various configurations other than the above can be adopted.
　For example, in the above embodiment, the extraction unit 102 performs the tracking person search process using one person area feature information registered in the person area feature information registration process, but in another example, the extraction unit 102 performs the tracking person search process. , A plurality of first person area feature information is generated from each of the plurality of first images 10, and these first person area feature information are associated with each other and registered in the person area feature information storage unit 112 as registered person area feature information. You may.
[0087]
　Then, the collation unit 104 can perform collation processing on each of at least one registrant area feature information or a plurality of registrant area feature information. The operator may specify which of the plurality of registrant area feature information is to be used, or the operator may specify to use all the registrant area feature information.
[0088]
　FIG. 15 is a diagram showing an example of the data structure of the human area feature information storage unit 112. In the example of FIG. 15, the registrant ID and a plurality of human area feature information n (n is a natural number) are associated and registered with the person area information ID.
[0089]
　As described above, since the human area characteristic information is mainly based on the characteristics of the person's clothes and the like, the tracking is premised on the person not changing clothes. However, even if the person changes clothes, it can be tracked by using different human domain characteristic information corresponding to different clothes.
[0090]
　Further, the extraction unit 102 may store an image including a human area corresponding to the generated first person area feature information in the person area feature information storage unit 112 in association with the first person area feature information. FIG. 16 is a schematic diagram for explaining an example of the data structure of the human area feature information storage unit 112.
[0091]
　In the example of FIG. 16A, the human area feature information storage unit 112 includes, for one person, the face image data of the person, the name, and one person area feature information (not shown). The image data of the human area corresponding to one human area feature information is registered. In the example of FIG. 16B, the human area feature information storage unit 112 contains, for one person, the face image data of the person, the name, and a plurality of human area feature information (not shown). A plurality of image data of the person area corresponding to the person area feature information are registered.
[0092]
　In the above embodiment, the information processing apparatus 100 performs the collation process using the face feature information in step S105 of FIG. 5 or FIG. 8 with the collation process using the human area feature information in step S109 of FIG. 5 or FIG. Although it is given higher priority, either of the face feature information matching process and the human area feature information matching process may be given priority. Which process should be prioritized may be changed by accepting the operator's instruction.
[0093]
　For example, in the information processing device 100, for a person who is likely to act by hiding his / her face intentionally, the first person area feature information and the second image extracted from the person area of ​​the person included in the first image 10. The collation process with the second person area feature information extracted from the acquisition area of ​​the person included in 20 is performed first, and then the collation process using the face feature information is performed independently of the result of the collation of the person area. You may. Further, in the situation where there are many people in similar clothes such as uniforms, the information processing device 100 performs face authentication first, and collation processing using human area feature information only for the person who could not be face-authenticated. May be done. Further, the information processing apparatus 100 may accept a designation as to which process is prioritized, and may change the processing order according to this designation. Further, the information processing apparatus 100 may automatically switch so as to give priority to the face recognition process when the similarity of the human area feature information in the second image 20 exceeds the reference value.
[0094]
　Further, in another embodiment, the information processing apparatus 100 uses the detection of a predetermined event as a trigger to detect at least one of the face region detection, the human region feature information extraction, and the person tracking based on the human region feature information. May be carried out. A predetermined event may be a predetermined action related to a person, that is, an action to be watched, such as being left behind, wandering around, wandering around, swinging an object, or invading a predetermined area. The predetermined event may be detected from the image captured by the surveillance camera by the image recognition process of the information processing device 100, or may be detected by another information processing device or sensor. In this case, the extraction unit 102 may identify a person in which a predetermined event is detected, and may extract the face area feature information and the person area feature information with the person as the tracking target person.
[0095]
　Although the present invention has been described above with reference to the embodiments and examples, the present invention is not limited to the above embodiments and examples. Various changes that can be understood by those skilled in the art can be made within the scope of the present invention in terms of the structure and details of the present invention.
　In addition, when information about a user is acquired and used in this invention, this shall be done legally.
[0096]
　Some or all of the above embodiments may also be described, but not limited to:
1. 1. Extraction means for extracting the first facial feature information from the face region of the person included in the first image and extracting the first human region feature information from the region including the region other than the face of the person, and
　two or more feature information. The
　extraction means includes a collation means for collating, and when the second face feature information cannot be extracted from the face area of ​​the person included in the second image, the extraction means other than the face of the person included in the second image. An
　information processing device that extracts a second human area feature information from a region including the area, and the matching means collates the first human area feature information with the second human area feature information.
2. 1. 1. In the information processing device according to the
　above, the second image is an information processing device generated by an imaging unit different from that of the first image.
3. 3. 1. 1. Or 2. In the information processing device according to the
　above, the second image is an information processing device that is an image captured after the first image.
4. 1. 1. From 3. In the information processing apparatus according to any one of the above, if the
　collation means can extract the second facial feature information from the person included in the second image, the first facial feature information and the first face feature information. When the second face feature information cannot be extracted from the person included in the second image by collating with the second face feature information, the first person area feature information and the second person area feature information An information processing device that collates with.
5. 1. 1. From 4. In the information processing apparatus described in any one of
　Extraction processing of the first face feature information, the second face feature information, the first human area feature information, and the second human area feature information by the extraction means, and the person by the collation means. An information processing device in which at least one of the area feature information collation processes is performed triggered by the detection of a predetermined event.
6. 1. 1. From 5. In the information processing apparatus according to any one of the above, the
　extracted first person area feature information is associated with an image including the person area corresponding to the first person area feature information, and the registered person area feature information. The
　collating means is an information processing device that collates the registered person area feature information stored in the storage unit with the second person area feature information.
7. 1. 1. From 6. The information processing apparatus according to any one of,
　using a plurality of the first image to extract a plurality of the first human region feature information, the registrant region said plurality of first human region feature information Further, a registration means for associating with each other as feature information and storing it in a storage unit is further provided, and the
　collation means collates at least one registrant area feature information stored in the storage unit with the second person area feature information. Information processing device.
8. 1. 1. From 7. In the information processing apparatus according to any one of the
　above, the collation means uses the first face feature information extracted from the face region of the person included in the first image and the first face stored in the storage unit. An information processing device that collates information and, when the collation results match, extracts the first person area feature information from the person included in the first image.
9. 8. In the information processing apparatus according to the
　above, the extraction means comprises the first face feature information extracted from the face region of a person included in the first image and the first face information stored in the storage unit. When the collation results are inconsistent, the third person area feature information is extracted from the person area including the person's face area other than the face area, and the
　extracted third person area feature information and the first face feature information are extracted. An information processing device further comprising a registration means for associating the information with each other and storing the information in a second storage unit.
10. 9. In the information processing apparatus according to the
　above, the registration means associates an image of the face region from which the first face feature information is extracted with the first face feature information and the third human region feature information. An information processing device that stores information in a second storage unit.
11. 1. 1. To 10. In the information processing apparatus according to any one of
　the above, the first facial feature information extracted from the person included in the first image and the first human area feature information correspond to the identification information of the person.
　Information further comprising a registration means to be attached and stored in the storage unit, and an output processing means for outputting the identification information of the person stored in the storage unit to the output unit when the collation results by the collation means match. Processing equipment.
12. 1. 1. From 11. In the information processing apparatus according to any one of the above, the information processing apparatus is
　connected to a plurality of imaging units via a network, and the
　extraction means acquires the second image from the plurality of imaging units and obtains the second image. An information processing device that performs extraction processing.
[0097]
13. Extraction means for extracting the first facial feature information from the face region of the person included in the first image and extracting the first human region feature information from the region including the region other than the face of the person, and
　two or more feature information. The
　extraction means includes a collation means for collating, and when the second face feature information cannot be extracted from the face area of ​​the person included in the second image, the extraction means other than the face of the person included in the second image. An
　information
　processing system that extracts a second human area feature information from a region including the area, and the matching means collates the first human area feature information with the second human area feature information .
14. 13. In the information processing system according to the
　above, the second image is an information processing system generated by an imaging unit different from that of the first image.
15. 13. Or 14. In the information processing system according to the
　above, the second image is an information processing system that is an image captured after the first image.
16. 13. To 15. In the information processing system according to any one of the above, if the
　collation means can extract the second facial feature information from the person included in the second image, the first facial feature information and the first face feature information. When the second face feature information cannot be extracted from the person included in the second image by collating with the second face feature information, the first person area feature information and the second person area feature information An information processing system that collates with.
17. 13. From 16. In the information processing system described in any one of
　The first face feature information, the second face feature information, the first human area feature information, and the second human area feature information extraction process by the extraction means, and the person by the collation means. An information processing system in which at least one of the area feature information collation processes is executed by detecting a predetermined event as a trigger.
18. 13. From 17. In the information processing system according to any one of the above, the
　extracted first person area feature information is associated with an image including the person area corresponding to the first person area feature information, and the registered person area feature information. The
　collating means is an information processing system that collates the registered person area feature information stored in the storage unit with the second person area feature information.
19. 13. To 18. In the information processing system according to any one of the above,
　a plurality of the first human area feature information is extracted using the plurality of the first images, and the plurality of first human area feature information is stored in the registered person area. Further, a registration means for associating with each other as feature information and storing it in a storage unit is further provided, and the
　collation means collates at least one registrant area feature information stored in the storage unit with the second person area feature information. Information processing system.
20. 13. From 19. In the information processing system according to any one of the
　above, the collation means uses the first face feature information extracted from the face region of the person included in the first image and the first face stored in the storage unit. An information processing system that collates information and extracts the first person area feature information from the person included in the first image when the collation results match.
21. 20. In the information processing system according to the
　above, the extraction means comprises the first face feature information extracted from the face region of a person included in the first image and the first face information stored in the storage unit. When the collation results are inconsistent, the third human area feature information is extracted from the person area including the person's face area other than the face area, and the
　extracted third person area feature information and the first face feature information are extracted. An information processing system further comprising a registration means for associating the information with each other and storing the information in a second storage unit.
22. 21. In the information processing system according to the
　above, the registration means associates an image of the face region from which the first face feature information is extracted with the first face feature information and the third human region feature information. An information processing system that stores information in a second storage unit.
23. 13. From 22. In the information processing system according to any one of
　the above, the first facial feature information extracted from the person included in the first image and the first human area feature information correspond to the identification information of the person.
　Information further comprising a registration means to be attached and stored in the storage unit, and an output processing means for outputting the identification information of the person stored in the storage unit to the output unit when the collation results by the collation means match. Processing system.
24. 13. From 23. In the information processing system according to any one of the above, the information processing system is
　connected to a plurality of imaging units via a network, and the
　extraction means acquires the second image from the plurality of imaging units and obtains the second image. An information processing system that performs extraction processing.
[0098]
25. The information processing device
　extracts the first facial feature information from the face region of the person included in the first image, extracts the first human region feature information from the region including the region other than the face of the person, and has
　two or more features. When the information is collated and
　the second facial feature information cannot be extracted from the face area of ​​the person included in the second image , the second human area feature is extracted from the area including other than the face of the person included in the second image. An information processing method for extracting
　information and collating the first human domain feature information with the second human domain feature information
.
26. 25. In the information processing method according to the
　above, the second image is an information processing method generated by an imaging unit different from that of the first image.
27. 25. Or 26. In the information processing method according to the
　above, the second image is an image captured after the first image.
28. 25. From 27. In the information processing method according to any one of the above, when the information
　processing apparatus
　　can extract the second facial feature information from the person included in the second image, the first facial feature information and the said When the second facial feature information is collated and the second facial feature information cannot be extracted from the person included in the second image, the first human domain feature information and the second human domain feature are collated. An information processing method that collates information.
29. 25. From 28. In the information processing method described in any one of
　The information processing device
　extracts the first face feature information, the second face feature information, the first human domain feature information, and the second human domain feature information, and the human domain feature. An information processing method in which at least one of information collation processing is performed by detecting a predetermined event as a trigger.
30. 25. From 29. In the information processing method according to any one of the above, the information
　processing apparatus further uses the
　extracted first human area feature information as an image including the human area corresponding to the first human area feature information.
　An information processing method in which the information is stored in a storage unit as registrant area feature information in association with the above, and the registrant area feature information stored in the storage unit is collated with the second person area feature information.
31. 25. From 30. In the information processing method according to any one of the above, the information
　processing apparatus further
　extracts a plurality of the first human region feature information using the plurality of the first images, and the plurality of first human region feature information. Information that associates human area feature information with each other as registered person area feature information
　and stores it in a storage unit, and collates at least one registered person area feature information stored in the storage unit with the second human area feature information. Processing method.
32. 25. From 31. In the information processing method according to any one of the above, the information
　processing apparatus
　The first facial feature information extracted from the face region of the person included in the first image is collated with the first face information stored in the storage unit, and when the collation results are in agreement, the first An information processing method for extracting the first person area feature information from the person included in one image.
33. 32. In the information processing method according to the
　above, the information processing apparatus
　includes the first face feature information extracted from the face region of a person included in the first image and the first face information stored in the storage unit. When the collation results of the above are inconsistent, the third person area feature information is extracted from the person area including the person's face area other than the face area of ​​the person, and
　further, the extracted third person area feature information and the first person area feature information are extracted . An information processing method in which facial feature information is associated with each other and stored in a second storage unit.
34. 33. In the information processing method according to the
　above, the information processing apparatus associates
　the image of the face region from which the first face feature information is extracted with the first face feature information and the third human region feature information. An information processing method for storing in the second storage unit.
35. 25. From 34. In the information processing method according to any one of the above, the information
　processing apparatus further
　comprises the first facial feature information and the first human area feature information extracted from a person included in the first image.
　Is stored in the storage unit in association with the identification information of the person, and when the collation results match, the information processing method in which the identification information of the person stored in the storage unit is output to the output unit.
36. 25. From 35. In the information processing method according to any one of the above, the information 　processing apparatus is
　connected to a plurality of imaging units via a network, and the
information processing apparatus
　acquires the second image from the plurality of imaging units, and the acquired second image. An information processing method that performs extraction processing on an image.
[0099]
37.
　A procedure for extracting the first facial feature information from the face area of ​​a person included in the first image to a computer and extracting the first human area feature information from an area including a region other than the face of the person,
　two or more feature information. When
　the second facial feature information could not be extracted from the face area of ​​the person included in the second image, the second human area feature from the area including the face other than the face of the person included in the second image. A program for executing a procedure for extracting
　information and a procedure for collating the first human area feature information with the second human area feature information
.
38. 37. In the program described in the
　above, the second image is a program generated by an imaging unit different from the first image.
39. 37. Or 38. In the program described in the
　above, the second image is an image captured after the first image.
40. 37. From 39. In the program described in any one of the
　above, when the second facial feature information can be extracted from the person included in the second image, the first facial feature information and the second facial feature information are obtained. If the second facial feature information cannot be extracted from the person included in the second image, the computer performs a procedure for collating the first human area feature information with the second human area feature information. A program to run on.
41. 37. From 40. In the program described in any one of
　Of the first face feature information, the second face feature information, the first human domain feature information, and the second human domain feature information extraction process, and the human domain feature information collation process. A program for causing a computer to execute at least one of the procedures triggered by the detection of a predetermined event.
42. 37. From 41. In the program described in any one of the above, the
　extracted first person area feature information is stored as registered person area feature information in association with an image including the person area corresponding to the first person area feature information.
　A program for causing a computer to perform a procedure of storing in a unit and a procedure of collating the registered person area feature information stored in the storage unit with the second person area feature information.
43. 37. From 42. In the program described in any one of the above, a
　plurality of the first human area feature information is extracted using the plurality of the first images, and the plurality of first human area feature information is registered as the registered person area feature information.
　A program for causing a computer to execute a procedure of associating with each other and storing the information in a storage unit, and a procedure of collating at least one registrant area feature information stored in the storage unit with the second person area feature information.
44. 37. From 43. In the program described in any one of
　the above, the first face feature information extracted from the face area of ​​the person included in the first image is collated with the first face information stored in the storage unit, and the said A program for causing a computer to execute a procedure of extracting the first person area feature information from the person included in the first image when the collation results are in agreement.
45. 44. In the program described in the
　above, the collation result of the first face feature information extracted from the face area of ​​the person included in the first image and the first face information stored in the storage unit is inconsistent. In the case, a procedure for extracting a third human area feature information from a person area including a person other than the face area of ​​the person, the
　extracted third person area feature information, and the first face feature information in association with each other are second. A program that allows a computer to execute the procedure of storing information in the storage unit of.
46. 45. In the program described in the
　above, the image of the face region from which the first face feature information is extracted is stored in the second storage unit in association with the first face feature information and the third human region feature information. A program that lets a computer perform the steps that make it happen.
47. 37. From 46. In the program described in any one of
　the above, the first facial feature information extracted from the person included in the first image and the first human area feature information are associated with the identification information of the person.
　A program for causing a computer to execute a procedure for storing in a storage unit and a procedure for outputting the identification information of the person stored in the storage unit to an output unit when the collation results match.
48. 37. From 47. In the program described in any one of the above 　, a procedure of
　connecting to a
plurality of imaging units via a network , acquiring the second image from the plurality of imaging units, and performing an extraction process on the acquired second image. A program that lets a computer run.
The scope of the claims
[Claim 1]
　Extraction means for extracting the first facial feature information from the face region of the person included in the first image and extracting the first human region feature information from the region including the region other than the face of the person, and
　two or more feature information. The
　extraction means includes a collation means for collating, and when the second face feature information cannot be extracted from the face area of ​​the person included in the second image, the extraction means other than the face of the person included in the second image. An
　information processing device that extracts a second human area feature information from a region including the area, and the matching means collates the first human area feature information with the second human area feature information.
[Claim 2]
　In the information processing device according to claim 1,
　the second image is an information processing device generated by an imaging unit different from the first image.
[Claim 3]
　In the information processing device according to claim 1 or 2,
　the second image is an information processing device that is an image captured after the first image.
[Claim 4]
　In the information processing apparatus according to any one of claims 1 to 3, if the
　collation means can extract the second facial feature information from the person included in the second image, the first face When the feature information is collated with the second face feature information and the second face feature information cannot be extracted from the person included in the second image, the first person area feature information and the second face feature information are checked. An information processing device that collates with the human area characteristic information of.
[Claim 5]
　In the information processing apparatus according to any one of claims 1 to 4,
　the first face feature information, the second face feature information, the first human area feature information, and the first person area feature information by the extraction means. At least one of the extraction process of the human area feature information of 2 and the collation process of the human area feature information by the collation means is performed by the detection of a predetermined event as a trigger, the information processing apparatus. ..
[Claim 6]
　In the information processing apparatus according to any one of claims 1 to 5, the
　extracted first human area feature information is associated with an image including the human area corresponding to the first human area feature information. Further provided with a registration means to be stored in the storage unit as the registrant area feature information, the
　collation means information processing for collating the registrant area feature information stored in the storage unit with the second person area feature information. Device.
[Claim 7]
　In the information processing apparatus according to any one of claims 1 to 6,
　a plurality of the first human domain feature information is extracted using the plurality of the first images, and the plurality of first human domain features are extracted. The
　collating means further includes a registration means for storing information as registrant area feature information in association with each other in a storage unit, and the collation means includes at least one registrant area feature information stored in the storage unit and the second person area. An information processing device that collates with feature information.
[Claim 8]
　In the information processing apparatus according to any one of claims 1 to 7, the
　collation means stores the first facial feature information extracted from the face region of a person included in the first image and a storage unit. An information processing device that collates the first face information and extracts the first person area feature information from the person included in the first image when the collation results match.
[Claim 9]
　In the information processing apparatus according to claim 8, the
　extraction means uses the first face feature information extracted from the face region of a person included in the first image and the first face stored in the storage unit. When the collation result with the information does not match, the third person area feature information is extracted from the person area including the person's face area other than the face area, and the
　extracted third person area feature information and the first person area feature information are extracted . An information processing device further comprising a registration means for associating facial feature information with each other and storing it in a second storage unit.
[Claim 10]
　In the information processing apparatus according to claim 9, the
　registration means uses the image of the face region from which the first face feature information is extracted as the first face feature information and the third human region feature information. An information processing device that is associated and stored in the second storage unit.
[Claim 11]
　In the information processing apparatus according to any one of claims 1 to 10,
　the first face feature information and the first human area feature information extracted from the person included in the first image are combined with the person. A registration means that is associated with the identification information of
　the above and stored in the storage unit, and an output processing means that outputs the identification information of the person stored in the storage unit to the output unit when the collation results by the collation means match. An information processing device further equipped with.
[Claim 12]
　In the information processing apparatus according to any one of claims 1 to 11, the information processing apparatus is
　connected to a plurality of imaging units via a network, and the
　extraction means acquires the second image from the plurality of imaging units and acquires the second image. An information processing device that performs extraction processing on the second image.
[Claim 13]
　Extraction means for extracting the first facial feature information from the face region of the person included in the first image and extracting the first human region feature information from the region including other than the face of the person, and
　two or more feature information. The
　extraction means includes a collation means for collating, and when the second face feature information cannot be extracted from the face area of ​​the person included in the second image, the extraction means other than the face of the person included in the second image. An
　information
　processing system that extracts a second human area feature information from a region including the area, and the matching means collates the first human area feature information with the second human area feature information .
[Claim 14]
　In the information processing system according to claim 13,
　the second image is an information processing system generated by an imaging unit different from that of the first image.
[Claim 15]
　In the information processing system according to claim 13 or 14,
　the second image is an information processing system that is an image captured after the first image.
[Claim 16]
　In the information processing system according to any one of claims 13 to 15, if the
　collation means can extract the second facial feature information from the person included in the second image, the first face When the feature information is collated with the second face feature information and the second face feature information cannot be extracted from the person included in the second image, the first person area feature information and the second face feature information are checked. An information processing system that collates with the human domain characteristic information of.
[Claim 17]
　In the information processing system according to any one of claims 13 to 16,
　the first face feature information, the second face feature information, the first human area feature information, and the first person area feature information by the extraction means. An information processing system in which at least one of the extraction process of the human domain feature information and the collation process of the human domain feature information by the collation means is performed triggered by the detection of a predetermined event. ..
[Claim 18]
　In the information processing system according to any one of claims 13 to 17, the
　extracted first human area feature information is associated with an image including the human area corresponding to the first human area feature information. Further provided with a registration means to be stored in the storage unit as the registrant area feature information, the
　collation means is information processing for collating the registrant area feature information stored in the storage unit with the second person area feature information. system.
[Claim 19]
　In the information processing system according to any one of claims 13 to 18,
　a plurality of the first human domain feature information is extracted using the plurality of the first images, and the plurality of first human domain features are extracted. The
　collating means further includes a registration means for storing information as registrant area feature information in association with each other in a storage unit, and the collation means includes at least one registrant area feature information stored in the storage unit and the second person area. An information processing system that collates with feature information.
[Claim 20]
　In the information processing system according to any one of claims 13 to 19, the
　collation means stores the first facial feature information extracted from the face region of a person included in the first image and a storage unit. An information processing system that collates with the first face information and extracts the first person area feature information from the person included in the first image when the collation results match.
[Claim 21]
　In the information processing system according to claim 20, the
　extraction means uses the first face feature information extracted from the face region of a person included in the first image and the first face stored in the storage unit. When the collation result with the information does not match, the third person area feature information is extracted from the person area including the person's face area other than the face area, and the
　extracted third person area feature information and the first person area feature information are extracted . An information processing system further including a registration means for associating facial feature information with each other and storing it in a second storage unit.
[Claim 22]
　In the information processing system according to claim 21, the
　registration means uses the image of the face region from which the first face feature information is extracted as the first face feature information and the third human region feature information. An information processing system that is associated and stored in the second storage unit.
[Claim 23]
　In the information processing system according to any one of claims 13 to 22,
　the first facial feature information and the first human area feature information extracted from the person included in the first image are obtained from the person. A registration means that is associated with the identification information of
　the above and stored in the storage unit, and an output processing means that outputs the identification information of the person stored in the storage unit to the output unit when the collation results by the collation means match. An information processing system further equipped with.
[Claim 24]
　In the information processing system according to any one of claims 13 to 23, the information processing system is
　connected to a plurality of imaging units via a network, and the
　extraction means acquires the second image from the plurality of imaging units and acquires the second image. An information processing system that performs extraction processing on the second image.
[Claim 25]
　The information processing device
　extracts the first facial feature information from the face region of the person included in the first image, extracts the first human region feature information from the region including other than the face of the person, and has
　two or more features. When the information is collated and
　the second facial feature information cannot be extracted from the face area of ​​the person included in the second image , the second human area feature is extracted from the area including other than the face of the person included in the second image. An information processing method for extracting
　information and collating the first human domain feature information with the second human domain feature information
.
[Claim 26]
　In the information processing method according to claim 25,
　the second image is an information processing method generated by an imaging unit different from the first image.
[Claim 27]
　The information processing method according to claim 25 or 26,
　wherein the second image is an image captured after the first image.
[Claim 28]
　In the information processing method according to any one of claims 25 to 27, if the information
　processing apparatus
　　can extract the second facial feature information from the person included in the second image, the first When the face feature information is collated with the second face feature information and the second face feature information cannot be extracted from the person included in the second image, the first person area feature information and the first person area feature information are collated. An information processing method that collates with 2 human domain characteristic information.
[Claim 29]
　In the information processing method according to any one of claims 25 to 28, the information
　processing apparatus uses
　the first face feature information, the second face feature information, the first human area feature information, and the information processing apparatus. An information processing method in which at least one of the second human area feature information extraction process and the human area feature information collation process is performed by detecting a predetermined event as a trigger.
[Claim 30]
　In the information processing method according to any one of claims 25 to 29, the information
　processing apparatus further uses the
　extracted first human area feature information to correspond to the first human area feature information.
　An information processing method in which the registered person area feature information is stored in a storage unit in association with an image including a human area, and the registered person area feature information stored in the storage unit is collated with the second human area feature information.
[Claim 31]
　In the information processing method according to any one of claims 25 to 30, the information
　processing apparatus further
　extracts a plurality of the first human region feature information using the plurality of the first images , and the information processing device is described. A plurality of first human area feature information is associated with each other as registered person area feature information and stored in a
　storage unit, and at least one of the registered person area feature information and the second human area feature information stored in the storage unit are stored. Information processing method to match with.
[Claim 32]
　In the information processing method according to any one of claims 25 to 31, the information
　processing apparatus is applied
　to the first facial feature information and a storage unit extracted from the face region of a person included in the first image. An information processing method in which the stored first face information is collated, and when the collation results are in agreement, the first person area feature information is extracted from the person included in the first image.
[Claim 33]
　In the information processing method according to claim 32, the information
　processing apparatus uses
　the first facial feature information extracted from the face region of a person included in the first image and the first stored in the storage unit. When the collation result with the face information does not match, the third person area feature information is extracted from the person area including the person other than the face area of ​​the person, and
　further, the extracted third person area feature information and the said third person area feature information. An information processing method in which the first facial feature information is associated with each other and stored in the second storage unit.
[Claim 34]
　In the information processing method according to claim 33, the information
　processing apparatus uses
　the image of the face region from which the first face feature information is extracted as the first face feature information and the third human region feature information. An information processing method in which the information is stored in the second storage unit in association with the above.
[Claim 35]
　In the information processing method according to any one of claims 25 to 34, the information
　processing apparatus further
　obtains the first facial feature information and the first facial feature information extracted from a person included in the first image. Information processing in which the human area feature information is stored in the storage unit in association with the identification information of the person, and when the
　collation results match, the identification information of the person stored in the storage unit is output to the output unit. Method.
[Claim 36]
　In the information processing method according to any one of claims 25 to 35, the information 　processing apparatus is
　connected to a plurality of image pickup units via a network, and the
information processing apparatus
　acquires the second image from the plurality of image pickup units. An information processing method for extracting the second image.
[Claim 37]

　A procedure for extracting the first facial feature information from the face area of ​​a person included 　in the first image to a computer and extracting the first human area feature information from an area including a region other than the face of the person,
　two or more feature information. When
　the second facial feature information could not be extracted from the face area of ​​the person included in the second image, the second human area feature from the area including the face other than the face of the person included in the second image. A program for executing a procedure for extracting
　information and a procedure for collating the first human area feature information with the second human area feature information
.
[Claim 38]
　In the program according to claim 37,
　the second image is a program generated by an imaging unit different from the first image.
[Claim 39]
　In the program according to claim 37 or 38,
　the second image is an image captured after the first image.
[Claim 40]
　In the program according to any one of claims 37 to 39, if
　the second facial feature information can be extracted from the person included in the second image, the first facial feature information and the second facial feature information can be extracted. When the face feature information is collated and the second face feature information cannot be extracted from the person included in the second image, the first human area feature information and the second human area feature information are displayed. A program that lets a computer perform matching steps.
[Claim 41]
　In the program according to any one of claims 37 to 40,
　the first face feature information, the second face feature information, the first human domain feature information, and the second human domain feature information. A program for causing a computer to execute a procedure of performing at least one of the extraction process and the collation process of the human area feature information by using the detection of a predetermined event as a trigger.
[Claim 42]
　In the program according to any one of claims 37 to 41,
　the registrant associates the extracted first person area feature information with an image including the person area corresponding to the first person area feature information.
　A program for causing a computer to execute a procedure of storing the area feature information in a storage unit and a procedure of collating the registrant area feature information stored in the storage unit with the second person area feature information.
[Claim 43]
　In the program according to any one of claims 37 to 42,
　a plurality of the first human area feature information is extracted using the plurality of the first images, and the plurality of first human area feature information is obtained. The
　computer executes a procedure of associating the registrant area feature information with each other and storing it in the storage unit, and a procedure of collating at least one registrant area feature information stored in the storage unit with the second person area feature information. Program to let you.
[Claim 44]
　In the program according to any one of claims 37 to 43,
　the first face feature information extracted from the face region of the person included in the first image and the first face information stored in the storage unit. A program for causing a computer to execute a procedure of extracting the first person area feature information from the person included in the first image when the matching results are matched.
[Claim 45]
　In the program according to claim 44,
　the collation result of the first face feature information extracted from the face area of ​​the person included in the first image and the first face information stored in the storage unit does not match. In the case of, the procedure for extracting the third human area feature information from the human area including the person's face area other than the face area, the
　extracted third human area feature information, and the first facial feature information are associated with each other. A program for causing a computer to execute a procedure for storing information in a second storage unit.
[Claim 46]
　In the program according to claim 45,
　the second storage is performed by associating the image of the face region from which the first face feature information is extracted with the first face feature information and the third human region feature information. A program that allows a computer to execute the procedure to be stored in the department.
[Claim 47]
　In the program according to any one of claims 37 to 46,
　the first facial feature information extracted from the person included in the first image and the first human area feature information are identified as the person.
　A program for causing a computer to execute a procedure of associating information with information and storing it in a storage unit, and a procedure of outputting the identification information of the person stored in the storage unit to an output unit when the collation results match.
[Claim 48]
　In the program according to any one of claims 37 to 47, 　the second image is acquired from
　the
plurality of imaging units by being connected to the plurality of imaging units via a network, and the acquired second image is obtained. A program that allows a computer to perform the extraction process.