Title of the invention: collation support device, collation support method, and computer-readable recording medium
Technical field
[0001]
The present invention relates to a collation support device and a collation support method that support collation, and further relates to a computer-readable recording medium that records a program for realizing these.
Background technology
[0002]
A collation device has been proposed that collates a target person's face image with a pre-registered face image and identifies the target person based on the collation result.
[0003]
For example, as a related technique, Patent Document 1 discloses an authentication system capable of performing authentication with high accuracy when authenticating the identity of a person. According to the certification system of Patent Document 1, a feature region corresponding to an individual feature region (hokuro, scratch, wrinkle) is automatically detected from an image of a person to be certified, and the feature amount of the detected feature region is recognized. Authenticate using the recognized feature quantity.
Prior art literature
Patent documents
[0004]
Patent Document 1: Japanese Unexamined Patent Publication No. 2007-304857
Outline of the invention
Problems to be solved by the invention
[0005]
However, since the authentication system of Patent Document 1 uses a pre-registered face image, if the orientation of the face of the registered face image and the orientation of the face of the person to be imaged are different, matching may not be possible. be. For example, when the face of the person to be imaged is not facing the front, the position of the individual feature portion to be seen changes due to the influence of the unevenness of the face, so that the face image facing the front must always be used.
[0006]
Therefore, when a face image that does not face the front is used, it becomes difficult to collate using the face image. Therefore, there is a desire to improve the collation accuracy even if a face image that does not face the front is used.
[0007]
An example of the object of the present invention is a collation support device, a collation support method, and a computer capable of collating by designating features visible on the skin surface of the face of a person to be collated according to the orientation of the face of the captured image. The purpose is to provide a readable recording medium.
Means to solve the problem
[0008]
In order to achieve the above object, the collation support device in one aspect of the present invention is
The skin of the person to be collated using the reference face development image display area for displaying the reference face development image generated based on the three-dimensional data of the reference head displayed on the screen of the display device. When a feature area that represents the features of the face of the person to be collated that can be seen on the surface is designated, a generation means for generating feature information about the feature area and a generation means.
A collation means for collating the feature information with the collation information associated with the collation face development image for each person registered in advance and the collation feature area.
Selection means to select a candidate person based on the collation result,
It is characterized by having.
[0009]
Further, in order to achieve the above object, the collation support method in one aspect of the present invention is:
(A) The reference face development image display area for displaying the reference face development image generated based on the three-dimensional data of the reference head displayed on the screen of the display device is used as a collation target. When a characteristic area representing the facial features of the person to be collated, which can be seen on the skin surface of the person, is specified, characteristic information regarding the characteristic area is generated.
(B) The feature information is collated with the collation information associated with the collation face development image for each person registered in advance and the collation feature area.
(C) Select a candidate person based on the collation result.
Characterized by that.
[0010]
Further, in order to achieve the above object, a computer-readable recording medium on which the program in one aspect of the present invention is recorded is
On the computer
(A) The reference face development image display area for displaying the reference face development image generated based on the three-dimensional data of the reference head displayed on the screen of the display device is used as a collation target. When a feature area representing the facial features of the person to be collated, which can be seen on the skin surface of the person, is specified, a step and a step of generating feature information about the feature area,
(B) A step of collating the feature information with the collation information associated with the collation face development image for each person registered in advance and the collation feature area.
(C) A step to select a candidate person based on the collation result, and
It is characterized by recording a program containing instructions to execute.
Effect of the invention
[0011]
As described above, according to the present invention, collation can be performed by designating the features visible on the skin surface of the face of the person to be collated according to the orientation of the face of the captured image.
A brief description of the drawing
[0012]
FIG. 1 is a diagram for explaining an example of a collation support device.
FIG. 2 is a diagram for explaining a user interface used for collation support.
FIG. 3 is a diagram for explaining an example of a system having a collation support device.
FIG. 4 is a diagram for explaining an example of displaying a candidate face image.
FIG. 5 is a diagram for explaining an example of the operation of the collation support device.
FIG. 6 is a diagram for explaining an example of a system having a collation support device.
FIG. 7 is a diagram for explaining a user interface used for collation support.
FIG. 8 is a diagram for explaining an example of a display in which a feature region on a reference face three-dimensional image is converted into a feature region on a reference face developed image face image.
FIG. 9 is a diagram for explaining an example of displaying a candidate face image.
FIG. 10 is a diagram for explaining an example of displaying a candidate face image.
FIG. 11 is a diagram for explaining an example of the operation of the collation support device.
FIG. 12 is a diagram for explaining an example of a computer that realizes a collation support device.
Embodiment for carrying out the invention
[0013]
(Embodiment)
Hereinafter, embodiments of the present invention will be described with reference to FIGS. 1 to 12.
[0014]
[Device configuration]
First, the configuration of the collation support device 10 in the present embodiment will be described with reference to FIGS. 1 and 2. FIG. 1 is a diagram for explaining an example of a collation support device. FIG. 2 is a diagram for explaining a user interface used for collation support.
[0015]
The collation support device 10 shown in FIG. 1 designates features that can be seen on the skin surface of the face of the person to be collated according to the orientation of the face of the captured image, and uses a feature area corresponding to the designated features. It is a device that supports collation. Further, as shown in FIG. 1, the collation support device 10 has a generation unit 11, a collation unit 12, and a selection unit 13.
[0016]
Of these, the generation unit 11 uses a reference face development image display area for displaying a reference face development image displayed on the screen of the display device based on the three-dimensional data of the reference head. , When a characteristic area representing the facial features of the person to be collated that can be seen on the skin surface of the person to be collated is designated, the feature information about the characteristic area is generated.
[0017]
The reference head is a head created by CG (Computer Graphics) based on the data of one or more heads measured or imaged in the past. Further, the reference head may be created based on the head of a specific person measured or imaged in the past.
[0018]
The reference face development image display area is, for example, an area for displaying the reference face development image 43 on the user interface 40 shown in FIG. The reference face development image 43 is a development image of a face projected in a cylindrical shape by, for example, performing a UV development process using the three-dimensional data of the reference head, which is stored in the storage device 21 in advance. However, the creation of the developed image of the face is not limited to the above-mentioned cylindrical projection. In the example of FIG. 2, the reference face development image 43 is displayed in the window 44. However, the configuration of the display screen is not limited to FIG.
[0019]
The characteristics of a person's face include, for example, parts that represent the characteristics of a person that can be seen on the surface of the skin, such as hokuro, freckles, tattoos, wrinkles, wrinkles, dents, scratches, blemishes, bumps, unevenness of the skin, and discolored parts of the skin. In the example of FIG. 2, since there is a skull 51 on the left cheek of the person captured in the collation image 41, this skull 51 is a feature.
[0020]
The feature area is a mark attached by the user on the standard face development image after the user recognizes the feature of the person's face, and is an area corresponding to the feature recognized by the user. In the example of FIG. 2, the region corresponding to the mark (x) on the reference face development image 43 is the feature region 52. In addition, a region having no feature on the face of a person may be used as a feature region.
[0021]
The feature information is, for example, texture information, position information, size information, shape information, feature type information indicating the type of feature, and the like regarding the designated feature area 52.
[0022]
The collation unit 12 collates the feature information with the collation information associated with the pre-registered collation face development image for each person and the collation feature area. The selection unit 13 selects a candidate person based on the collation result.
[0023]
The collation face development image is a face development image generated based on three-dimensional data of the head of each of a plurality of persons registered in advance. The collation feature area is a pre-registered feature area representing features visible on the skin surface of the head of each of a plurality of persons.
[0024]
As described above, in the present embodiment, the user observes the collation image 41 and the user designates the feature area 52 on the reference face development image 43, so that the person in the collation image 41 does not face the front. Even in this case, the designated feature area 52 can be used to provide collation support that leads to the identification of the person to be collated. The collation support is a process of selecting a person having a feature at the same position as the person to be collated by using the feature area 52 designated on the reference face development image 43.
[0025]
Further, by designating the feature area 52 using the reference face development image 43, the influence of the unevenness of the face due to the change in the face orientation can be reduced, so that even if the position for only the hokuro 51 changes, the user can use it. , The feature area 52 corresponding to the hokuro 51 can be easily designated on the reference face development image 43.
[0026]
[System configuration]
Subsequently, the configuration of the collation support device 10 in the present embodiment will be described more specifically with reference to FIG. FIG. 3 is a diagram for explaining an example of a system having a collation support device.
[0027]
As shown in FIG. 3, the system in the present embodiment includes a collation support device 10, an image pickup device 20, a storage device 21, an input device 22, and a display device 30. For example, the system may be a monitoring system, an authentication system, or the like. Further, in the collation support device 10 in FIG. 3, in addition to the generation unit 11, the collation unit 12, and the selection unit 13, the first display information generation unit 61, the second display information generation unit 62, and the user interface display information It has a generation unit 63, a candidate face image display information generation unit 64, a detection unit 14, and an association unit 15.
[0028]
Explain the system.
The collation support device 10 is, for example, an information processing device such as a server computer, a personal computer, or a mobile terminal equipped with a CPU (Central Processing Unit), an FPGA (Field-Programmable Gate Array), or both.
[0029]
The image pickup device 20 is a device for capturing the face of a person to be collated. Specifically, the image pickup device 20 transmits the captured image to the collation support device 10 via the communication network. The image pickup device 20 is, for example, an image pickup device such as a camera.
[0030]
The storage device 21 stores the above-mentioned reference three-dimensional data of the head and the collation information. The collation information is information in which the collation face development image and the collation feature area are associated with each other. The collation face development image is a face development image generated based on three-dimensional data of the head of each of a plurality of persons registered in advance. The collation feature area is a pre-registered feature area representing features visible on the skin surface of the head of each of a plurality of persons. In addition, the reference face development image or the reference face three-dimensional image is stored in the storage device 21 in advance. The image, or both, may be remembered.
[0031]
Specifically, when displaying the reference face development image in the second display area, the storage device 21 transmits the three-dimensional data of the reference head to the collation support device 10 via the communication network. When the reference face development image is stored, the storage device 21 transmits the reference face development image to the collation support device 10.
[0032]
Further, when a candidate person is selected, the storage device 21 collates the collation face development image corresponding to the selected person with the collation information associated with the collation feature area via the communication network. It is transmitted to the support device 10.
[0033]
The storage device 21 is, for example, a storage device such as a database. Further, information such as the above-mentioned reference three-dimensional data of the head and collation information may be stored separately in a plurality of storage devices. Further, the storage device 21 may be provided inside the collation support device 10 or may be provided outside.
[0034]
The input device 22 is a physical user interface such as a mouse, a touch panel, and a keyboard. Specifically, the input device 22 is used by the user when performing collation support using the user interface displayed on the display device 30.
[0035]
The display device 30 acquires various display information and displays the generated image or the like on the screen based on the display information. The display device 30 is, for example, a device using a liquid crystal, an organic EL (Electro Luminescence), a CRT (Cathode Ray Tube), or the like. Further, the display device 30 may include an audio output device such as a speaker. The display device 30 may be a printing device such as a printer.
[0036]
The collation support device will be explained.
The first display information generation unit 61 generates the first display information for displaying the first display area for displaying the collation image captured by the image pickup device 20 on the screen of the display device 30. ..
[0037]
The first display area is, for example, an area for displaying the collation image 41 on the user interface 40 shown in FIG. 2. The collated image 41 is a frame image of a still image or a moving image. In the example of FIG. 2, the collation image 41 is displayed in the window 42. However, the configuration of the display screen is not limited to FIG.
[0038]
Specifically, the first display information generation unit 61 acquires an image of a person captured by the image pickup device 20. Subsequently, the first display information generation unit 61 generates the first display information for displaying the collation image 41 as shown in FIG. 2 on the screen of the display device 30 based on the acquired image. .. After that, the first display information generation unit 11 transmits the first display information to the display device 30.
[0039]
Note that only the frame image whose features are easy to see may be used as the collation image. Further, a frame image in which a feature that was not visible before the direction of the face is changed may be used as a collation image.
[0040]
The second display information generation unit 62 sets the reference face development image display area (second display area) for displaying the reference face development image based on the three-dimensional data of the reference head of the display device 30. Generate a second display information to be displayed on the screen.
[0041]
The second display area is, for example, an area for displaying the reference face development image 43 on the user interface 40 shown in FIG. The reference face development image 43 is a development image of a face projected in a cylindrical shape by, for example, performing a UV development process using the three-dimensional data of the reference head, which is stored in the storage device 21 in advance. However, the creation of the developed image of the face is not limited to the above-mentioned cylindrical projection. In the example of FIG. 2, the reference face development image 43 is displayed in the window 44. However, the configuration of the display screen is not limited to FIG.
[0042]
Specifically, the second display information generation unit 62 acquires the three-dimensional data of the reference head from the storage device 21.
[0043]
Subsequently, the second display information generation unit 62 generates a reference face development image using the three-dimensional data of the reference head. Subsequently, based on the generated reference face development image, a second display information for displaying the reference face development image 43 as shown in FIG. 2 on the screen of the display device 30 is generated. After that, the second display information generation unit 62 transmits the second display information to the display device 30. When the reference face development image is stored in the storage device 21, the second display information generation unit 62 may acquire the reference face development image directly from the storage device 21.
[0044]
The user interface display information generation unit 63 makes the user refer to the first display area and displays the first user interface for designating the feature area as the second display area on the screen of the display device 30. Generates the first user interface display information for.
[0045]
Specifically, the user interface display information generation unit 63 displays the user interface 40 as shown in FIG. 2 as the first user interface, and causes the user to specify a feature area on the reference face development image.
[0046]
The first user interface corresponds to the user interface 40 in the example of FIG. For example, when the feature area 52 is drawn (specified) using the user interface 40 and the input device 22 which is a physical user interface such as a mouse, a touch panel, and a keyboard, it is displayed on the user interface 40. When the "Add" button 45 is selected, the drawn feature area 52 is added to the reference face development image 43.
[0047]
Note that the user may observe one collation image and specify a plurality of feature areas on the reference face development image. Further, the user may observe a plurality of collated images (a plurality of frame images) having different face orientations and specify one or more feature areas on the reference face development image. For example, one or more feature regions may be designated on the reference face development image by observing a plurality of collated images having different face orientations. By doing so, the position of the feature area becomes accurate and the number of feature areas can be increased, so that the collation accuracy is improved.
[0048]
Further, as shown in FIG. 2, for example, it is conceivable to provide an "add" button 45, a "delete" button 46, a "save" button 47, a "feature extraction" button 48, an "enlargement / reduction" button 49, and the like. .. When the "Add" button 45 is selected, the feature area 52 drawn on the reference face development image 43 can be added. When the "Delete" button 46 is selected, the feature area 52 drawn on the reference face development image 43 can be deleted. When the "Save" button 47 is selected, feature information (for example, texture information, position information, size information, shape information, feature type information indicating the type of feature, etc.) related to the specified feature area 52 is stored in the storage unit. Will be done. When the "feature extraction" button 48 is selected, features are automatically extracted from the collation image 41. When the "enlargement / reduction" button 49 is selected, the display of the selected collation image 41 or the reference face development image 43 is enlarged or reduced. However, the editing function is not limited to the above-mentioned function.
[0049]
The generation unit 11 is a reference face development image display area (second display) for displaying a reference face development image displayed on the screen of the display device 30 based on the three-dimensional data of the reference head. Area) is used to generate characteristic information about the characteristic area when a characteristic area representing the facial features of the person to be collated that can be seen on the skin surface of the person to be collated is specified.
[0050]
Specifically, the generation unit 11 first uses the first user interface to generate feature information for each of the designated feature regions when one or more feature regions are designated for the reference face development image. .. After that, the generation unit 11 outputs the feature information to the collation unit 12.
[0051]
In the example of FIG. 2, when the feature area 52 is designated for the reference face development image 43, the generation unit 11 generates the feature information of the designated feature area 52 by using the user interface 40.
[0052]
The collation unit 12 collates the feature information with the collation information associated with the pre-registered collation face development image for each person and the collation feature area. Specifically, the collation unit 12 first acquires feature information from the generation unit 11. Subsequently, the collation unit 12 refers to each of the collation information stored in the storage device 21 by using the acquired feature information, executes a collation process, and calculates a collation result. After that, the collation unit 12 associates the collation information with the calculated collation result.
[0053]
In the matching process, for example, the number in which the position of the specified feature area and the position of the matching feature area are close to each other, or the deviation (distance) between the position of the specified feature area and the position of the matching feature area, or , They are combined to calculate a collation index (score) as a collation result. Further, when a plurality of feature regions are specified, the collation index may be calculated using the mutual positional relationship of the plurality of feature regions and the mutual positional relationship of the plurality of collation feature regions. The collation index can be, for example, the Euclidean distance between the two position coordinates of the specified feature area and the collation feature area, the similarity due to the normalization correlation of the texture information of the two feature areas, or the overlapping area of ​​the two feature areas. Can be used.
[0054]
The selection unit 13 selects a candidate person based on the collation result. Specifically, the selection unit 13 selects collation information having a collation index equal to or higher than a preset threshold. Subsequently, the selection unit 13 outputs the selected collation information to the candidate face image display information generation unit 64.
[0055]
The candidate face image display information generation unit 64 includes collation information in which a feature area designated by using the first user interface is associated with a pre-registered collation face development image for each person and a collation feature area. When a candidate person is selected based on the matching result, the matching face development image corresponding to the selected candidate person is displayed on the screen of the display device 30 together with the matching feature area. Generate candidate face image display information.
[0056]
A specific explanation will be given using FIG. FIG. 4 is a diagram for explaining an example of displaying a candidate face image.
[0057]
First, when a candidate person is selected by the selection unit 13 based on the collation result of the collation unit 12, the candidate face image display information generation unit 64 corresponds to the selected candidate person from the storage device 21. Acquires the matching information in which the matching face development image and the matching feature area are associated with each other.
[0058]
Subsequently, the candidate face image display information generation unit 64 displays the collation face development image 71 and the collation feature area 73 as shown in the window 72 of FIG. 4 on the display device 30 based on the acquired collation information. Generate candidate face image display information to be displayed on the screen. After that, the candidate face image display information generation unit 64 transmits the candidate face image display information to the display device 30.
[0059]
If there are multiple candidate persons, the collation face development images are displayed in order according to the collation result. For example, the collation index (score) represented by the collation result is displayed in descending order.
[0060]
The detection unit 14 automatically detects a feature area from an image including the face of a person to be collated displayed on the screen of the display device 30. Specifically, the detection unit 14 uses a collation image to display features of a person's face (for example, hokuro, freckles, tattoos, itch, wrinkles, dents, scratches, blemishes, bumps, uneven skin, and discolored parts of the skin. It automatically detects the characteristic area corresponding to the characteristics of a person that can be seen on the surface of the skin. For the detection of the feature region, for example, it is conceivable to use a technique such as segmentation processing.
[0061]
The association unit 15 may automatically associate the position of the detected feature area with the corresponding position of the reference face development image. The association may be automatically associated with the corresponding position of the reference face development image from the relative positional relationship with the face part such as eyes, nose, and mouth.
[0062]
[Device operation]
Next, the operation of the collation support device according to the embodiment of the present invention will be described with reference to FIG. FIG. 5 is a diagram for explaining an example of the operation of the collation support device. In the following description, FIGS. 1 to 4 will be referred to as appropriate. Further, in the present embodiment, The collation support method is implemented by operating the collation support device. Therefore, the description of the collation support method in the present embodiment is replaced with the following operation description of the collation support device.
[0063]
As shown in FIG. 5, first, the first display information generation unit 61 causes the screen of the display device 30 to display the first display area for displaying the collation image captured by the image pickup device 20. First display information for the purpose is generated (step A1). Specifically, in step A1, the first display information generation unit 61 acquires an image of a person captured by the image pickup device 20.
[0064]
Subsequently, in step A1, the first display information generation unit 61 displays the collation image 41 as shown in FIG. 2 on the screen of the display device 30 based on the acquired image. Generate information. After that, in step A1, the first display information generation unit 61 transmits the first display information to the display device 30.
[0065]
Note that only the frame image whose features are easy to see may be used as the collation image. Further, a frame image in which a feature that was not visible before the direction of the face is changed may be used as a collation image.
[0066]
Further, the second display information generation unit 62 displays a second display area for displaying the reference face development image based on the three-dimensional data of the reference head on the screen of the display device 30. Generate the second display information (step A2). Specifically, in step A2, the second display information generation unit 62 acquires three-dimensional data of the reference head from the storage device 21.
[0067]
Subsequently, in step A2, the second display information generation unit 62 generates a reference face development image using the three-dimensional data of the reference head. Subsequently, in step A2, the second display information generation unit 62 displays the reference face development image 43 as shown in FIG. 2 on the screen of the display device 30 based on the generated reference face development image. Generates the second display information of. After that, in step A2, the second display information generation unit 62 transmits the second display information to the display device 30.
[0068]
When the reference face development image is stored in the storage device 21, the second display information generation unit 62 may directly acquire the reference face development image from the storage device 21.
[0069]
The order of the process of step A1 and the process of step A2 described above may be reversed, or the processes may be executed in parallel.
[0070]
Next, the user interface display information generation unit 63 makes the user refer to the first display area, and sets the first user interface for designating the feature area as the second display area on the screen of the display device 30. Generates the first user interface display information to be displayed in (step A3).
[0071]
Specifically, in step A3, the user interface display information generation unit 63 displays the user interface 40 as shown in FIG. 2 as the first user interface, and causes the user to display the feature area on the reference face development image. To be specified.
[0072]
Note that the user may observe one collation image and specify a plurality of feature areas on the reference face development image. Further, the user may observe a plurality of collated images (a plurality of frame images) having different face orientations and specify one or more feature areas on the reference face development image. For example, one or more feature regions may be designated on the reference face development image by observing a plurality of collated images having different face orientations. By doing so, the position of the feature area becomes accurate, and the number of feature areas can be increased.
[0073]
Furthermore, the feature area may be specified automatically. In that case, the detection unit 14 automatically detects the feature area from the image including the face of the person to be collated displayed on the screen of the display device 30, and the association unit 15 determines the position of the detected feature area. It is automatically associated with the corresponding position of the reference face development image.
[0074]
Next, the generation unit 11 uses a second display area for displaying the reference face development image generated based on the three-dimensional data of the reference head, which is displayed on the screen of the display device 30. When a characteristic area representing the facial features of the person to be collated that can be seen on the skin surface of the person to be collated is designated, the feature information regarding the characteristic area is generated (step A4).
[0075]
Specifically, in step A4, first, when one or more feature regions are designated for the reference face development image by using the first user interface, the generation unit 11 features each of the designated feature regions. Generate information. After that, in step A4, the generation unit 11 outputs the feature information to the collation unit 12.
[0076]
In the example of FIG. 2, when the feature area 52 is designated for the reference face development image 43, the generation unit 11 generates the feature information of the designated feature area 52 by using the user interface 40.
[0077]
The collation unit 12 collates the feature information with the collation information associated with the collation face development image for each person registered in advance and the collation feature area (step A5). Specifically, in step A5, the collation unit 12 first acquires feature information from the generation unit 11. Subsequently, in step A5, the collation unit 12 refers to each of the collation information stored in the storage device 21 using the acquired feature information, executes a collation process, and calculates a collation result. After that, in step A5, the collation unit 12 associates the collation information with the calculated collation result.
[0078]
The selection unit 13 selects a candidate person based on the collation result (step A6). Specifically, in step A6, the selection unit 13 selects collation information having a collation index equal to or higher than a preset threshold. Subsequently, in step A6, the selection unit 13 outputs the selected collation information to the candidate face image display information generation unit 64.
[0079]
The candidate face image display information generation unit 64 includes collation information in which a feature area designated by using the first user interface is associated with a pre-registered collation face development image for each person and a collation feature area. When a candidate person is selected based on the matching result, the matching face development image corresponding to the selected candidate person is displayed on the screen of the display device 30 together with the matching feature area. Candidate face image display information is generated (step A7).
[0080] [0080]
Specifically, in step A7, the candidate face image display information generation unit 64 first receives from the storage device 21 when a candidate person is selected by the selection unit 13 based on the collation result of the collation unit 12. Acquires the matching information in which the matching face development image corresponding to the selected candidate person and the matching feature area are associated with each other.
[0081]
Subsequently, in step A7, the candidate face image display information generation unit 64 uses the acquired collation information to obtain the collation face development image 71 and the collation feature area 73 as shown in the window 72 of FIG. Candidate face image display information for display on the screen of the display device 30 is generated. After that, the candidate face image display information generation unit 64 transmits the candidate face image display information to the display device 30.
[0082]
If there are multiple candidate persons, the collation face development images are displayed in order according to the collation result. For example, the collation index (score) represented by the collation result is displayed in descending order.
[0083]
Next, the display device 30 acquires the candidate face image display information and displays the matching face development image 71 and the matching feature area 73 as shown in the window 72 of FIG. 4 on the screen (step A8). ).
[0084]
[Effect of this embodiment]
As described above, according to the present embodiment, the user observes the collated image and the user specifies the feature area on the reference face development image, so that even if the person in the collated image does not face the front. Using the specified feature area, it is possible to provide collation support that leads to the identification of the person to be collated.
[0085]
In addition, by designating the feature area using the reference face development image, the influence of the unevenness of the face due to the change in the face orientation can be reduced, so even if the position for only the feature changes, the user can use the feature. The corresponding feature area can be easily specified on the reference face development image.
[0086]
Further, a collation device that extracts an image corresponding to a person from a frame image captured by the image pickup device and performs collation using the extracted image may be linked with a collation support device. In that case, when an image corresponding to the feature is detected from any of the frame images, processing is performed from a collation processing mode in which collation is performed using a collation device to a collation support processing mode in which collation support is performed using a collation support device. May be switched.
[0087]
In addition, when it is determined that the person registered in advance and the person captured are the same person, the matching face development image and the matching feature corresponding to the person are based on the characteristics of the captured person. You may edit the area.
[0088]
In addition, when it is determined that the pre-registered person and the imaged person are not the same person, the person is dealt with based on the characteristic showing the difference between the pre-registered person and the imaged person. You may edit the collation face development image and the collation feature area.
[0089]
[program]
The program according to the embodiment of the present invention may be any program that causes a computer to execute steps A1 shown in FIG. By installing and executing this program on a computer, the collation support device and the collation support method in the present embodiment can be realized. In this case, the processor of the computer has a generation unit 11, a collation unit 12, a selection unit 13, a detection unit 14, an association unit 15, a first display information generation unit 61, a second display information generation unit 62, and a user interface display information. It functions as a generation unit 63 and a candidate face image display information generation unit 64, and performs processing.
[0090]
Further, the program in the present embodiment may be executed by a computer system constructed by a plurality of computers. In this case, for example, each computer has its own processor, which is a generation unit 11, a collation unit 12, a selection unit 13, a detection unit 14, an association unit 15, a first display information generation unit 61, and a second display. It may function as any of an information generation unit 62, a user interface display information generation unit 63, and a candidate face image display information generation unit 64.
[0091]
(Variation example)
Hereinafter, modifications of the present invention will be described with reference to FIGS. 6 to 11.
[0092]
[System configuration]
A modified example will be described with reference to FIG. FIG. 6 is a diagram for explaining an example of a system having a collation support device.
[0093]
As shown in FIG. 6, the collation support device 10 in the modified example includes a generation unit 11, a collation unit 12, a selection unit 13, a detection unit 14, an association unit 15, a first display information generation unit 61, and a second display information. In addition to the generation unit 62, the user interface display information generation unit 63, and the candidate face image display information generation unit 64, the conversion unit 16 and the third display information generation unit 65 are included.
[0094]
In the modified example, when the storage device 21 further displays the reference face three-dimensional image in the third display area, the storage device 21 transmits the reference three-dimensional data of the head to the collation support device 10 via the communication network. .. The third display area will be described later. When the reference face three-dimensional image is stored, the storage device 21 transmits the reference face development image to the collation support device 10.
[0095]
The third display information generation unit 65 sets the face orientation of the reference face three-dimensional image generated based on the three-dimensional data of the reference head as the person to be collated displayed in the first display area. A third display information for displaying the third display area to be displayed according to the orientation of the face on the screen of the display device 30 is generated. Specifically, the third display information generation unit 65 acquires three-dimensional data of the reference head from the storage device 21.
[0096]
Subsequently, the third display information generation unit 65 generates a reference face exhibition three-dimensional image using the three-dimensional data of the reference head. Subsequently, based on the generated reference face three-dimensional image, a third display information for displaying the reference face three-dimensional image 81 as shown in FIG. 6 on the screen of the display device 30 is generated.After that, the third display information generation unit 65 transmits the third display information to the display device 30. When the reference face three-dimensional image is stored in the storage device 21, the third display information generation unit 65 may directly acquire the reference face three-dimensional image from the storage device 21.
[0097]
The orientation of the face may be adjusted manually by the user using the user interface, or may be adjusted automatically.
[0098]
In the modification, the user interface display information generation unit 63 displays the feature area in the third display area and the second user interface used for the operation for designating the user on the screen of the display device 30. Generates the second user interface display information of.
[0099]
A specific explanation will be given using FIG. FIG. 7 is a diagram for explaining a user interface used for collation support.
[0100]
The user interface display information generation unit 63 displays the user interface 80 as shown in FIG. 7 as the second user interface, and causes the user to specify the feature area on the reference face three-dimensional image. In the example of FIG. 7, the reference face three-dimensional image 81 is displayed in the window 82. However, the configuration of the display screen is not limited to FIG. 7.
[0101]
Note that the user may observe one collation image and specify a plurality of feature areas on the reference face three-dimensional image. Further, the user may observe a plurality of collated images (a plurality of frame images) having different face orientations and specify one or more feature regions on the reference face three-dimensional image. For example, one or more feature regions may be designated on the reference face three-dimensional image by observing a plurality of collated images having different face orientations. By doing so, the position of the feature area becomes accurate and the number of feature areas can be increased, so that the collation accuracy is improved.
[0102]
Further, in the modified example, for example, when the user selects the reference face three-dimensional image 81 shown in FIG. 7, and then the “add” button 45 is selected, the feature drawn on the reference face three-dimensional image 81 is selected. Region 83 can be added. When the "Delete" button 46 is selected, the feature area 83 drawn on the reference face three-dimensional image 81 can be deleted. When the "save" button 47 is selected, feature information (for example, texture information, position information, feature type information, etc.) related to the designated feature area 83 is stored in the storage unit. When the "feature extraction" button 48 is selected, features are automatically extracted from the collation image 41. When the "enlargement / reduction" button 49 is selected, the display of the selected collation image 41, the reference face development image 43, or the reference face three-dimensional image 81 is enlarged or reduced. However, the editing function is not limited to the above-mentioned function.
[0103]
In the modification, the generation unit 11 provides a third display area for displaying the reference face three-dimensional image generated based on the reference three-dimensional data of the head, which is displayed on the screen of the display device 30. When a characteristic area representing the facial features of the person to be collated that can be seen on the skin surface of the person to be collated is designated, the feature information about the characteristic area is generated.
[0104]
Specifically, the generation unit 11 first uses the second user interface to generate feature information for each of the designated feature regions when one or more feature regions are designated for the reference face three-dimensional image. do. After that, the generation unit 11 outputs the feature information to the collation unit 12. In the example of FIG. 7, the generation unit 11 uses the user interface 80 to generate the feature information of the designated feature region 83 when the feature region 83 is designated for the reference face three-dimensional image 81.
[0105]
The conversion unit 16 displays the face orientation of the reference face three-dimensional image generated based on the three-dimensional data of the reference head in accordance with the orientation of the face of the person to be collated. When a feature area is designated using the display area, the feature information related to the feature area designated in the reference face three-dimensional image is converted into the feature information used in the reference face development image.
[0106]
The explanation of the collation unit 12 and the selection unit 13 has been described above and will be omitted.
[0107]
A specific explanation will be given using FIG. FIG. 8 is a diagram for explaining an example of a display in which a feature region on a reference face three-dimensional image is converted into a feature region on a reference face developed image face image.
[0108]
In FIG. 8, when the feature area 83 is added, the feature area 86 corresponding to the feature area 83 is also added on the reference face development image 84. In the example of FIG. 8, the reference face development image 84 is displayed in the window 85. However, the configuration of the display screen is not limited to FIG.
[0109]
In the modification, the candidate face image display information generation unit 64 associates the feature area designated by using the second user interface with the pre-registered matching face development image for each person and the matching feature area. When a candidate person is selected based on the collation result, the collation face development image corresponding to the selected candidate person is displayed on the screen of the display device 30 together with the feature area. Generate candidate face image display information for making.
[0110]
A specific explanation will be given using FIG. FIG. 9 is a diagram for explaining an example of displaying a candidate face image.
[0111]
First, when a candidate person is selected by the selection unit 13 based on the collation result of the collation unit 12, the candidate face image display information generation unit 64 corresponds to the selected candidate person from the storage device 21. Acquires the matching information in which the matching face development image and the matching feature area are associated with each other.
[0112]
Subsequently, the candidate face image display information generation unit 64 displays the collation face development image 91 and the collation feature area 93 as shown in the window 92 of FIG. 9 on the display device 30 based on the acquired collation information. Generate candidate face image display information to be displayed on the screen. After that, the candidate face image display information generation unit 64 transmits the candidate face image display information to the display device 30.
[0113]
If there are multiple candidate persons, the collation face development images are displayed in order according to the collation result. For example, the collation index (score) represented by the collation result is displayed in descending order.
[0114]
Further, in the modified example, the candidate face image display information generation unit 64 displays a collation face three-dimensional image corresponding to the candidate person based on the three-dimensional data of the head of the selected candidate person. Generates candidate face three-dimensional image display information to be displayed on the screen of the device 30.
[0115]
A specific explanation will be given with reference to FIG. FIG. 10 is a diagram for explaining an example of displaying a candidate face image.
[0116]
First, when a candidate person is selected by the selection unit 13 based on the collation result of the collation unit 12, the candidate face image display information generation unit 64 corresponds to the selected candidate person from the storage device 21. Acquires the matching information in which the matching face three-dimensional image and the matching feature area are associated with each other.
[0117]
Subsequently, the candidate face image display information generation unit 64, based on the acquired collation information, creates a collation face three-dimensional image 91'as shown in the window 92'of FIG. 10 and a collation feature area 93'. Candidate face image display information for display on the screen of the display device 30 is generated. After that, the candidate face image display information generation unit 64 transmits the candidate face image display information to the display device 30.
[0118]
In addition, one or more of the above-mentioned collation image, reference face development image, reference face three-dimensional image, collation face expansion image including the collation feature area, and collation face three-dimensional image including the collation feature area are combined. It may be displayed on the user interface.
[0119]
[Device operation]
The operation of the modified example will be described with reference to FIG. FIG. 11 is a diagram for explaining an example of the operation of the collation support device. In the following description, FIGS. 1 to 10 will be referred to as appropriate. Further, in the modified example, the collation support method is implemented by operating the collation support device. Therefore, the description of the collation support method in the modified example is replaced with the following operation description of the collation support device.
[0120]
As shown in FIG. 11, first, the process of step A1 described above is executed.
[0121]
The third display information generation unit 65 sets the face orientation of the reference face three-dimensional image generated based on the three-dimensional data of the reference head as the person to be collated displayed in the first display area. A third display information for displaying the third display area to be displayed according to the orientation of the face on the screen of the display device 30 is generated (step B1). Specifically, the third display information generation unit 65 acquires three-dimensional data of the reference head from the storage device 21.
[0122]
Subsequently, in step B1, the third display information generation unit 65 generates a reference face exhibition three-dimensional image using the three-dimensional data of the reference head. Subsequently, in step B1, the third display information generation unit 65 displays the reference face three-dimensional image 81 as shown in FIG. 7 on the screen of the display device 30 based on the generated reference face three-dimensional image. Generate a third display information to make it. After that, in step B1, the third display information generation unit 65 transmits the third display information to the display device 30.
[0123]
When the reference face three-dimensional image is stored in the storage device 21, the third display information generation unit 65 may directly acquire the reference face three-dimensional image from the storage device 21.
[0124]
The orientation of the face may be adjusted manually by the user or automatically.
[0125]
The order of the process of step A1 and the process of step B1 described above may be reversed, or the processes may be executed in parallel.
[0126]
Next, the user interface display information generation unit 63 displays the feature area in the third display area and the second user interface used for the operation for designating the user on the screen of the display device 30. Generate the second user interface display information (step B2).
[0127]
Specifically, in step B2, the user interface display information generation unit 63 displays the user interface 80 as shown in FIG. 7 as the second user interface, and features the user on the reference face three-dimensional image. Have the area specified. In the example of FIG. 7, the reference face three-dimensional image 81 is displayed in the window 82. However, the configuration of the display screen is not limited to FIG. 7.
[0128]
Note that the user may observe one collation image and specify a plurality of feature areas on the reference face three-dimensional image. Further, the user may observe a plurality of collated images (a plurality of frame images) having different face orientations and specify one or more feature regions on the reference face three-dimensional image. For example, one or more feature regions may be designated on the reference face three-dimensional image by observing a plurality of collated images having different face orientations. By doing so, the position of the feature area becomes accurate, and the number of feature areas can be increased.
[0129]
Furthermore, the feature area may be specified automatically. In that case, the detection unit 14 automatically detects the feature area from the image including the face of the person to be collated displayed on the screen of the display device 30, and the association unit 15 determines the position of the detected feature area. It is automatically associated with the corresponding position of the reference face three-dimensional image.
[0130]
Next, the generation unit 11 uses a third display area for displaying the reference face three-dimensional image generated based on the reference three-dimensional data of the head, which is displayed on the screen of the display device 30. When a characteristic area representing the facial features of the person to be collated that can be seen on the skin surface of the person to be collated is designated, the feature information related to the characteristic area is generated (step B3).
[0131]
Specifically, in step B3, the generation unit 11 first uses the second user interface to specify one or more feature regions in the reference face three-dimensional image, for each of the designated feature regions. Generate feature information. After that, in step B3, the generation unit 11 outputs the feature information to the collation unit 12. In the example of FIG. 7, the generation unit 11 uses the user interface 80 to generate the feature information of the designated feature region 83 when the feature region 83 is designated for the reference face three-dimensional image 81.
[0132]
Next, the conversion unit 16 is based on the three-dimensional data of the reference head. When the feature area is specified using the reference face 3D image display area that displays the face orientation of the generated reference face 3D image according to the orientation of the face of the person to be collated, the reference face 3D The feature information about the feature region specified in the image is converted into the feature information used in the reference face development image (step B4).
[0133]
Next, the processes of steps A5 and A6 described above are executed.
[0134]
Next, in the candidate face image display information generation unit 64, the feature area designated by using the second user interface is associated with the collation face development image and the collation feature area for each person registered in advance. When a candidate person is selected based on the collation result by collating with the collation information, the collation face development image corresponding to the selected candidate person is displayed on the screen of the display device 30 together with the feature area. Generate candidate face image display information for this purpose (step B5).
[0135]
In step B5, when the candidate face image display information generation unit 64 first selects a candidate person by the selection unit 13 based on the collation result of the collation unit 12, the candidate face image display information generation unit 64 selects the candidate from the storage device 21. Acquires the matching information in which the matching face development image corresponding to the person and the matching feature area are associated with each other.
[0136]
Subsequently, in step B5, the candidate face image display information generation unit 64 uses the acquired collation information to obtain the collation face development image 91 and the collation feature area 93 as shown in the window 92 of FIG. Candidate face image display information for display on the screen of the display device 30 is generated. After that, the candidate face image display information generation unit 64 transmits the candidate face image display information to the display device 30.
[0137]
If there are multiple candidate persons, the collation face development images are displayed in order according to the collation result. For example, the collation index (score) represented by the collation result is displayed in descending order.
[0138]
Further, in step B5, the candidate face image display information generation unit 64 displays a collation face three-dimensional image corresponding to the candidate person based on the three-dimensional data of the head of the selected candidate person. Candidate face three-dimensional image display information for display on the screen of the device 30 may be generated.
[0139]
In step B5, when the candidate face image display information generation unit 64 first selects a candidate person by the selection unit 13 based on the collation result of the collation unit 12, the candidate face image display information generation unit 64 selects the candidate from the storage device 21. Acquires the matching information in which the matching face three-dimensional image corresponding to the person and the matching feature area are associated with each other.
[0140]
Subsequently, in step B5, the candidate face image display information generation unit 64 bases the collation face three-dimensional image 91'as shown in the window 92'of FIG. 10 and the collation feature area 93 based on the acquired collation information. ´ and are generated as candidate face image display information for displaying on the screen of the display device 30. After that, the candidate face image display information generation unit 64 transmits the candidate face image display information to the display device 30.
[0141]
If there are multiple candidate persons, the collation face development images are displayed in order according to the collation result. For example, the collation index (score) represented by the collation result is displayed in descending order.
[0142]
Next, the display device 30 acquires the candidate face image display information and displays the matching face development image 91 and the matching feature area 93 as shown in the window 92 of FIG. 9 on the screen (step B6). ). Alternatively, in step B6, the display device 30 may display the matching face development image 91'and the matching feature area 93'as shown in the window 92'of FIG. 10 on the screen.
[0143]
[Effect of modified example]
According to the modified example as described above, the user observes the collated image and the user specifies the feature area on the reference face three-dimensional image, so that even if the person in the collated image does not face the front. Using the specified feature area, it is possible to provide collation support that leads to the identification of the person to be collated.
[0144]
In addition, by designating the feature area using the reference face three-dimensional image, the influence of the unevenness of the face due to the change in the face orientation can be reduced, so even if the position for only the feature changes, the user can use the feature. The feature area corresponding to can be easily specified on the reference face three-dimensional image.
[0145]
Further, a collation device that extracts an image corresponding to a person from a frame image captured by the image pickup device and performs collation using the extracted image may be linked with a collation support device. In that case, when an image corresponding to the feature is detected from any of the frame images, the collation support processing mode that supports collation using the collation support device from the collation processing mode that collates using the current collation device. The process may be switched to.
[0146]
In addition, when it is determined that the person registered in advance and the person captured are the same person, the matching face development image and the matching feature corresponding to the person are based on the characteristics of the captured person. You may edit the area.
[0147]
In addition, when it is determined that the pre-registered person and the imaged person are not the same person, the person is dealt with based on the characteristic showing the difference between the pre-registered person and the imaged person. You may edit the collation face development image and the collation feature area.
[0148]
[program]
The program in the modified example of the present invention may be any program that causes a computer to execute from step A1 shown in FIG. By installing and executing this program on a computer, the collation support device and the collation support method in the present embodiment can be realized. In this case, the processor of the computer includes a generation unit 11, a collation unit 12, a selection unit 13, a detection unit 14, an association unit 15, a conversion unit 16, a first display information generation unit 61, and a second display information generation unit 62. It functions as a user interface display information generation unit 63, a candidate face image display information generation unit 64, and a third display information generation unit 65, and performs processing.
[0149]
Further, the program in the present embodiment may be executed by a computer system constructed by a plurality of computers. In this case, for example, each computer has a computer processor such as a generation unit 11, a collation unit 12, a selection unit 13, a detection unit 14, an association unit 15, a conversion unit 16, and a first display information generation unit 61. It may function as any of a second display information generation unit 62, a user interface display information generation unit 63, a candidate face image display information generation unit 64, and a third display information generation unit 65.
[0150]
[Physical configuration]
Here, a computer that realizes a collation support device by executing the programs in the embodiment and the modified example will be described with reference to FIG. 12. FIG. 12 is a block diagram showing an example of a computer that realizes a collation support device according to an embodiment and a modification of the present invention.
[0151]
As shown in FIG. 12, the computer 110 includes a CPU 111, a main memory 112, a storage device 113, an input interface 114, a display controller 115, a data reader / writer 116, and a communication interface 117. Each of these parts is connected to each other via a bus 121 so as to be capable of data communication. The computer 110 may include a GPU (Graphics Processing Unit) or an FPGA in addition to or in place of the CPU 111.
[0152]
The CPU 111 expands the program (code) in the present embodiment stored in the storage device 113 into the main memory 112, and executes these in a predetermined order to perform various calculations. The main memory 112 is typically a volatile storage device such as a DRAM (Dynamic Random Access Memory). Further, the program in the present embodiment is provided in a state of being stored in a computer-readable recording medium 120. The program in the present embodiment may be distributed on the Internet connected via the communication interface 117. The recording medium 120 is a non-volatile recording medium.
[0153]
Further, specific examples of the storage device 113 include a semiconductor storage device such as a flash memory in addition to a hard disk drive. The input interface 114 mediates data transmission between the CPU 111 and an input device 118 such as a keyboard and a mouse. The display controller 115 is connected to the display device 119 and controls the display on the display device 119.
[0154]
The data reader / writer 116 mediates the data transmission between the CPU 111 and the recording medium 120, reads the program from the recording medium 120, and writes the processing result in the computer 110 to the recording medium 120. The communication interface 117 mediates data transmission between the CPU 111 and another computer.
[0155]
Specific examples of the recording medium 120 include general-purpose semiconductor storage devices such as CF (Compact Flash (registered trademark)) and SD (Secure Digital), magnetic recording media such as Flexible Disk, or CD-. Examples include optical recording media such as ROM (Compact Disk Read Only Memory).
[0156]
The collation support device in the present embodiment can also be realized by using the hardware corresponding to each part instead of the computer in which the program is installed. Further, the collation support device may be partially realized by a program and the rest by hardware.
[0157]
[Additional Notes]
Regarding the above embodiments, the following additional notes will be further disclosed. A part or all of the above-described embodiments can be expressed by the following descriptions (Appendix 1) to (Appendix 12), but the description is not limited to the following.
[0158]
(Appendix 1)
The skin of the person to be collated using the reference face development image display area for displaying the reference face development image generated based on the three-dimensional data of the reference head displayed on the screen of the display device. When a feature area that represents the features of the face of the person to be collated that can be seen on the surface is specified, a generation unit that generates feature information about the feature area and a generator.
A collation unit that collates the feature information with the collation information associated with the collation face development image for each person registered in advance and the collation feature area.
The selection section, which selects candidate persons based on the collation results,
A collation support device characterized by having.
[0159]
(Appendix 2)
The collation support device described in Appendix 1
A reference face three-dimensional image display area for displaying the face orientation of the reference face three-dimensional image generated based on the three-dimensional data of the reference head according to the orientation of the face of the person to be collated. When the feature area is specified by using, the conversion unit that converts the feature information about the feature area specified in the reference face three-dimensional image into the feature information used in the reference face development image.
A collation support device characterized by having.
[0160]
(Appendix 3)
The collation support device described in Appendix 1 or 2,
The collation unit has a number in which the position of the feature area and the position of the collation feature area are close to each other, the deviation between the position of the feature area and the position of the collation feature area, or the position of the feature area. And the relationship with the position of the collation feature area, or a combination thereof, and the collation index is calculated as the collation result.
A collation support device characterized by that.
[0161]
(Appendix 4)
The collation support device described in any one of Supplementary notes 1 to 3 and
A detection unit that detects the feature area from the image including the face of the person to be collated displayed on the screen of the display device.
With the association part that associates the detected position of the feature area with the corresponding position of the reference face development image.
A collation support device characterized by having.
[0162]
(Appendix 5)
(A) To display a reference face development image generated based on the three-dimensional data of the reference head displayed on the screen of the display device.When a characteristic area representing the facial features of the person to be collated, which can be seen on the skin surface of the person to be collated, is designated by using the reference face development image display area, the feature information about the feature area is generated. Steps and
(B) A step of collating the feature information with the collation information associated with the collation face development image for each person registered in advance and the collation feature area.
(C) A step to select a candidate person based on the collation result, and
A collation support method characterized by having.
[0163]
(Appendix 6)
The collation support method described in Appendix 5
(D) A reference face three-dimensional image in which the face orientation of the reference face three-dimensional image generated based on the three-dimensional data of the reference head is displayed according to the orientation of the face of the person to be collated. When a feature area is designated using the display area, the step of converting the feature information about the feature area specified in the reference face three-dimensional image into the feature information used in the reference face development image.
A collation support method characterized by having.
[0164]
(Appendix 7)
The collation support method described in Appendix 5 or 6
In (b), the number in which the position of the feature region and the position of the matching feature region are close to each other, the deviation between the position of the feature region and the position of the matching feature region, or the position of the feature region A step of calculating a collation index as the collation result by the relationship between the position and the position of the collation feature area or a combination thereof.
A collation support method characterized by having.
[0165]
(Appendix 8)
The collation support method described in any one of Appendix 5 to 7
(E) The feature area is detected from the image including the face of the person to be collated displayed on the screen of the display device.
(F) The detected position of the feature area is associated with the corresponding position of the reference face development image.
A collation support method characterized by that.
[0166]
(Appendix 9)
On the computer
(A) The reference face development image display area for displaying the reference face development image generated based on the three-dimensional data of the reference head displayed on the screen of the display device is used as a collation target. When a feature area representing the facial features of the person to be collated, which can be seen on the skin surface of the person, is specified, a step and a step of generating feature information about the feature area,
(B) A step of collating the feature information with the collation information associated with the collation face development image for each person registered in advance and the collation feature area.
(C) A step to select a candidate person based on the collation result, and
A computer-readable recording medium recording a program, including instructions to execute.
[0167]
(Appendix 10)
The computer-readable recording medium described in Appendix 9
The program is on the computer
(D) A reference face three-dimensional image in which the face orientation of the reference face three-dimensional image generated based on the three-dimensional data of the reference head is displayed according to the orientation of the face of the person to be collated. When a feature area is specified using the display area, an instruction to execute a step of converting the feature information about the feature area specified in the reference face three-dimensional image into the feature information used in the reference face development image is issued. Including,
A computer-readable recording medium on which the program is recorded.
[0168]
(Appendix 11)
A computer-readable recording medium according to Appendix 9 or 10.
In the step (b), the number in which the position of the feature region and the position of the matching feature region are close to each other, or the deviation between the position of the feature region and the position of the matching feature region, or the feature. The collation index is calculated as the collation result by the relationship between the position of the region and the position of the collation feature region or a combination thereof.
A computer-readable recording medium characterized by that.
[0169]
(Appendix 12)
A computer-readable recording medium according to any one of Supplementary note 9 to 11.
The program is on the computer
(E) A step of detecting the feature area from an image including the face of the person to be collated displayed on the screen of the display device.
(F) Further includes an instruction to execute a step, associating the detected position of the feature region with the corresponding position of the reference face development image.
A computer-readable recording medium on which the program is recorded.
[0170]
Although the present invention has been described above with reference to the embodiment, the present invention is not limited to the above embodiment. Various changes that can be understood by those skilled in the art can be made within the scope of the present invention in terms of the structure and details of the present invention.
Industrial availability
[0171]
As described above, according to the present invention, collation can be performed by designating the features visible on the skin surface of the face of the person to be collated according to the orientation of the face of the captured image. The present invention is useful in fields such as monitoring systems and authentication systems that require verification.
Description of the sign
[0172]
10 Collation support device
11 Generation unit
12 Collation section
13 Selection section
14 Detection unit
15 Association part
16 Conversion unit
20 Imaging device
21 Storage device
22 Input device
30 display device
61 First display information generation unit
62 Second display information generation unit
63 User interface display information generation unit
64 Candidate face image display information generation unit
65 Third display information generation unit
110 computer
111 CPU
112 Main memory
113 Storage device
114 input interface
115 Display controller
116 Data reader / writer
117 communication interface
118 Input device
119 Display device
120 Recording medium
121 Bus
The scope of the claims
[Claim 1]
The skin of the person to be collated using the reference face development image display area for displaying the reference face development image generated based on the three-dimensional data of the reference head displayed on the screen of the display device. When a feature area that represents the features of the face of the person to be collated that can be seen on the surface is specified, a generation means for generating feature information about the feature area and a generation means.
A collation means for collating the feature information with the collation information associated with the collation face development image for each person registered in advance and the collation feature area.
Selection means to select a candidate person based on the collation result,
A collation support device characterized by having.
[Claim 2]
The collation support device according to claim 1.
A reference face three-dimensional image display area for displaying the face orientation of the reference face three-dimensional image generated based on the three-dimensional data of the reference head according to the orientation of the face of the person to be collated. When the feature area is designated by using, the conversion means for converting the feature information about the feature area specified in the reference face three-dimensional image into the feature information used in the reference face development image.
A collation support device characterized by having.
[Claim 3]
The collation support device according to claim 1 or 2.
The collation means has a number in which the position of the feature region and the position of the collation feature region are close to each other, the deviation between the position of the feature region and the position of the collation feature region, or the position of the feature region. And the relationship with the position of the collation feature area, or a combination thereof, and the collation index is calculated as the collation result.
A collation support device characterized by that.
[Claim 4]
The collation support device according to any one of claims 1 to 3.
A detection means for detecting the feature area from an image including the face of the person to be collated displayed on the screen of the display device.
With an association means that associates the detected position of the feature area with the corresponding position of the reference face development image.
A collation support device characterized by having.
[Claim 5]
(A) The reference face development image display area for displaying the reference face development image generated based on the three-dimensional data of the reference head displayed on the screen of the display device is used as a collation target. When a characteristic area representing the facial features of the person to be collated, which can be seen on the skin surface of the person, is specified, the characteristic information regarding the characteristic area is generated.
(B) The feature information is collated with the collation information associated with the collation face development image for each person registered in advance and the collation feature area.
(C) Select a candidate person based on the collation result.
A collation support method characterized by that.
[Claim 6]
The collation support method described in claim 5
(D) A reference face three-dimensional image in which the face orientation of the reference face three-dimensional image generated based on the three-dimensional data of the reference head is displayed according to the orientation of the face of the person to be collated. When a feature area is specified using the display area, the feature information about the feature area specified in the reference face three-dimensional image is converted into the feature information used in the reference face development image.
A collation support method characterized by that.
[Claim 7]
The collation support method according to claim 5 or 6.
In (b), the number in which the position of the feature region and the position of the matching feature region are close to each other, the deviation between the position of the feature region and the position of the matching feature region, or the position of the feature region The collation index is calculated as the collation result by the relationship between the position and the position of the collation feature area or a combination thereof.
A collation support method characterized by that.
[Claim 8]
The collation support method according to any one of claims 5 to 7.
(E) The feature area is detected from the image including the face of the person to be collated displayed on the screen of the display device.
(F) The detected position of the feature area is associated with the corresponding position of the reference face development image.
A collation support method characterized by that.
[Claim 9]
On the computer
(A) The reference face development image display area for displaying the reference face development image generated based on the three-dimensional data of the reference head displayed on the screen of the display device is used as a collation target. When a feature area representing the facial features of the person to be collated, which can be seen on the skin surface of the person, is specified, a step and a step of generating feature information about the feature area,
(B) A step of collating the feature information with the collation information associated with the collation face development image for each person registered in advance and the collation feature area.
(C) A step to select a candidate person based on the collation result, and
A computer-readable recording medium recording a program, including instructions to execute.
[Claim 10]
The computer-readable recording medium according to claim 9.
The program is on the computer
(D) A reference face three-dimensional image in which the face orientation of the reference face three-dimensional image generated based on the three-dimensional data of the reference head is displayed according to the orientation of the face of the person to be collated. When a feature area is specified using the display area, an instruction to execute a step of converting the feature information about the feature area specified in the reference face three-dimensional image into the feature information used in the reference face development image is issued. Including,
A computer-readable recording medium on which the program is recorded.
[Claim 11]
The computer-readable recording medium according to claim 9 or 10.
In the step (b), the number in which the position of the feature region and the position of the matching feature region are close to each other, or the deviation between the position of the feature region and the position of the matching feature region, or the feature. The collation index is calculated as the collation result by the relationship between the position of the region and the position of the collation feature region or a combination thereof.
A computer-readable recording medium characterized by that.
[Claim 12]
The computer-readable recording medium according to any one of claims 9 to 11.
The program is on the computer
(E) A step of detecting the feature area from an image including the face of the person to be collated displayed on the screen of the display device.
(F) Further includes an instruction to execute a step, associating the detected position of the feature region with the corresponding position of the reference face development image.
A computer-readable recording medium on which the program is recorded.