Title of the invention: collation support device, collation support method, and computer-readable recording medium
Technical field
[0001]
The present invention relates to a collation support device and a collation support method that support collation, and further relates to a computer-readable recording medium that records a program for realizing these.
Background technology
[0002]
A collation device has been proposed that collates a target person's face image with a pre-registered face image and identifies the target person based on the collation result.
[0003]
For example, as a related technique, Patent Document 1 discloses an authentication system capable of performing authentication with high accuracy when authenticating the identity of a person. According to the certification system of Patent Document 1, a feature region corresponding to an individual feature region (hokuro, scratch, wrinkle) is automatically detected from an image of a person to be certified, and the feature amount of the detected feature region is recognized. Authenticate using the recognized feature quantity.
Prior art literature
Patent documents
[0004]
Patent Document 1: Japanese Unexamined Patent Publication No. 2007-304857
Outline of the invention
Problems to be solved by the invention
[0005]
However, since the authentication system of Patent Document 1 uses a pre-registered face image, if the orientation of the face of the registered face image and the orientation of the face of the person to be imaged are different, matching may not be possible. be. For example, when the face of the person to be imaged is not facing the front, the position of the individual feature portion to be seen changes due to the influence of the unevenness of the face, so that the face image facing the front must always be used.
[0006]
Therefore, when a face image that does not face the front is used, it becomes difficult to collate using the face image. Therefore, there is a desire to be able to specify a feature area corresponding to an individual feature portion even if a face image that does not face the front is used.
[0007]
An example of an object of the present invention is a collation support device, a collation support method, and a computer-readable recording that can specify features that can be seen on the skin surface of the face of a person to be collated according to the orientation of the face of the captured image. It is to provide the medium.
Means to solve the problem
[0008]
In order to achieve the above object, the collation support device in one aspect of the present invention is
The first display information for displaying the first display area for displaying the image including the face of the person to be collated captured by the image pickup device on the screen of the display device is generated. Display information generation means and
A second display information for displaying a second display area for displaying a reference face development image generated based on three-dimensional data of a reference head on the screen of the display device is generated. Second display information generation means,
A first user interface used for an operation for allowing a user to specify a feature area representing the facial features of the person to be collated, which can be seen on the skin surface of the person to be collated, in the second display area. To generate the first user interface display information for displaying on the screen of the display device, and the user interface display information generation means.
It is characterized by having.
[0009]
Further, in order to achieve the above object, the collation support method in one aspect of the present invention is:
(A) The first display information for displaying the first display area for displaying the image including the face of the person to be collated captured by the image pickup device on the screen of the display device is generated.
(B) Generate a second display information for displaying a second display area for displaying a reference face development image generated based on the three-dimensional data of the reference head on the screen of the display device. death,
(C) A first operation used for an operation for allowing a user to specify a feature area representing the facial features of the person to be collated, which can be seen on the skin surface of the person to be collated, in the second display area. Generates first user interface display information for displaying the user interface of the above on the screen of the display device.
Characterized by that.
[0010]
Further, in order to achieve the above object, a computer-readable recording medium on which the program in one aspect of the present invention is recorded is
On the computer
(A) Generate the first display information for displaying the first display area for displaying the image including the face of the person to be collated captured by the image pickup device on the screen of the display device. Steps and
(B) Generate a second display information for displaying a second display area for displaying a reference face development image generated based on the three-dimensional data of the reference head on the screen of the display device. To do, step and
(C) A first operation used for an operation for allowing a user to specify a feature area representing the facial features of the person to be collated, which can be seen on the skin surface of the person to be collated, in the second display area. To generate the first user interface display information for displaying the user interface of the above on the screen of the display device, the step and
It is characterized by recording a program containing instructions to execute.
Effect of the invention
[0011]
As described above, according to the present invention, it is possible to specify the features that can be seen on the skin surface of the face of the person to be collated according to the orientation of the face of the captured image.
A brief description of the drawing
[0012]
FIG. 1 is a diagram for explaining an example of a collation support device.
FIG. 2 is a diagram for explaining a user interface used for collation support.
FIG. 3 is a diagram for explaining an example of a system having a collation support device.
FIG. 4 is a diagram for explaining an example of displaying a candidate face image.
FIG. 5 is a diagram for explaining an example of the operation of the collation support device.
FIG. 6 is a diagram for explaining an example of a system having a collation support device.
FIG. 7 is a diagram for explaining a user interface used for collation support.
FIG. 8 is a diagram for explaining an example of a display in which a feature region on a reference face three-dimensional image is converted into a feature region on a reference face developed image face image.
FIG. 9 is a diagram for explaining an example of displaying a candidate face image.
FIG. 10 is a diagram for explaining an example of displaying a candidate face image.
FIG. 11 is a diagram for explaining an example of the operation of the collation support device.
FIG. 12 is a diagram for explaining an example of a computer that realizes a collation support device.
Embodiment for carrying out the invention
[0013]
(Embodiment)
Hereinafter, embodiments of the present invention will be described with reference to FIGS. 1 to 12.
[0014]
[Device configuration]
First, the configuration of the collation support device 1 in the present embodiment will be described with reference to FIGS. 1 and 2. FIG. 1 is a diagram for explaining an example of a collation support device. FIG. 2 is a diagram for explaining a user interface used for collation support.
[0015]
The collation support device 10 shown in FIG. 1 is a device capable of designating features that can be seen on the skin surface of the face of a person to be collated according to the orientation of the face of the captured image. Further, as shown in FIG. 1, the collation support device 10 has a first display information generation unit 11, a second display information generation unit 12, and a user interface display information generation unit 13. Further, as shown in FIG. 1, the collation support device 10 is connected to an image pickup device 20, a storage device 21, an input device 22, and a display device 30.
[0016]
Of these, the first display information generation unit 11 displays a first display area for displaying a collation image including the face of a person to be collated imaged by the image pickup device 20 on the screen of the display device 30. Generate the first display information for display.
[0017]
The first display area is, for example, an area for displaying the collation image 41 on the user interface 40 shown in FIG. 2. The collated image 41 is a frame image of a still image or a moving image. In the example of FIG. 2, the collation image 41 is displayed in the window 42. However, the configuration of the display screen is not limited to FIG.
[0018]
The second display information generation unit 12 displays a second display area for displaying a reference face development image generated based on the three-dimensional data of the reference head on the screen of the display device 30. Generates the second display information of.
[0019]
The reference head is a head created by CG (Computer Graphics) based on the data of one or more heads measured or imaged in the past. Further, the reference head may be created based on the head of a specific person measured or imaged in the past.
[0020]
The second display area is, for example, an area for displaying the reference face development image 43 on the user interface 40 shown in FIG. The reference face development image 43 is a development image of a face projected in a cylindrical shape by, for example, performing a UV development process using the three-dimensional data of the reference head, which is stored in the storage device 21 in advance. However, the creation of the developed image of the face is not limited to the above-mentioned cylindrical projection. In the example of FIG. 2, the reference face development image 43 is displayed in the window 44. However, the configuration of the display screen is not limited to FIG.
[0021]
The user interface display information generation unit 13 is an operation for causing the user to specify a feature area representing the facial features of the person to be collated, which can be seen on the skin surface of the person to be collated, in the second display area. The first user interface display information for displaying the first user interface to be used on the screen of the display device 30 is generated.
[0022]
The characteristics of a person's face include, for example, parts that represent the characteristics of a person that can be seen on the surface of the skin, such as hokuro, freckles, tattoos, wrinkles, wrinkles, dents, scratches, blemishes, bumps, unevenness of the skin, and discolored parts of the skin. In the example of FIG. 2, since there is a skull 51 on the left cheek of the person captured in the collation image 41, this skull 51 is a feature.
[0023]
The feature area is a mark attached by the user on the standard face development image after the user recognizes the feature of the person's face, and is an area corresponding to the feature recognized by the user. In the example of FIG. 2, the region corresponding to the mark (x) on the reference face development image 43 is the feature region 52. In addition, a region having no feature on the face of a person may be used as a feature region.
[0024]
The first user interface corresponds to the user interface 40 in the example of FIG. For example, when the feature area 52 is drawn (specified) using the user interface 40 and the input device 22 which is a physical user interface such as a mouse, a touch panel, and a keyboard, it is displayed on the user interface 40. When the "Add" button 45 is selected, the drawn feature area 52 is added to the reference face development image 43.
[0025]
As described above, in the present embodiment, the user observes the collation image 41 and the user designates the feature area 52 on the reference face development image 43, so that the person in the collation image 41 does not face the front. Even in this case, the designated feature area 52 can be used to provide collation support that leads to the identification of the person to be collated. The collation support is a process of selecting a person having a feature at the same position as the person to be collated by using the feature area 52 designated on the reference face development image 43.
[0026]
Further, by designating the feature area 52 using the reference face development image 43, the influence of the unevenness of the face due to the change in the face orientation can be reduced, so that even if the position for only the hokuro 51 changes, the user can use it. , The feature area 52 corresponding to the hokuro 51 can be easily designated on the reference face development image 43.
[0027]
[System configuration]
Subsequently, the configuration of the collation support device 10 in the present embodiment will be described more specifically with reference to FIG. FIG. 3 is a diagram for explaining an example of a system having a collation support device.
[0028]
As shown in FIG. 3, the system in the present embodiment includes a collation support device 10, an image pickup device 20, a storage device 21, an input device 22, and a display device 30. For example, the system may be a monitoring system, an authentication system, or the like. Further, the collation support device 10 in FIG. 3 includes a candidate face image display information generation unit 14 in addition to the first display information generation unit 11, the second display information generation unit 12, and the user interface display information generation unit 13. Collate with the generator 61 The first display information for displaying the first display area for displaying the collation image captured by using 20 on the screen of the display device 30 is generated (step A1). Specifically, in step A1, the first display information generation unit 11 acquires an image of a person captured by the image pickup device 20.
[0060]
Subsequently, in step A1, the first display information generation unit 11 displays the collation image 41 as shown in FIG. 2 on the screen of the display device 30 based on the acquired image. Generate information. After that, in step A1, the first display information generation unit 11 transmits the first display information to the display device 30.
[0061]
Note that only the frame image whose features are easy to see may be used as the collation image. Further, a frame image in which a feature that was not visible before the direction of the face is changed may be used as a collation image.
[0062]
Further, the second display information generation unit 12 displays a second display area for displaying the reference face development image based on the three-dimensional data of the reference head on the screen of the display device 30. Generate the second display information (step A2). Specifically, in step A2, the second display information generation unit 12 acquires three-dimensional data of the reference head from the storage device 21.
[0063]
Subsequently, in step A2, the second display information generation unit 12 generates a reference face development image using the three-dimensional data of the reference head. Subsequently, in step A2, the second display information generation unit 12 displays the reference face development image 43 as shown in FIG. 2 on the screen of the display device 30 based on the generated reference face development image. Generates the second display information of. After that, in step A2, the second display information generation unit 12 transmits the second display information to the display device 30.
[0064]
When the reference face development image is stored in the storage device 21, the second display information generation unit 12 may directly acquire the reference face development image from the storage device 21.
[0065]
The order of the process of step A1 and the process of step A2 described above may be reversed, or the processes may be executed in parallel.
[0066]
Next, the user interface display information generation unit 13 makes the user refer to the first display area, and sets the first user interface for designating the feature area as the second display area on the screen of the display device 30. Generates the first user interface display information to be displayed in (step A3).
[0067]
Specifically, in step A3, the user interface display information generation unit 13 displays the user interface 40 as shown in FIG. 2 as the first user interface, and causes the user to display the feature area on the reference face development image. To be specified.
[0068]
Note that the user may observe one collation image and specify a plurality of feature areas on the reference face development image. Further, the user may observe a plurality of collated images (a plurality of frame images) having different face orientations and specify one or more feature areas on the reference face development image. For example, one or more feature regions may be designated on the reference face development image by observing a plurality of collated images having different face orientations. By doing so, the position of the feature area becomes accurate, and the number of feature areas can be increased.
[0069]
Furthermore, the feature area may be specified automatically. In that case, the detection unit 64 automatically detects the feature area from the image including the face of the person to be collated displayed on the screen of the display device 30, and the association unit 65 determines the position of the detected feature area. It is automatically associated with the corresponding position of the reference face development image.
[0070]
Next, the generation unit 61 uses a second display area for displaying the reference face development image generated based on the three-dimensional data of the reference head, which is displayed on the screen of the display device 30. When a characteristic area representing the facial features of the person to be collated that can be seen on the skin surface of the person to be collated is designated, the feature information regarding the characteristic area is generated (step A4).
[0071]
Specifically, in step A4, first, when one or more feature regions are designated for the reference face development image by using the first user interface, the generation unit 61 features each of the designated feature regions. Generate information. After that, in step A4, the generation unit 61 outputs the feature information to the collation unit 62.
[0072]
In the example of FIG. 2, when the feature area 52 is designated for the reference face development image 43, the generation unit 61 generates the feature information of the designated feature area 52 by using the user interface 40.
[0073]
The collation unit 62 collates the feature information with the collation information associated with the collation face development image for each person registered in advance and the collation feature area (step A5). Specifically, in step A5, the collation unit 62 first acquires feature information from the generation unit 61. Subsequently, in step A5, the collation unit 62 refers to each of the collation information stored in the storage device 21 using the acquired feature information, executes a collation process, and calculates a collation result. After that, in step A5, the collation unit 62 associates the collation information with the calculated collation result.
[0074]
The selection unit 63 selects a candidate person based on the collation result (step A6). Specifically, in step A6, the selection unit 63 selects collation information having a collation index equal to or higher than a preset threshold. Subsequently, in step A6, the selection unit 63 outputs the selected collation information to the candidate face image display information generation unit 14.
[0075]
The candidate face image display information generation unit 14 includes collation information in which a feature area designated by using the first user interface is associated with a pre-registered collation face development image for each person and a collation feature area. When a candidate person is selected based on the matching result, the matching face development image corresponding to the selected candidate person is displayed on the screen of the display device 30 together with the matching feature area. Candidate face image display information is generated (step A7).
[0076]
Specifically, in step A7, when the candidate face image display information generation unit 14 first selects a candidate person by the selection unit 63 based on the collation result of the collation unit 62, the candidate face image display information generation unit 14 from the storage device 21. Acquires the matching information in which the matching face development image corresponding to the selected candidate person and the matching feature area are associated with each other.
[0077]
Subsequently, in step A7, the candidate face image display information generation unit 14 uses the acquired collation information to obtain the collation face development image 71 and the collation feature area 73 as shown in the window 72 of FIG. Candidate face image display information for display on the screen of the display device 30 is generated. After that, the candidate face image display information generation unit 14 transmits the candidate face image display information to the display device 30.
[0078]
If there are multiple candidate persons, the collation face development images are displayed in order according to the collation result. For example, the collation index (score) represented by the collation result is displayed in descending order.
[0079]
Next, the display device 30 acquires the candidate face image display information and displays the matching face development image 71 and the matching feature area 73 as shown in the window 72 of FIG. 4 on the screen (step A8). ).
[0080] [0080]
[Effect of this embodiment]
As described above, according to the present embodiment, the user observes the collated image and the user specifies the feature area on the reference face development image, so that even if the person in the collated image does not face the front. Using the specified feature area, it is possible to provide collation support that leads to the identification of the person to be collated.
[0081]
In addition, by designating the feature area using the reference face development image, the influence of the unevenness of the face due to the change in the face orientation can be reduced, so even if the position for only the feature changes, the user can use the feature. The corresponding feature area can be easily specified on the reference face development image.
[0082]
Further, a collation device that extracts an image corresponding to a person from a frame image captured by the image pickup device and performs collation using the extracted image may be linked with a collation support device. In that case, when an image corresponding to the feature is detected from any of the frame images, processing is performed from a collation processing mode in which collation is performed using a collation device to a collation support processing mode in which collation support is performed using a collation support device. May be switched.
[0083]
In addition, when it is determined that the person registered in advance and the person captured are the same person, the matching face development image and the matching feature corresponding to the person are based on the characteristics of the captured person. You may edit the area.
[0084]
In addition, when it is determined that the pre-registered person and the imaged person are not the same person, the person is dealt with based on the characteristic showing the difference between the pre-registered person and the imaged person. You may edit the collation face development image and the collation feature area.
[0085]
[program]
The program according to the embodiment of the present invention may be any program that causes a computer to execute steps A1 shown in FIG. By installing and executing this program on a computer, the collation support device and the collation support method in the present embodiment can be realized. In this case, the computer processor has a first display information generation unit 11, a second display information generation unit 12, a user interface display information generation unit 13, a candidate face image display information generation unit 14, a generation unit 61, and a collation unit 62. , Functions as a selection unit 63, a detection unit 64, and an association unit 65, and performs processing.
[0086]
Further, the program in the present embodiment may be executed by a computer system constructed by a plurality of computers. In this case, for example, each computer has its own processor, that is, the first display information generation unit 11, the second display information generation unit 12, the user interface display information generation unit 13, and the candidate face image display information generation unit. It may function as any of 14, the generation unit 61, the collation unit 62, the selection unit 63, the detection unit 64, and the association unit 65.
[0087]
(Variation example)
Hereinafter, modifications of the present invention will be described with reference to FIGS. 6 to 11.
[0088]
[System configuration]
A modified example will be described with reference to FIG. FIG. 6 is a diagram for explaining an example of a system having a collation support device.
[0089]
As shown in FIG. 6, the collation support device 10 in the modified example includes a first display information generation unit 11, a second display information generation unit 12, a user interface display information generation unit 13, and a candidate face image display information generation unit 14. In addition to the generation unit 61, the collation unit 62, the selection unit 63, the detection unit 64, and the association unit 65, the third display information generation unit 15 and the conversion unit 66 are included.
[0090]
In the modified example, when the storage device 21 further displays the reference face three-dimensional image in the third display area, the storage device 21 transmits the reference three-dimensional data of the head to the collation support device 10 via the communication network. .. The third display area will be described later. When the reference face three-dimensional image is stored, the storage device 21 transmits the reference face development image to the collation support device 10.
[0091]
The third display information generation unit 15 sets the face orientation of the reference face three-dimensional image generated based on the three-dimensional data of the reference head to the person to be collated displayed in the first display area. A third display information for displaying the third display area to be displayed according to the orientation of the face on the screen of the display device 30 is generated. Specifically, the third display information generation unit 15 acquires three-dimensional data of the reference head from the storage device 21.
[0092]
Subsequently, the third display information generation unit 15 generates a reference face three-dimensional image using the three-dimensional data of the reference head. Subsequently, based on the generated reference face three-dimensional image, a third display information for displaying the reference face three-dimensional image 81 as shown in FIG. 6 on the screen of the display device 30 is generated. After that, the third display information generation unit 15 transmits the third display information to the display device 30. When the reference face three-dimensional image is stored in the storage device 21, the third display information generation unit 15 may directly acquire the reference face three-dimensional image from the storage device 21.
[0093] The orientation of the face may be adjusted manually by the user using the user interface or may be adjusted automatically.
[0094]
In the modification, the user interface display information generation unit 13 displays the feature area in the third display area and the second user interface used for the operation for designating the user on the screen of the display device 30. Generates the second user interface display information of.
[0095]
A specific explanation will be given using FIG. FIG. 7 is a diagram for explaining a user interface used for collation support.
[0096]
The user interface display information generation unit 13 displays the user interface 80 as shown in FIG. 7 as the second user interface, and causes the user to specify the feature area on the reference face three-dimensional image. In the example of FIG. 7, the reference face three-dimensional image 81 is displayed in the window 82. However, the configuration of the display screen is not limited to FIG. 7.
[0097]
Note that the user may observe one collation image and specify a plurality of feature areas on the reference face three-dimensional image. Further, the user may observe a plurality of collated images (plural frame images) having different face orientations and specify one or more feature regions on the reference face three-dimensional image. For example, one or more feature regions may be designated on the reference face three-dimensional image by observing a plurality of collated images having different face orientations. By doing so, the position of the feature area becomes accurate, and the number of feature areas can be increased.
[0098]
Further, in the modified example, for example, when the user selects the reference face three-dimensional image 81 shown in FIG. 7, and then the “add” button 45 is selected, the feature drawn on the reference face three-dimensional image 81 is selected. Region 83 can be added. When the "Delete" button 46 is selected, the feature area 83 drawn on the reference face three-dimensional image 81 can be deleted. When the "save" button 47 is selected, feature information (for example, texture information, position information, feature type information, etc.) related to the designated feature area 83 is stored in the storage unit. When the "feature extraction" button 48 is selected, features are automatically extracted from the collation image 41. When the "enlargement / reduction" button 49 is selected, the display of the selected collation image 41, the reference face development image 43, or the reference face three-dimensional image 81 is enlarged or reduced. However, the editing function is not limited to the above-mentioned function.
[0099]
In the modification, the generation unit 61 provides a third display area for displaying the reference face three-dimensional image generated based on the reference three-dimensional data of the head, which is displayed on the screen of the display device 30. When a characteristic area representing the facial features of the person to be collated that can be seen on the skin surface of the person to be collated is designated, the characteristic information about the characteristic area is generated.
[0100]
Specifically, the generation unit 61 first uses the second user interface to generate feature information for each of the designated feature regions when one or more feature regions are designated for the reference face three-dimensional image. do. After that, the generation unit 61 outputs the feature information to the collation unit 62. In the example of FIG. 7, the generation unit 61 uses the user interface 80 to generate the feature information of the designated feature region 83 when the feature region 83 is designated for the reference face three-dimensional image 81.
[0101]
The conversion unit 66 displays the face orientation of the reference face three-dimensional image generated based on the three-dimensional data of the reference head in accordance with the orientation of the face of the person to be collated. When a feature area is designated using the display area, the feature information related to the feature area designated in the reference face three-dimensional image is converted into the feature information used in the reference face development image.
[0102]
The explanation of the collation unit 62 and the selection unit 63 has been described above and will be omitted.
[0103]
A specific explanation will be given using FIG. FIG. 8 is a diagram for explaining an example of a display in which a feature region on a reference face three-dimensional image is converted into a feature region on a reference face developed image face image.
[0104]
In FIG. 8, when the feature area 83 is added, the feature area 86 corresponding to the feature area 83 is also added on the reference face development image 84. In the example of FIG. 8, the reference face development image 84 is displayed in the window 85. However, the configuration of the display screen is not limited to FIG.
[0105]
In the modification, the candidate face image display information generation unit 14 associates the feature area designated by using the second user interface with the pre-registered matching face development image for each person and the matching feature area. When a candidate person is selected based on the collation result, the collation face development image corresponding to the selected candidate person is displayed on the screen of the display device 30 together with the feature area. Generate candidate face image display information for making.
[0106]
A specific explanation will be given using FIG. FIG. 9 is a diagram for explaining an example of displaying a candidate face image.
[0107]
First, when a candidate person is selected by the selection unit 63 based on the collation result of the collation unit 62, the candidate face image display information generation unit 14 corresponds to the selected candidate person from the storage device 21. Acquires the matching information in which the matching face development image and the matching feature area are associated with each other.
[0108]
Subsequently, the candidate face image display information generation unit 14 displays the collation face development image 91 and the collation feature area 93 as shown in the window 92 of FIG. 9 on the display device 30 based on the acquired collation information. Generate candidate face image display information to be displayed on the screen. After that, the candidate face image display information generation unit 14 transmits the candidate face image display information to the display device 30.
[0109]
If there are multiple candidate persons, the collation face development images are displayed in order according to the collation result. For example, the collation index (score) represented by the collation result is displayed in descending order.
[0110]
Further, in the modified example, the candidate face image display information generation unit 14 displays a collation face three-dimensional image corresponding to the candidate person based on the three-dimensional data of the head of the selected candidate person. Generates candidate face three-dimensional image display information to be displayed on the screen of the device 30.
[0111]
A specific explanation will be given with reference to FIG. FIG. 10 is a diagram for explaining an example of displaying a candidate face image.
[0112]
First, when a candidate person is selected by the selection unit 63 based on the collation result of the collation unit 62, the candidate face image display information generation unit 14 corresponds to the selected candidate person from the storage device 21. Acquires the matching information in which the matching face three-dimensional image and the matching feature area are associated with each other.
[0113]
Subsequently, the candidate face image display information generation unit 14 creates a matching face three-dimensional image 91'and a matching feature area 93'as shown in the window 92'of FIG. 10 based on the acquired matching information. Candidate face image display information for display on the screen of the display device 30 is generated. After that, the candidate face image display information generation unit 14 transmits the candidate face image display information to the display device 30.
[0114]
In addition, one or more of the above-mentioned collation image, reference face development image, reference face three-dimensional image, collation face expansion image including the collation feature area, and collation face three-dimensional image including the collation feature area are combined. It may be displayed on the user interface.
[0115]
[Device operation]
The operation of the modified example will be described with reference to FIG. FIG. 11 is a diagram for explaining an example of the operation of the collation support device. In the following description, FIGS. 1 to 10 will be referred to as appropriate. Further, in the modified example, the collation support method is implemented by operating the collation support device. Therefore, the description of the collation support method in the modified example is replaced with the following operation description of the collation support device.
[0116]
As shown in FIG. 11, first, the process of step A1 described above is executed.
[0117]
The third display information generation unit 15 sets the face orientation of the reference face three-dimensional image generated based on the three-dimensional data of the reference head to the person to be collated displayed in the first display area. A third display information for displaying the third display area to be displayed according to the orientation of the face on the screen of the display device 30 is generated (step B1). Specifically, the third display information generation unit 15 acquires three-dimensional data of the reference head from the storage device 21.
[0118]
Subsequently, in step B1, the third display information generation unit 15 generates a reference face three-dimensional image using the three-dimensional data of the reference head. Subsequently, in step B1, the third display information generation unit 15 displays the reference face three-dimensional image 81 as shown in FIG. 7 on the screen of the display device 30 based on the generated reference face three-dimensional image. Generate a third display information to make it. After that, in step B1, the third display information generation unit 15 transmits the third display information to the display device 30.
[0119]
When the reference face three-dimensional image is stored in the storage device 21, the third display information generation unit 15 may directly acquire the reference face three-dimensional image from the storage device 21.
[0120]
The orientation of the face may be adjusted manually by the user or automatically.
[0121]
The order of the process of step A1 and the process of step B1 described above may be reversed, or the processes may be executed in parallel.
[0122]
Next, the user interface display information generation unit 13 is for displaying the feature area in the third display area and the second user interface used for the operation for designating the user on the screen of the display device 30. Generate the second user interface display information (step B2).
[0123]
Specifically, in step B2, the user interface display information generation unit 13 displays the user interface 80 as shown in FIG. 7 as the second user interface, and features the user on the reference face three-dimensional image. Have the area specified. In the example of FIG. 7, the reference face three-dimensional image 81 is displayed in the window 82. However, the configuration of the display screen is not limited to FIG. 7.
[0124]
Note that the user may observe one collation image and specify a plurality of feature areas on the reference face three-dimensional image. Further, the user may observe a plurality of collated images (plural frame images) having different face orientations and specify one or more feature regions on the reference face three-dimensional image. For example, one or more feature regions may be designated on the reference face three-dimensional image by observing a plurality of collated images having different face orientations. By doing so, the position of the feature area becomes accurate and the number of feature areas can be increased, so that the collation accuracy is improved.
[0125]
Furthermore, the feature area may be specified automatically. In that case, the detection unit 64 automatically detects the feature area from the image including the face of the person to be collated displayed on the screen of the display device 30, and the association unit 65 determines the position of the detected feature area. It is automatically associated with the corresponding position of the reference face three-dimensional image.
[0126]
Next, the generation unit 61 uses a third display area for displaying the reference face three-dimensional image generated based on the reference three-dimensional data of the head, which is displayed on the screen of the display device 30. When a characteristic area representing the facial features of the person to be collated that can be seen on the skin surface of the person to be collated is designated, the feature information related to the characteristic area is generated (step B3).
[0127]
Specifically, in step B3, the generation unit 61 first uses the second user interface to specify one or more feature regions in the reference face three-dimensional image, for each of the designated feature regions. Generate feature information. After that, in step B3, the generation unit 61 outputs the feature information to the collation unit 62. In the example of FIG. 7, the generation unit 61 uses the user interface 80 to generate the feature information of the designated feature region 83 when the feature region 83 is designated for the reference face three-dimensional image 81.
[0128]
Next, the conversion unit 66 displays the orientation of the face of the reference face three-dimensional image generated based on the three-dimensional data of the reference head in accordance with the orientation of the face of the person to be collated. When a feature area is specified using the three-dimensional image display area, the feature information related to the feature area specified in the reference face three-dimensional image is used in the reference face development image.Is converted to (step B4).
[0129]
Next, the processes of steps A5 and A6 described above are executed.
[0130]
Next, the candidate face image display information generation unit 14 associates the feature area designated by using the second user interface with the collation face development image and the collation feature area for each person registered in advance. When a candidate person is selected based on the collation result by collating with the collation information, the collation face development image corresponding to the selected candidate person is displayed on the screen of the display device 30 together with the feature area. Generate candidate face image display information for this purpose (step B5).
[0131]
In step B5, when the candidate face image display information generation unit 14 first selects a candidate person by the selection unit 63 based on the collation result of the collation unit 62, the candidate face image display information generation unit 14 selects the candidate from the storage device 21. Acquires the matching information in which the matching face development image corresponding to the person and the matching feature area are associated with each other.
[0132]
Subsequently, in step B5, the candidate face image display information generation unit 14 uses the acquired collation information to obtain the collation face development image 91 and the collation feature area 93 as shown in the window 92 of FIG. Candidate face image display information for display on the screen of the display device 30 is generated. After that, the candidate face image display information generation unit 14 transmits the candidate face image display information to the display device 30.
[0133]
If there are multiple candidate persons, the collation face development images are displayed in order according to the collation result. For example, the collation index (score) represented by the collation result is displayed in descending order.
[0134]
Further, in step B5, the candidate face image display information generation unit 14 displays a collation face three-dimensional image corresponding to the candidate person based on the three-dimensional data of the head of the selected candidate person. Candidate face three-dimensional image display information for display on the screen of the device 30 may be generated.
[0135]
In step B5, when the candidate face image display information generation unit 14 first selects a candidate person by the selection unit 63 based on the collation result of the collation unit 62, the candidate face image display information generation unit 14 selects the candidate from the storage device 21. Acquires the matching information in which the matching face three-dimensional image corresponding to the person and the matching feature area are associated with each other.
[0136]
Subsequently, in step B5, the candidate face image display information generation unit 14 has the matching face three-dimensional image 91 ′ and the matching feature area 93 ′ as shown in the window 92 ′ of FIG. 10 based on the acquired matching information. ´ and are generated as candidate face image display information for displaying on the screen of the display device 30. After that, the candidate face image display information generation unit 14 transmits the candidate face image display information to the display device 30.
[0137]
If there are multiple candidate persons, the collation face development images are displayed in order according to the collation result. For example, the collation index (score) represented by the collation result is displayed in descending order.
[0138]
Next, the display device 30 acquires the candidate face image display information and displays the matching face development image 91 and the matching feature area 93 as shown in the window 92 of FIG. 9 on the screen (step B6). ). Alternatively, in step B6, the display device 30 may display the matching face development image 91'and the matching feature area 93'as shown in the window 92'of FIG. 10 on the screen.
[0139]
[Effect of modified example]
According to the modified example as described above, the user observes the collated image and the user specifies the feature area on the reference face three-dimensional image, so that even if the person in the collated image does not face the front. Using the specified feature area, it is possible to provide collation support that leads to the identification of the person to be collated.
[0140]
In addition, by designating the feature area using the reference face three-dimensional image, the influence of the unevenness of the face due to the change in the face orientation can be reduced, so even if the position for only the feature changes, the user can use the feature. The feature area corresponding to can be easily specified on the reference face three-dimensional image.
[0141]
Further, a collation device that extracts an image corresponding to a person from a frame image captured by the image pickup device and performs collation using the extracted image may be linked with a collation support device. In that case, when an image corresponding to the feature is detected from any of the frame images, the collation support processing mode that supports collation using the collation support device from the collation processing mode that collates using the current collation device. The process may be switched to.
[0142]
In addition, when it is determined that the person registered in advance and the person captured are the same person, the matching face development image and the matching feature corresponding to the person are based on the characteristics of the captured person. You may edit the area.
[0143]
In addition, when it is determined that the pre-registered person and the imaged person are not the same person, the person is dealt with based on the characteristic showing the difference between the pre-registered person and the imaged person. You may edit the collation face development image and the collation feature area.
[0144]
[program]
The program in the modified example of the present invention may be any program that causes a computer to execute from step A1 shown in FIG. By installing and executing this program on a computer, the collation support device and the collation support method in the present embodiment can be realized. In this case, the computer processor has a first display information generation unit 11, a second display information generation unit 12, a user interface display information generation unit 13, a candidate face image display information generation unit 14, and a third display information generation unit. 15, It functions as a generation unit 61, a collation unit 62, a selection unit 63, a detection unit 64, an association unit 65, and a conversion unit 66, and performs processing.
[0145]
Further, the program in the present embodiment may be executed by a computer system constructed by a plurality of computers. In this case, for example, each computer has its own processor, that is, the first display information generation unit 11, the second display information generation unit 12, the user interface display information generation unit 13, and the candidate face image display information generation unit. It may function as any of 14, the third display information generation unit 15, the generation unit 61, the collation unit 62, the selection unit 63, the detection unit 64, the association unit 65, and the conversion unit 66.
[0146]
[Physical configuration]
Here, a computer that realizes a collation support device by executing the programs in the embodiment and the modified example will be described with reference to FIG. 12. FIG. 12 is a block diagram showing an example of a computer that realizes a collation support device according to an embodiment and a modification of the present invention.
[0147]
As shown in FIG. 12, the computer 110 includes a CPU 111, a main memory 112, a storage device 113, an input interface 114, a display controller 115, a data reader / writer 116, and a communication interface 117. Each of these parts is connected to each other via a bus 121 so as to be capable of data communication. The computer 110 may include a GPU (Graphics Processing Unit) or an FPGA in addition to or in place of the CPU 111.
[0148]
The CPU 111 expands the program (code) in the present embodiment stored in the storage device 113 into the main memory 112, and executes these in a predetermined order to perform various calculations. The main memory 112 is typically a volatile storage device such as a DRAM (Dynamic Random Access Memory). Further, the program in the present embodiment is provided in a state of being stored in a computer-readable recording medium 120. The program in the present embodiment may be distributed on the Internet connected via the communication interface 117. The recording medium 120 is a non-volatile recording medium.
[0149]
Further, specific examples of the storage device 113 include a semiconductor storage device such as a flash memory in addition to a hard disk drive. The input interface 114 mediates data transmission between the CPU 111 and an input device 118 such as a keyboard and a mouse. The display controller 115 is connected to the display device 119 and controls the display on the display device 119.
[0150]
The data reader / writer 116 mediates the data transmission between the CPU 111 and the recording medium 120, reads the program from the recording medium 120, and writes the processing result in the computer 110 to the recording medium 120. The communication interface 117 mediates data transmission between the CPU 111 and another computer.
[0151]
Specific examples of the recording medium 120 include general-purpose semiconductor storage devices such as CF (Compact Flash (registered trademark)) and SD (Secure Digital), magnetic recording media such as Flexible Disk, or CD-. Examples include optical recording media such as ROM (Compact Disk Read Only Memory).
[0152]
The collation support device in the present embodiment can also be realized by using the hardware corresponding to each part instead of the computer in which the program is installed. Further, the collation support device may be partially realized by a program and the rest by hardware.
[0153]
[Additional Notes]
Regarding the above embodiments, the following additional notes will be further disclosed. A part or all of the above-described embodiments can be expressed by the following descriptions (Appendix 1) to (Appendix 18), but the description is not limited to the following.
[0154]
(Appendix 1)
The first display information for displaying the first display area for displaying the image including the face of the person to be collated captured by the image pickup device on the screen of the display device is generated. Display information generator and
A second display information for displaying a second display area for displaying a reference face development image generated based on three-dimensional data of a reference head on the screen of the display device is generated. The second display information generator and
A first user interface used for an operation for causing a user to specify a feature area showing features of the face of the person to be collated, which can be seen on the skin surface of the person to be collated, in the second display area. With a user interface display information generation unit that generates a first user interface display information for displaying on the screen of the display device.
A collation support device characterized by having.
[0155]
(Appendix 2)
The collation support device described in Appendix 1
The feature area specified by using the first user interface is collated with the collation information associated with the collation face development image for each person registered in advance and the collation feature area, and based on the collation result. When a candidate person is selected, a candidate face image display information for displaying a matching face development image corresponding to the selected candidate person on the screen of the display device together with the matching feature area is generated. Candidate face image display information generation unit
A collation support device characterized by having.
[0156]
(Appendix 3)
The collation support device described in Appendix 2
The orientation of the face of the reference face three-dimensional image generated based on the three-dimensional data of the reference head is matched with the orientation of the face of the person to be collated displayed in the first display area. A third display information generation unit that generates a third display information for displaying the third display area to be displayed on the screen of the display device.
A collation support device characterized by having.
[0157]
(Appendix 4)
The collation support device described in Appendix 3
The user interface display information generation unit is for displaying the feature area in the third display area and the second user interface used for the operation for designating the user on the screen of the display device. Generate second user interface display information
A collation support device characterized by that.
[0158]
(Appendix 5)
The collation support device described in Appendix 4,
The candidate face image display information generation unit is the first The feature area specified using the second user interface is collated with the collation information associated with the collation face development image for each person registered in advance and the collation feature area, and a candidate is selected based on the collation result. When a person is selected, a candidate face image display information for displaying a matching face development image corresponding to the selected candidate person on the screen of the display device together with the feature area is generated.
A collation support device characterized by that.
[0159]
(Appendix 6)
The collation support device described in any one of Appendix 2 to 5.
The candidate face image display information generation unit displays a matching face three-dimensional image corresponding to the candidate person on the screen of the display device based on the three-dimensional data of the head of the selected candidate person. Generate candidate face 3D image display information for display in
A collation support device characterized by that.
[0160]
(Appendix 7)
(A) Generate the first display information for displaying the first display area for displaying the image including the face of the person to be collated captured by the image pickup device on the screen of the display device. Steps and
(B) Generate a second display information for displaying a second display area for displaying a reference face development image generated based on the three-dimensional data of the reference head on the screen of the display device. To do, step and
(C) A first operation used for an operation for allowing a user to specify a feature area representing the facial features of the person to be collated, which can be seen on the skin surface of the person to be collated, in the second display area. To generate the first user interface display information for displaying the user interface of the above on the screen of the display device, the step and
A collation support method characterized by having.
[0161]
(Appendix 8)
The collation support method described in Appendix 7
(D) The feature area designated by using the first user interface is collated with the collation information associated with the collation face development image for each person registered in advance and the collation feature area, and the collation result is obtained. When a candidate person is selected based on the above, a candidate face image display for displaying a matching face development image corresponding to the selected candidate person on the screen of the display device together with the matching feature area. Steps to generate information
A collation support method characterized by having.
[0162]
(Appendix 9)
The collation support method described in Appendix 8
(E) The orientation of the face of the reference face three-dimensional image generated based on the three-dimensional data of the reference head is the orientation of the face of the person to be collated displayed in the first display area. A step of generating a third display information for displaying the third display area to be displayed according to the above on the screen of the display device, and
A collation support method characterized by having.
[0163]
(Appendix 10)
The collation support method described in Appendix 9
(F) Second user interface display for displaying the second user interface used for the operation for designating the feature area in the third display area to the user on the screen of the display device. Steps to generate information
A collation support method characterized by having.
[0164]
(Appendix 11)
The collation support method described in Appendix 10
(G) The feature area specified by using the second user interface is collated with the collation information associated with the collation face development image for each person registered in advance and the collation feature area, and the collation result is obtained. When a candidate person is selected based on the above, the candidate face image display information for displaying the matching face development image corresponding to the selected candidate person on the screen of the display device together with the feature area is displayed. Generate, step
A collation support method characterized by having.
[0165]
(Appendix 12)
The collation support method described in any one of Appendix 8 to 11
(H) Candidate face for displaying a collation face three-dimensional image corresponding to the candidate person on the screen of the display device based on the three-dimensional data of the head of the selected candidate person. Steps to generate 3D image display information
A collation support method characterized by having.
[0166]
(Appendix 13)
On the computer
(A) Generate the first display information for displaying the first display area for displaying the image including the face of the person to be collated captured by the image pickup device on the screen of the display device. Steps and
(B) Generate a second display information for displaying a second display area for displaying a reference face development image generated based on the three-dimensional data of the reference head on the screen of the display device. To do, step and
(C) A first operation used for an operation for allowing a user to specify a feature area representing the facial features of the person to be collated, which can be seen on the skin surface of the person to be collated, in the second display area. To generate the first user interface display information for displaying the user interface of the above on the screen of the display device, the step and
A computer-readable recording medium recording a program, including instructions to execute.
[0167]
(Appendix 14)
The computer-readable recording medium described in Appendix 13, which is a computer-readable recording medium.
The program is on the computer
(D) The feature area designated by using the first user interface is collated with the collation information associated with the collation face development image for each person registered in advance and the collation feature area, and the collation result is obtained. When a candidate person is selected based on the above, a candidate face image display for displaying a matching face development image corresponding to the selected candidate person on the screen of the display device together with the matching feature area. Generate information, include additional instructions to perform steps,
A computer-readable recording medium on which the program is recorded.
[0168]
(Appendix 15)
The computer-readable recording medium described in Appendix 14,
The program is on the computer
(E) The orientation of the face of the reference face three-dimensional image generated based on the three-dimensional data of the reference head is the orientation of the face of the person to be collated displayed in the first display area. Further includes an instruction to execute a step, which generates a third display information for displaying the third display area to be displayed in accordance with the screen of the display device.
A computer-readable recording medium on which the program is recorded.
[0169]
(Appendix 16)
The computer-readable recording medium described in Appendix 15, which is a computer-readable recording medium.
The program is on the computer
(F) Second user interface display for displaying the second user interface used for the operation for designating the feature area in the third display area to the user on the screen of the display device. Generate information, include additional instructions to perform steps,
A computer-readable recording medium on which the program is recorded.
[0170]
(Appendix 17)
The computer-readable recording medium described in Appendix 16
The program is on the computer
(G) The feature area specified by using the second user interface is collated with the collation information associated with the collation face development image for each person registered in advance and the collation feature area, and the collation result is obtained. When a candidate person is selected based on the above, the candidate face image display information for displaying the matching face development image corresponding to the selected candidate person on the screen of the display device together with the feature area is displayed. Generate, include more instructions to execute the step,
A computer-readable recording medium on which the program is recorded.
[0171]
(Appendix 18)
A computer-readable recording medium according to any one of Supplementary note 14 to 17.
The program is on the computer
(H) Candidate face for displaying a collation face three-dimensional image corresponding to the candidate person on the screen of the display device based on the three-dimensional data of the head of the selected candidate person. Generates 3D image display information, further includes instructions to execute steps,
A computer-readable recording medium on which the program is recorded.
[0172]
Although the present invention has been described above with reference to the embodiment, the present invention is not limited to the above embodiment. Various changes that can be understood by those skilled in the art can be made within the scope of the present invention in terms of the structure and details of the present invention.
Industrial availability
[0173]
As described above, according to the present invention, it is possible to specify the features that can be seen on the skin surface of the face of the person to be collated according to the orientation of the face of the captured image. The present invention is useful in fields such as monitoring systems and authentication systems that require verification.
Description of the sign
[0174]
10 Collation support device
11 First display information generation unit
12 Second display information generation unit
13 User interface display information generation unit
14 Candidate face image display information generation unit
15 Third display information generation unit
20 Imaging device
21 Storage device
22 Input device
30 display device
61 generator
62 Collation section
63 Selection section
64 detector
65 Association part
66 conversion unit
110 computer
111 CPU
112 Main memory
113 Storage device
114 input interface
115 Display controller
116 Data reader / writer
117 communication interface
118 Input device
119 Display device
120 Recording medium
121 Bus
The scope of the claims
[Claim 1]
The first display information for displaying the first display area for displaying the image including the face of the person to be collated captured by the image pickup device on the screen of the display device is generated. Display information generation means and
A second display information for displaying a second display area for displaying a reference face development image generated based on three-dimensional data of a reference head on the screen of the display device is generated. Second display information generation means,
A first user interface used for an operation for allowing a user to specify a feature area representing the facial features of the person to be collated, which can be seen on the skin surface of the person to be collated, in the second display area. To generate the first user interface display information for displaying on the screen of the display device, and the user interface display information generation means.
A collation support device characterized by having.
[Claim 2]
The collation support device according to claim 1.
The feature area specified by using the first user interface is collated with the collation information associated with the collation face development image for each person registered in advance and the collation feature area, and based on the collation result. When a candidate person is selected, a candidate face image display information for displaying a matching face development image corresponding to the selected candidate person on the screen of the display device together with the matching feature area is generated. Candidate face image display information generation means
A collation support device characterized by having.
[Claim 3]
The collation support device according to claim 2,
The orientation of the face of the reference face three-dimensional image generated based on the three-dimensional data of the reference head is matched with the orientation of the face of the person to be collated displayed in the first display area. A third display information generation means for generating a third display information for displaying the third display area to be displayed on the screen of the display device.
A collation support device characterized by having.
[Claim 4]
The collation support device according to claim 3
The user interface display information generation means is for displaying the feature area in the third display area and the second user interface used for the operation for designating the user on the screen of the display device. Generate second user interface display information
A collation support device characterized by that.
[Claim 5]
The collation support device according to claim 4,
The candidate face image display information generation means is collation information in which a feature area designated by using the second user interface is associated with a pre-registered collation face development image for each person and a collation feature area. A candidate who collates with and based on the collation result When an object is selected, a candidate face image display information for displaying a matching face development image corresponding to the selected candidate person on the screen of the display device together with the feature area is generated.
A collation support device characterized by that.
[Claim 6]
The collation support device according to any one of claims 2 to 5.
The candidate face image display information generation means displays a matching face three-dimensional image corresponding to the candidate person on the screen of the display device based on the three-dimensional data of the head of the selected candidate person. Generate candidate face 3D image display information for display in
A collation support device characterized by that.
[Claim 7]
(A) The first display information for displaying the first display area for displaying the image including the face of the person to be collated captured by the image pickup device on the screen of the display device is generated.
(B) Generate a second display information for displaying a second display area for displaying a reference face development image generated based on the three-dimensional data of the reference head on the screen of the display device. death,
(C) A first operation used for an operation for allowing a user to specify a feature area representing the facial features of the person to be collated, which can be seen on the skin surface of the person to be collated, in the second display area. Generates first user interface display information for displaying the user interface of the above on the screen of the display device.
A collation support method characterized by that.
[Claim 8]
The collation support method described in claim 7
(D) The feature area designated by using the first user interface is collated with the collation information associated with the collation face development image for each person registered in advance and the collation feature area, and the collation result is obtained. When a candidate person is selected based on the above, a candidate face image display for displaying a matching face development image corresponding to the selected candidate person on the screen of the display device together with the matching feature area. Generate information
A collation support method characterized by that.
[Claim 9]
The collation support method described in claim 8
(E) The orientation of the face of the reference face three-dimensional image generated based on the three-dimensional data of the reference head is the orientation of the face of the person to be collated displayed in the first display area. Generates a third display information for displaying the third display area to be displayed according to the above on the screen of the display device.
A collation support method characterized by that.
[Claim 10]
The collation support method described in claim 9
(F) Second user interface display for displaying the second user interface used for the operation for designating the feature area in the third display area to the user on the screen of the display device. Generate information
A collation support method characterized by that.
[Claim 11]
The collation support method according to claim 10,
(G) The feature area specified by using the second user interface is collated with the collation information associated with the collation face development image for each person registered in advance and the collation feature area, and the collation result is obtained. When a candidate person is selected based on the above, the candidate face image display information for displaying the matching face development image corresponding to the selected candidate person on the screen of the display device together with the feature area is displayed. Generate
A collation support method characterized by that.
[Claim 12]
The collation support method according to any one of claims 8 to 11.
(H) Candidate face for displaying a collation face three-dimensional image corresponding to the candidate person on the screen of the display device based on the three-dimensional data of the head of the selected candidate person. Generate 3D image display information
A collation support method characterized by that.
[Claim 13]
On the computer
(A) Generate the first display information for displaying the first display area for displaying the image including the face of the person to be collated captured by the image pickup device on the screen of the display device. Steps and
(B) Generate a second display information for displaying a second display area for displaying a reference face development image generated based on the three-dimensional data of the reference head on the screen of the display device. To do, step and
(C) A first operation used for an operation for allowing a user to specify a feature area representing the facial features of the person to be collated, which can be seen on the skin surface of the person to be collated, in the second display area. To generate the first user interface display information for displaying the user interface of the above on the screen of the display device, the step and
A computer-readable recording medium recording a program, including instructions to execute.
[Claim 14]
The computer-readable recording medium according to claim 13,
The program is on the computer
(D) The feature area designated by using the first user interface is collated with the collation information associated with the collation face development image for each person registered in advance and the collation feature area, and the collation result is obtained. When a candidate person is selected based on the above, a candidate face image display for displaying a matching face development image corresponding to the selected candidate person on the screen of the display device together with the matching feature area. Generate information, include additional instructions to perform steps,
A computer-readable recording medium on which the program is recorded.
[Claim 15]
The computer-readable recording medium according to claim 14,
The program is on the computer
(E) The orientation of the face of the reference face three-dimensional image generated based on the three-dimensional data of the reference head is the orientation of the face of the person to be collated displayed in the first display area. Further includes an instruction to execute a step, which generates a third display information for displaying the third display area to be displayed in accordance with the screen of the display device.
A computer-readable recording medium on which the program is recorded.
[Claim 16]
The computer-readable recording medium according to claim 15,
The program is on the computer
(F) Second user interface display for displaying the second user interface used for the operation for designating the feature area in the third display area to the user on the screen of the display device. Generate information, include additional instructions to perform steps,
A computer-readable recording medium on which the program is recorded.
[Claim 17]
The computer-readable recording medium according to claim 16.
The program is on the computer
(G) The feature area specified by using the second user interface is collated with the collation information associated with the collation face development image for each person registered in advance and the collation feature area, and the collation result is obtained. When a candidate person is selected based on the above, the candidate face image display information for displaying the matching face development image corresponding to the selected candidate person on the screen of the display device together with the feature area is displayed. Generate, include more instructions to execute the step,
A computer-readable recording medium on which the program is recorded.
[Claim 18]
The computer-readable recording medium according to any one of claims 14 to 17.
The program is on the computer
(H) Candidate face for displaying a collation face three-dimensional image corresponding to the candidate person on the screen of the display device based on the three-dimensional data of the head of the selected candidate person. Generates 3D image display information, further includes instructions to execute steps,
A computer-readable recording medium on which the program is recorded.