Title of invention: Information processing device, terminal device, information processing system, information processing method and recording medium
Technical field
[0001]
The present invention relates to an information processing device, a terminal device, an information processing system, an information processing method and a recording medium.
Background technology
[0002]
Patent Document 1 discloses a face authentication system that determines whether or not a gate can pass through a person who moves in the authentication area set in the vicinity of the gate by performing face authentication.
Prior art literature
Patent documents
[0003]
Patent Document 1: Japanese Unexamined Patent Publication No. 2015-1790
Outline of the invention
Problems to be solved by the invention
[0004]
The system described in Patent Document 1 is based on the position and size of the person's face in the photographed image after collating the person's face image photographed in the authentication area with the registered face image registered in advance in the database. To determine whether to open the gate. Since it is necessary to complete the collation process and the determination process in order while the person is moving toward the gate, for example, when the person is moving at a high speed, the gate may not be opened in time. That is, it was difficult to allow a moving person to pass through the gate at an appropriate timing.
[0005]
Therefore, in view of the above-mentioned problems, the present invention provides an information processing device, a terminal device, an information processing system, an information processing method, and a recording medium capable of permitting a moving person to pass through a gate at an appropriate timing. With the goal.
Means to solve the problem
[0006]
According to one aspect of the present invention, whether or not the detection unit that detects the biometric information of the person from the input captured image and the condition for starting the process of determining whether or not the person can pass through the gate are satisfied. A first determination unit that determines based on the captured image, a collation unit that collates the biometric information with the registered biometric information in parallel with the processing of the first determination unit, a determination result of the first determination unit, and An information processing apparatus including a second determination unit that executes the determination process based on the collation result of the collation unit is provided.
[0007]
According to another aspect of the present invention, the biometric information is output to a detection unit that detects the biometric information of a person from the input captured image and a collation device that collates the biometric information with the registered biometric information. 1 The output unit, a determination unit that determines whether or not the conditions for starting the gate passability determination process for the person are satisfied, based on the captured image, the determination result of the determination unit, and the determination unit. A terminal device including a second output unit that outputs the determination result to the determination device that executes the determination process based on the collation result of the collation device executed in parallel with the process of the above is provided.
[0008]
According to still another aspect of the present invention, it is determined whether or not the conditions for detecting the biological information of the person from the input photographed image and starting the process of determining whether or not the person can pass through the gate are satisfied. A first determination device that makes a determination based on an image, a collation device that collates the biometric information with the registered biometric information in parallel with the processing of the first determination device, a determination result of the first determination device, and the collation. An information processing system including a second determination device that executes the determination process based on a collation result of the device is provided.
[0009]
According to still another aspect of the present invention, whether or not the step of detecting the biometric information of the person from the input captured image and the condition for starting the passage determination process of whether or not the person can pass through the gate are satisfied. , A step of executing a condition determination process for determining based on the captured image, a step of executing a collation process of the biological information and the registered biological information in parallel with the condition determination process, the condition determination process, and the above. An information processing method including a step of executing the passage determination process based on the result of the collation process is provided.
[0010]
According to still another aspect of the present invention, a step of detecting the biometric information of a person from the input captured image and a step of outputting the biometric information to a collating device for collating the biometric information with the registered biometric information. A step of executing a condition determination process for determining whether or not the condition for starting the pass determination process of whether or not the person can pass through the gate is satisfied based on the captured image, and a determination result of the condition determination process. Information processing including a step of outputting the determination result to the determination device that executes the passage determination process based on the collation result of the collation process executed in the collation device in parallel with the condition determination process. The method is provided.
[0011]
According to still another aspect of the present invention, is the condition for starting the step of detecting the biometric information of the person from the input photographed image and the process of determining whether or not the person can pass through the gate are satisfied in the computer? A step of executing a condition determination process for determining whether or not based on the captured image, a step of executing a collation process between the biological information and the registered biological information in parallel with the condition determination process, and the condition determination. A recording medium in which a step for executing the passage determination process based on the result of the process and the collation process and a program for executing the process is recorded is provided.
Effect of the invention
[0012]
According to the present invention, it is possible to provide an information processing device, a terminal device, an information processing system, an information processing method and a recording medium capable of permitting a moving person to pass through a gate at an appropriate timing.
A brief description of the drawing
[0013]
FIG. 1 is a block diagram showing an overall configuration example of the walk-through biometric authentication system according to the first embodiment.
FIG. 2 is an image diagram of a person in the first embodiment at the time of authentication processing.
FIG. 3 is a diagram showing an example of registrant data stored in the storage unit in the first embodiment.
FIG. 4 is a diagram showing an example of certifier data stored in the storage unit according to the first embodiment.
FIG. 5 is a block diagram showing a hardware configuration example of a management server according to the first embodiment.
FIG. 6 is a sequence diagram showing an example of processing of a management server according to the first embodiment.
FIG. 7 is a diagram showing a setting example of a trigger determination range in the first embodiment.
FIG. 8 is a diagram illustrating the relationship between the distance between the eyes of a person and the front-back position in the first embodiment.
FIG. 9 is a block diagram showing an overall configuration example of the walk-through biometric authentication system according to the second embodiment.
FIG. 10 is a sequence diagram showing an example of processing of the walk-through biometric authentication system in the second embodiment.
FIG. 11 is a block diagram showing a configuration of an information processing apparatus according to a third embodiment.
FIG. 12 is a block diagram showing a configuration of a terminal device according to a fourth embodiment.
FIG. 13 is a block diagram showing a configuration of an information processing system according to a fifth embodiment.
Embodiment for carrying out the invention
[0014]
Hereinafter, an exemplary embodiment of the present invention will be described with reference to the drawings. Similar elements or corresponding elements may be designated by the same reference numerals in the drawings, and the description thereof may be omitted or simplified.
[0015]
[First Embodiment]
First, the configuration of the walk-through biometric authentication system 1 in the present embodiment will be described with reference to the drawings. FIG. 1 is a block diagram showing an overall configuration example of the walk-through biometric authentication system 1 in the present embodiment. Further, FIG. 2 is an image diagram of a person (user U) in the present embodiment at the time of authentication processing. The walk-through biometric authentication system 1 is an information processing system in which a management server 10, a camera 20, a gate device 30, and a notification device 40 are connected via a network NW such as a LAN (Local Area Network) or the Internet.
[0016]
The walk-through biometric authentication system 1 in the present embodiment is, for example, identity verification for entry / exit at an airport or the like, identity verification at an administrative agency, identity verification for entry / exit to a factory / business establishment, entry to an event venue, etc. It can be applied to identity verification at the time.
[0017]
The management server 10 bio-authenticates whether or not the user U detected from the captured image is a registrant registered in advance in the database, and based on the authentication result, determines whether or not the user U can pass through the gate. It is an information processing device for judgment.
[0018]
The camera 20 is, for example, a shooting device such as a security camera installed in an arbitrary number in the authentication area of ​​the facility, and the shot image data is sequentially transmitted to the management server 10. As the camera 20, a digital camera using a CMOS (Complementary Metal Oxide Semiconductor) image sensor, a CCD (Charge Coupled Device) image sensor, or the like can be used so as to be suitable for image processing after shooting. The camera 20 may include a light source that irradiates the user U with illumination light. In FIG. 1, the camera 20 is connected to the management server 10 by wire via the network NW, but the connection method is not limited to the wired connection. The camera 20 may be wirelessly connected to the management server 10.
[0019]
When the gate device 30 succeeds in confirming the identity of the user U on the management server 10, the gate device 30 is controlled by the management server 10 (gate control unit 19), and the user U is blocked from the closed state during standby to block the passage of the user U. It shifts to the open state that allows the passage of U. The opening / closing method of the gate device 30 is not particularly limited, and is, for example, a flapper gate that opens / closes by flappers provided from one side or both sides of the passage, a turn-style gate in which three bars rotate, and the like.
[0020]
The notification device 40 is a device that gives various notifications and alerts to the user U based on the notification control information from the management server 10. The notification device 40 includes a display 41, an LED 42 and a speaker 43.
[0021]
The display 41 displays a face image and a text message of the user U in the display area to notify that the person is the target person for determining whether or not to pass through the gate. The LED 42 notifies whether or not the gate can be passed by switching on / off and switching the lighting color. For example, in the LED 42, when the lighting color is green, it can be notified that it is passable, when it is yellow, it is in the determination process, and when it is red, it is not passable.
[0022]
The speaker 43 outputs an alarm sound and a guidance voice to the user U moving in the authentication area in order to improve the accuracy of face authentication. For example, it is advisable to output guidance sounds such as "Look at the camera with the lamp lit" and "Please shift your line of sight slightly to the right."
[0023]
Further, as shown in FIG. 1, the management server 10 includes a face detection unit 11, a tracking unit 12, a face image selection unit 13, a trigger determination unit 14, a storage unit 15, a feature amount extraction unit 16, a collation unit 17, and a gate. It includes an open / close determination unit 18 and a gate control unit 19. Here, each unit other than the storage unit 15 is classified into a first processing group G1, a second processing group G2, and a third processing group G3 by a broken line portion.
[0024]
The first processing group G1 is a module for determining whether or not the trigger condition for starting the gate passage determination processing in the third processing group G3 is satisfied. The first processing group G1 includes a face detection unit 11, a tracking unit 12, a face image selection unit 13, and a trigger determination unit 14.
[0025]
The second processing group G2 is a module that executes biometric authentication of the user U in parallel with the first processing group. The second processing group G2 includes a feature amount extraction unit 16 and a collation unit 17.
[0026]
The third processing group G3 is a module that executes the gate passage determination processing based on the two processing results in the first processing group G1 and the second processing group G2. The third processing group G3 includes a gate open / close determination unit 18 and a gate control unit 19.
[0027]
The storage unit 15 stores various data necessary for operation in the management server 10. For example, the storage unit 15 has registrant data of a plurality of persons (registrants) who have the authority to pass to the controlled area, tracker data including a tracking ID issued to each person detected from the captured image, and a face. Stores the certifier data of the user U (certifier) ​​who has been authenticated as a registrant by collation.
[0028]
Figure 3 Is a diagram showing an example of registrant data stored in the storage unit 15. The registrant data includes a registrant ID that identifies the registrant, attribute information of the registrant (name, age, gender, etc.), a facial image, and a facial feature amount in the data items. The facial feature amount is an amount indicating facial features such as the positions of characteristic parts such as eyes, nose, and mouth edge, and is extracted from a facial image. The biological information is not limited to the facial image and the amount of facial features.
[0029]
FIG. 4 is a diagram showing an example of certifier data stored in the storage unit 15. The certifier data includes a tracking ID, a registrant ID, a face image, a facial feature amount, and an authentication date and time in the data items. The tracking ID is an identifier given by the tracking unit 12 to each person detected from the captured image. In a plurality of continuously acquired images, the same tracking ID is given to the user U who is regarded as the same person. The authentication date and time is a time stamp when the user U is authenticated as a registrant in the second processing group G2. The management server 10 (gate open / close determination unit 18) of the present embodiment determines whether or not to pass through the gate based on the relationship between the authentication date and time of the biometric authentication stored in the storage unit 15 and the time stamp included in the trigger activation request. judge. The details of the functions of each unit other than the storage unit 15 will be described later.
[0030]
FIG. 5 is a block diagram showing a hardware configuration example of the management server 10 in this embodiment. The management server 10 includes a CPU (Central Processing Unit) 151, a RAM (Random Access Memory) 152, a ROM (Read Only Memory) 153, and an HDD (Hard Disk Drive) 154 as a computer that performs calculation, control, and storage. Further, the management server 10 includes a communication I / F (interface) 155, a display device 156, and an input device 157. The CPU 151, RAM 152, ROM 153, HDD 154, communication I / F 155, display device 156 and input device 157 are connected to each other via a bus line 158. The display device 156 and the input device 157 may be connected to the bus line 158 via a drive device (not shown) for driving these devices.
[0031]
The CPU 151 is a processor having a function of performing a predetermined operation according to a program stored in the ROM 153, the HDD 154, etc., and controlling each part of the management server 10. The RAM 152 is composed of a volatile storage medium and provides a temporary memory area necessary for the operation of the CPU 151. The ROM 153 is composed of a non-volatile storage medium and stores necessary information such as a program used for the operation of the management server 10. The HDD 154 is a storage device composed of a non-volatile storage medium and storing data necessary for processing, an operation program of the management server 10, and the like.
[0032]
Communication I / F155 is a communication interface based on standards such as Ethernet (registered trademark), Wi-Fi (registered trademark), and 4G, and is a module for communicating with other devices. The display device 156 is a liquid crystal display, an OLED display, or the like, and is used for displaying images, characters, interfaces, and the like. The input device 157 is a keyboard, a pointing device, or the like, and is used for the user to operate the management server 10. Examples of pointing devices include mice, trackballs, touch panels, pen tablets and the like. The display device 156 and the input device 157 may be integrally formed as a touch panel.
[0033]
The CPU 151 loads the program stored in the ROM 153, the HDD 154, etc. into the RAM 152 and executes it. As a result, the CPU 151 includes the above-mentioned face detection unit 11, tracking unit 12, face image selection unit 13, trigger determination unit 14, feature amount extraction unit 16, collation unit 17, gate open / close determination unit 18, gate control unit 19, and the like. Realize the function.
[0034]
The hardware configuration shown in FIG. 5 is an example, and devices other than these may be added or some devices may not be provided. Further, some devices may be replaced with other devices having similar functions. Further, a part of the functions of the present embodiment may be provided by another device via the network NW, or the functions of the present embodiment may be distributed and realized by a plurality of devices. For example, the HDD 154 may be replaced with an SSD (Solid State Drive) using a semiconductor memory, or may be replaced with a cloud storage.
[0035]
Subsequently, the operation of the walk-through biometric authentication system 1 configured as described above will be described with reference to the drawings.
[0036]
FIG. 6 is a sequence diagram showing an example of processing of the management server 10. The processes of the first process group G1, the second process group G2, and the third process group G3 in the management server 10 are executed in parallel.
[0037]
First, the processing of the first processing group G1 in the management server 10 will be described. When the management server 10 (face detection unit 11) acquires a captured image from the camera 20 (step S101), it detects the facial images of all the persons included in the captured image (step S102).
[0038]
Next, the management server 10 (tracking unit 12) issues a unique tracking ID for each detected person (step S103). When the captured images are continuously obtained, the tracking unit 12 of the present embodiment determines whether or not they are the same person based on the positional relationship of the persons in the captured images. Then, when the tracking unit 12 considers that they are the same person, the tracking unit 12 assigns the same tracking ID. As a result, the tracking unit 12 tracks the same person over a plurality of captured images.
[0039]
Next, the management server 10 (face image selection unit 13) analyzes the position of the person in the captured image, the direction of the person's face in the face image, the sharpness, the brightness, the size of the display area of ​​the predetermined portion, and the like ( Step S104).
[0040]
Next, the management server 10 (face image selection unit 13) determines whether or not to select the analyzed face image for collation in the second processing group G2 (step S105). Specifically, the face image selection unit 13 has a direction, sharpness, brightness, and display area of ​​a person's feature extraction portion in the face image from a plurality of face images (biological information) detected by the face detection unit 11. A face image for collation is selected based on at least one of the above and output to the collation unit 17. Here, when the management server 10 (face image selection unit 13) determines that the face image is selected for collation (step S105: YES), the process proceeds to step S106.
[0041]
On the other hand, when the management server 10 (face image selection unit 13) determines that the face image is not selected for collation (step S105: NO), the process returns to step S101. For example, if the face of a person in the captured image is not facing the front, it is considered inappropriate as an image for collation, and the face image is not selected. Similarly, (A) when it is regarded as a person who does not need to be collated from the positional relationship of the person in the captured image, (B) when the brightness of the face image is low, (C) the body part from which the feature amount is extracted (feature extraction). Even when the part) is covered with a shield (for example, a mask), the face image of the person is not selected.
[0042]
In step S106, the management server 10 (face image selection unit 13) outputs a face image collation request to the second processing group G2. The collation request (request data) includes a person's face image and a tracking ID. When face images of a plurality of people are selected from the captured images, a collation request is made for each person.
[0043]
Next, the management server 10 (trigger determination unit 14) determines whether or not each person included in the captured image satisfies a predetermined trigger condition (step S107). In the present embodiment, the trigger condition in the trigger determination unit 14 is set based on the body size, which is the size or length of a predetermined body part of the person in the captured image. As the body size, the distance between the two eyes of a person (hereinafter referred to as "inter-eye distance") is used. The trigger determination unit 14 satisfies the trigger condition for the person when the distance between the eyes of the person whose face image is detected in the captured image is the maximum and the distance between the eyes satisfies a predetermined threshold. Is determined.
[0044]
When the management server 10 (trigger determination unit 14) determines that each person included in the captured image satisfies a predetermined trigger condition (step S107: YES), the process proceeds to step S108. On the other hand, when the management server 10 (trigger determination unit 14) determines that each person included in the captured image does not satisfy the predetermined trigger condition (step S107: NO), the process returns to step S101.
[0045]
FIG. 7 is a diagram showing a setting example of the trigger determination range A2 in the captured image IMG_01. Here, a range (hereinafter, referred to as “collation range”) A1 for detecting a face image for collation is set inside the captured image IMG_01. Further, inside the collation range A1, a range (hereinafter, referred to as “trigger determination range”) A2 for determining whether or not the trigger condition is satisfied is set. The collation range A1 and the trigger determination range A2 can be arbitrarily set based on the positional relationship between the camera 20 and the authentication area, the movement direction of the user U, and the like.
[0046]
Further, the reference numerals F1 to F3 indicate facial images of the same person sequentially detected from the continuous captured images IMG_01. It is assumed that the face images F1 to F3 are detected in the order of the face image F1, the face image F2, and the face image F3. The reference numerals D11 to D13 indicate the distance between the two eyes (distance between the eyes) in each of the facial images F1 to F3.
[0047]
FIG. 7 shows that the camera 20 is shooting from diagonally upward to the left with respect to the authentication area. Therefore, the trigger determination range A2 is set at the lower right of the captured image IMG_01. The face image F1 shows when the person is at the farthest position from the camera 20. Since the face image F1 is included in the collation range A1, it is subject to collation processing. Next, when the person moves the authentication area toward the gate, the face image F2 can be detected. At this time, the inter-eye distance of the person is D12, which is longer than the inter-eye distance D11 in the face image F1. Since the face image F2 is included in the collation range A1, it is the target of the collation process. However, the face image F2 includes only a part of the face in the trigger determination range A2. Therefore, the face image F2 is not a target of the trigger determination.
[0048]
Then, when the person moves the authentication area toward the gate, the face image F3 can be detected. At this time, the inter-eye distance of the person is D13, which is longer than the inter-eye distance D12 in the face image F2. The face image F3 includes the entire face in the collation range A1 and the trigger determination range A2. Therefore, the face image F3 is subject to the collation request and the trigger activation request. In this way, by setting the collation range A1 and the trigger determination range A2 inside the captured image IMG_01, the collation image selection process and the trigger determination process can be efficiently performed only for the person who approaches the gate. Can be done.
[0049]
FIG. 8 is a diagram illustrating the relationship between the distance between the eyes of a person and the front-back position. Here, four people P1 to P4 are detected in the captured image IMG_02. All of the persons P1 to P4 are included in the collation range A1. Further, the trigger determination range A2 includes two persons, a person P1 and a person P2. In such a case, the inter-eye distance is compared between the persons included in the trigger determination range A2. In FIG. 8, the eye-to-eye distance of the person P1 is D1, the eye-to-eye distance of the person P2 is D2, and D1 is longer than D2. By comparing the distance between the eyes, it can be seen that the person P1 is in front of the person P2. When the eye distance D1 of the person P1 is longer than a predetermined threshold value, the management server 10 (trigger determination unit 14) considers the person to satisfy the trigger condition, and the person P1 Make a trigger activation request for.
[0050]
In step S108, the management server 10 (trigger determination unit 14) calculates a liveness score for a person who satisfies the trigger condition.
[0051]
Next, the management server 10 (trigger determination unit 14) outputs a trigger activation request to the third processing group G3 (step S109). The trigger activation request is data including a person's tracking ID and liveness score.
[0052]
Next, when the management server 10 (trigger determination unit 14) transmits a control signal to the notification device 40 to display a screen display of a gate passability determination target person satisfying the trigger condition (step S110), the process returns to step S101. ..
[0053]
Subsequently, the processing of the second processing group G2 in the management server 10 will be described. The management server 10 (feature amount extraction unit 16) determines whether or not a collation request has been input from the first processing group G1 (step S201).
[0054]
When the management server 10 (feature amount extraction unit 16) determines that the collation request has been input from the first processing group G1 (step S201: YES), the processing proceeds to step S202. On the other hand, when the management server 10 (feature amount extraction unit 16) determines that the collation request has not been input from the first processing group G1 (step S201: NO), the standby state is maintained.
[0055]
In step S202, the management server 10 (feature amount extraction unit 16) extracts the face feature amount from the face image included in the collation request (request data) input from the first processing group G1.
[0056]
Next, the management server 10 (verification unit 17) collates the input face image with the registered face image (registered biometric information) of the registrant stored in advance in the storage unit 15 (step S203).
[0057]
Then, when the management server 10 (verification unit 17) outputs the collation result to the third processing group G3 (gate open / close determination unit 18) (step S204), the processing returns to step S201.
[0058]
Finally, the processing in the third processing group G3 of the management server 10 will be described. The management server 10 (gate open / close determination unit 18) determines whether or not the collation result data has been input from the second processing group G2 (step S301).
[0059]
When the management server 10 (gate open / close determination unit 18) determines that the collation result data has been input from the second processing group G2 (step S301: YES), the processing proceeds to step S302. On the other hand, when the management server 10 (gate open / close determination unit 18) determines that the collation result data has not been input from the second processing group G2 (step S301: NO), the processing proceeds to step S303.
[0060]
In step S302, the management server 10 (gate open / close determination unit 18) stores the collation result input from the second processing group G2 in the storage unit 15. In the present embodiment, the data relating to the person (certifier) ​​whose collation result matches the collation match is stored in the storage unit 15 as the certifier data (see FIG. 4).
[0061]
In step S303, the management server 10 (gate open / close determination unit 18) determines whether or not the trigger activation request from the first processing group G1 has been input. Here, when the management server 10 (gate open / close determination unit 18) determines that the trigger activation request from the first processing group G1 has been input (step S303: YES), the processing proceeds to step S304.
[0062]
On the other hand, when the management server 10 (gate open / close determination unit 18) determines that the trigger activation request from the first processing group G1 has not been input (step S303: NO), the processing returns to step S301.
[0063]
In step S304, the management server 10 (gate open / close determination unit 18) determines whether or not the person who satisfies the trigger condition is a person who has been authenticated within a predetermined time. Here, when the management server 10 (gate open / close determination unit 18) determines that the person who satisfies the trigger condition is a person who has been authenticated within a predetermined time (step S304: YES), the process proceeds to step S305. ..
[0064]
For example, for the same person whose tracking ID is "0001", the processing time when the trigger determination unit 14 determines that the trigger condition is satisfied is T1, and the matching unit 17 authenticates that the match is matched with the registrant. A case where the processing time is T2 will be described. T1 can be obtained from the time stamp of the trigger activation request. On the other hand, T2 can be acquired from the time stamp (authentication time) of the certifier data. In this case, if the processing time (authentication time) T2 is within a certain time from the processing time (trigger activation time) T1, the gate open / close determination unit 18 determines to open the gate.
[0065]
On the other hand, when the management server 10 (gate open / close determination unit 18) determines that the person who satisfies the trigger condition is not the authenticated person within a predetermined time (step S304: NO), the process returns to step S301. ..
[0066]
In step S305, when the management server 10 (gate control unit 19) outputs a gate control signal for opening the gate to the gate device 30, the process returns to step S301.
[0067]
As described above, according to the present embodiment, the processes of the first processing group G1, the second processing group G2, and the third processing group G3 in the management server 10 are executed in parallel, respectively. Therefore, it is possible to determine whether or not the user U can pass through the gate at an appropriate timing, and control the opening and closing of the gate based on the determination result.
[0068]
[Second Embodiment]
Hereinafter, the walk-through biometric authentication system 2 in the second embodiment will be described. In addition, the code common to the code given in the figure of the first embodiment indicates the same object. The description of the parts common to the first embodiment will be omitted, and the different parts will be described in detail.
[0069]
FIG. 9 is a block diagram showing an overall configuration example of the walk-through biometric authentication system 2 in the present embodiment. As shown in the figure, in the walk-through biometric authentication system 2, the functions of the management server 10 shown in FIG. 1 are distributed to three devices, an edge terminal 110, a collation server 120, and a gate control server 130. It is different from the first embodiment.
[0070]
Further, the storage unit 15 shown in FIG. 1 is divided into a first storage unit 15A and a second storage unit 15B. The first storage unit 15A stores tracker data including a tracking ID and position information in the image regarding the tracking target person detected from the captured image. On the other hand, the second storage unit 15B stores registrant data (see FIG. 3) and certifier data (see FIG. 4).
[0071]
Subsequently, the operation of the walk-through biometric authentication system 2 configured as described above will be described with reference to the drawings.
[0072]
FIG. 10 is a sequence diagram showing an example of processing of the walk-through biometric authentication system 2. The processes of the edge terminal 110, the collation server 120, and the gate control server 130 are executed in parallel.
[0073]
First, the processing of the edge terminal 110 will be described. When the edge terminal 110 (face detection unit 11) acquires a captured image from the camera 20 (step S401), the edge terminal 110 detects the facial images of all the persons included in the captured image (step S402).
[0074]
Next, the edge terminal 110 (tracking unit 12) issues a unique tracking ID for each detected person (step S403). When the captured images are continuously obtained, the tracking unit 12 of the present embodiment determines whether or not they are the same person based on the positional relationship of the persons in the captured images. Then, when the tracking unit 12 considers that they are the same person, the tracking unit 12 assigns the same tracking ID.
[0075]
Next, the edge terminal 110 (face image selection unit 13) analyzes the position of the person in the captured image, the direction of the person's face in the face image, the sharpness, the brightness, the size of the display area of ​​the predetermined portion, and the like ( Step S404).
[0076]
Next, the edge terminal 110 (face image selection unit 13) determines whether or not to select the analyzed face image for collation in the collation server 120 (step S405). Here, when the management server 10 (face image selection unit 13) determines that the face image is selected for collation (step S405: YES), the process proceeds to step S406.
[0077]
On the other hand, when the edge terminal 110 (face image selection unit 13) determines that the face image is not selected for collation (step S405: NO), the process returns to step S401.
[0078]
In step S406, the edge terminal 110 (face image selection unit 13) transmits a face image matching request to the matching server 120. The collation request (request data) includes a person's face image and a tracking ID. When face images of a plurality of people are selected from the captured images, a collation request is made for each person.
[0079]
Next, the edge terminal 110 (trigger determination unit 14) determines whether or not each person included in the captured image satisfies a predetermined trigger condition (step S407). In the present embodiment, the trigger determination unit 14 satisfies the trigger condition when the inter-eye distance of the person whose face image is detected in the captured image is the maximum and the inter-eye distance satisfies a predetermined threshold value. Is determined.
[0080] [0080]
When the edge terminal 110 (trigger determination unit 14) determines that each person included in the captured image satisfies a predetermined trigger condition (step S407: YES), the process proceeds to step S408. On the other hand, when the edge terminal 110 (trigger determination unit 14) determines that each person included in the captured image does not satisfy the predetermined trigger condition (step S407: NO), the process returns to step S401.
[0081]
In step S408, the edge terminal 110 (trigger determination unit 14) calculates a liveness score for a person who satisfies the trigger condition.
[0082]
Next, the edge terminal 110 (trigger determination unit 14) transmits a trigger activation request to the gate control server 130 (step S409). The trigger activation request is data including a person's tracking ID and liveness score.
[0083]
Next, when the edge terminal 110 (trigger determination unit 14) transmits a control signal to the notification device 40 to display a screen display of a gate passability determination target person satisfying the trigger condition (step S410), the process returns to step S401. ..
[0084]
Subsequently, the processing of the collation server 120 in the management server 10 will be described. The management server 10 (feature amount extraction unit 16) determines whether or not a collation request has been received from the edge terminal 110 (step S501).
[0085]
When the collation server 120 (feature amount extraction unit 16) determines that the collation request has been received from the edge terminal 110 (step S501: YES), the process proceeds to step S502. On the other hand, when the collation server 120 (feature amount extraction unit 16) determines that the collation request has not been received from the edge terminal 110 (step S501: NO), the standby state is maintained.
[0086]
In step S502, the collation server 120 (feature amount extraction unit 16) extracts the face feature amount from the face image included in the collation request (request data) received from the edge terminal 110.
[0087]
Next, the collation server 120 (verification unit 17) collates the received face image with the registered face image (registered biometric information) of the registrant stored in advance in the second storage unit 15B (step S503).
[0088]
Then, when the collation server 120 (verification unit 17) transmits the collation result to the gate control server 130 (gate open / close determination unit 18) (step S504), the process returns to step S501.
[0089]
Finally, the processing in the gate control server 130 will be described. The gate control server 130 (gate open / close determination unit 18) has a collation result from the collation server 120.It is determined whether or not the data of is received (step S601).
[0090]
When the gate control server 130 (gate open / close determination unit 18) determines that the collation result data has been received from the collation server 120 (step S601: YES), the process proceeds to step S602. On the other hand, when the management server 10 (gate open / close determination unit 18) determines that the collation result data has not been received from the collation server 120 (step S601: NO), the process proceeds to step S603.
[0091]
In step S602, the gate control server 130 (gate open / close determination unit 18) stores the collation result received from the collation server 120 in the storage area. In the present embodiment, the data relating to the person (certifier) ​​whose collation result matches the collation match is stored as the certifier data in the storage area of ​​the second storage unit 15B of the collation server 120 and the gate control server 130 (FIG. FIG. 4).
[0092]
In step S603, the gate control server 130 (gate open / close determination unit 18) determines whether or not the trigger activation request from the edge terminal 110 has been received. Here, when the gate control server 130 (gate open / close determination unit 18) determines that the trigger activation request from the edge terminal 110 has been received (step S603: YES), the process proceeds to step S604.
[0093]
On the other hand, when the gate control server 130 (gate open / close determination unit 18) determines that the trigger activation request from the edge terminal 110 has not been received (step S603: NO), the process returns to step S601.
[0094]
In step S604, the gate control server 130 (gate open / close determination unit 18) determines whether or not the person who satisfies the trigger condition is a person who has been authenticated within a predetermined time. Here, when the management server 10 (gate open / close determination unit 18) determines that the person who satisfies the trigger condition is a person who has been authenticated within a predetermined time (step S604: YES), the process proceeds to step S605. ..
[0095]
On the other hand, when the gate control server 130 (gate open / close determination unit 18) determines that the person who satisfies the trigger condition is not the authenticated person within a predetermined time (step S604: NO), the process proceeds to step S601. return.
[0096]
In step S605, when the gate control server 130 (gate control unit 19) transmits a gate control signal for opening the gate to the gate device 30, the process returns to step S601.
[0097]
As described above, according to the present embodiment, the processes of the edge terminal 110, the collation server 120, and the gate control server 130 are executed in parallel. Therefore, as in the case of the first embodiment, it is possible to determine whether or not the user U can pass through the gate at an appropriate timing, and control the opening and closing of the gate based on the determination result.
[0098]
[Third Embodiment]
FIG. 11 is a block diagram showing the configuration of the information processing apparatus 100 according to the third embodiment. The information processing apparatus 100 includes a detection unit 100A, a first determination unit 100B, a collation unit 100C, and a second determination unit 100D. The detection unit 100A detects the biological information of a person from the input captured image. The first determination unit 100B determines whether or not the condition for starting the gate passability determination process for a person is satisfied based on the captured image. The collation unit 100C collates the biometric information with the registered biometric information in parallel with the processing of the first determination unit 100B. The second determination unit 100D executes a gate passability determination process based on the determination result of the first determination unit 100B and the collation result of the collation unit 100C. According to the present embodiment, it is possible to allow a moving person to pass through the gate at an appropriate timing.
[0099]
[Fourth Embodiment]
FIG. 12 is a block diagram showing the configuration of the terminal device 200 according to the fourth embodiment. The terminal device 200 includes a detection unit 200A, a first output unit 200B, a determination unit 200C, and a second output unit 200D. The detection unit 200A detects the biological information of a person from the input captured image. The first output unit 200B outputs the biometric information to the collating device for collating the biometric information with the registered biometric information. The determination unit 200C determines whether or not the condition for starting the gate passability determination process for a person is satisfied based on the captured image. The second output unit 200D outputs the determination result to the determination device that executes the determination process based on the determination result of the determination unit 200C and the collation result of the collation device executed in parallel with the process of the determination unit 200C. According to the present embodiment, it is possible to allow a moving person to pass through the gate at an appropriate timing.
[0100]
[Fifth Embodiment]
FIG. 13 is a block diagram showing the configuration of the information processing system 300 according to the fifth embodiment. The information processing system 300 includes a first determination device 300A, a collation device 300B, and a second determination device 300C. The first determination device 300A detects the biological information of the person from the input photographed image, and determines whether or not the condition for starting the gate passability determination process for the person is satisfied based on the photographed image. .. The collation device 300B collates the biometric information with the registered biometric information in parallel with the processing of the first determination device 300A. The second determination device 300C executes the determination process based on the determination result of the first determination device 300A and the collation result of the collation device 300B. According to the present embodiment, it is possible to allow a moving person to pass through the gate at an appropriate timing.
[0101]
[Modification Embodiment]
Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above-described embodiments. The configuration and details of the present invention can be modified in various ways that can be understood by those skilled in the art, without departing from the gist of the present invention. For example, an embodiment in which a partial configuration of any of the embodiments is added to another embodiment or replaced with a partial configuration of another embodiment is also an embodiment to which the present invention can be applied. Should be understood.
[0102]
In the above-described embodiment, the case where the tracking unit 12 tracks a person by determining whether or not the person is the same person based on the position in the image of the person among a plurality of captured images has been described. However, the method of tracking a person is not limited to this. For example, the tracking unit 12 may track a person by collating biometric information among a plurality of captured images and determining whether or not the person is the same person. In this case, the face image may be transmitted to the collation engine (second processing group G2) of the management server 10 and the collation score returned from the collation engine may be used.
[0103]
Further, in the above-described embodiment, the gate control unit 19 outputs a gate open / close signal based on the result of the determination process in the gate open / close determination unit 18, but even if the determination is made by further combining other conditions. good. For example, the gate control unit 19 controls the opening / closing of the gate based on the result of the determination process in the gate opening / closing determination unit 18 and the identification information acquired from the medium (for example, an IC card for authentication) owned by the person. May be good.
[0104]
Each embodiment also has a processing method in which a program for operating the configuration of the embodiment is recorded on a recording medium so as to realize the functions of the above-described embodiment, the program recorded on the recording medium is read out as a code, and the program is executed by a computer. Is included in the category of. That is, a computer-readable recording medium is also included in the scope of each embodiment. Further, not only the recording medium on which the above-mentioned program is recorded but also the program itself is included in each embodiment.
[0105]
As the recording medium, for example, a floppy (registered trademark) disk, a hard disk, an optical disk, a photomagnetic disk, a CD-ROM, a magnetic tape, a non-volatile memory card, or the like can be used. Further, not only the program recorded on the recording medium that executes the process alone, but also the program that operates on the OS and executes the process in cooperation with the functions of other software and the expansion board is also an embodiment. Is included in the category of.
[0106]
A part or all of the above-described embodiment may be described as in the following appendix, but is not limited to the following.
[0107]
(Appendix 1)
A detector that detects the biometric information of a person from the input captured image,
The first determination unit that determines whether or not the conditions for starting the gate passability determination process for the person are satisfied based on the captured image, and
In parallel with the processing of the first determination unit, a collation unit that collates the biometric information with the registered biometric information,
A second determination unit that executes the determination process based on the determination result of the first determination unit and the collation result of the collation unit.
Information processing device equipped with.
[0108]
(Appendix 2)
Further equipped with a tracking unit that tracks the same person over a plurality of captured images,
The second determination unit executes the determination process based on the determination result and the collation result for the same person tracked by the tracking unit.
The information processing device described in Appendix 1.
[0109]
(Appendix 3)
The tracking unit tracks the person by determining whether or not the person is the same person among a plurality of captured images based on the position in the image of the person.
The information processing device described in Appendix 2.
[0110]
(Appendix 4)
The tracking unit tracks the person by collating the biometric information among the plurality of captured images and determining whether or not the person is the same person.
The information processing device described in Appendix 2.
[0111]
(Appendix 5)
The conditions are set based on the body size, which is the size or length of a predetermined body part of the person in the captured image.
The information processing device according to any one of Supplementary note 1 to 4.
[0112]
(Appendix 6)
The body size is the distance between the two eyes of the person,
The information processing device described in Appendix 5.
[0113]
(Appendix 7)
The first determination unit satisfies the above conditions for a target person whose body size is maximum and whose body size is equal to or larger than a threshold among the persons included in a predetermined determination area set in the captured image. Judging,
The information processing device according to Appendix 5 or 6.
[0114]
(Appendix 8)
From the plurality of biometric information detected by the detection unit, the biometric information for collation is obtained based on at least one of the direction, sharpness, brightness, and display area of ​​the characteristic extraction site of the person in the biometric information. A sorting unit that sorts and outputs to the collation unit,
The information processing device according to any one of Supplementary note 1 to 7, further comprising.
[0115]
(Appendix 9)
The detection unit detects the biological information of the person included in a predetermined detection area in the captured image, and detects the biometric information of the person.
The first determination unit determines whether or not the person satisfies the above conditions in a predetermined determination area set inside the detection area.
The information processing device according to any one of Supplementary notes 1 to 8.
[0116]
(Appendix 10)
The second determination unit is when the first time when the matching result of the matching match is obtained by the matching unit is within a certain time from the second time when the first determination unit determines that the condition is satisfied. Allow the person to pass through the gate,
The information processing device according to any one of Supplementary note 1 to 9.
[0117]
(Appendix 11)
A gate control unit that controls the opening and closing of the gate based on the result of the determination process.
The information processing apparatus according to any one of Supplementary note 1 to 10, further comprising.
[0118]
(Appendix 12)
The gate control unit controls the opening and closing of the gate based on the result of the determination process and the identification information acquired from the medium owned by the person.
The information processing device according to Appendix 11.
[0119]
(Appendix 13)
Display control unit that displays the target person on the display device,
The information processing device according to Appendix 7, further comprising.
[0120]
(Appendix 14)
The biometric information is a facial image of the person or a feature amount extracted from the facial image.
The information processing device according to any one of Supplementary note 1 to 13.
[0121]
(Appendix 15)
A detector that detects the biometric information of a person from the input captured image,
The biometric information is opposed to a collation device that collates the biometric information with the registered biometric information.The first output unit that outputs
A determination unit that determines whether or not the conditions for starting the gate passability determination process for the person are satisfied based on the captured image, and
A second output unit that outputs the determination result to the determination device that executes the determination process based on the determination result of the determination unit and the collation result of the collation device executed in parallel with the processing of the determination unit. ,
Terminal device equipped with.
[0122]
(Appendix 16)
A first determination device that detects the biological information of a person from the input captured image and determines whether or not the conditions for starting the gate passability determination process for the person are satisfied based on the captured image. ,
In parallel with the processing of the first determination device, a collation device for collating the biometric information with the registered biometric information,
A second determination device that executes the determination process based on the determination result of the first determination device and the collation result of the collation device.
Information processing system equipped with.
[0123]
(Appendix 17)
The step of detecting the biometric information of a person from the input captured image,
A step of executing a condition determination process for determining whether or not the condition for starting the passage determination process for passing through the gate for the person is satisfied based on the captured image, and a step of executing the condition determination process.
In parallel with the condition determination process, a step of executing a collation process between the biometric information and the registered biometric information,
The step of executing the passage determination process based on the results of the condition determination process and the collation process, and
Information processing method with.
[0124]
(Appendix 18)
The step of detecting the biometric information of a person from the input captured image,
The step of outputting the biometric information to the collation device for collating the biometric information with the registered biometric information,
A step of executing a condition determination process for determining whether or not the condition for starting the passage determination process for passing through the gate for the person is satisfied based on the captured image, and a step of executing the condition determination process.
The determination result is output to the determination device that executes the passage determination process based on the determination result of the condition determination process and the collation result of the collation process executed in parallel with the condition determination process. Steps and
Information processing method with.
[0125]
(Appendix 19)
On the computer
The step of detecting the biometric information of a person from the input captured image,
A step of executing a condition determination process for determining whether or not the condition for starting the passage determination process for passing through the gate for the person is satisfied based on the captured image, and a step of executing the condition determination process.
In parallel with the condition determination process, a step of executing a collation process between the biometric information and the registered biometric information,
The step of executing the passage determination process based on the results of the condition determination process and the collation process, and
A recording medium on which a program for executing is recorded.
Description of the sign
[0126]
NW ・ ・ ・ Network
1, 2, ... Walk-through biometric authentication system
10 ... Management server
11 ... Face detection unit
12 ... Tracking unit
13 ... Face image selection section
14 ... Trigger judgment unit
15 ... Memory
16 ... Feature amount extraction unit
17 ... Collation section
18 ... Gate open / close judgment unit
19 ... Gate control unit
20 ... Camera
30 ... Gate device
40 ... Notification device
110 ・ ・ ・ Edge terminal
120 ... Matching server
130 ... Gate control server
151 ... CPU
152 ... RAM
153 ... ROM
154 ... HDD
155 ... Communication I / F
156 ... Display device
157 ... Input device
158 ... Bus line
The scope of the claims
[Claim 1]
A detector that detects the biometric information of a person from the input captured image,
The first determination unit that determines whether or not the conditions for starting the gate passability determination process for the person are satisfied based on the captured image, and
In parallel with the processing of the first determination unit, a collation unit that collates the biometric information with the registered biometric information,
A second determination unit that executes the determination process based on the determination result of the first determination unit and the collation result of the collation unit.
Information processing device equipped with.
[Claim 2]
Further equipped with a tracking unit that tracks the same person over a plurality of captured images,
The second determination unit executes the determination process based on the determination result and the collation result for the same person tracked by the tracking unit.
The information processing device according to claim 1.
[Claim 3]
The tracking unit tracks the person by determining whether or not the person is the same person among a plurality of captured images based on the position in the image of the person.
The information processing device according to claim 2.
[Claim 4]
The tracking unit tracks the person by collating the biometric information among the plurality of captured images and determining whether or not the person is the same person.
The information processing device according to claim 2.
[Claim 5]
The conditions are set based on the body size, which is the size or length of a predetermined body part of the person in the captured image.
The information processing device according to any one of claims 1 to 4.
[Claim 6]
The body size is the distance between the two eyes of the person,
The information processing device according to claim 5.
[Claim 7]
The first determination unit satisfies the above conditions for a target person whose body size is maximum and whose body size is equal to or larger than a threshold among the persons included in a predetermined determination area set in the captured image. Judging,
The information processing device according to claim 5 or 6.
[Claim 8]
From the plurality of biometric information detected by the detection unit, the biometric information for collation is obtained based on at least one of the direction, sharpness, brightness, and display area of ​​the characteristic extraction site of the person in the biometric information. A sorting unit that sorts and outputs to the collation unit,
The information processing apparatus according to any one of claims 1 to 7, further comprising.
[Claim 9]
The detection unit detects the biological information of the person included in a predetermined detection area in the captured image, and detects the biometric information of the person.
The first determination unit determines whether or not the person satisfies the above conditions in a predetermined determination area set inside the detection area.
The information processing device according to any one of claims 1 to 8.
[Claim 10]
The second determination unit is when the first time when the matching result of the matching match is obtained by the matching unit is within a certain time from the second time when the first determination unit determines that the condition is satisfied. Allow the person to pass through the gate,
The information processing device according to any one of claims 1 to 9.
[Claim 11]
A gate control unit that controls the opening and closing of the gate based on the result of the determination process.
The information processing apparatus according to any one of claims 1 to 10, further comprising.
[Claim 12]
The gate control unit controls the opening and closing of the gate based on the result of the determination process and the identification information acquired from the medium owned by the person.
The information processing device according to claim 11.
[Claim 13]
Display control unit that displays the target person on the display device,
The information processing apparatus according to claim 7, further comprising.
[Claim 14]
The biometric information is a facial image of the person or a feature amount extracted from the facial image.
The information processing device according to any one of claims 1 to 13.
[Claim 15]
A detector that detects the biometric information of a person from the input captured image,
The first output unit that outputs the biometric information to the collation device that collates the biometric information with the registered biometric information,
A determination unit that determines whether or not the conditions for starting the gate passability determination process for the person are satisfied based on the captured image, and
A second output unit that outputs the determination result to the determination device that executes the determination process based on the determination result of the determination unit and the collation result of the collation device executed in parallel with the processing of the determination unit. ,
Terminal device equipped with.
[Claim 16]
A first determination device that detects the biological information of a person from the input captured image and determines whether or not the conditions for starting the gate passability determination process for the person are satisfied based on the captured image. ,
In parallel with the processing of the first determination device, a collation device for collating the biometric information with the registered biometric information,
A second determination device that executes the determination process based on the determination result of the first determination device and the collation result of the collation device.
Information processing system equipped with.
[Claim 17]
The step of detecting the biometric information of a person from the input captured image,
A step of executing a condition determination process for determining whether or not the condition for starting the passage determination process for passing through the gate for the person is satisfied based on the captured image, and a step of executing the condition determination process.
In parallel with the condition determination process, a step of executing a collation process between the biometric information and the registered biometric information,
The step of executing the passage determination process based on the results of the condition determination process and the collation process, and
Information processing method with.
[Claim 18]
The step of detecting the biometric information of a person from the input captured image,
The step of outputting the biometric information to the collation device for collating the biometric information with the registered biometric information,
A step of executing a condition determination process for determining whether or not the condition for starting the passage determination process for passing through the gate for the person is satisfied based on the captured image, and a step of executing the condition determination process.
The determination result is output to the determination device that executes the passage determination process based on the determination result of the condition determination process and the collation result of the collation process executed in parallel with the condition determination process. Steps and
Information processing method with.
[Claim 19]
On the computer
The step of detecting the biometric information of a person from the input captured image,
A step of executing a condition determination process for determining whether or not the condition for starting the passage determination process for passing through the gate for the person is satisfied based on the captured image, and a step of executing the condition determination process.
In parallel with the condition determination process, a step of executing a collation process between the biometric information and the registered biometric information,
The step of executing the passage determination process based on the results of the condition determination process and the collation process, and
A recording medium on which a program for executing is recorded.