[Technical Field]
5 [0001]
The some non-limiting embodiments relates to a gate device, an
authentication system, a method of controlling the gate device, and a
storage medium.
[Background Art]
10 [0002]
Services using face authentication have started to spread (see PTLs
1 to 3). For example, face authenti cation has started to be applied to
various procedures (for example, check in, baggage check, security check,
and the like) at airports. A service using face authentication is often
15 provided using a gate device.
[Citation List]
[Patent Literature]
[0003]
[PTL 1] JP 2006-236244 A
20 [PTL 2] JP 2018-128970 A
[PTL 3] JP 2018-109935 A
[Summary of Invention]
[Technical Problem]
[0004]
25 As described above, services using face authentication have started
to be provided at airports and the like . For example, a security check
using face authentication is performed at a security check site at an
airport. Here, the device installation space of the existing airport is
limited, and the gate device for face authentication is often installed in a
3
narrow space.
[0005]
Therefore, when the gate device acquires an image (the image
including the face region), the image may include not only the face image
5 of a user in the forward-most row (the user to be authenticated) but also
the face image of a user behind. That is, since the gate device for the
face authentication is installed in a narrow space, the distance between
the users who line up before the gate device is short, and a plurality of
people may appear in the same image.
10 [0006]
In this case, the gate device is required to select one face image
from the plurality of face images and set the selected face image as the
authentication target. For example, it is conceivable that the gate device
calculates an area (size) for each of a plurality of face images and sets a
15 face image having the largest area as an authentication target . This
handling is based on the premise that the face image of a user who lines
up in the forward-most row for the gate device is larger than the face
image of a user who lines up behind. However, the size of the face has a
large individual difference, and in the above handling (determination of
20 the authentication target based on the area of the face image), a user who
lines up behind a user who lines up in the forward -most row may be set as
the authentication target.
[0007]
A main object of the some non-limiting embodiments is to provide
25 a gate device, an authentication system, a method of controlling the gate
device, and a storage medium that contribute to accurately determining a
user to be authenticated.
[Solution to Problem]
[0008]
4
According to a first aspect of the some non-limiting embodiments,
there is provided a gate device including an acquisition unit that acquires
an upper image and a lower image from an upper camera and a lower
camera installed below the upper camera in a vertical direction,
5 respectively, a setting unit that, when a plurality of users appears in both
of the upper image and the lower image, identifies a user in a forward -
most row from among the plurality of users by calculating a distance
between each of the plurality of users and a camera installation face, and
sets the identified user as a person to be authenticated, and a gate control
10 unit that controls a gate based on a result of authentication of the person
to be authenticated.
[0009]
According to a second aspect of the some non-limiting
embodiments, there is provided an authentication system including a
15 server device that stores biological information about each of a plurality
of system users and performs an authentication process using the plurality
of pieces of biological information, and a gate device connected to the
server device, wherein the gate device includes an acquisition unit that
acquires an upper image and a lower image from an upper camera and a
20 lower camera installed below the upper camera in a vertical direction,
respectively, a setting unit that, when a plurality of users appears in both
of the upper image and the lower image, identifies a user in a forward -
most row from among the plurality of users by calculating a distance
between each of the plurality of users and a camera installation face, and
25 sets the identified user as a person to be authenticated, a n authentication
request unit that requests the server device to authenticate the person to
be authenticated, and a gate control unit that controls a gate based on a
result of authentication of the person to be authenticated.
[0010]
5
According to a third aspect of the some non-limiting embodiments,
there is provided a method of controlling a gate device, the method
including the steps, performed by the gate device, of acquiring an upper
image and a lower image from an upper camera and a lower camera
5 installed below the upper camera in a vertical direction, respectively,
when a plurality of users appears in both of the upper image and the lower
image, identifying a user in a forward-most row from among the plurality
of users by calculating a distance between each of the plurality of users
and a camera installation face, and setting the identifie d user as a person
10 to be authenticated, and controlling a gate based on a result of
authentication of the person to be authenticated.
[0011]
According to a fourth aspect of the some non-limiting
embodiments, there is provided a computer-readable storage medium
15 storing a program for causing a computer mounted on a gate device to
execute the steps of acquiring an upper image and a lower image from an
upper camera and a lower camera installed below the upper camera in a
vertical direction, respectively, when a plurality of users appears in both
of the upper image and the lower image, identifying a user in a forward -
20 most row from among the plurality of users by calculating a distance
between each of the plurality of users and a camera installation face, and
setting the identified user as a person to be authenticated, and controlling
a gate based on a result of authentication of the person to be
authenticated.
25 [Advantageous Effects of Invention]
[0012]
According to each aspect of the some non-limiting embodiments, a
gate device, an authentication system, a method of controlling the gate
device, and a storage medium that contribute to accurately determining a
6
user to be authenticated are provided. The effects of the some nonlimiting embodiments are not limited to the above. According to the
some non-limiting embodiments, other effects may be exhibited instead of
or in addition to the effects.
5 [Brief Description of Drawings]
[0013]
Fig. 1 is a diagram for describing an outline of an example
embodiment.
Fig. 2 is a diagram illustrating an example of a schematic
10 configuration of an authentication system according to the first example
embodiment.
Fig. 3 is a view illustrating an example of an appearance of the
gate device according to the first example embodiment.
Fig. 4 is a view illustrating an example of a plan view of the gate
15 device according to the first example embodiment.
Fig. 5 is a diagram illustrating an example of a processing
configuration of the gate device according to the first example
embodiment.
Fig. 6 is a diagram for explaining the operation of a biological
20 information acquisition unit according to the first example embodiment.
Fig. 7 is a flowchart illustrating an example of the operation of an
authentication target setting unit according to the first example
embodiment.
Fig. 8 is a diagram for explaining the operation of the
25 authentication target setting unit according to the first example
embodiment.
Fig. 9 is a diagram for explaining the operation of the
authentication target setting unit according to the first example
embodiment.
7
Fig. 10 is a diagram for explaining the operation of the
authentication target setting unit according to the first example
embodiment.
Fig. 11 is a flowchart illustrating an example of a front/rear
5 relationship identification process of the authentication target setting unit
according to the first example embodiment.
Fig. 12 is a diagram for explaining the operation of the
authentication target setting unit according to the first example
embodiment.
10 Fig. 13 is a diagram for explaining the operation of the
authentication target setting unit according to the first example
embodiment.
Fig. 14 is a diagram illustrating an example of a graph illustrating
a correspondence relationship between a distance between a camera
15 installation face and a target object and a position of the target object in a
common region.
Fig. 15 is a diagram illustrating an example of an authentication
request.
Fig. 16 is a flowchart illustrating an example of the operation of
20 the gate device according to the first example embodiment.
Fig. 17 is a diagram illustrating an example of a processing
configuration of a server device according to the first example
embodiment.
Fig. 18 is a diagram illustrating an example of a user data base.
25 Fig. 19 is a sequence diagram illustrating an example of the
operation of the authentication system according to the first example
embodiment.
Fig. 20 is a diagram for explaining the relationship between the
actual position of the user and the position of the user appearing in the
8
image.
Fig. 21 is a flowchart illustrating an example of the operation of
an authentication target setting unit according to the second example
embodiment.
5 Fig. 22 is a diagram illustrating an example of a hardware
configuration of the gate device.
[Example Embodiment]
[0014]
First, an outline of an example embodiment will be described .
10 The reference numerals in the drawings attached to this outline are
attached to respective elements for convenience as an example for
assisting understanding, and the description of this outline is not intended
to be any limitation. In a case where there is no particular explanation,
the block described in each drawing represents not a configuration of a
15 hardware unit but a configuration of a functional unit. Connection lines
between blocks in each drawing include both bidirectional and
unidirectional lines. The unidirectional arrow schematically indicates a
flow of a main signal (data), and does not exclude bidirectionality. In
the present specification and the drawings, elements that can be similarly
20 described are denoted by the same reference numerals, and redundant
description can be omitted.
[0015]
A gate device 100 according to an example embodiment includes
an acquisition unit 101, a setting unit 102, and a gate control unit 103 (see
25 Fig. 1). The acquisition unit 101 acquires an upper image and a lower
image from an upper camera and a lower camera, respectively, that is
installed vertically below the upper camera . When a plurality of users
appears in both the upper image and the lower image, the setting unit 102
identifies a user in the forward-most row from among the plurality of
9
users by calculating the distance between each of the plurality of users
and a camera installation face, and then sets the identified user as a
person to be authenticated. The gate control unit 103 controls a gate
based on the result of authentication of the person to be authenticated.
5 [0016]
Gate device 100 acquires images from two cameras (upper camera,
lower camera). In a case where a plurality of users appears in the image,
the gate device 100 identifies a person to be authenticated . The gate
device 100 calculates the distance between each user and the camera
10 installation face, for example, using the parallax of the target object (the
user commonly appearing in the two images) in the upper image and the
lower image. The gate device 100 identifies a user closest to the gate
device based on the calculated distance, and sets the identified user as a
person to be authenticated. As described above, the gate device 100
15 identifies the user in the forward-most row regardless of the size of the
face region having a large individual difference, so that the user to be
authenticated can be accurately determined.
[0017]
Hereinafter, specific example embodiments will be described in
20 more detail with reference to the drawings.
[0018]
(First example embodiment)
The first example embodiment will be described in more detail
with reference to the drawings.
25 [0019]
Fig. 2 is a diagram illustrating an example of a schematic
configuration of the authentication system according to the first example
embodiment. Referring to Fig. 2, the authentication system includes a
plurality of gate devices 10-1 to 10-3 and a server device 20.
10
[0020]
In the following description, in a case where there is no particular
reason to distinguish the gate devices 10-1 to 10-3, they are simply
referred to as a “gate device 10”. Similarly, for other components,
5 reference numerals on the le ft side divided by hyphens are used to
represent the components.
[0021]
The configuration illustrated in Fig. 2 is an example and is not
intended to limit the number of gate devices 10 or the like . The
10 authentication system may include at least one or more gate devices 10.
[0022]
Gate device 10 and server device 20 are configured to be able to
communicate with each other by wired or wireless communication means.
Server device 20 may be installed in the same building as gate device 10,
15 or may be installed on a network (cloud).
[0023]
The gate device 10 is, for example, a device installed in an airport
or a station. Gate device 10 controls passage of a user. The gate
device 10 permits passage of a user (hereinafter, it is described as an
20 authentication successful person) who has been successfully authenticated
by the server device 20. The gate device 10 does not permit passage of a
user who has not been authenticated by the server device 20 or a user who
has failed in authentication.
[0024]
25 The server device 20 is a device that performs an authentication
process. Specifically, server device 20 stores biological information
about a user (system user) who is permitted to pass through the gate
device 10. The server device 20 performs the authentication process
using the biological information acquired from the gate device 10 and the
11
biological information stored in advance, and notifies the gate device 10
of a result of authentication (authentication succeeded, authentication
failed).
[0025]
5 The gate device 10 opens the gate to permit passage of the
authentication successful person when the authentication successful
person enters the internal area of the gate device (the area surrounded by
the main body of the gate device 10).
[0026]
10 Examples of the biological information about the user include data
(feature amount) calculated from physical features unique to an individual
such as a face and an iris pattern (pattern). Alternatively, the biological
information about the user may be image da ta such as a face image and an
iris image. The biological information about the user may include the
15 physical characteristics of the user as information.
[0027]
Next, details of each device included in the authentication system
according to the first example embodiment will be described. In the
following description, a “face image” of the user will be described as an
20 example of the biological information.
[0028]
[Gate device]
Fig. 3 is a view illustrating an example of an appearance of the
gate device 10 according to the first example embodiment.
25 [0029]
Referring to Fig. 3, the gate device 10 includes a face
authentication module 11. The face authentication module 11 is attached
to the main body of the gate device 10. The face authentication module
11 includes two cameras (an upper camera 12-1 and a lower camera 12-2).
12
The “main body” of the gate device 10 is a structure forming the center of
the gate device 10, and is a member that is in contact with the floor and to
which a face authentication modul e and the like are attached.
[0030]
5 The face authentication module 11 is installed to be inclined at a
predetermined angle with respect to the direction in which the user walks
toward the gate device 10. Fig. 4 is a view (plan view) illustrating a
case where the gate device 10 is viewed from above.
[0031]
10 As illustrated in Fig. 4, the face authentication module 11 is
installed at an angle θ with respect to the traveling direction of the user (a
direction perpendicular to the gate 17). The angle θ is selected
(adjusted) in such a way that the upper camera 12 -1 and the lower camera
12-2 can image the user U in the authentication area.
15 [0032]
When the angle θ is reduced, only the vicinity of the gate device
10 can be imaged, and there is a possibility that the user enters the inside
(internal area) of the gate device 10 before the authentication process
using the face image is ended. In this case, the user is required to stop
20 and stand by until the authentication process is completed, and it cannot
be said that comfortable walk-through can be achieved. On the other
hand, when the angle θ is increased, it is possible to capture an image far
away from the gate device 10. In this case, although it is advantageous
for achieving a comfortable walk-through, there is a possibility that even
25 a user who is not walking toward the gate device may be authenticated .
For example, in the example of Fig. 2, the gate device 10 -1 captures an
image of a user who is not walking toward gate device 10 -1, and requests
the server device 20 for authentication. As described above, when the
angle θ is too large, there is a possibility that the useless authentication
13
process occurs. It is desirable that the administrator and the operator of
the system determine the optimum angle in consideration of the
advantages and disadvantages caused by the setting angle θ of the face
authentication module 11 as described above . As a result of intensive
5 studies by the inventors, it has been found that the angle θ is desirably
selected from a range of 30 to 60 degrees.
[0033]
Although Figs. 3 and 4 illustrate an example in which the face
authentication module 11 is installed on the right side with respect to the
10 traveling direction of the user, it is a matter of course that the installation
position of the face authentication module 11 is not limited to the right
side. The face authentication module 11 may be installed on the left side
with respect to the traveling direction of the user. Although Figs. 3 and
4 illustrate an example in which the face authentication module 11 is
15 directly installed in the main body of the gate device 10, the face
authentication module 11 may be installed away from the main body of the
gate device 10. Alternatively, when the gate device 10 includes the
ceiling portion (roof; not illustrated), the face authentication module 11
may be installed on the ceiling portion. The face authentication module
20 11 may be installed at a predetermined angle with respect to the traveling
direction of the user.
[0034]
The upper camera 12-1 is attached to the vertically upper side of
the face authentication module 11. The lower camera 12-2 is attached to
25 the vertically lower side of the face authentication module 11 . That is,
as illustrated in Fig. 3, the lower camera 12 -2 is installed below the upper
camera 12-1 in the vertical direction.
[0035]
On the surface (camera installation face) of the face authentication
14
module, the positions of the two cameras in the vertical direction (vertical
direction) are different as described a bove, but are attached at the same
position in the horizontal direction (horizontal position). Similarly, in
the depth direction of the two cameras, the two cameras are mounted on
5 the same surface (camera installation face). The meaning of “the
positions of the two cameras in the horizontal direction are the same ”
does not mean that the positions of the two cameras in the horizontal
direction strictly coincide with each other, but allows some deviation .
That is, the positions of the two cameras in the ho rizontal direction may
10 be substantially the same.
[0036]
The upper camera 12-1 and the lower camera 12-2 are cameras
having substantially the same performance, and parameters such as an
angle of view and a focal distance are substantially the same . According
15 to the method of installing the two cameras or the like as described above,
the images obtained from the two cameras have different photographing
ranges in the height direction, but have substantially the same
photographing ranges in the horizontal direction.
[0037]
20 Two cameras (upper camera 12-1 and lower camera 12-2) included
in gate device 10 are provided to authenticate every person (subject) with
a short height to a high height. The upper camera 12-1 is installed to
acquire a face image of a tall person to be authenticated. The lower
camera 12-2 is installed to acquire a face image of a short person to be
25 authenticated.
[0038]
With these cameras, the face image of the tall person to be
authenticated is acquired by the upper camera 12 -1, and the face image of
the short person to be authenticated is acquired by the lower camera 1 2-2.
15
The face image of the medium-height person to be authenticated is
acquired from both the upper camera 12-1 and the lower camera 12-2.
Most of the users are classifi ed as medium height, and it is rare that a face
image is acquired from one of the upper camera 12 -1 and the lower
5 camera 12-2.
[0039]
The upper camera 12-1 and the lower camera 12-2 operate in
conjunction with each other. Specifically, the two cameras image the
user at substantially the same timing.
10 [0040]
As illustrated in Fig. 3, the face authentication module 11 includes
a light emitting diode (LED) (not illustrated) for dimming and a display
13. The gate device 10 controls an environment (light irra diated to the
user) at the time of acquiring the user’s face image using the LED for
15 dimming. Gate device 10 notifies the user of a necessary message or the
like by using display 13. For example, the gate device 10 displays a
result of authentication (authentication succeeded, authentication failed)
by the server device 20.
[0041]
20 The gate device 10 includes a display 14. Gate device 10 notifies
the user of a necessary message or the like by using the display 14 . For
example, the gate device 10 displays a result of authentication
(authentication succeeded, authentication failed) by the server device 20 .
Alternatively, the gate device 10 may notify the user of a necessary
25 message by using a speaker (not illustrated).
[0042]
The gate device 10 includes an area sensor 15. The area sensor
15 is a sensor that detects whether a person is present in a predetermined
area (an area surrounded by a dotted line in Fig. 3) set in front of the gate
16
device 10. The area sensor 15 can be configured using a sensor (so -
called motion sensor) using infrared rays.
[0043]
When confirming that a person is present in the predetermined area
5 based on the detection signal from the area sensor 15, the gate device 10
controls two cameras (upper camera 12-1 and lower camera 12-2) to
acquire two images (image data). In the following description, an area
where the area sensor 15 detects the presence of a person is referred to as
an “authentication area”. When detecting that the user is present in the
10 authentication area, the gate device 10 captures an image of the user
located in the authentication area.
[0044]
The gate device 10 includes an in-gate sensor 16. The in-gate
sensor 16 is a sensor that detects that a person enters the inside of the
15 gate device (gate device 10). As the in-gate sensor 16, for example, a
sensor (so-called passage sensor using light) including an optical
transmission device and an optical reception device can be used . For
example, the optical transmission device and the optical reception device
are installed to face each other (two devices are installed in an inner wall
20 of the main body). The transmission device constantly transmits light,
and the reception device receives the transmitted light. The gate device
10 determines that a person enters gate device 10 when the reception
device cannot receive the light. Fig. 3 illustrates one of the two devices
constituting the in-gate sensor 16. A plurality of pairs of the in-gate
25 sensors 16 may be provided at substantially equal intervals in the
traveling direction of the person.
[0045]
The gate device 10 includes a gate 17. When confirming that the
user is present in the authentication area, the gate device 10 acquires a
17
face image of the user. The gate device 10 generates a feature amount
(biological information) from the acquired image to transmit an
authentication request including the generated feature amount to the
server device 20. The gate device 10 opens the gate 17 when the
5 authentication successful person enters the internal area of the gate
device. The gate device 10 closes the gate 17 after the authentication
successful person passes through the gate 17.
[0046]
The type of the gate 17 is not particularly limited, and is, for
10 example, a flapper gate in which a flapper provided from one side o r both
sides of the passage opens and closes, a turn coil gate in which three bars
rotate, or the like.
[0047]
Fig. 5 is a diagram illustrating an example of a processing
15 configuration (processing module) of the gate device 10 according to the
first example embodiment. Referring to Fig. 5, the gate device 10
includes a communication control unit 201, a user detection unit 202, a
biological information acquisition unit 203, an authentication target
setting unit 204, an authentication request unit 205, an a pproaching
20 person determination unit 206, a gate control unit 207, a message output
unit 208, and a storage unit 209.
[0048]
The communication control unit 201 is a means configured to
control communication with another device . Specifically, the
25 communication control unit 201 receives data (packet) from the server
device 20. The communication control unit 201 transmits data to the
server device 20. The communication control unit 201 delivers data
received from another device to another processin g module. The
communication control unit 201 transmits data acquired from another
18
processing module to another device . In this manner, the other
processing modules transmit and receive data to and from another device
via the communication control unit 201.
[0049]
5 The user detection unit 202 is a means configured to detect a user
located in the authentication area . The user detection unit 202 detects a
user present in the authentication area based on a detection signal from
the area sensor 15. In a case where the output of the area sensor 15
indicates that “there is a user in the authentication area ”, the user
10 detection unit 202 notifies the biological information acquisition unit 203
of the fact.
[0050]
The biological information acquisition unit 203 is a means
configured to acquire the biological information about the user present in
15 the authentication area. Upon acquiring the notification related to “there
is a user in the authentication area ” from the user detection unit 202, the
biological information acquisition unit 203 controls the two cameras
(upper camera 12-1 and lower camera 12-2) and captures an image of the
user located in the authentication area.
20 [0051]
The biological information acquisition unit 203 acquires an image
from each of the two cameras. As described above, the upper camera 12-
1 is attached to the upper side of the face authentication module 11 .
Therefore, the image (hereinafter, it is referred to as an upper image)
25 obtained from the upper camera 12-1 includes the upper side of the
authentication area. For example, when a medium-height user is located
in the authentication area, an upper image surrounded by a broken line in
Fig. 6A is obtained.
[0052]
19
On the other hand, the lower camera 12-2 is attached to the lower
side of the face authentication module 11. Therefore, the image
(hereinafter, it is referred to as a lower image) obtained from the lower
camera 12-2 includes the lower side of the authentication area . For
5 example, when a medium-height user is located in the authentication area,
a lower image surrounded by a broken line in Fig. 6B is obtained.
[0053]
The upper image and the lower image include a common region .
In the example of Fig. 6, a region indicated by a dot-and-dash line in Fig.
10 6C is a region common to the upper image and the lower image . In the
following description, a region common to the upper image and the lower
image is referred to as a “common region”.
[0054]
The biological information acquisition unit 203 delivers two
15 images (upper image, lower image) captured by the upper camera 12-1 and
the lower camera 12-2 to the authentication target setting unit 204.
[0055]
The authentication target setting unit 204 is a means configured to
set (determine) a user (person to be authenticated) to be authenticat ed
20 from users appearing in an image . When there is a plurality of users in
the authentication area, the authentication target setting unit 204 selects
one user from the plurality of users and sets the selected user as the
“person to be authenticated”. The operation of the authentication target
setting unit 204 will be described with reference to the flowchart
25 illustrated in Fig. 7.
[0056]
The authentication target setting unit 204 extracts a face image
(step S101). The authentication target setting unit 204 extracts a face
image from the upper image . The authentication target setting unit 204
20
extracts a face image from the lower image . The authentication target
setting unit 204 extracts a face image from the image using various
methods. For example, the authentication target setting unit 204 extracts
a face image (face region) from the image data using a learning model
5 learned by a convolutional neural network (CNN). Alternatively, the
authentication target setting unit 204 may extract the face image using a
method such as template matching.
[0057]
The authentication target setting unit 204 determines the
10 distribution status of the face images in the two images (step S102). The
authentication target setting unit 204 determines whether the face image is
included only in the upper image or the lower image or whether the face
image is included in both the upper image and the lower image . In the
former case, the process in and after step S103 is performed . In the
15 latter case, the process in and after st ep S110 is performed.
[0058]
When step S102 is performed and it is determined that the face
image is distributed in the upper image or the lower image, the
authentication target setting unit 204 determines whether the number of
20 face images included only in the upper image or the lower image is one
(step S103).
[0059]
When one face image is included in the upper image or the lower
image (step S103: Yes branch), the authentication target setting unit 204
25 sets the one face image as the authentication target (step S104).
[0060]
For example, as illustrated in Fig. 8A, the face image of the tall
person to be authenticated is included only in the upper image . As
illustrated in Fig. 8B, the face image of the short person to be
21
authenticated is included only in the lower image. In the case illustrated
in Fig. 8, the authentication target setting unit 204 sets the face image
included in the upper image and the face image included in the lower
image as the authentication target (the face image of the person to be
5 authenticated). In the drawings including Fig. 8, the range of each
image is horizontally moved in order to facilitate distinction between the
upper image and the lower image.
[0061]
When two or more face images are included in the upper image or
10 the lower image (step S103: No branch), the authentication target setting
unit 204 sets an error (step S105).
[0062]
For example, as illustrated in Figs. 9A and 9B, an error is set when
a plurality of face images is included only in the upper image or the lower
15 image. In the case illustrated in Fig. 9, it is difficult to determine which
of the plurality of face images is set as the authentication target .
However, the situation illustrated in Fig. 9 rarely occurs. When such a
situation occurs, the gate device 10 takes a measure such as instructing
the user to leave the authentication area.
20 [0063]
As described above, in a case where the face image is distributed
in each of the upper image and the lower image, the process in and after
step S110 illustrated in Fig. 7 is performed.
[0064]
25 The authentication target setting unit 204 determines whether one
face image is included in each of the upper image and the lower image
(step S110).
[0065]
Specifically, the authentication target setting unit 204 determines
22
whether a situation as illustrated in Fig. 10A has occurred or a situation as
illustrated in Fig. 10B has occurred.
[0066]
When one face image is included in each of the upper image and
5 the lower image (step S110: Yes branch), the authentication target se tting
unit 204 sets a face image included in either the upper image or the lower
image as the authentication target (step S111). In the example of Fig.
10A, a face image extracted from either the upper image or the lower
image is set as the authentication target.
10 [0067]
When two or more face images are included in each of the upper
image and the lower image (step S110: No branch), the authentication
target setting unit 204 identifies the front/rear relationship of each user
(step S112). In the example of Fig. 10B, which of the user U1 and the
15 user U2 is walking in front is identified (determined). Details regarding
the identification process of the front/rear relationship will be described
later.
[0068]
The authentication target setting unit 204 sets a user in the
20 forward-most row (the forefront) among a plurality of users (a plurality of
users present in the authentication area) as the authentication target (step
S113). In the example of Fig. 10B, in a case where the user U1 is
walking in front of the user U2 (in a case where the user U1 is closer to
the gate device 10 than the user U2), the face image of the user U1 is set
25 as the authentication target.
[0069]
As described above, when one user appears in both the upper
image and the lower image, the authentication target setting unit 204 sets
the one user as the person to be authenticated . When one user appears in
23
one of the upper image and the lower image, the authentication target
setting unit 204 sets the one user as the person to be authenticated.
[0070]
The authentication target setting unit 204 desirably confirms
5 whether the users appearing in the upper image and the lower image are
identical before setting the authentication target in step S111 .
Specifically, the authentication target setting u nit 204 calculates the
feature amount from each of the two face images. The authentication
target setting unit 204 determines whether the two feature amounts
10 substantially match. The authentication target setting unit 204
calculates similarity between two feature amounts (feature vectors).
When the calculated similarity is equal to or more than a predetermined
value, the authentication target setting unit 204 determines that the two
feature amounts substantially match. That is, the authentication target
15 setting unit 204 determines whether the two face images are the face
image of the identical person by one -to-one collation using the feature
amounts generated from the upper face image and the lower face image .
When the two feature amounts do not match, the authentication target
setting unit 204 may set an error. That is, a situation in which one face
20 image is included in each of the upper image and the lower image and
each face image is not the face image of the identical person does not
usually occur. Therefore, it is appropriate to set the situation as an
error.
[0071]
25 In a case where there are one person to be authenticated with a
high height and one person to be authenticated with a short height in the
authentication area, the processing related to step S112 is performed.
However, in this case, since there is no common region between the upper
image and the lower image, the front/rear relationship in step S112 cannot
24
be identified. Therefore, when the above situation occurs, the
authentication target setting unit 204 sets an error.
[0072]
Next, the process (the identification process of the front/rear
5 relationship) according to step S112 in Fig. 7 will be described . Fig. 11
is a flowchart illustrating an example of the front/rear relationship
identification process of the authentication target setting unit 204.
[0073]
The authentication target setting unit 204 sets a “distance
10 measurement band” in the common region of the two images (upper
image, lower image) (step S201). For example, in a case where the
upper image and the lower image illustrated in Fig. 12 are obtained, the
region colored in gray is set as the “distance measurement band”. As
illustrated in Fig. 12, the distance measurement band is desirably set on
15 the lower side of the upper image. The distance measurement band is set
to a region in which two or more users simultaneously appear in the
common region. In an extreme example, the distance measurement band
may be set at the lowermost stage of the upper image, or the entire
common region may be set as the distance measurement band . The
20 authentication target setting unit 204 sets all or part of the upper image
and the lower image in the distance measurement band.
[0074]
The authentication target setting unit 204 calculates the dist ance to
the camera installation face for the target object appearing in the distance
25 measurement band (all or part of the common region common to the upper
image and the lower image) (step S202). Specifically, the authentication
target setting unit 204 calculates the distance to the camera (upper camera
12-1 and lower camera 12-2) installation face for each small region
included in the distance measurement band . The authentication target
25
setting unit 204 calculates the distance between each small region (target
object) and the camera installation face using the parallax (deviation
between upper image and lower image) of the two images.
[0075]
5 The authentication target setting unit 204 calculates a deviation
amount (parallax) between the upper image and th e lower image regarding
the target object included in the distance measurement band . The
authentication target setting unit 204 calculates the distance between the
target object and the camera installation face using the calculated
10 parallax. That is, the authentication target setting unit 204 calculates
the distance between the camera installation face and the target object
(small region of the distance measurement band) using a so -called stereo
method. Although the calculation of the distance using the stereo
method is obvious to those skilled in the art and detailed description is
15 omitted, the authentication target setting unit 204 calculates the distance
between the camera installation face and the target object by the
following method.
[0076]
The authentication target setting unit 204 divides the distance
20 measurement band into a plurality of small regions. For example, the
authentication target setting unit 204 divides the distance measurement
band such that the size of one region is A1 × A2 pixels. The
authentication target setting unit 204 calculates the distance D between
the small region (target object) and the camera installation face by the
25 following Expression (1).
[0077]
[Math. 1]
D = B × f/Z
••• (1)
26
[0078]
In Expression (1), B is a distance (camera interval) between the
upper camera 12-1 and the lower camera 12-2. f is a focal distance of
the two cameras. Z is parallax (deviation amount). Fig. 13 illustrates
5 the relationship between the parameters expressed by Expression (1).
[0079]
In Expression (1), the camera interval B and the focal distance f
are values determined in advance according to the installation and
selection of the two cameras. Therefore, the authentication target
10 setting unit 204 can calculate the distance D by ob taining the parallax Z.
[0080]
The authentication target setting unit 204 searches for a region in
which patterns match in a small region forming a common region band of
the reference camera (the upper camera 12 -1) with respect to one small
15 region in the image (distance measurement band) of the reference camera
(the lower camera 12-2 in Fig. 13). For example, the authentication
target setting unit 204 calculates a city block distance for each small
region, and selects a region having the shortest distance as a region in
which the patterns match. The authentication target setting unit 204
20 calculates a deviation between a small region on the reference camera side
and a small region on the reference camera side selected as described
above as the parallax Z.
[0081]
For example, when the distance D between the target object
25 included in the distance measurement band illustrated in Fig. 12 and the
camera installation face is calculated, and the position (position in the
horizontal direction) of the target object is set to the X axis and the
calculated distance D is set to the Y axis, a graph as illustrated in Fig. 14
is obtained.
27
[0082]
In order to obtain a waveform as illustrated in Fig. 14, the
authentication target setting unit 204 desirably performs a predet ermined
data process or the like on the calculated distance D. For example, in
5 order to remove a high frequency component (a high frequency component
of the distance D) caused by noise included in the image, the
authentication target setting unit 204 may apply a low pass filter (LPF) to
the calculated distance D. Alternatively, in a case where the variation of
the distance D falls within a predetermined range, the authentication
10 target setting unit 204 may perform a process of converging respective
values of the distance D to a predetermined value (for example, the
average value of the variation values).
[0083]
As illustrated in Fig. 14, when a plurality of users is imaged by the
15 upper camera 12-1 and the lower camera 12-2, peaks according to the
number of people appearing in the two images appear. In the examples
of Figs. 12 and 14, two peaks appear.
[0084]
The authentication target setting unit 204 identifies a position (X
20 coordinate) where each peak appears (step S203). In the example of Fig.
14, coordinates X1 and X2 are identified. That is, the authentication
target setting unit 204 identifies (searches for) peaks according to the
number of the plurality of users in the position of the target object (the
position of the target object in the image) and the distance (for example, a
25 graph of Fig. 14 illustrating the relationship between the position and the
distance) to the camera installation face for the target object.
[0085]
The authentication target setting unit 204 identifies the peak of the
distance D using any method. For example, the authentication target
28
setting unit 204 calculates a difference between data regarding the
distance D, and sets a point at which the calculated difference changes
from positive to negative as a peak. It is desirable that the
authentication target setting unit 204 identify the peak from the data to
5 which the filtering process or the convergence process is applied.
[0086]
The authentication target setting unit 204 selects the X coordinate
related to the minimum peak value (step S204). The user (face image)
located on the vertically upper side of the selected X coordinate is
10 identified as the person in the forward-most row of the authentication
area. In the examples of Figs. 12 and 14, the user U1 located above the
coordinate X1 is identified as the user in the forward -most row. That is,
the authentication target setting unit 204 regards a user at the position
related to the minimum peak value among the plurality of identified peaks
15 as the user in the forward-most row.
[0087]
The authentication target setting unit 204 sets the face image of
the user in the forward-most row as the authentication target (step S113 in
Fig. 7). As described above, in a case where a plurality of users appears
20 in both the upper image and the lower image, the authentication target
setting unit 204 calculates the distance between each of the plurality of
users and the camera installation face to identify a user in the forward -
most row from among the plurality of users. The authentication target
setting unit 204 sets the identified user as a person to be authenticated.
25 [0088]
The authentication target setting unit 204 delivers the face image
(biological information) of the person to be authenticated to the
authentication request unit 205.
[0089]
29
The authentication request unit 205 is a means configured to
request the server device 20 to authenticate the person to be
authenticated. The authentication request unit 205 generates a feature
amount (a feature vector including a plurality of feature amounts) from
5 the face image acquired from the biological information acquisition unit
203. The authentication request unit 205 transmits an authentication
request including the generated feature amount (biological information) to
the server device 20.
[0090]
10 An existing technique can be used for the feature amount
generation process by the authentication request unit 205, and thus a
detailed description thereof will be omitted . For example, the
authentication request unit 205 extracts eyes, a no se, a mouth, and the like
as feature points from the face image . Thereafter, the authentication
15 request unit 205 calculates the position of each feature point and the
distance between the feature points as feature amounts to generate a
feature vector (vector information characterizing the face image)
including a plurality of feature amounts.
[0091]
20 The authentication request unit 205 generates an authentication
request including an identifier (hereinafter, referred to as a gate
identifier) of the gate device, a feature amount, and the like (see Fig. 15).
As the gate identifier, a media access control (MAC) address or an
Internet protocol (IP) address of the gate device 10 can be used.
25 [0092]
When transmitting the authentication request to the server devi ce
20, the authentication request unit 205 delivers the feature amount
included in the request to the approaching person determination unit 206.
[0093]
30
The authentication request unit 205 receives a response to the
authentication request from the server device 20. The authentication
request unit 205 delivers a response (authentication succeeded,
authentication failed) from the server device 20 to the gate control unit
5 207.
[0094]
The approaching person determination unit 206 is a means
configured to detect an approaching person using an in-gate sensor 16 that
detects an approaching person into the gate device (gate device 10) and
10 determines whether the approaching person and the person to be
authenticated are identical. The approaching person determination unit
206 monitors (polls) the output of the in-gate sensor 16. When the
output of the in-gate sensor 16 indicates “there is an approaching person
inside”, the approaching person determination unit 206 determines
15 whether the approaching person and the p erson to be authenticated match.
Specifically, similarly to the biological information acquisition unit 203,
the approaching person determination unit 206 controls at least one
camera to acquire a face image . The approaching person determination
unit 206 calculates a feature amount (a feature amount of an approaching
20 person) from the obtained face image.
[0095]
The approaching person determination unit 206 determines whether
the calculated feature amount (the feature amount of the approaching
person) and the feature amount (the feature amount of the person to be
25 authenticated) acquired from the authentication request unit 205
substantially match. Specifically, the approaching person determination
unit 206 calculates the similarity between the two feature amounts. A
chi-square distance, a Euclidean distance, or the like can be used as the
similarity. The similarity is lower as the distance is longer, and the
31
similarity is higher as the distance is shorter. When the similarity is
equal to or more than a predetermined value, the approaching person
determination unit 206 determines that the two feature amounts
substantially match (the approaching person and the person to be
5 authenticated are identical).
[0096]
In this manner, the approaching person determi nation unit 206
detects an approaching person based on the detection signal from the
passage sensor (the in-gate sensor 16) using light. Thereafter, the
10 approaching person determination unit 206 performs one -to-one collation
using two feature amounts (fea ture amount of approaching person, feature
amount of person to be authenticated). Whether the approaching person
and the person to be authenticated are identical is determined by the one -
to-one collation (one-to-one collation using the biological information
15 about the approaching person and the biological information about the
person to be authenticated).
[0097]
When determining that the approaching person and the person to be
authenticated are identical, the approaching person determination unit 206
20 notifies the gate control unit 207 of the determination.
[0098]
The gate control unit 207 is a means configured to control the gate
17 included in the gate device 10. The gate control unit 207 controls the
gate 17 based on the result of authentication of the person to be
25 authenticated. The gate control unit 207 opens the gate 17 when the
result of authentication of the server device 20 is “authentication
succeeded” and the approaching person and the person to be authenticated
are identical. In other words, in principle, the gate control unit 207 does
not open the gate 17 unless the above condition (authentication succeeded,
32
the approaching person and the person to be authenticated are identical) is
satisfied. The gate control unit 207 closes the gate 17 after a user
permitted to pass by using a distance sensor or the like passes through the
gate 17.
5 [0099]
The message output unit 208 is a means configured to output a
message to be notified to the user (person to be authenticated,
authentication successful person, etc.). The message output unit 208
notifies the user of a necessary message using the displays 13 and 14, a
10 speaker, or the like.
[0100]
The storage unit 209 is a means configured to store information
necessary for the operation of the gate de vice 10.

WE CLAIM:
[Claim 1]
A gate device comprising:
an acquisition unit that acquires an upper image and a lower image
5 from an upper camera and a lower camera installed below the upper
camera in a vertical direction, respectively;
a setting unit that, when a plurality of users appears in both of the
upper image and the lower image, identifies a user in a forward -most row
from among the plurality of users by calculating a distance between each
10 of the plurality of users and a camera installation face, and sets the
identified user as a person to be authenticated; and
a gate control unit that controls a gate based on a result of
authentication of the person to be authenticat ed.
[Claim 2]
15 The gate device according to claim 1, wherein the setting unit
calculates a distance to the camera installation face from a target object
appearing in a common region common to the upper image and the lower
image, and identifies the user in the forward-most row.
[Claim 3]
20 The gate device according to claim 2, wherein the setting unit
calculates parallax between the upper image and the lower image of the
target object, and calculates a distance between the target object and the
camera installation face by a stereo method using the calculated parallax.
[Claim 4]
25 The gate device according to claim 2 or 3, wherein the setting unit
identifies peaks according to the number of the plurality of users in a
relationship between a position of the target object and a distance to a
camera installation face from the target ob ject, and identifies a user at a
position related to a minimum peak value among the plurality of identified
59
peaks as the user in the forward-most row.
[Claim 5]
The gate device according to any one of claims 1 to 4, wherein,
when one user appears in both the upper image and the lower image, the
5 setting unit sets the one user as the person to be authenticated.
[Claim 6]
The gate device according to any one of claims 1 to 4, wherein,
when one user appears in one of the upper image and the lower image, the
setting unit sets the one user as the person to be authenticated.
10 [Claim 7]
The gate device according to any one of claims 1 to 6, further
comprising an authentication request unit that transmits an authentication
request including biological information about the person to be
authenticated to a server device.
15 [Claim 8]
The gate device according to claim 7, wherein the authentication
request unit transmits the authentication request including a feature
amount generated from a face image of the person to be authenticated to
the server device.
20 [Claim 9]
The gate device according to any one of claims 1 to 8, wherein a
position of the upper camera and a position of the lower camera on the
camera installation face in a horizontal direction are identical.
[Claim 10]
25 An authentication system comprising:
a server device that stores biological information about each of a
plurality of system users and performs an authentication process using the
plurality of pieces of biological information; and
a gate device connected to the server device, wherein
60
the gate device includes:
an acquisition unit that acquires an upper image and a lower image
from an upper camera and a lower camera installed below the upper
camera in a vertical direction, respectively;
5 a setting unit that, when a plurality of users appears in both of the
upper image and the lower image, identifies a user in a forward -most row
from among the plurality of users by calculating a distance between each
of the plurality of users and a camera installatio n face, and sets the
identified user as a person to be authenticated ;
10 an authentication request unit that requests the server device to
authenticate the person to be authenticated; and
a gate control unit that controls a gate based on a result of
authentication of the person to be authenticated.
[Claim 11]
15 The authentication system according to claim 10, wherein
the authentication request unit transmits an authentication request
including biological information about the person to be authenticated to
the server device, and
the server device authenticates the person to be authenticated by
20 using biological information acquired from the authentication request and
biological information about the plurality of system users.
[Claim 12]
A method of controlling a gate device, the method comprising:
by the gate device,
25 acquiring an upper image and a lower image from an upper camera
and a lower camera installed below the upper camera in a vertical
direction, respectively;
when a plurality of users appears in both of the upper image and
the lower image, identifying a user in a forward -most row from among the
61
plurality of users by calculating a distance between each of the plurality
of users and a camera installation face, and setting the identified use r as a
person to be authenticated; and
controlling a gate based on a result of authentication of the person
5 to be authenticated.
[Claim 13]
A computer-readable storage medium storing a program for causing
a computer mounted on a gate device to execute pr ocesses of:
acquiring an upper image and a lower image from an upper camera
10 and a lower camera installed below the upper camera in a vertical
direction, respectively;
when a plurality of users appears in both of the upper image and
the lower image, identifying a user in a forward-most row from among the
plurality of users by calculating a distance between each of the plurality
15 of users and a camera installation face, and setting the identified user as a
person to be authenticated; and
controlling a gate based on a result of authentication of the person
to be authenticated.