CLIAMS:1. A people identification system (102) for pose and sub-pose clustering-based identification of individuals, the people identification system (102) comprising:
a processor (202); and
a gait-pose modeler(110) coupled to the processor (202), to
obtain static feature vectors and dynamic feature vectors for skeleton data frames of each of the individuals performing a step activity;
create, for the each of the individuals, a first predefined number of clusters based on clustering of the dynamic feature vectors for the skeleton data frames, wherein the first predefined number is a number of poses to be identified in the step activity;
create, for the each of the individuals, a second predefined number of sub-clusters within the each of the clusters based on clustering of the dynamic feature vectors for the skeleton data frames associated with the each of the clusters, wherein the second predefined number is a number of sub-poses to be identified in the poses; and
determine, for the each of the individuals, a gait-pose feature data set based on computation of a center of the dynamic feature vectors for the skeleton data frames associated with the each of the sub-clusters, and a mean of the static feature vectors for the skeleton data frames associated with the each of the clusters for identifying the individuals uniquely.

2. The people identification system (102) as claimed in claim 1, wherein the gait-pose modeler (110) creates the clusters based on a predefined clustering method comprising one of K-means clustering method and Fuzzy C-means clustering method.

3. The people identification system (102) as claimed in claim 1, wherein the gait-pose modeler (110) creates the sub-clusters based on a predefined clustering method comprising one of K-means clustering method and Fuzzy C-means clustering method.

4. The people identification system (102) as claimed in claim 1 further comprising a feature extraction module (212) coupled to the processor (202), to
receive the skeleton data frames for the each of the individuals from a skeleton recording device (106), and
extract the static feature vectors and the dynamic feature vectors from each of the skeleton data frames received for the each of the individuals.

5. The people identification system (102) as claimed in claim 1 further comprising a noise remover (214) coupled to the processor (202), to
detect, for the each of the individuals, noisy frames, from amongst the skeleton data frames, based on clustering of the static feature vectors obtained for the skeleton data frames for the each of the individuals, wherein the noisy frames are the skeleton data frames in which static gait features are sparsely distributed; and
remove the noisy frames from the skeleton data frames.

6. The people identification system (102) as claimed in claim 5, wherein, to detect the noisy frames, the noise remover (214)
computes absolute difference values of the static feature vectors of successive skeleton data frames;
performs a histogram analysis for the absolute difference values to populate counts in predefined bins;
determines a first starting center as a data point corresponding to center of a bin having maximum count;
determines a second starting center as a data point corresponding to a static feature vector farthest from the first starting center; and
clusters the static feature vectors into two clusters, based on 2-cluster clustering method performed with the first starting center and the second starting center.

7. The people identification system (102) as claimed in claim 1 further comprising an identification module (112) coupled to the processor (202), to
train a classifier with the gait-pose feature data set determined for the each of the individuals, and
identifying the individuals uniquely using the classifier.

8. The people identification system (102) as claimed in claim 1, wherein each of the static feature vectors comprises multiple static gait features, and each of the dynamic feature vectors comprises multiple dynamic gait features.

9. A method for identifying individuals based on pose and sub-pose clustering, the computer-implemented method comprising:
obtaining, by a processor (202), static feature vectors and dynamic feature vectors for skeleton data frames of each of the individuals performing a step activity with an arbitrary pattern and in a random path;
creating, by a processor (202), for the each of the individuals, a first predefined number of clusters based on clustering of the dynamic feature vectors for the skeleton data frames, wherein the first predefined number is a number of poses to be identified in the step activity;
creating, by a processor (202), for the each of the individuals, a second predefined number of sub-clusters within the each of the clusters based on clustering of the dynamic feature vectors for the skeleton data frames associated with the each of the clusters, wherein the second predefined number is a number of sub-poses to be identified in the poses;
determining, by a processor (202), for the each of the individuals, a gait-pose feature data set based on computation of a center of the dynamic feature vectors for the skeleton data frames associated with the each of the sub-clusters, and a mean of the static feature vectors for the skeleton data frames associated with the each of the clusters; and
training, by a processor (202), a classifier with the gait-pose feature data set determined for the each of the individuals, for identification of the individuals uniquely.

10. The method as claimed in claim 9, wherein the first predefined number of clusters is a number within a range from 2 to 10.

11. The method as claimed in claim 9, wherein the second predefined number of sub-clusters is a number within a range from 2 to 10.

12. The method as claimed in claim 9, wherein the clustering is based on a predefined clustering methods comprising one of K-means clustering method and Fuzzy C-means clustering method.

13. The method as claimed in claim 9, wherein the obtaining comprises:
receiving the skeleton data frames for the each of the individuals from a skeleton recording device (106), and
extracting the static feature vectors and the dynamic feature vectors from each of the skeleton data frames received for the each of the individuals.

14. The method as claimed in claim 9 further comprising:
detecting, by a processor (202), for the each of the individuals, noisy frames, from amongst the skeleton data frames, based on clustering of the static feature vectors obtained for the skeleton data frames for the each of the individuals, wherein the noisy frames are the skeleton data frames in which static gait features are sparsely distributed; and
removing, by a processor (202), the noisy frames from the skeleton data frames before the clustering.

15. A non-transitory computer readable medium having a set of computer readable instructions that, when executed, cause a computing system to:
obtain static feature vectors and dynamic feature vectors for skeleton data frames of individuals performing a step activity;
create, for each of the individuals, a first predefined number of clusters based on clustering of the dynamic feature vectors for the skeleton data frames, wherein the first predefined number is a number of poses to be identified in the step activity;
create, for the each of the individuals, a second predefined number of sub-clusters within the each of the clusters based on clustering of the dynamic feature vectors for the skeleton data frames associated with the each of the clusters, wherein the second predefined number is a number of sub-poses to be identified in the poses; and
determine, for the each of the individuals, a gait-pose feature data set based on computation of a center of the dynamic feature vectors for the skeleton data frames associated with the each of the sub-clusters, and a mean of the static feature vectors for the skeleton data frames associated with the each of the clusters for identifying the individuals uniquely.
,TagSPECI

PD010296IN-SC
FORM 2
THE PATENTS ACT, 1970
(39 of 1970)
&
THE PATENTS RULES, 2003
COMPLETE SPECIFICATION
(See section 10, rule 13)
1. Title of the invention: POSE AND SUB-POSE CLUSTERING-BASED IDENTIFICATION
OF INDIVIDUALS
2. Applicant(s)
NAME NATIONALITY ADDRESS
TATA CONSULTANCY
SERVICES LIMITED
Indian Nirmal Building, 9th Floor, Nariman
Point, Mumbai, Maharashtra 400021,
India
3. Preamble to the description
COMPLETE SPECIFICATION
The following specification particularly describes the invention and the manner in which it
is to be performed.
1
2
TECHNICAL FIELD
[0001] The present subject matter relates, in general, to identification of
individuals and in particular, to identification of individuals based on pose and
sub-pose clustering.
BACKGROUND
[0002] Individuals are generally identified by recognizing the identity of the
individuals through their physical and behavioral characteristics. This act of
recognizing the identity of the individuals is referred to as identification of
individuals or people identification. The individuals may have to be identified for
a variety of purposes, such as human surveillance, security control, and people
counting. In such scenarios, the individuals have to be identified distinctly and
robustly with substantial efficiency and accuracy.
BRIEF DESCRIPTION OF DRAWINGS
[0003] The detailed description is described with reference to the
accompanying figures. In the figures, the left-most digit(s) of a reference number
identifies the figure in which the reference number first appears. The same
numbers are used throughout the drawings to refer features and components.
[0004] Figure 1 illustrates a system environment implementing a people
identification system, in accordance with an implementation of the present subject
matter.
[0005] Figure 2 illustrates the people identification system, in accordance with
an implementation of the present subject matter.
[0006] Figure 3 illustrates a method for identification of individuals based on
pose and sub-pose clustering, in accordance with an implementation of the present
subject matter.
[0007] It should be appreciated by those skilled in the art that any block
diagrams herein represent conceptual views of illustrative systems embodying the
3
principles of the present subject matter. Similarly, it will be appreciated that any
flow charts, flow diagrams, state transition diagrams, pseudo code, and the like
represent various processes which may be substantially represented in computer
readable medium and so executed by a computing device or processor, whether or
not such computing device or processor is explicitly shown.
DETAILED DESCRIPTION
[0008] The subject matter disclosed herein relates to system(s) and method(s)
for identification of individuals based on pose and sub-pose based clustering. For
the purposes of the present subject matter, identification refers to an act of
establishing or recognizing people or individuals performing a step activity, such
as walking, jogging, and running. The motion of the individual is related to two
aspects: gait and pose. Gait is a spatiotemporal phenomenon related to walking
style, or jogging style, or running style of the individual, and pose refers to a
posture of the individual while performing the step activity. The pose further
includes sub-postures, referred to as sub-poses.
[0009] Individuals are commonly identified automatically based on their
physical and behavioral characteristics, such as voice or speech, face, iris, and
fingerprint. Systems and methods for identification of individuals based on such
characteristics are intrusive as they require a direct human interaction. Moreover,
it is substantially difficult to extract biometric features like iris, speech, and
fingerprints, from a large distance. As a result, such systems and methods fail to
identify the individuals from a distance.
[0010] The identification of individuals based on their walking patterns that
define gait is known. Gait-based identification is non-intrusive, as the gait features
of the individuals can be captured or extracted from a relatively large distance
without any direct interaction. In gait-based identification, a system is first trained
with the characteristic gait features of the individuals for their distinct
identification. The system is then used to identify the individuals based on their
gait features captured or extracted in real-time. During the training phase, each
individual is required to perform walking activity. While the individual is
4
walking, his skeleton data is captured using a skeleton recording device or a
sensor. The skeleton data is processed to extract gait features, including static and
dynamic gait features. Conventionally, the systems are trained for the gait features
extracted for walking patterns of individuals with predefined walking paths.
During the identification of individuals in real-time, when the individual performs
the walking activity with an arbitrary pattern in a path other than the predefined
walking paths, the extraction of the gait features is error-prone and the system
may give an erroneous identification result. Thus, the conventional systems are
not able to identify individuals with substantial accuracy for the case when they
are performing walking activity with an arbitrary pattern and following a random
path.
[0011] Further, the skeleton data signals captured from a skeleton recording
device may have noise. Conventional systems and methods do not consider the
effect of noisy signals. The noise in the signals may result in extraction of false
features which may adversely affect the identification of individuals.
[0012] Further, some conventional systems and methodologies analyze
silhouette shapes during motion of the individual to identify gait characteristics.
The silhouette is an image of the individual depicted as an outline of that
individual. Such systems and methodologies utilize representations such as
motion energy image, motion history image, gait energy image, gait history
image, and frame difference energy image. These representations require motion
videos and silhouettes of substantially high quality which may not be available in
an open and unrestricted environment. This may also result into an inaccurate
identification of the individuals.
[0013] The present subject matter describes system(s) and method(s) for
identification of individuals based on pose and sub-pose clustering. The systems
and the methods of the present subject matter are capable of identifying
individuals performing a step activity, such as walking, jogging, or running, with
an arbitrary activity pattern and in an unconstrained environment. The arbitrary
activity pattern may refer to a step activity pattern with any style of movement
and following any path. By unconstrained environment, it may be understood that
5
the region or the area where the individual is performing the step activity is
neither predefined nor restricted.
[0014] The systems and the methods of the present subject matter are based
on correlation of gait features in gait cycles with the poses and sub-poses of the
individuals, which is exploited for identification of individuals distinctly. As
mentioned earlier, gait is a spatio-temporal phenomenon related to motion of the
individual. A gait cycle or pattern, starting with one foot forward and ending with
the same foot forward, is composed of distinct poses. A pose refers to a posture of
the individual, for example, in which the legs are maximum spread or are
together. Each pose may have variations which can be referred to as sub-postures
or sub-poses. So, the arbitrary pattern of the step activity has poses which further
have sub-poses. The poses and the sub-poses may occur periodically in successive
gait cycles. By correlating the gait features over multiple gait cycles with poses
and subs-poses of an individual, a set of gait cycle independent features
corresponding to characteristic poses and sub-poses of that individual can be
obtained. This set of gait cycle independent features can then be used to identify
that individual distinctly.
[0015] In an implementation, for the identification of individuals performing a
step activity with an arbitrary activity pattern and in an unconstrained
environment, signals containing skeleton data for multiple skeleton joints are
obtained. The signals can be obtained from a skeleton recording device. These
signals are generated by the skeleton recording device in the form of frames,
referred to as skeleton data frames. Each of the frames is processed to extract a
static feature vector and a dynamic feature vector. The static feature vector is a
vector comprising multiple static gait features, and the dynamic feature vector is a
vector comprising multiple dynamic gait features. In an example, the static feature
vector may comprise static gait features such as height, lengths of both legs, torso,
both lower legs, both thighs, both upper arms, and both forearms of the individual.
In the example, the dynamic feature vector may comprise dynamic gait features
such as velocity, step-length, area-related features, angle-related features, and
dynamic centroid distance-related features.
6
[0016] In an implementation, from amongst the frames for each individual,
noisy frames are identified. The noisy frames can be identified based on analysis
of static feature vectors for the frames, as the mutual Euclidean distance between
two skeleton joints or, in other words, between two static feature vectors for any
two frames should not vary. In case the distance is more than a predefined
threshold value, the frames can be understood to have noise. In an
implementation, the identification of the noisy frames is based on clustering of the
static feature vectors for all the frames. For this, the static feature vectors may be
partitioned into two clusters using a predefined clustering method. The cluster in
which points representing the static feature vectors are sparsely distributed is
identified as a noisy cluster, and the frames associated with the sparsely
distributed static feature vectors are referred to as the noisy frames. The noisy
frames are not considered for further processing. In an implementation, the noisy
frames may be removed from amongst the frames before further processing.
Unlike the conventional methodologies, the identification of the noisy frames
facilitates in increasing the accuracy and reliability of identification of the
individuals.
[0017] Further, for each individual, after the removal of the noisy frames, a
first predefined number of clusters are created based on clustering of the dynamic
feature vectors extracted from the frames. The clustering can be done using a
predefined clustering method. The first predefined number is indicative of number
of poses that are to be identified in the arbitrary activity pattern of the individual.
As mentioned earlier, the pose refers to a posture of the individual within his
activity pattern. The dynamic feature vectors are clustered into poses, as the
dynamic feature vectors contain most of the spatio-temporal variations within the
activity pattern with respect to time. The dynamic feature vectors represented by
points in one cluster are characteristic of one pose of the individual in his activity
pattern. Thus, with a first predefined number of clusters, the dynamic feature
vectors, and thus the frames, are segregated into the first predefined number of
characteristic poses of the individual.
7
[0018] Further, as mentioned earlier, each distinct pose of the individual may
be composed of multiple sub-poses. Thus, for each individual, a second
predefined number of sub-clusters within each cluster are created based on
clustering of the dynamic feature vectors for the frames associated that cluster.
The clustering can be done using a predefined clustering method. The second
predefined number is indicative of number of sub-poses that are to be identified in
the poses. The dynamic feature vectors represented by points in one sub-cluster
are characteristic of one sub-pose within a pose of the individual in his activity
pattern. Thus, with a second predefined number of sub-clusters in each cluster, the
dynamic feature vectors, and thus the frames, are segregated into the second
predefined number times the first predefined number of characteristic sub-poses
of the individual.
[0019] Once the clusters and sub-clusters of the dynamic feature vectors are
created for the individual performing the step activity with the arbitrary activity
pattern, a gait-pose feature data set is determined for the individual. The gait-pose
feature data set is based on computation of a center of the dynamic feature vectors
for the frames associated with the each of the sub-clusters. The gait-pose data set
is appended with a mean of the static feature vectors for the frames associated
with the each of the clusters to form an updated gait-pose feature data set.
[0020] Further, a classifier can be trained with the gait-pose feature data set
determined for each of the individuals for the identification of the individuals
uniquely.
[0021] The identification of individuals based on gait-pose and sub-pose
correlation makes the systems and the methods of the present subject matter
agnostic of the activity pattern of the individuals and the path followed while
performing the step activity. Thus, the system can be trained with the gait-pose
feature data set for each individual following any arbitrary activity pattern in any
path, and the system can uniquely identify individuals performing a step activity
with any other arbitrary activity pattern and following any path. In addition, since
the noisy frames are identified and not considered for processing, the present
subject matter provides a robust methodology of identifying the individuals with a
8
substantial accuracy. Also, since the present subject matter is agnostic of activity
patterns and paths following during the step activity, the systems and the methods
can be implemented to identify individuals performing a step activity in an
unconstrained indoor/outdoor environment.
[0022] The manner in which the system(s) and method(s) shall be
implemented has been explained in details with respect to Figure 1 to Figure 3.
Although the description herein is with reference to laptop(s), the method(s) and
system(s) may be implemented in other computing device(s) as well, albeit with a
few variations, as will be understood by a person skilled in the art. While aspects
of described methods can be implemented in any number of different computing
devices, transmission environments, and/or configurations, the implementations
are described in the context of the following computing device(s).
[0023] Figure 1 illustrates a system environment 100 implementing a people
identification system 102, in accordance with an implementation of the present
subject matter. For the purpose of simplicity of description, the people
identification system 102 is hereinafter referred to as the system 102. The system
102 may be implemented in a variety of computing systems, such as a laptop
computer, a desktop computer, a notebook, a workstation, a mainframe computer,
a server, a network server, and the like. The system is enabled to identify people
or individuals 104-1, 104-2, … , 104-N, based on their walking patterns for
walking activity. The walking pattern of the individual 104-1, 104-2, … , 104-N
may be arbitrary, and the individuals 104-1, 104-2, … , 104-N may be following
any walking path while performing the walking activity. The individuals 104-1,
104-2, … , 104-N, hereinafter are collectively referred to as individuals 104, and
individually referred to as an individual 104. Although the description herein is
described for identification of individuals 104 based on their walking patterns for
walking activity, the system 102 can be configured to identify the individuals 104
based on their activity patterns for any step activity, such as jogging, or running.
[0024] As shown in the figure, the system environment 100 includes a
skeleton recording device 106 that captures skeleton data points of each individual
104 performing a walking activity with an arbitrary walking pattern and following
9
any walking path. The skeleton recording device 106 captures the skeleton data
points for a predefined time period, while the individuals 104 are within a field of
view 108 of the skeleton recording device 106. The field of view 108 is an area
observable by the skeleton recording device 106. The skeleton data points of the
individual 104 contain information about the skeleton joints of the individual 104.
In an implementation, the skeleton recording device 106 may capture data for 20
skeleton joints. After capturing the skeleton data points, the skeleton recording
device 106 generates signals that may be provided to the system 102 for
processing.
[0025] In an example, the skeleton recording device 106 may be a sensor,
such as a Kinect sensor. The skeleton recording device 106 may be fixed or
rotating. The rotating skeleton recording device provides a wider field of view and
facilitates in capturing the walking activity of the individual 104 over a wider
area.
[0026] In an implementation, the system 102 comprises a gait-pose modeler
110 and an identification module 112. The signals received by the system 102
from the skeleton recording device 106 are processed to extract multiple static and
dynamic gait features from the signals for each individual 104. The gait-pose
modeler 110 obtains the gait features in the form of vectors, and, for each
individual 104, creates a predefined number of clusters for the gait feature vectors.
These clusters are characteristics of poses that can be identified in the walking
pattern of the individual 104. The gait-pose modeler 110 creates another
predefined number of sub-clusters for the gait feature vectors associated with each
of the clusters. The sub-clusters within each cluster are characteristics of subposes
that can be identified in the corresponding pose of the individual 104. Based
on the clustering of the gait feature vectors, the gait-pose modeler 110 determines
a gait-pose feature data set which is used by the identification module 112 for
training a classifier and subsequent identification of the individuals 104 uniquely.
[0027] Figure 2 illustrates the people identification system 102, in accordance
with an implementation of the present subject matter. In an implementation, the
system 102 includes processor(s) 202. The processor(s) 202 may be implemented
10
as one or more microprocessors, microcomputers, microcontrollers, digital signal
processors, central processing units, state machines, logic circuitries, and/or any
devices that manipulate signals based on operational instructions. Among other
capabilities, the processor(s) 202 is configured to fetch and execute computerreadable
instructions stored in a memory.
[0028] The system 102 includes interface(s) 204. The interfaces may include a
variety of machine readable instruction-based and hardware-based interfaces that
allow the system 102 to communicate with other devices, including servers, data
sources and external repositories. Further, the interface(s) 204 may enable the
system 102 to communicate with other communication devices, such as network
entities, over a communication network.
[0029] Further, the system 102 includes a memory 206. The memory 206 may
be coupled to the processor(s) 202. The memory 206 can include any computerreadable
medium known in the art including, for example, volatile memory, such
as static random access memory (SRAM) and dynamic random access memory
(DRAM), and/or non-volatile memory, such as read only memory (ROM),
erasable read only memory (ROM), erasable programmable ROM, flash
memories, hard disks, optical disks, and magnetic tapes.
[0030] Further, the system 102 includes module(s) 208 and data 210. The
module(s) 208 and the data 210 may be coupled to the processor(s) 202. The
modules 208, amongst other things, include routines, programs, objects,
components, data structures, etc., which perform particular tasks or implement
particular abstract data types. The modules 208 may also be implemented as,
signal processor(s), state machine(s), logic circuitries, and /or any other device or
component that manipulate signals based on operational instructions. The data
210 serves, amongst other things, as a repository for storing data that may be
fetched, processed, received, or generated by the module(s) 208. Although the
data is shown internal to the mobile device, it may be understood that the data 210
can reside in an external repository (not shown in the Figure), which may be
coupled to the system 102. The system 102 may communicate with the external
repository through the interface(s) 204.
11
[0031] Further, the module(s) 208 can be implemented in hardware,
instructions executed by a processing unit, or by a combination thereof. The
processing unit can comprise a computer, a processor, a state machine, a logic
array or any other suitable devices capable of processing instructions. The
processing unit can be a general-purpose processor which executes instructions to
cause the general-purpose processor to perform the required tasks or, the
processing unit can be dedicated to perform the required functions. In another
aspect of the present subject matter, the module(s) 208 may be machine-readable
instructions which, when executed by a processor/processing unit, perform any of
the desired functionalities. The machine-readable instructions may be stored on an
electronic memory device, hard disk, optical disk or other machine-readable
storage medium or non-transitory medium. In an implementation, the machinereadable
instructions can also be downloaded to the storage medium via a network
connection.
[0032] In an implementation, the module(s) 208 include a feature extraction
module 212, a noise remover 214, the gait-pose modeler 110, the identification
module 112 and other module(s) 216. The other module(s) 216 may include
programs or coded instructions that supplement applications or functions
performed by the system 102. In said implementation, the data 210 includes
skeleton data and extracted feature data 218, cluster data 220, classifier data 222
and other data 224. The other data 224 amongst other things, may serve as a
repository for storing data that is processed, received, or generated as a result of
the execution of one or more modules in the module(s) 208.
[0033] The following description describes the procedure of identification of
individuals 104 based on pose and sub-pose clustering by the system 102. The
procedure is described for one individual performing a walking activity and can
be carried out in a similar manner for multiple individuals performing any step
activity.
[0034] As described earlier, the skeleton recording device 106 captures the
skeleton data points of the individual 104 performing the walking activity with an
arbitrary walking pattern and following any walking path within the field of view
12
108 of the skeleton recording device 106. The skeleton recording device 106
captures the data over a predefined period and generates signals based on the
captured skeleton data points. The skeleton data points provide information about
gait of the individual 104. The predefined time period is such that the complete
walking pattern of the individual 104 is captured. The system 102 receives the
signals from the skeleton recording device 106 in the form of skeleton data frames
comprising skeleton data points of the individual 104. In an example, the
predefined time period, for which the data of the individual 104 is captured, is
such that the skeleton recording device 106 generates signals to provide 3000
skeleton data frames to the system 102. The system 102 receives 3000 skeleton
data frames, for further processing, from skeleton recording device 106. For the
sake of simplicity of the specification, the skeleton data frames hereinafter are
referred to as frames. Let the frames be denoted by f1, f2, f3, … , f3000.
[0035] Based on the frames received by the system 102, the feature extraction
module 212 extracts a static feature vector and a dynamic feature vector from
each of the frames. In an implementation, the static feature vector includes static
gait features including, but not limited to, height, length of both legs, torso, both
lower legs, both thighs, both upper arms, and both forearms of the individual. In
the implementation, the dynamic feature vector includes dynamic gait features
including, but not limited to, velocity, step length, area-related features, anglerelated
features, and dynamic centroid distance-related features.
[0036] The area-related features may include, but are not limited to, mean of
area occupied by upper body portion and mean of area occupied by lower body
portion of the individual. The angle-related features may include, but are not
limited to, angle of upper left leg relative to vertical axis, angle between left foot
relative to horizontal axis, angle of lower left leg relative to the upper left leg,
angle of upper right leg relative to the vertical axis, angle of right foot relative to
the horizontal axis, and angle of lower right leg relative to the upper right leg. The
dynamic centroid-distance related features may include, but are not limited to,
distance between centroid of body and centroids of right hand, left hand, right leg,
and left leg.
13
[0037] The values of the static gait features from each frame form the static
feature vector for the frame, and the values of the dynamic gait features from each
frame form the dynamic feature vector for the frame. The data related to the
skeleton data points, the static feature vector, and the dynamic feature vector are
stored in the skeleton data and extracted features data 218.
[0038] In an example, the static feature vector includes 12 static gait features
and the dynamic feature vector includes 14 dynamic gait features. Let the static
feature vector for each frame be denoted by a 12-dimensional vector of 12
elements [ds1... ds12], and the dynamic feature vector for each frame be denoted
by a 14-dimensional vector of 14 elements [df1... df14]. So, for all the frames
from f1 to f3000, there are 3000 static feature vectors and 3000 dynamic feature
vectors. The extracted static and dynamic feature vectors for all the frames f1 to
f3000 can be represented by a two-dimensional (2-D) matrix of order of [3000 x
(14+12)].
[0039] The skeleton data points captured by the skeleton recording device 106
may include noise owing to, for example, the limitation of the skeleton recording
device 106. This noise is included in the frames f1 to f3000, and hence in the
static and dynamic feature vectors extracted therefrom. The noise remover 214
detects the noisy frames, from amongst all the frames, based on variations in the
static feature vectors extracted from the frames f1 to f3000. The static feature
vectors have values corresponding to distances between skeleton joints of the
individual 104 and therefore, should not vary with time. If this distance varies
substantially from one frame to another frame, then the static feature vectors have
noise therein. In the other words, the frames containing those static features are
noisy.
[0040] For this, in an implementation, the noise remover 214 clusters the
static feature vectors of the frames f1 to f3000 into two clusters. In the context of
the present subject matter, a cluster formed by the noise remover 214 includes
points, where each point represents one static feature vector in a multidimensional
space. The distribution of points in such a cluster is indicative of the
distribution and variation of static feature vectors for the individual 104. Initially
14
the absolute differences of static feature vectors of successive frames are
computed. Histogram analysis is done for the values of the absolute differences to
populate counts in predefined bins. The static feature data point corresponding to
the center of the bin having maximum count is taken as the first starting center.
The second starting center is the data point which is at a farthest distance from the
first starting center. These two starting centers are considered as initial cluster
centers for a 2-cluster clustering method performed for the static feature vectors.
After the 2-cluster clustering method is performed two clusters with their cluster
centers are obtained. The data points associated with the final location of the
cluster center of one of the clusters are good data, and the data points associated
with the final location of the cluster center of the other cluster correspond to the
noisy data. It is observed that the cluster with the noisy data has sparsely
distributed data points representing the static feature vectors having noise. The
frames corresponding to those static feature vectors are referred to as noisy
frames. The noisy frames are not considered for further processing in the system
102.
[0041] In an implementation, the noise remover 214 removes the noisy frames
from the frames f1 to f3000, thereby reducing the number of frames for further
processing. The removal of noisy frames is understood as the removal of static
and dynamic feature vectors corresponding to the noisy frames from the 2-D
matrix of order [3000 x (14+12)]. Let there be 500 noisy frames. Let the
remaining 2500 skeleton data frames be denoted by f1, f2, f3, ... , f2500. The
static gait features and the dynamic gait features for the reduced number of frames
f1 to f2500 can be represented by a 2-D matrix of order of [2500 x (14+12)]. After
the removal of the noisy frames and the corresponding feature vectors, the static
and dynamic feature vectors for the remaining frames f1 to f2500 are processed
by the gait-pose modeler 110 for gait-pose and sub-pose modeling.
[0042] For the gait-pose and sub-pose modeling, the gait-pose modeler 110
creates a first predefined number of clusters of the dynamic feature vectors for the
frames f1 to f2500.The first predefined number of clusters is a number of poses to
15
be identified in the walking activity of the individual 104. In an implementation,
the first predefined number may range from 2 to 10.
[0043] Let us consider an example, where the number of poses to be identified
is 3. The gait-pose modeler 110 performs K-means clustering method over the
dynamic feature vectors for the frames f1 to f2500 for creating three clusters. The
gait-pose modeler 110 randomly selects three dynamic feature vectors, amongst
all the dynamic feature vectors, as the starting centers, and computes the
Euclidean distances of the remaining dynamic feature vectors from each of these
three starting centers. Based on the distances, the gait-pose modeler 110 creates
clusters around each of the three starting centers, such that each cluster has points,
in a multi-dimensional space, representing the dynamic feature vectors that are
closer to the starting center of that cluster. The gait-pose modeler 110 then
modifies the starting centers by computing a revised center of each of the three
clusters. Based on the modified starting centers, the gait-pose modeler 110 again
creates clusters around each of the three modified centers in manner described
above. The gait-pose modeler 110 continuously modifies the starting centers for
the three clusters and creates revised clusters around the modified centers until the
starting centers stop changing. With this, three clusters are obtained, where each
cluster has points representing segregated dynamic feature vectors. The dynamic
feature vectors for each cluster correspond to one pose that can be identified in the
walking pattern of the individual 104. In an implementation, the clustering may be
performed through any clustering method including, but not restricted to, Kmeans
or Fuzzy C-means clustering method.
[0044] Now, as mentioned earlier, each pose of the individual 104 may have
variations which can represent sub-poses in the walking pattern. The dynamic
feature vectors for the frames associated with each cluster are processed
separately by the gait-pose modeler 110 for pose-sub-pose modeling. For the
pose-sub-pose modeling, the gait-pose modeler 110 creates a second predefined
number of sub-clusters of the dynamic feature vectors for the frames associated
with each cluster. The second predefined number of sub-clusters is a number of
16
sub-poses to be identified in each of the poses. In an implementation, the second
predefined number may range from 2 to 10.
[0045] Let us consider an example, where the number of sub-poses to be
identified is 8. The gait-pose modeler 110 performs K-means clustering method
separately over the dynamic feature vectors for the frames associated with each
cluster for creating eight sub-clusters in the each cluster. The creation of the subclusters
within the each cluster is carried out in the same manner as described
earlier for the creation of the clusters. The dynamic feature vectors for each subcluster
of a cluster correspond to one sub-pose that can be identified within a pose
in the walking pattern of the individual 104. The data related to pose and sub-pose
clusters is stored in the cluster data 220. In an implementation, the clustering may
be performed through any clustering method including, but not restricted to, Kmeans
or Fuzzy C-means clustering method.
[0046] After the clusters and the sub-clusters are created, the gait-pose
modeler 110 determines a gait-pose feature data set. For this, the gait-pose
modeler 110 computes a center of the dynamic feature vectors for the frames
associated with each sub-cluster within a cluster. The center for a sub-cluster can
be represented as a 14-dimensional vector of 14 elements. Thus, for 8 sub-clusters
within a cluster, the centers of the dynamic feature vectors for the frames
associated with the sub-clusters within the cluster can be represented as 2-D
matrix of the order of [8 x 14]. The gait-pose modeler 110 also computes a mean
of the static feature vectors for the frames associated with each cluster. The mean
of the static feature vectors for the frames associated with a cluster can be
represented as a 12-dimensional vector of 12 elements. This mean, computed for
each cluster, is appended to the center vector computed for each sub-cluster of
that cluster. This gives the data set for each cluster, which can be represented as a
2-D matrix of the order of [8 x (14+12)]. Thus, for 3 clusters, the complete data
set for the individual 104 can be represented as a 2-D matrix of the order of [3 x 8
x (14+12)], i.e., [24 x 26]. This data set is referred to as the gait-pose feature data
set. In general, the gait-pose feature data set can be represented as a 2-D matrix of
order of [(number of poses to be identified in the walking pattern) x (number of
17
sub-poses to be identified in a pose in the walking pattern) x (total number of
static gait features representing a pose and a sub-pose + total number of dynamic
gait features representing a pose and a sub-pose)]. The data related to the gaitpose
feature data is stored in the classifier data 222.
[0047] Based on the gait-pose feature data set determined for the individual
104, the identification module 112 trains a classifier with the gait-pose feature
data set for the individual 104. The classifier may include a supervised learning
algorithm, such as a support vector machine (SVM) classifier with a radial basis
function, a naïve bayes, a decision tree, a neural network, and the like. The
identification module 112 then uses the classifier to identify the individual 104
uniquely in real-time in future.
[0048] The gait-pose modeler 110 of the system 102 can determine the gaitpose
feature data sets for all the individuals 104 in a similar manner, and the
identification module 112 can train the classifier with the gait-pose feature data
sets for the individuals 104 to identify them uniquely.
[0049] Figure 3 illustrates a method 300 for identification of individuals based
on pose and sub-pose clustering, in accordance with an implementation of the
present subject matter. The method 300 can be implemented in the people
identification system 102. The order in which the method 300 is described is not
intended to be construed as a limitation, and any number of the described method
blocks can be combined in any order to implement the method 300, or any
alternative methods. Additionally, individual blocks may be deleted from the
method 300 without departing from the spirit and scope of the subject matter
described herein. Furthermore, the method 300 can be implemented in any
suitable hardware.
[0050] The method 300 may be described in the general context of computer
executable instructions. Generally, computer executable instructions can include
routines, programs, objects, components, data structures, procedures, modules,
functions, etc., that perform particular functions or implement particular abstract
data types. Further, although the method 300 may be implemented in any
computing device; in an example described in Figure 3, the method 300 is
18
explained in context of the aforementioned people identification system 102, for
the ease of explanation.
[0051] Referring to Figure 3, at block 302, static feature vectors and dynamic
feature vectors for skeleton data frames for individuals 104 are obtained. The
static and dynamic feature vectors are obtained by capturing skeleton data points
of the individuals 104 using a skeleton recording device 106. The skeleton
recording device 106 provides information of skeleton joints in the form of
skeleton data frames, hereinafter referred to as frames. The skeleton data points
are captured by the skeleton recording device 106 while the individuals 106 are
performing a step activity, such as a walking activity, jogging activity, or running
activity, with an arbitrary step activity pattern and following a random path. The
static and dynamic feature vectors are obtained for a predefined number of
frames, for example 3000 frames. Each static feature vector includes multiple
static gait features, and each dynamic feature vector includes multiple dynamic
gait features, of the individuals 104.
[0052] In an implementation, after obtaining the static and dynamic data
feature sets from the frames, for each individual 104 noisy frames are detected, at
block 304, by clustering the static feature vectors for all the frames into two
clusters. As mentioned earlier, the cluster having sparsely distributed points is
referred to as a noisy cluster and the points in that cluster represent the static
feature vectors of noisy frames. The noisy frames, and the static and dynamic
feature vectors of the noisy frames, are not considered for further processing.
[0053] At block 306, a first predefined number of clusters are created based
on clustering of dynamic feature vectors for the frames for each individual
104.The first predefined number is indicative of number of poses that are to be
identified in the activity pattern of the individuals 104. The dynamic feature
vectors may be clustered using K-means clustering method or Fuzzy C-means
clustering method or any other clustering method, as described earlier in the
description. Each cluster, and the dynamic feature vectors corresponding to that
cluster, corresponds to one pose that can be identified in the step activity pattern.
19
[0054] Further, at block 308, a second predefined number of sub-clusters are
created within each of the clusters based on clustering of dynamic feature vectors
for the frames associated with the each cluster. The second predefined number is
indicative of number of sub-poses that are to be identified within each pose in the
activity pattern of the individuals 104. The dynamic feature vectors may be subclustered
using K-means clustering method or Fuzzy C-means clustering method
or any other clustering method, as described earlier in the description. Each subcluster,
and the dynamic feature vectors corresponding to that sub-cluster,
corresponds to one sub-pose that can be identified within a pose in the step
activity pattern.
[0055] After clustering and sub-clustering the dynamic feature vectors for
each individual 104, a gait-pose feature data set is determined, at block 310. The
gait-pose feature data set is determined based on computation of a center of the
dynamic feature vectors for the frames associated with each sub-cluster, and a
mean of the static feature vectors for the frames associated with each cluster. As
described earlier, the gait-pose feature data set is a 2-D matrix of the order of
[(number of poses to be identified in the activity pattern) x (number of sub-poses
to be identified in a pose in the activity pattern) x (total number of static gait
features representing a pose and a sub-pose + total number of dynamic gait
features representing a pose and a sub-pose)]. The gait-pose feature data set is
unique for each individual 104 and is characterized based on the poses and the
sub-poses that define the activity pattern for the step activity performed by the
each individual 104. The characterization is done through gait-pose modeling by
clustering the gait features extracted for each individual 104.
[0056] At block 312, a classifier is trained with the gait-pose feature data set
determined for each individual 104. The trained classifier can then be used for
identification of individuals uniquely, while they are performing a step activity
with any arbitrary pattern and following any random path in any direction.
[0057] Although implementations for system(s) and method(s) for
identification of individuals are described, it is to be understood that the present
subject matter is not necessarily limited to the specific features or methods
described. Rather, the specific features and methods are disclosed as
implementations to identify individuals uniquely.