FORM 2
THE PATENTS ACT, 1970
(39 of 1970)
&
THE PATENTS RULES, 2003
COMPLETE SPECIFICATION
[See section 10, Rule 13]
SYSTEM AND METHOD FOR ANOMALY DETECTION USING AN
ATTENTION MODEL;
MITSUBISHI ELECTRIC CORPORATION, A CORPORATION
ORGANISED AND EXISTING UNDER THE LAWS OF JAPAN, WHOSE
ADDRESS IS 7-3, MARUNOUCHI 2-CHOME, CHIYODA-KU, TOKYO
100-8310, JAPAN
THE FOLLOWING SPECIFICATION PARTICULARLY DESCRIBES
THE INVENTION AND THE MANNER IN WHICH IT IS TO BE
PERFORMED.
38
We Claim :
[Claim 1]
An anomaly detector for controlling a system including a plurality of
actuators assisting one or multiple tools to perform one or multiple tasks,
comprising:5
at least one processor; and
a memory having instructions stored thereon that, when executed by at
least one processor, cause the anomaly detector to:
collect a feedforward signal indicative of a sequence of control
inputs to the plurality of actuators at a plurality of control steps;10
collect a feedback signal indicative of a sequence of outputs of the
system caused by the plurality of actuators operated based on the
sequence of control inputs at corresponding control steps of the plurality
of control steps;
combine input state variables extracted from the feedforward15
signal and output state variables extracted from the feedback signal to
form a sequence of extended states of the system;
transmit the sequence of extended states as a set of queries to an
attention model,
wherein each extended state of the sequence of extended20
states corresponds to a respective query of the set of queries to the
attention model, and
wherein the attention model is configured to:
compare each of the transmitted set of queries with a
set of keys to produce a plurality of attention weights; and25
encode each of the set of queries as a weighted linear
combination of a set of first values weighted with a
corresponding attention weight of the plurality of attention
39
weights to produce an encoding of each extended state of
the sequence of extended states in a latent space,
wherein each key of the set of keys is
associated with a normal extended state of a normal
operation of the system,5
wherein each first value of the set of first
values is an encoding of a corresponding key of the
set of keys into the latent space,
wherein each attention weight of the plurality
of attention weights is produced by comparing a10
plurality of first entities of a query of the set of
queries with a plurality of second entities of a key of
the set of keys in their respective states by combining
the similarity distances between a first entity of the
plurality of first entities and a corresponding second15
entity of the plurality of second entities, and
wherein the similarity distance for comparing
the first entity and the second entity is obtained from
a mathematical space used to describe the first entity
and the second entity;20
detect an anomaly in a current operation of the system based on
the encoded set of queries; and
generate a result of the anomaly detection.
[Claim 2]
The anomaly detector of claim 1, wherein, during training of the anomaly25
detector, the at least one processor is further configured to learn a dictionary
comprising extremum values of the set of first values.
[Claim 3]
40
The anomaly detector of claim 2, wherein the at least one processor is
further configured to:
generate a predetermined convex subspace in the latent space based on
the set of first values and the dictionary, wherein a boundary of the
predetermined convex subspace is determined based on the extremum values5
of the set of first values; and
detect the anomaly in the current operation of the system when at least
one encoded query of the encoded set of queries is outside the predetermined
convex subspace of the latent space.
[Claim 4]10
The anomaly detector of claim 1, wherein the at least one processor is
further configured to:
encode each key of the set of keys into a respective first value of the set
of first values into the latent space during training of the anomaly detector; and
store the set of keys and the set of first values into the memory.15
[Claim 5]
The anomaly detector of claim 1, wherein the state variables extracted
from the feedforward signal corresponds to at least one physical entity provided
to the system.
[Claim 6]20
The anomaly detector of claim 1, wherein the state variables extracted
from the feedback signal corresponds to at least one output of the system sensed
by a sensor.
[Claim 7]
The anomaly detector of claim 1, wherein each of the normal operation25
and current operation of the system includes at least one of an assembly
operation performed by synchronized actions of different robotic manipulators,
moving an object through a sequence of locations, or machining a workpiece
41
by different tools in a predetermined sequential or partially overlapping order.
[Claim 8]
The anomaly detector of claim 1, wherein the at least one processor is
further configured to transmit a first extended state of the sequence of extended
states for a first control step of the plurality of control steps of the current5
operation of the system as a first query of the set of queries to the attention
model.
[Claim 9]
The anomaly detector of claim 8,
wherein the at least one processor is further configured to compare the10
first query with a first key of the set of keys for the first control step,
wherein the first key is associated with the normal operation of the
system for the first control step.
[Claim 10]
The anomaly detector of claim 1, wherein the attention model is based15
on a multi-headed attention mechanism, and wherein the at least one processor
is further configured to encode the set of queries into the latent space based on
the multi-headed attention mechanism.
[Claim 11]
The anomaly detector of claim 1, wherein the at least one processor is20
further configured to output the result of anomaly detection to a user-interface.
[Claim 12]
The anomaly detector of claim 1, wherein the at least one processor is
further configured to:
generate control commands based on the result of anomaly detection;25
and
transmit the control commands to the system.
[Claim 13]
42
An anomaly detection method, comprising:
collecting a feedforward signal indicative of a sequence of control inputs
to the plurality of actuators at a plurality of control steps;
collecting a feedback signal indicative of a sequence of outputs of the
system caused by the plurality of actuators operated based on the sequence of5
control inputs at corresponding control steps of the plurality of control steps;
combining input state variables extracted from the feedforward signal
and output state variables extracted from the feedback signal to form a sequence
of extended states of the system;
transmitting the sequence of extended states as a set of queries to an10
attention model,
wherein each extended state of the sequence of extended states
corresponds to a respective query of the set of queries to the attention
model, and
wherein the attention model is configured to:15
compare each of the transmitted set of queries with a set of
keys to produce a plurality of attention weights; and
encode each of the set of queries as a weighted linear
combination of a set of first values weighted with a corresponding
attention weight of the plurality of attention weights to produce an20
encoding of each extended state of the sequence of extended states
in a latent space,
wherein each key of the set of keys is associated with
a normal extended state of a normal operation of the system,
wherein each first value of the set of first values is an25
encoding of a corresponding key of the set of keys into the
latent space,
wherein each attention weight of the plurality of
43
attention weights is produced by comparing a plurality of
first entities of a query of the set of queries with a plurality
of second entities of a key of the set of keys in their
respective states by combining the similarity distances
between a first entity of the plurality of first entities and a5
corresponding second entity of the plurality of second
entities, and
wherein the similarity distance for comparing the
first entity and the second entity is obtained from a
mathematical space used to describe the first entity and the10
second entity;
detecting an anomaly in a current operation of the system based on the
encoded set of queries; and
generating a result of the anomaly detection.
[Claim 14]15
The anomaly detection method of claim 13, further comprising during
training of the anomaly detector, learning a dictionary comprising extremum
values of the set of first values.
[Claim 15]
The anomaly detection method of claim 14, further comprising:20
generating a predetermined convex subspace in the latent space based on
the set of first values and the dictionary, wherein a boundary of the
predetermined convex subspace is determined based on the extremum values
of the set of first values; and
detecting the anomaly in the current operation of the system when at least25
one encoded query of the encoded set of queries is outside the predetermined
convex subspace of the latent space.
[Claim 16]
44
The anomaly detection method of claim 13, further comprising:
encoding each key of the set of keys into a respective first value of the
set of first values into the latent space during training of the anomaly detector;
and
storing the set of keys and the set of first values into the memory.5
[Claim 17]
The anomaly detection method of claim 13, further comprising
transmitting a first extended state of the sequence of extended states for a first
control step of the plurality of control steps of the current operation of the
system as a first query of the set of queries to the attention model.10
[Claim 18]
The anomaly detection method of claim 17, further comprising
comparing the first query with a first key of the set of keys for the first control
step, wherein the first key is associated with the normal operation of the system
for the first control step.15
[Claim 19]
The anomaly detection method of claim 13, wherein the attention model
is based on a multi-headed attention mechanism, and wherein the at least one
processor is further configured to encode the set of queries into the latent space
based on the multi-headed attention mechanism.20
[Claim 20]
A non-transitory computer-readable medium having stored thereon
computer-executable instructions, which when executed by a computer, cause
the computer to execute operations, the operations comprising: collecting a
feedforward signal indicative of a sequence of control inputs to the plurality of25
actuators at a plurality of control steps;
collecting a feedforward signal indicative of a sequence of control inputs
to the plurality of actuators at a plurality of control steps;
45
collecting a feedback signal indicative of a sequence of outputs of the
system caused by the plurality of actuators operated based on the sequence of
control inputs at corresponding control steps of the plurality of control steps;
combining input state variables extracted from the feedforward signal
and output state variables extracted from the feedback signal to form a sequence5
of extended states of the system;
transmitting the sequence of extended states as a set of queries to an
attention model,
wherein each extended state of the sequence of extended states
corresponds to a respective query of the set of queries to the attention10
model, and
wherein the attention model is configured to:
compare each of the transmitted set of queries with a set of
keys to produce a plurality of attention weights; and
encode each of the set of queries as a weighted linear15
combination of a set of first values weighted with a corresponding
attention weight of the plurality of attention weights to produce an
encoding of each extended state of the sequence of extended states
in a latent space,
wherein each key of the set of keys is associated with20
a normal extended state of a normal operation of the system,
wherein each first value of the set of first values is an
encoding of a corresponding key of the set of keys into the
latent space,
wherein each attention weight of the plurality of25
attention weights is produced by comparing a plurality of
first entities of a query of the set of queries with a plurality
of second entities of a key of the set of keys in their
46
respective states by combining the similarity distances
between a first entity of the plurality of first entities and a
corresponding second entity of the plurality of second
entities, and
wherein the similarity distance for comparing the5
first entity and the second entity is obtained from a
mathematical space used to describe the first entity and the
second entity;
detecting an anomaly in a current operation of the system based on the
encoded set of queries; and10
generating a result of the anomaly detection.