FORM 2
THE PATENTS ACT, 1970
(39 of 1970)
&
THE PATENT RULES, 2003
COMPLETE SPECIFICATION
(See Section 10 and Rule 13)
Title of invention:
METHOD AND SYSTEM FOR QUANTUM-ENHANCED
SEMANTIC SEGMENTATION OF VOXEL-BASED 3D-OBJECT
Applicant
Tata Consultancy Services Limited
A company Incorporated in India under the Companies Act, 1956
Having address:
Nirmal Building, 9th floor,
Nariman point, Mumbai 400021,
Maharashtra, India
Preamble to the description:
The following specification particularly describes the invention and the manner in
which it is to be performed.
2
TECHNICAL FIELD
[001] The disclosure herein generally relates to semantic segmentation,
and, more particularly, to method and system for quantum-enhanced semantic
segmentation of voxel-based 3D-object.
5
BACKGROUND
[002] Three-dimensional (3D) semantic segmentation is one of the major
steps in 3D object analysis tasks and is used for categorizing each voxel value of an
object to a particular class. Semantic segmentation is being used as solution for
10 problems such as location of objects in 3D scans, locating tumors in medical 3D
scans and so on. There are various algorithms available in the art for semantic
segmentation. Simple object processing methods like threshold based segmentation
methods is one approach for semantic segmentation which is used largely.
However, this method ignores voxel location in the 3D objects, and they result in
15 incoherent segmentation. Another approach for semantic segmentation is
traditional machine learning method like clustering based technique. However, the
clustering based technique performs efficiently and provides good results only with
a good choice of initial settings such as a number of clusters and initial cluster center
locations. Also, the whole 3D object needs to be processed in each iteration of the
20 clustering based technique and which makes it inefficient.
[003] There have been attempts where neural network models have been
used for semantic segmentation. However, these methods involve long learning
process, and require a huge and diverse dataset for training the model. If the model
is not trained properly, the results for the semantic segmentation will be poor.
25 Nowadays, classical CNN based architecture like U-Nets are used for semantic
segmentation. Even though the classical U-Nets perform almost accurately, training
time for the classical U-Nets is high. There is existing research that shows quantum
kernels transform classical data into Hilbert space that are classically intractable
and hard to simulate at large scales. Recently, a combination of quantum and
30 classical deep learning models is explored for two-dimensional image
segmentation. However, implementation of combined quantum and classical deep
3
learning models for semantic segmentation of three dimensional semantic is
challenging.
SUMMARY
5 [004] Embodiments of the present disclosure present technological
improvements as solutions to one or more of the above-mentioned technical
problems recognized by the inventors in conventional systems. For example, in one
aspect, a processor implemented method is provided. The processor implemented
method includes receiving, via one or more hardware processors, (i) a set of voxel10
based three-dimensional (3D) objects, and (ii) a set of annotated voxel-based threedimensional
(3D) objects corresponding to the set of voxel-based three-dimensional
(3D) objects as input data for training a quantum-classical hybrid three-dimensional
(3D) object segmentation model, wherein the quantum-classical hybrid 3D object
segmentation model includes a U-shaped encoder-decoder neural network
15 architecture comprising a bottleneck layer including (i) a classical path and (ii) a
quantum path, and wherein the quantum path is designed using a quantum circuit
that comprises a set of eight qubits where a central qubit is absent, and wherein a
plurality of voxel values associated with the set of voxel-based 3D objects are
embedded with 2-qubit gates for each of a plurality of combinations of pair of qubits
20 in the quantum circuit and a commutative property of 2-qubit gates is exhibited;
training, via one or more hardware processors, the quantum-classical hybrid 3D
object segmentation model using the input data to obtain a trained quantumclassical
hybrid 3D object segmentation model, wherein the steps for training
comprises: (i) preprocessing the set of voxel-based 3D objects using one or more
25 preprocessing techniques; (ii) encoding the set of voxel-based 3D objects using a
set of convolutional and pooling layers present in an encoder in the quantumclassical
hybrid 3D object segmentation model to obtain an encoded set of voxelbased
3D objects; (iii) obtaining, from each of the encoded set of voxel-based 3D
objects, (i) a first tensor from the classical path and (ii) a second tensor from the
30 quantum path of the bottleneck layer of the quantum-classical hybrid 3D object
segmentation model; (iv) concatenating the first tensor and the second tensor to
4
obtain a merged tensor corresponding to each voxel-based 3D object from the set
of voxel-based 3D objects; (v) decoding the merged tensor using a set of
convolutional and pooling layers present in a decoder of the quantum-classical
hybrid 3D object segmentation model to obtain a decoded set of voxel-based 3D
5 objects; (vi) determining a training loss associated with the quantum-classical
hybrid 3D object segmentation model based on a comparison of each of the decoded
set of voxel-based 3D objects with a set of corresponding ground truth data; (vii)
backpropagating the training loss to the encoder through at least one of (a) the
classical path, and (b) the quantum path of the bottleneck layer of the quantum10
classical hybrid 3D object segmentation model; and (viii) tuning a set of parameters
in the quantum-classical hybrid 3D object segmentation model based on the training
loss to obtain the trained quantum-classical hybrid 3D object segmentation model;
performing, via the one or more hardware processors, steps (i) through (iv) for an
incoming voxel-based three-dimensional (3D) object on the trained quantum15
classical hybrid 3D object segmentation model; and decoding, via the one or more
hardware processors, the merged tensor using a set of convolutional and pooling
layers present in the decoder of the trained quantum-classical hybrid 3D object
segmentation model to obtain a segmented voxel-based 3D object corresponding to
the incoming voxel-based three-dimensional (3D) object
20 [005] In another aspect, there is provided a system . The system includes
a memory storing instructions; one or more communication interfaces; and one or
more hardware processors coupled to the memory via the one or more
communication interfaces, wherein the one or more hardware processors are
configured by the instructions to: receive (i) a set of voxel-based three-dimensional
25 (3D) objects, and (ii) a set of annotated voxel-based three-dimensional (3D) objects
corresponding to the set of voxel-based three-dimensional (3D) objects as input
data for training a quantum-classical hybrid three-dimensional (3D) object
segmentation model, wherein the quantum-classical hybrid 3D object segmentation
model includes a U-shaped encoder-decoder neural network architecture
30 comprising a bottleneck layer including (i) a classical path and (ii) a quantum path,
and wherein the quantum path is designed using a quantum circuit that comprises a
5
set of eight qubits where a central qubit is absent, and wherein a plurality of voxel
values associated with the set of voxel-based 3D objects are embedded with 2-qubit
gates for each of a plurality of combinations of pair of qubits in the quantum circuit
and a commutative property of 2-qubit gates is exhibited; train the quantum-
5 classical hybrid 3D object segmentation model using the input data to obtain a
trained quantum-classical hybrid 3D object segmentation model, wherein the steps
for training comprises: (i) preprocessing the set of voxel-based 3D objects using
one or more preprocessing techniques; (ii) encoding the set of voxel-based 3D
objects using a set of convolutional and pooling layers present in an encoder in the
10 quantum-classical hybrid 3D object segmentation model to obtain an encoded set
of voxel-based 3D objects; (iii) obtaining, from each of the encoded set of voxelbased
3D objects, (i) a first tensor from the classical path and (ii) a second tensor
from the quantum path of the bottleneck layer of the quantum-classical hybrid 3D
object segmentation model; (iv) concatenating the first tensor and the second tensor
15 to obtain a merged tensor corresponding to each voxel-based 3D object from the set
of voxel-based 3D objects; (v) decoding the merged tensor using a set of
convolutional and pooling layers present in a decoder of the quantum-classical
hybrid 3D object segmentation model to obtain a decoded set of voxel-based 3D
objects; (vi) determining a training loss associated with the quantum-classical
20 hybrid 3D object segmentation model based on a comparison of each of the decoded
set of voxel-based 3D objects with a set of corresponding ground truth data; (vii)
backpropagating the training loss to the encoder through at least one of (a) the
classical path, and (b) the quantum path of the bottleneck layer of the quantumclassical
hybrid 3D object segmentation model; and (viii) tuning a set of parameters
25 in the quantum-classical hybrid 3D object segmentation model based on the training
loss to obtain the trained quantum-classical hybrid 3D object segmentation model;
perform steps (i) through (iv) for an incoming voxel-based three-dimensional (3D)
object on the trained quantum-classical hybrid 3D object segmentation model; and
decode the merged tensor using a set of convolutional and pooling layers present in
30 the decoder of the trained quantum-classical hybrid 3D object segmentation model
6
to obtain a segmented voxel-based 3D object corresponding to the incoming voxelbased
three-dimensional (3D) object.
[006] In yet another aspect, there are provided one or more non-transitory
machine readable information storage mediums comprising one or more
5 instructions which when executed by one or more hardware processors causes at
least one of: receiving (i) a set of voxel-based three-dimensional (3D) objects, and
(ii) a set of annotated voxel-based three-dimensional (3D) objects corresponding to
the set of voxel-based three-dimensional (3D) objects as input data for training a
quantum-classical hybrid three-dimensional (3D) object segmentation model,
10 wherein the quantum-classical hybrid 3D object segmentation model includes a Ushaped
encoder-decoder neural network architecture comprising a bottleneck layer
including (i) a classical path and (ii) a quantum path, and wherein the quantum path
is designed using a quantum circuit that comprises a set of eight qubits where a
central qubit is absent, and wherein a plurality of voxel values associated with the
15 set of voxel-based 3D objects are embedded with 2-qubit gates for each of a
plurality of combinations of pair of qubits in the quantum circuit and a commutative
property of 2-qubit gates is exhibited; training the quantum-classical hybrid 3D
object segmentation model using the input data to obtain a trained quantumclassical
hybrid 3D object segmentation model, wherein the steps for training
20 comprises: (i) preprocessing the set of voxel-based 3D objects using one or more
preprocessing techniques; (ii) encoding the set of voxel-based 3D objects using a
set of convolutional and pooling layers present in an encoder in the quantumclassical
hybrid 3D object segmentation model to obtain an encoded set of voxelbased
3D objects; (iii) obtaining, from each of the encoded set of voxel-based 3D
25 objects, (i) a first tensor from the classical path and (ii) a second tensor from the
quantum path of the bottleneck layer of the quantum-classical hybrid 3D object
segmentation model; (iv) concatenating the first tensor and the second tensor to
obtain a merged tensor corresponding to each voxel-based 3D object from the set
of voxel-based 3D objects; (v) decoding the merged tensor using a set of
30 convolutional and pooling layers present in a decoder of the quantum-classical
hybrid 3D object segmentation model to obtain a decoded set of voxel-based 3D
7
objects; (vi) determining a training loss associated with the quantum-classical
hybrid 3D object segmentation model based on a comparison of each of the decoded
set of voxel-based 3D objects with a set of corresponding ground truth data; (vii)
backpropagating the training loss to the encoder through at least one of (a) the
5 classical path, and (b) the quantum path of the bottleneck layer of the quantumclassical
hybrid 3D object segmentation model; and (viii) tuning a set of parameters
in the quantum-classical hybrid 3D object segmentation model based on the training
loss to obtain the trained quantum-classical hybrid 3D object segmentation model;
performing steps (i) through (iv) for an incoming voxel-based three-dimensional
10 (3D) object on the trained quantum-classical hybrid 3D object segmentation model;
and decoding the merged tensor using a set of convolutional and pooling layers
present in the decoder of the trained quantum-classical hybrid 3D object
segmentation model to obtain a segmented voxel-based 3D object corresponding to
the incoming voxel-based three-dimensional (3D) object
15 [007] In accordance with an embodiment of the present disclosure, the
steps for the training are performed until the training loss associated with the
quantum-classical hybrid 3D object segmentation model converges to an optimal
value.
[008] In accordance with an embodiment of the present disclosure,
20 obtaining the second tensor from each voxel-based 3D object comprises: passing a
set of patches of the encoded voxel-based 3D object to a quantum feature map
comprised in the quantum circuit with a stride value in each axis for one channel at
a time in the quantum path; flattening each patch from the set of patches into an
array comprising a set of elements; mapping the set of elements in the array to a set
25 of qubits using a quantum circuit corresponding to the quantum feature map;
obtaining a set of expectation values by calculating an expectation value associated
with each qubit of the set of qubits; and mapping the set of expectation values to a
channel to obtain the second tensor from each voxel-based 3D object.
[009] In accordance with an embodiment of the present disclosure, the
30 quantum feature map is a 2 􀵈 2 􀵈 2 qubit three- dimensional (3D) feature map
8
which provides an all-to-all-connectivity to each element from the set of elements
in the quantum circuit.
[010] It is to be understood that both the foregoing general description and
the following detailed description are exemplary and explanatory only and are not
5 restrictive of the invention, as claimed.
BRIEF DESCRIPTION OF THE DRAWINGS
[011] The accompanying drawings, which are incorporated in and
constitute a part of this disclosure, illustrate exemplary embodiments and, together
10 with the description, serve to explain the disclosed principles:
[012] FIG. 1 illustrates a system for quantum-enhanced semantic
segmentation of voxel-based 3D-object, according to some embodiments of the
present disclosure.
[013] FIGS. 2A and 2B depicts an exemplary flow diagram illustrating
15 method for quantum-enhanced semantic segmentation of voxel-based 3D-object,
according to some embodiments of the present disclosure.
[014] FIG. 3 is an architectural diagram of a quantum-classical hybrid 3D
object segmentation model for quantum-enhanced semantic segmentation of voxelbased
3D-object, according to some embodiments of the present disclosure.
20 [015] FIG. 4 is a block diagram illustrating a process along the quantum
path of the quantum-classical hybrid 3D object segmentation model for quantumenhanced
semantic segmentation of voxel-based 3D-object, according to some
embodiments of the present disclosure.
[016] FIG. 5 is a schematic diagram of the quantum circuit of the quantum25
classical hybrid 3D object segmentation model for quantum-enhanced semantic
segmentation of voxel-based 3D-object, according to some embodiments of the
present disclosure.
[017] FIG. 6 depicts an example of a quantum circuit illustrating
connections between every possible combination of pairs of elements in the
30 quantum circuit, according to some embodiments of the present disclosure.
9
DETAILED DESCRIPTION OF EMBODIMENTS
[018] Exemplary embodiments are described with reference to the
accompanying drawings. In the figures, the left-most digit(s) of a reference number
identifies the figure in which the reference number first appears. Wherever
5 convenient, the same reference numbers are used throughout the drawings to refer
to the same or like parts. While examples and features of disclosed principles are
described herein, modifications, adaptations, and other implementations are
possible without departing from the scope of the disclosed embodiments.
[019] The embodiments herein provide a method and system for quantum10
enhanced semantic segmentation of voxel-based 3D-object. The embodiments
disclose a quantum-classical hybrid 3D object segmentation model which enhances
the performance of a classical 3D U-shaped deep neural network architecture (UNet)
by interleaving quantum layers with classical ones. The quantum-classical
hybrid 3D object segmentation model comprises quantum layers and layers of
15 classical 3D U-Net architecture which are trained together for performing object
segmentation. The U-Net architecture may include but not restricted to a threedimensional
(3D) U-Net and its variants. The example model uses feature maps
which perform quantum convolution at bottleneck layers of the U-Net and in future
with the larger and quality quantum computers become available, the quantum layer
20 can be utilized in other layers of classical 3D U-Net also, not restricting to
bottleneck layer alone. The example model uses a quantum circuit involving
quantum convolution at bottleneck layers of the 3D U-Net. However, the quantum
layer can be utilized in other layers of classical U-Net also, and not restricted to
bottleneck layer alone. The quantum-classical hybrid nature of the invention is used
25 for segmenting large size real-life 3D scans of practical interest. The disclosed
model is configurable depending on the capability of the available quantum
computer at that time.
[020] Conventionally, in quantum enhanced neural network architectures
for semantic segmentation, features are extracted from pretrained deep learning
30 models from 3D objects. These features are then transformed to another feature
space by embedding them into a quantum circuit and measuring an appropriate
10
basis. The quantum transformed features are then either directly used to draw
inference or are fed into a multilayered perceptron for single/multi-class
classification.
[021] The quantum circuit of the system of the present disclosure
comprises 8 qubits and is 5 more suitable and efficient for transforming the features
of 3D-objects with comparatively less time. The classically extracted features are
transformed to another feature space by embedding them into the quantum circuit
and measuring in an appropriate basis learnt through use of a feature map. The
quantum transformed features are then concatenated with classical features and fed
10 into the subsequent layers/ modules. This quantum circuit can be used even with
larger 3-dimensional tensors. In the present disclosure, the quantum circuit utilizes
a quantum feature map for 3D-object segmentation that is modified to surpass mIoU
(mean Intersection over Union) score by around 3 percent.
[022] Referring now to the drawings, and more particularly to FIGS. 1
15 through 6, where similar reference characters denote corresponding features
consistently throughout the figures, there are shown preferred embodiments and
these embodiments are described in the context of the following exemplary system
and/or method.
[023] FIG. 1 illustrates a system 100 for quantum-enhanced semantic
20 segmentation of voxel-based 3D-object, according to an embodiment of the present
disclosure. In an embodiment, the system 100 includes or is otherwise in
communication with one or more hardware processors 104, communication
interface device(s) or input/output (I/O) interface(s) 106, and one or more data
storage devices or memory 102 operatively coupled to the one or more hardware
25 processors 104. The one or more hardware processors 104, the memory 102, and
the I/O interface(s) 106 may be coupled to a system bus 108 or a similar mechanism.
[024] The I/O interface(s) 106 may include a variety of software and
hardware interfaces, for example, a web interface, a graphical user interface, and
the like. The I/O interface(s) 106 may include a variety of software and hardware
30 interfaces, for example, interfaces for peripheral device(s), such as a keyboard, a
mouse, an external memory, a plurality of sensor devices, a printer, and the like.
11
Further, the I/O interface(s) 106 may enable the system 100 to communicate with
other devices, such as web servers and external databases.
[025] The I/O interface(s) 106 can facilitate multiple communications
within a wide variety of networks and protocol types, including wired networks, for
example, local area 5 network (LAN), cable, etc., and wireless networks, such as
Wireless LAN (WLAN), cellular, or satellite. For the purpose, the I/O interface(s)
106 may include one or more ports for connecting a number of computing systems
with one another or to another server computer. Further, the I/O interface(s) 106
may include one or more ports for connecting a number of devices to one another
10 or to another server.
[026] The one or more hardware processors 104 may be implemented as
one or more microprocessors, microcomputers, microcontrollers, digital signal
processors, central processing units, state machines, logic circuitries, and/or any
devices that manipulate signals based on operational instructions. Among other
15 capabilities, the one or more hardware processors 104 are configured to fetch and
execute computer-readable instructions stored in the memory 102. In the context of
the present disclosure, the expressions ‘processors’ and ‘hardware processors’ may
be used interchangeably. In an embodiment, the system 100 can be implemented in
a variety of computing systems, such as laptop computers, portable computer,
20 notebooks, hand-held devices, workstations, mainframe computers, servers, a
network cloud and the like.
[027] The memory 102 may include any computer-readable medium
known in the art including, for example, volatile memory, such as static random
access memory (SRAM) and dynamic random access memory (DRAM), and/or
25 non-volatile memory, such as read only memory (ROM), erasable programmable
ROM, flash memories, hard disks, optical disks, and magnetic tapes. The memory
102 may also include quantum random access memory (qRAM) which uses qubits
to address any quantum superposition of memory cells. The quantum modules
explained further can be implemented on quantum processors based on various
30 technologies such as superconducting qubits, ion trapped qubits and so on. In an
embodiment, the memory 102 includes a plurality of modules 102a and a repository
12
102b for storing data processed, received, and generated by one or more of the
plurality of modules 102a. The plurality of modules 102a may include routines,
programs, objects, components, data structures, and so on, which perform particular
tasks or implement particular abstract data types.
5 [028] The plurality of modules 102a may include programs or computerreadable
instructions or coded instructions that supplement applications or
functions performed by the system 100. The plurality of modules 102a may also be
used as, signal processor(s), state machine(s), logic circuitries, and/or any other
device or component that manipulates signals based on operational instructions.
10 Further, the plurality of modules 102a can be used by hardware, by computerreadable
instructions executed by the one or more hardware processors 104, or by
a combination thereof. Further, the memory 102 may include information pertaining
to input(s)/output(s) of each step performed by the processor(s) 104 of the system
100 and methods of the present disclosure.
15 [029] The repository 102b may include a database or a data engine.
Further, the repository 102b amongst other things, may serve as a database or
includes a plurality of databases for storing the data that is processed, received, or
generated as a result of the execution of the plurality of modules 102a. Although
the repository 102b is shown internal to the system 100, it will be noted that, in
20 alternate embodiments, the repository 102b can also be implemented external to the
system 100, where the repository 102b may be stored within an external database
(not shown in FIG. 1) communicatively coupled to the system 100. The data
contained within such external database may be periodically updated. For example,
new data may be added into the external database and/or existing data may be
25 modified and/or non-useful data may be deleted from the external database. In one
example, the data may be stored in an external system, such as a Lightweight
Directory Access Protocol (LDAP) directory and a Relational Database
Management System (RDBMS). In another embodiment, the data stored in the
repository 102b may be distributed between the system 100 and the external
30 database. Functions of the components of the system 100 are now explained with
13
reference to architecture as depicted in steps in flow diagrams in FIGS. 2A and 2B,
and FIG. 3.
[030] FIGS. 2A and 2B, with reference to FIG. 1, depict an exemplary
flow diagram illustrating method for quantum-enhanced semantic segmentation of
5 voxel-based 3D-object, according to an embodiment of the present disclosure.
Referring to FIGS. 2A and 2B, in an embodiment, the system(s) 100 comprises one
or more data storage devices or the memory 102 operatively coupled to the one or
more hardware processors 104 and is configured to store instructions for execution
of steps of the method by the one or more processors 104. The steps of the method
10 200 of the present disclosure will now be explained with reference to components
of the system 100 of FIG. 1, the flow diagram as depicted in FIGS. 2A and 2B,
architectural diagram of FIG. 3, and one or more examples. Although steps of the
method 300 including process steps, method steps, techniques or the like may be
described in a sequential order, such processes, methods, and techniques may be
15 configured to work in alternate orders. In other words, any sequence or order of
steps that may be described does not necessarily indicate a requirement that the
steps be performed in that order. The steps of processes described herein may be
performed in any practical order. Further, some steps may be performed
simultaneously, or some steps may be performed alone or independently.
20 [031] In an embodiment, at step 202 of the present disclosure, the one or
more hardware processors 104 are configured to receive (i) a set of voxel-based
three-dimensional (3D) objects, and (ii) a set of annotated voxel-based threedimensional
(3D) objects corresponding to the set of voxel-based three-dimensional
objects as input data for training a quantum-classical hybrid three-dimensional (3D)
25 object segmentation model. FIG. 3 is an architectural diagram of the quantumclassical
hybrid 3D object segmentation model for quantum-enhanced semantic
segmentation of voxel-based 3D-object, according to some embodiments of the
present disclosure. The U-shaped encoder-decoder neural network architecture may
include but not restricted to a three-dimensional U-Net architecture and its variants.
30 As shown in FIG. 3, the quantum-classical hybrid 3D object segmentation model
includes a U-shaped encoder-decoder neural network architecture comprising a
14
bottleneck layer including (i) a classical path and (ii) a quantum path. The quantum
path is designed using a quantum circuit that comprises a set of eight qubits where
a central qubit is absent. In an embodiment, a plurality of voxel values associated
with the set of voxel-based 3D objects are embedded with 2-qubit gates for each of
a plurality of combinations of pair of 5 qubits in the quantum circuit and a
commutative property of 2-qubit gates is exhibited.
[032] In an embodiment, at step 204 of the present disclosure, the one or
more hardware processors 104 are configured to train the quantum-classical hybrid
3D object segmentation model using the input data to obtain a trained quantum10
classical hybrid 3D object segmentation model. the steps for training comprises: (i)
preprocessing the set of voxel-based 3D objects using one or more preprocessing
techniques, (ii) encoding the set of voxel-based 3D objects using a set of
convolutional and pooling layers present in an encoder in the quantum-classical
hybrid 3D object segmentation model to obtain an encoded set of voxel-based 3D
15 objects, (iii) obtaining, from each of the encoded set of voxel-based 3D objects, (i)
a first tensor from the classical path and (ii) a second tensor from the quantum path
of the bottleneck layer of the quantum-classical hybrid 3D object segmentation
model, (iv) concatenating, the first tensor and the second tensor to obtain a merged
tensor corresponding to each voxel-based 3D object from the set of voxel-based 3D
20 objects, (v) decoding, the merged tensor using a set of convolutional and pooling
layers present in a decoder of the quantum-classical hybrid 3D object segmentation
model to obtain a decoded set of voxel-based 3D objects, (vi) determining, via the
one or more hardware processors, a training loss associated with the quantumclassical
hybrid 3D object segmentation model based on a comparison of each of
25 the decoded set of voxel-based 3D objects with a set of corresponding ground truth
data, (vii) backpropagating, the training loss to the encoder through at least one of
(a) the classical path, and (b) the quantum path of the bottleneck layer of the
quantum-classical hybrid 3D object segmentation model, and (viii) tuning a set of
parameters in the quantum-classical hybrid 3D object segmentation model based on
30 the training loss to obtain the trained quantum-classical hybrid 3D object
segmentation model.
15
[033] In other words, for training the quantum-classical hybrid 3D object
segmentation model, first the set of voxel-based 3D objects is preprocessed using
one or more preprocessing techniques. The preprocessing techniques includes
rescaling to a pre-defined resolution, translation, resizing each of the voxel-based
5 3D objects to a specific dimension, normalizing with in a predefined range such as
0 to 1, and/or the like which are performed using conventional methods. Further,
the set of voxel-based 3D objects are encoded using a set of convolutional and
pooling layers present in an encoder in the quantum-classical hybrid 3D object
segmentation model to obtain an encoded set of voxel-based 3D objects. In another
10 embodiment, the set of voxel-based 3D objects are encoded through a sequence of
convolution and pooling layers to gradually reduce its dimensions with a
corresponding increase in the number of channels. Further, in the training process,
a (i) a first tensor from the classical path and (ii) a second tensor from the quantum
path of the bottleneck layer of the quantum-classical hybrid 3D object segmentation
15 model are obtained from each of the encoded set of voxel-based 3D objects as
shown in FIG. 3. The process of obtaining the second tensor from each voxel-based
3D object is explained hereafter. First a set of patches of the encoded voxel-based
3D object is passed to a quantum feature map comprised in the quantum circuit with
a stride value in each axis for one channel at a time in the quantum path. The stride
20 value could be 1 or 2. Further, each patch from the set of patches are flattened into
an array comprising a set of elements. Size of each patch considered in the present
disclosure is 2 􀵈 2 􀵈 2, however such exemplary size shall not be construed as
limiting the scope of the present disclosure. The size of the array is 𝑤􀬶. The set of
elements in the array are mapped to a set of qubits using a quantum circuit
25 corresponding to the quantum feature map. Further, a set of expectation values is
obtained by calculating an expectation value associated with each qubit of the set
of qubits. The set of qubits comprises 8 qubits. The set of expectation values are
mapped to a channel to obtain the second tensor from each voxel-based 3D object.
[034] The disclosed model introduces the concept of parallel paths, or
30 subnetworks with unidirectional propagation. As shown in FIG. 3, the bottleneck
layer of the quantum-classical hybrid 3D object segmentation model, the tensor size
16
reduces to 𝑚 􀵈 𝑛 􀵈 𝑝 􀵈 𝑞 and bifurcates into a classical and a unidirectional
quantum path. The encoded voxel-based 3D object is acted upon by classical
convolution to get a 𝑚 􀵈 𝑛 􀵈 𝑝 first tensor. Simultaneously, the same encoded
voxel-based 3D object is passed as an input to the quantum path. Along the classical
5 path, there is a convolutional layer with the first tensor as output with a size of
𝑚 􀵈 𝑛 􀵈 𝑝 􀵈 𝑞. FIG. 4 is a block diagram illustrating a process along the quantum
path of the quantum-classical hybrid 3D object segmentation model for quantumenhanced
semantic segmentation of voxel-based 3D-object, according to some
embodiments of the present disclosure. Along the quantum path, 2 􀵈 2 􀵈 2 patches
10 of the input 3D tensor of shape 𝑚 􀵈 𝑛 􀵈 𝑝 􀵈 𝑞 are passed to the quantum feature
map in the quantum circuit with stride 1 or 2 in each axis, one channel at a time,
which results in a 𝑚 􀵈 𝑛 􀵈 𝑝 􀵈 𝑞sized tensor as output. If needed, this second
tensor is further convolved to a dimension of the first tensor using a down
convolution layer present in the quantum path of the bottleneck layer to finally
15 obtain a similar 𝑚 × 𝑛 × 𝑝× q sized output tensor. In other words, the resultant
tensor obtained by measuring the quantum circuits are brought down to the second
tensor with a size of 𝑚 􀵈 𝑛 􀵈 𝑝 􀵈 𝑞 using down convolution to match the shape of
the first tensor in the classical path. When the dimension of second tensor matches
the first tensor, no convolution is needed. However, when the dimension of second
20 tensor does not match with the first tensor, a classical convolution is applied on the
second tensor to match it with the dimension of the first tensor.
[035] FIG. 5 is a schematic diagram of the quantum circuit of the quantumclassical
hybrid 3D object segmentation model for quantum-enhanced semantic
segmentation of voxel-based 3D-object, according to some embodiments of the
25 present disclosure. The quantum circuit disclosed in the present disclosure is a
2 􀵈 2 􀵈 2 qubit circuit that requires only 8 qubits which reduces the computational
time by a huge amount. Also, the quantum circuit does not need to have a central
or middle qubit. As shown in FIGS. 4 and 5, each patch is flattened, and each of the
elements is mapped to a quantum bit or qubit. As shown in FIG. 5, the quantum
30 circuit comprises a plurality of gates with a fixed circuit depth and commutative
𝑅􀯭􀯭 operations between neighbors. The quantum circuit is expected to act as a
17
potential 2 􀵈 2 􀵈 2 quantum receptive field that can be utilized for implementing
3D convolutions or feature maps on 3D voxel-based object. In the present
disclosure, the quantum feature map is a 2 􀵈 2 􀵈 2 qubit three- dimensional (3D)
feature map which provides an all-to-all-connectivity to each element from the set
of elements in the quantum 5 circuit. The all-to-all-connectivity represents
connections between every possible combination of pairs of elements from the set
of elements in the quantum circuit. FIG. 6 depicts an example of a quantum circuit
illustrating connections between every possible combination of pairs of elements in
the quantum circuit, according to some embodiments of the present disclosure. The
10 quantum circuit shown in FIG. 6 is a 4-qubit quantum circuit. The embedding is
done via the use of 𝐻, 𝑅􀯭 and 𝑅􀯭􀯭 gates.
[036] As shown in FIG. 5, in the quantum feature map, feature embedding
of input data along the classical path is done using Hadamard 􁈺𝐻􁈻, 𝑅􀯭 and 𝑅􀯭􀯭 gates.
In case the quantum path is non-trainable then the 𝑅􀯭􀯭 connectivity between two
15 input element values or input features represented by 􁈺𝑥􀯜 , 𝑥􀯝􁈻 is obtained by
multiplying input values of two features along the classical path (also referred as
two classical features input values), scaled in a range 􁈾0, 𝜋􁈿 such that angle of the
gate is 𝑖𝑗 􀵌 𝜋 𝑥􀯜 𝑥􀯝. This operation achieves connectivity between each input voxel
with its neighboring input voxel and introduces non-linearity. Moreover, this 𝑅􀯭􀯭
20 coupling is performed between any two pair of qubits to achieve an all-to-all
connectivity in the quantum feature map. The 𝑅􀯭􀯭 gates are chosen as they are
commutative and can be applied in any order.
[037] Further, in the training process, the first tensor and the second tensor
are concatenated to obtain a merged tensor corresponding to each voxel-based 3D
25 object from the set of voxel-based 3D objects. The output tensor (i.e., the second
tensor) is concatenated with the resultant tensor from the classical path (i.e., the
first tensor) and fed into the subsequent layers. The quantum circuit can be used
even with larger 4-dimensional tensors. The tensors from the classical and quantum
paths are concatenated along the channel axis to get the merged tensor of size
30 𝑚 􀵈 𝑛 􀵈 𝑝 􀵈 2𝑞 tensor. The merged tensor is then further down convolved and
18
pooled in the rest of the bottleneck layer. In the present disclosure, 𝑚 􀵌 𝑛 􀵌 𝑝 􀵌
4 𝑎𝑛𝑑 𝑞 􀵌 128.
[038] Furthermore, the merged tensor is decoded using a set of
convolutional and pooling layers present in a decoder of the quantum-classical
5 hybrid 3D object segmentation model to obtain a decoded set of voxel-based 3D
objects. The output of the bottleneck layer is decoded using up and down
convolutions and pooling. After obtaining the decoded set of voxel-based 3D
objects, a training loss associated with the quantum-classical hybrid 3D object
segmentation model is determined based on a comparison of each of the decoded
10 set of voxel-based 3D objects with a set of corresponding ground truth data. The set
of corresponding ground truth data is one hot encoded. The training of the quantumclassical
hybrid 3D object segmentation model further comprises, backpropagating
the training loss to the encoder through at least one of (a) the classical path, and (b)
the quantum path of the bottleneck layer of the quantum-classical hybrid 3D object
15 segmentation model, and further tuning a set of parameters in the quantum-classical
hybrid 3D object segmentation model based on the training loss to obtain the trained
quantum-classical hybrid 3D object segmentation model. Based on the training loss
value, backward propagation takes place normally along the decoder, until the
bottleneck layer. At the bottleneck layer, the backpropagation passes unhindered
20 along the classical path, but stops at quantum module on the quantum path.
Backpropagation continues in the encoder through the classical path.
[039] The steps for the training are performed until the training loss
associated with the quantum-classical hybrid 3D object segmentation model
converges to a predefined value. The predefined value could be any positive integer
25 based on depth of the U-Net encoder-decoder architecture and application.
However, in the present dislcoure, it ranges between 32 􀵈 32 􀵈 32 to
1024 􀵈 1024 􀵈 1024. This means that the steps (i) through (viii) are repeated for
multiple epochs for training the quantum-classical hybrid 3D object segmentation
model until the training loss is minimized and it converges to an optimal value,
30 tuning the parameters in the model.
19
[040] In an embodiment, at step 206 of the present disclosure, the one or
more hardware processors 104 are configured to perform steps (i) through (iv) for
an incoming voxel-based three-dimensional (3D) object on the trained quantumclassical
hybrid 3D object segmentation model. The incoming voxel-based threedimensional
(5 3D) object represents a test data. These steps are performed for
inferencing to segment the voxel-based 3D object. The incoming voxel-based threedimensional
(3D) object are preprocessed and fed into the trained quantum-classical
hybrid 3D object segmentation model. The same set of preprocessing techniques as
used in the training process of the quantum-classical hybrid 3D object segmentation
10 model is being used. As an output, a merged corresponding to each incoming voxelbased
three-dimensional (3D) object is obtained.
[041] In an embodiment, at step 208 of the present disclosure, the one or
more hardware processors 104 are configured to decode the merged tensor using a
set of convolutional and pooling layers present in a decoder of the trained quantum15
classical hybrid 3D object segmentation model to obtain a segmented voxel-based
3D object corresponding to the incoming voxel-based three-dimensional (3D)
object
[042] The written description describes the subject matter herein to enable
any person skilled in the art to make and use the embodiments. The scope of the
20 subject matter embodiments is defined by the claims and may include other
modifications that occur to those skilled in the art. Such other modifications are
intended to be within the scope of the claims if they have similar elements that do
not differ from the literal language of the claims or if they include equivalent
elements with insubstantial differences from the literal language of the claims.
25 [043] The embodiments of present disclosure herein address the problem
of object segmentation using quantum-classical hybrid 3D object segmentation
model. The embodiment thus provides the method steps for performing 3D object
segmentation using the quantum-classical hybrid 3D object segmentation model.
The disclosed quantum-classical hybrid 3D object segmentation model includes U30
Net encoder-decoder architecture comprising a bottleneck layer of classical path
and quantum path. Moreover, the quantum-classical hybrid 3D object segmentation
20
model comprises quantum layers and layers of classical U-Net encoder-decoder
architecture which are trained together for performing 3D object segmentation.
[044] It is to be understood that the scope of the protection is extended to
such a program and in addition to a computer-readable means having a message
therein; such 5 computer-readable storage means contain program-code means for
implementation of one or more steps of the method, when the program runs on a
server or mobile device or any suitable programmable device. The hardware device
can be any kind of device which can be programmed including e.g., any kind of
computer like a server or a personal computer, or the like, or any combination
10 thereof. The device may also include means which could be e.g., hardware means
like e.g., an application-specific integrated circuit (ASIC), a field-programmable
gate array (FPGA), or a combination of hardware and software means, e.g., an
ASIC and an FPGA, or at least one microprocessor and at least one memory with
software processing components located therein. Thus, the means can include both
15 hardware means, and software means. The method embodiments described herein
could be implemented in hardware and software. The device may also include
software means. Alternatively, the embodiments may be implemented on different
hardware devices, e.g., using a plurality of CPUs.
[045] The embodiments herein can comprise hardware and software
20 elements. The embodiments that are implemented in software include but are not
limited to, firmware, resident software, microcode, etc. The functions performed by
various components described herein may be implemented in other components or
combinations of other components. For the purposes of this description, a
computer-usable or computer readable medium can be any apparatus that can
25 comprise, store, communicate, propagate, or transport the program for use by or in
connection with the instruction execution system, apparatus, or device.
[046] The illustrated steps are set out to explain the exemplary
embodiments shown, and it should be anticipated that ongoing technological
development will change the manner in which particular functions are performed.
30 These examples are presented herein for purposes of illustration, and not limitation.
Further, the boundaries of the functional building blocks have been arbitrarily
21
defined herein for the convenience of the description. Alternative boundaries can
be defined so long as the specified functions and relationships thereof are
appropriately performed. Alternatives (including equivalents, extensions,
variations, deviations, etc., of those described herein) will be apparent to persons
5 skilled in the relevant art(s) based on the teachings contained herein. Such
alternatives fall within the scope of the disclosed embodiments. Also, the words
“comprising,” “having,” “containing,” and “including,” and other similar forms are
intended to be equivalent in meaning and be open ended in that an item or items
following any one of these words is not meant to be an exhaustive listing of such
10 item or items or meant to be limited to only the listed item or items. It must also be
noted that as used herein and in the appended claims, the singular forms “a,” “an,”
and “the” include plural references unless the context clearly dictates otherwise.
[047] Furthermore, one or more computer-readable storage media may be
utilized in implementing embodiments consistent with the present disclosure. A
15 computer-readable storage medium refers to any type of physical memory on which
information or data readable by a processor may be stored. Thus, a computerreadable
storage medium may store instructions for execution by one or more
processors, including instructions for causing the processor(s) to perform steps or
stages consistent with the embodiments described herein. The term “computer20
readable medium” should be understood to include tangible items and exclude
carrier waves and transient signals, i.e., be non-transitory. Examples include
random access memory (RAM), read-only memory (ROM), volatile memory,
nonvolatile memory, hard drives, CD ROMs, DVDs, flash drives, disks, and any
other known physical storage media.
25 [048] It is intended that the disclosure and examples be considered as
exemplary only, with a true scope of disclosed embodiments being indicated by the
following claims.
We Claim:
1. A processor implemented method (200), comprising:
receiving (202), via one or more hardware processors, (i) a set of voxel-based three-dimensional (3D) objects, and (ii) a set of annotated voxel-based three-dimensional (3D) objects corresponding to the set of voxel-based three-dimensional (3D) objects as input data for training a quantum-classical hybrid three-dimensional (3D) object segmentation model, wherein the quantum-classical hybrid 3D object segmentation model includes a U-shaped encoder-decoder neural network architecture comprising a bottleneck layer including (i) a classical path and (ii) a quantum path, and wherein the quantum path is designed using a quantum circuit that comprises a set of eight qubits where a central qubit is absent, and wherein a plurality of voxel values associated with the set of voxel-based 3D objects are embedded with 2-qubit gates for each of a plurality of combinations of pair of qubits in the quantum circuit and a commutative property of 2-qubit gates is exhibited;
training (204), via one or more hardware processors, the quantum-classical hybrid 3D object segmentation model using the input data to obtain a trained quantum-classical hybrid 3D object segmentation model, wherein the steps for training comprises:
(i) preprocessing the set of voxel-based 3D objects using one or more preprocessing techniques;
(ii) encoding the set of voxel-based 3D objects using a set of convolutional and pooling layers present in an encoder in the quantum-classical hybrid 3D object segmentation model to obtain an encoded set of voxel-based 3D objects; (iii) obtaining, from each of the encoded set of voxel-based 3D objects, (i) a first tensor from the classical path and (ii) a second tensor from the quantum path of the bottleneck layer of the quantum-classical hybrid 3D object segmentation model;

(iv) concatenating the first tensor and the second tensor to obtain a merged tensor corresponding to each voxel-based 3D object from the set of voxel-based 3D objects; (v) decoding the merged tensor using a set of convolutional and pooling layers present in a decoder of the quantum-classical hybrid 3D object segmentation model to obtain a decoded set of voxel-based 3D objects;
(vi) determining a training loss associated with the quantum-classical hybrid 3D object segmentation model based on a comparison of each of the decoded set of voxel-based 3D objects with a set of corresponding ground truth data; (vii) backpropagating the training loss to the encoder through at least one of (a) the classical path, and (b) the quantum path of the bottleneck layer of the quantum-classical hybrid 3D object segmentation model; and
(viii) tuning a set of parameters in the quantum-classical hybrid
3D object segmentation model based on the training loss to obtain
the trained quantum-classical hybrid 3D object segmentation model;
performing (206), via the one or more hardware processors, steps (i)
through (iv) for an incoming voxel-based three-dimensional (3D) object on
the trained quantum-classical hybrid 3D object segmentation model; and
decoding (208), via the one or more hardware processors, the
merged tensor using a set of convolutional and pooling layers present in the
decoder of the trained quantum-classical hybrid 3D object segmentation
model to obtain a segmented voxel-based 3D object corresponding to the
incoming voxel-based three-dimensional (3D) object.
2. The processor implemented method as claimed in claim 1, wherein the steps for the training are performed until the training loss associated with the quantum-classical hybrid 3D object segmentation model converges to an optimal value.

3. The processor implemented method as claimed in claim 1, wherein
obtaining the second tensor from each voxel-based 3D object comprises:
passing a set of patches of the encoded voxel-based 3D object to a quantum feature map comprised in the quantum circuit with a stride value in each axis for one channel at a time in the quantum path;
flattening each patch from the set of patches into an array comprising a set of elements;
mapping the set of elements in the array to a set of qubits using a quantum circuit corresponding to the quantum feature map;
obtaining a set of expectation values by calculating an expectation value associated with each qubit of the set of qubits; and
mapping the set of expectation values to a channel to obtain the second tensor from each voxel-based 3D object.
4. The processor implemented method as claimed in claim 3, wherein the quantum feature map is a 2 x 2 x 2 qubit three- dimensional (3D) feature map which provides an all-to-all-connectivity to each element from the set of elements in the quantum circuit.
5. A system (100), comprising:
a memory (102) storing instructions;
one or more communication interfaces (106); and
one or more hardware processors (104) coupled to the memory (102) via the
one or more communication interfaces (106), wherein the one or more
hardware processors (104) are configured by the instructions to:
receive (i) a set of voxel-based three-dimensional (3D) objects, and (ii) a set of annotated voxel-based three-dimensional (3D) objects corresponding to the set of voxel-based three-dimensional (3D) objects as input data for training a quantum-classical hybrid three-dimensional (3D) object segmentation model, wherein the quantum-classical hybrid 3D object segmentation model includes a U-shaped encoder-decoder neural network

architecture comprising a bottleneck layer including (i) a classical path and (ii) a quantum path, and wherein the quantum path is designed using a quantum circuit that comprises a set of eight qubits where a central qubit is absent, and wherein a plurality of voxel values associated with the set of voxel-based 3D objects are embedded with 2-qubit gates for each of a plurality of combinations of pair of qubits in the quantum circuit and a commutative property of 2-qubit gates is exhibited;
train the quantum-classical hybrid 3D object segmentation model using the input data to obtain a trained quantum-classical hybrid 3D object segmentation model, wherein the steps for training comprises:
(i) preprocessing the set of voxel-based 3D objects using one or more preprocessing techniques;
(ii) encoding the set of voxel-based 3D objects using a set of convolutional and pooling layers present in an encoder in the quantum-classical hybrid 3D object segmentation model to obtain an encoded set of voxel-based 3D objects; (iii) obtaining from each of the encoded set of voxel-based 3D objects, (i) a first tensor from the classical path and (ii) a second tensor from the quantum path of the bottleneck layer of the quantum-classical hybrid 3D object segmentation model; (iv) concatenating the first tensor and the second tensor to obtain a merged tensor corresponding to each voxel-based 3D object from the set of voxel-based 3D objects; (v) decoding the merged tensor using a set of convolutional and pooling layers present in a decoder of the quantum-classical hybrid 3D object segmentation model to obtain a decoded set of voxel-based 3D objects;
(vi) determining a training loss associated with the quantum-classical hybrid 3D object segmentation model based on a comparison of each of the decoded set of voxel-based 3D objects with a set of corresponding ground truth data;

(vii) backpropagating the training loss to the encoder through at least one of (a) the classical path, and (b) the quantum path of the bottleneck layer of the quantum-classical hybrid 3D object segmentation model; and
(viii) tuning a set of parameters in the quantum-classical hybrid 3D object segmentation model based on the training loss to obtain the trained quantum-classical hybrid 3D object segmentation model; perform steps (i) through (iv) for an incoming voxel-based three-dimensional (3D) object on the trained quantum-classical hybrid 3D object segmentation model; and
decode the merged tensor using a set of convolutional and pooling layers present in the decoder of the trained quantum-classical hybrid 3D object segmentation model to obtain a segmented voxel-based 3D object corresponding to the incoming voxel-based three-dimensional (3D) object.
6. The system (100) as claimed in claim 5, wherein the steps for the training are performed until the training loss associated with the quantum-classical hybrid 3D object segmentation model converges to an optimal value.
7. The system (100) as claimed in claim 5, wherein obtaining the second tensor from each voxel-based 3D object comprises:
passing a set of patches of the encoded voxel-based 3D object to a quantum feature map comprised in the quantum circuit with a stride value in each axis for one channel at a time in the quantum path;
flattening each patch from the set of patches into an array comprising a set of elements;
mapping the set of elements in the array to a set of qubits using a quantum circuit corresponding to the quantum feature map;
obtaining a set of expectation values by calculating an expectation value associated with each qubit of the set of qubits; and

mapping the set of expectation values to a channel to obtain the second tensor from each voxel-based 3D object.
8. The system (100) as claimed in claim 7, wherein the quantum feature map
is a 2 x 2 x 2 qubit three- dimensional (3D) feature map which provides an all-to-all-connectivity to each element from the set of elements in the quantum circuit.