Claims:1. A method (200) for generating a representation learning of data corresponding to an arbitrary domain, comprising:
obtaining (202), via a representation learning subsystem (110), at least one dataset corresponding to the arbitrary domain, wherein the at least one dataset comprises one or more entities;
building (204), via a representation learning subsystem (110), a similarity matrix corresponding to each of the one or more entities of the at least one dataset;
generating (206), via a representation learning subsystem (110), an adjacency matrix corresponding to each of the one or more entities based at least on a corresponding similarity matrix and a graph kernel function; and
re-ordering (210), via a representation learning subsystem (110), the adjacency matrix corresponding to each of the one or more entities based on an order advisory (208) to generate a corresponding image grid (212), wherein each image grid (212) is the representation learning corresponding to each of the one or more entities.
2. The method (200) of claim 1, wherein each of the one or more entities comprises a graph representation of a raw data entity corresponding to the arbitrary domain.
3. The method (200) of claim 2, wherein the at least one dataset further comprises a pre-determined score corresponding to each of the one or more entities.
4. The method (200) of claim 3, further comprising:
dividing (214) the image grids (212) and the pre-determined scores corresponding to the one or more entities into a training set and a test set;
training (216) a convolutional neural network with one or more image grids (212) and pre-determined scores corresponding to the training set;
testing (218) the convolutional neural network with one or more image grids (212) corresponding to the test set to generate one or more corresponding predicted scores; and
evaluating (220) a performance of the convolutional neural network based on a comparison of a predicted score and a pre-determined score corresponding to each of the image grids of the test set.
5. The method (200) of claim 3, wherein building (204) the similarity matrix comprises generating the similarity matrix based on a similarity function, and wherein the similarity function comprises a correlation function, a spatial distance function, a spectral translation function, or combinations thereof.
6. The method (200) of claim 3, wherein the graph kernel function comprises a Gaussian kernel, an RBF kernel, a shortest path kernel, a random walk kernel, or combinations thereof.
7. The method (200) of claim 3, wherein the order advisory (208) comprises an optimization function of a row and column order configuration corresponding to the adjacency matrix.
8. A representation learning subsystem (110) for representation learning of data corresponding to an arbitrary domain, comprising:
a processor unit (112) configured to obtain (202) at least one dataset corresponding to the arbitrary domain, wherein the at least one dataset comprises one or more entities;
a modeling unit (116) operatively coupled to the processor unit (112) and configured to:
build (204) a similarity matrix corresponding to each of the one or more entities of the at least one dataset;
generate (206) an adjacency matrix corresponding to each of the one or more entities based at least on a corresponding similarity matrix and a graph kernel function; and
an ordering unit (118) operatively coupled to the processor unit (112) and configured to re-order (210) the adjacency matrix corresponding to each of the one or more entities based on an order advisory (208) to generate an image grid (212) corresponding to each of the one or more entities, wherein each image grid (212) is the representation learning corresponding to each of the one or more entities.
9. The representation learning subsystem (110) of claim 8, wherein each of the one or more entities comprises a graph representation of at least one raw data entity corresponding to the arbitrary domain.
10. The representation learning subsystem (110) of claim 9, wherein the at least one dataset further comprises a pre-determined score corresponding to each of the one or more entities.
11. The representation learning subsystem (110) of claim 10, wherein the modeling unit (116) is configured to build the similarity matrix based on at least one similarity function, and wherein the at least one similarity function comprises a correlation function, a spatial distance function, a spectral translation function, or combinations thereof.
12. The representation learning subsystem (110) of claim 10, wherein the graph kernel function comprises a Gaussian kernel, an RBF kernel, a shortest path kernel, a random walk kernel, or combinations thereof.
13. The representation learning subsystem (110) of claim 10, wherein the order advisory (208) comprises an optimization function of a row and column order configuration corresponding to the adjacency matrix.
14. A system (100), comprising:
a representation learning subsystem (110), comprising:
a processor unit (112) configured to:
obtain (202) at least one dataset corresponding to an arbitrary domain, wherein the at least one dataset comprises one or more entities, and wherein each of the one or more entities comprises a graph representation of at least one raw data entity corresponding to the arbitrary domain and a pre-determined score corresponding to each of the one or more entities;
a modeling unit (116) operatively coupled to the processor unit (112) and configured to:
build (204) a similarity matrix corresponding to each of the one or more entities of the at least one dataset;
generate (206) an adjacency matrix corresponding to each of the one or more entities based at least on a corresponding similarity matrix and a graph kernel function;
an ordering unit (118) operatively coupled to the processor unit (112) and configured to re-order (210) the adjacency matrix corresponding to each of the one or more entities based on an order advisory (208) to generate an image grid (212) corresponding to each of the one or more entities, wherein each image grid (212) is the representation learning corresponding to each of the one or more entities;
a convolutional neural network subsystem (124) communicatively coupled to the representation learning subsystem (110) and configured to:
divide (214) the image grids and pre-determined scores corresponding to the one or more entities of the at least one dataset into a training set and a test set;
train (216) a convolutional neural network running on the convolutional neural network subsystem (124) with one or more image grids (212) and pre-determined scores corresponding to the training set;
test (218) the convolutional neural network with one or more image grids (212) corresponding to the test set to obtain one or more corresponding predicted scores; and
evaluate (220) a performance of the convolutional neural network based on a comparison of a predicted score and a pre-determined score corresponding to each of the image grids (212) corresponding to the test set.
15. A method (200), comprising:
obtaining (202) at least one dataset corresponding to an arbitrary domain, wherein the at least one dataset comprises one or more entities, and wherein each of the one or more entities comprises a graph representation of at least one raw data entity corresponding to the arbitrary domain and a pre-determined score corresponding to each of the one or more entities;
building (204) a similarity matrix corresponding to each of the one or more entities of the at least one dataset;
generating (206) an adjacency matrix corresponding to the each of the one or more entities based at least on a corresponding similarity matrix and a graph kernel function;
re-ordering (210) the adjacency matrix corresponding to each of the one or more entities based on an order advisory (208) to generate an image grid (212) corresponding to each of the one or more entities, wherein the image grid (212) is the representation learning corresponding to each of the one or more entities; and
evaluating a performance of a convolutional neural network based on the image grids corresponding to the one or more entities.
16. The method of claim 15, wherein evaluating the performance of the convolutional neural network comprises:
dividing (214) the image grids (212) and pre-determined scores corresponding to each of the one or more entities of the at least one dataset into a training set and a test set;
training (216) a convolutional neural network with one or more image grids (212) and pre-determined scores corresponding to the training set;
testing (218) the convolutional neural network with one or more image grids (212) corresponding to the test set to obtain one or more corresponding predicted scores; and
comparing at least one predicted score and a corresponding pre-determined score associated with each of the image grids (212) corresponding to the test set to assess the performance of the convolutional neural network.
, Description:BACKGROUND
[0001] Embodiments of the present specification relate generally to a system and method for representation learning of data on arbitrary domains for deep convolutional architectures. Specifically, the system and method are directed to determining an optimized and ordered representation learning of raw data from arbitrary domain spaces for use by deep convolutional architectures.
[0002] As will be appreciated, machine learning is the subfield of computer science that “gives computers the ability to learn without being explicitly programmed.” Machine learning explores the study and construction of algorithms that can learn from and make predictions on data. In machine learning, feature learning or representation learning is a set of techniques that transform raw data input into a representation that can be effectively exploited in machine learning tasks. Representation learning is motivated by the fact that machine learning tasks such as classification often require input that is mathematically and computationally convenient to process. However, real-world data such as images, video, and sensor measurement is usually complex, redundant, and highly variable. Thus, it is desirable to identify useful features or representations from raw data. Currently, manual feature identification methods require expensive human labor and rely on expert knowledge. Also, manually generated representations normally do not lend themselves well to generalization, thereby motivating the design of efficient representation learning techniques to automate and generalize feature or representation learning.
[0003] As will also be appreciated, in machine learning, a convolutional neural network (CNN or ConvNet) or a deep convolutional neural network is a type of feed-forward artificial neural network in which the connectivity pattern between its neurons is inspired by the organization of the animal visual cortex. Convolutional neural networks (CNNs) are biologically inspired variants of multilayer perceptrons, designed to emulate the behavior of a visual cortex with minimal amounts of preprocessing. It may be noted that a multilayer perceptron network (MLP) is a feed-forward artificial neural network model that maps sets of input data onto a set of appropriate outputs. An MLP includes multiple layers of nodes in a directed graph, with each layer fully connected to the next one. In current deployments, CNNs have wide applications in image and video recognition, recommender systems, and natural language processing. CNNs are also known
as shift invariant or space invariant artificial neural networks (SIANNs), based on their shared weights architecture and translation invariance characteristics.
[0004] It may be noted that in deep CNN architectures, a convolutional layer is the core building block. Parameters associated with a convolutional layer include a set of learnable filters or kernels. During a forward pass, each filter is convolved across the width and height of the input volume, computing the dot product between the entries of the filter and the input and producing a two-dimensional (2D) activation map of that filter. Thus, the network learns filters that are activated when the network detects a specific type of feature at a given spatial position in the input.
[0005] CNN architectures mitigate the challenges posed by MLP architecture by exploiting the strong spatially local correlation also known as neighborhood properties present in natural images. Moreover, in CNNs, each filter is replicated across the entire visual field, causing all the neurons in a convolutional layer to detect the same feature. This allows for features to be detected regardless of their position in the visual field, thus constituting the property of shift and translation invariance. Thus, convolutions are meaningful operations for image data due to their shift invariant property and their strong spatially local correlation. Consequently, deep CNNs have had tremendous success in computer vision and image processing domains with classification and regression problems.
[0006] However, for data on arbitrary domains, such as text document corpora or bioinformatic data, data defined on three-dimensional (3D) meshes such as surface tension or temperature, measurements from a network of meteorological stations, data coming from social networks or collaborative filtering, and the like, convolution operations are not effective due to the absence of neighborhood properties or strong spatially local correlations and shift invariance. In such cases, raw data is typically modeled via graph representations. It may be noted that graph properties may include structural and spectral properties. More particularly, spectral properties of a graph pertain to characteristic polynomials, eigenvalues, and eigenvectors of matrices associated with the graph, such as its adjacency matrix or Laplacian matrix.
BRIEF DESCRIPTION
[0007] In accordance with certain aspects of the present specification, a method for generating a representation learning of data corresponding to an arbitrary domain is presented. The method includes obtaining, via a representation learning subsystem, at least one dataset corresponding to the arbitrary domain, where the at least one dataset includes one or more entities. The method also includes building, via a representation learning subsystem, a similarity matrix corresponding to each of the one or more entities of the at least one dataset. Additionally, the method includes generating, via a representation learning subsystem, an adjacency matrix corresponding to each of the one or more entities based at least on a corresponding similarity matrix and a graph kernel function. Moreover, the method includes re-ordering the adjacency matrix corresponding to each of the one or more entities based on an order advisory to generate a corresponding image grid, where each image grid is the representation learning corresponding to each of the one or more entities.
[0008] In accordance with another aspect of the present specification, a representation learning subsystem for representation learning of data corresponding to an arbitrary domain is presented. The representation learning subsystem includes a processor unit configured to obtain at least one dataset corresponding to the arbitrary domain, where the at least one dataset comprises one or more entities. Also, the representation learning subsystem includes a modeling unit operatively coupled to the processor unit and configured to build a similarity matrix corresponding to each of the one or more entities of the at least one dataset and generate an adjacency matrix corresponding to each of the one or more entities based at least on a corresponding similarity matrix and a graph kernel function. Furthermore, the representation learning subsystem includes an ordering unit operatively coupled to the processor unit and configured to re-order the adjacency matrix corresponding to each of the one or more entities based on an order advisory to generate an image grid corresponding to each of the one or more entities, where each image grid is the representation learning corresponding to each of the one or more entities.
[0009] In accordance with yet another aspect of the present specification, a system is presented. The system includes a representation learning subsystem, where the representation learning subsystem includes a processor unit configured to obtain at least one dataset corresponding to the arbitrary domain, where the at least one dataset includes one or more entities, and where each of the one or more entities includes a graph representation of at least one raw data entity
corresponding to the arbitrary domain and a pre-determined score corresponding to each of the one or more entities. Also, the representation learning subsystem includes a modeling unit operatively coupled to the processor unit and configured to build a similarity matrix corresponding to each of the one or more entities of the at least one dataset and generate an adjacency matrix corresponding to each of the one or more entities based at least on a corresponding similarity matrix and a graph kernel function. Furthermore, the representation learning subsystem includes an ordering unit operatively coupled to the processor unit and configured to re-order the adjacency matrix corresponding to each of the one or more entities based on an order advisory to generate an image grid corresponding to each of the one or more entities, where each image grid is the representation learning corresponding to each of the one or more entities. Moreover, the system includes a convolutional neural network subsystem communicatively coupled to the representation learning subsystem and configured to divide the image grids and pre-determined scores corresponding to the one or more entities of the at least one dataset into a training set and a test set, train a convolutional neural network running on the convolutional neural network subsystem with one or more image grids and pre-determined scores corresponding to the training set, test the convolutional neural network with one or more image grids corresponding to the test set to obtain one or more corresponding predicted scores, and evaluate a performance of the convolutional neural network based on a comparison of a predicted score and a pre-determined score corresponding to each of the image grids corresponding to the test set.
[0010] In accordance with yet another aspect of the present specification, a method is presented. The method includes obtaining at least one dataset corresponding to an arbitrary domain, where the at least one dataset comprises one or more entities, where each of the one or more entities comprises a graph representation of at least one raw data entity corresponding to the arbitrary domain and a pre-determined score corresponding to each of the one or more entities. The method further includes building a similarity matrix corresponding to each of the one or more entities of the at least one dataset. Moreover, the method includes generating an adjacency matrix corresponding to the each of the one or more entities based at least on a corresponding similarity matrix and a graph kernel function. Additionally, the method includes re-ordering the adjacency matrix corresponding to each of the one or more entities based on an order advisory to generate an image grid corresponding to each of the one or more entities, where the image grid is the representation learning corresponding to each of the one or more entities. Furthermore, the method
includes evaluating a performance of a convolutional neural network based on the image grids corresponding to the one or more entities.
DRAWINGS
[0011] These and other features and aspects of embodiments of the present specification will become better understood when the following detailed description is read with references to the accompanying drawings in which like characters represent like parts throughout the drawings, wherein:
[0012] FIG. 1 is a schematic diagram of an exemplary system for representation learning of data on arbitrary domains for a deep convolutional architecture, in accordance with aspects of the present specification;
[0013] FIG. 2 is a flowchart illustrating a method for representation learning of data on arbitrary domains for a deep convolutional architecture, in accordance with aspects of the present specification;
[0014] FIG. 3A is a diagrammatic representation of an exemplary re-ordered adjacency matrix or image grid corresponding to functional magnetic resonance imaging (fMRI) data, generated as a training sample using the systems and methods of FIGs. 1-2, in accordance with aspects of the present specification; and
[0015] FIG. 3B is a diagrammatic representation of an exemplary re-ordered adjacency matrix or image grid corresponding to fMRI data, generated as a test sample using the systems and methods of FIGs. 1-2, in accordance with aspects of the present specification.
DETAILED DESCRIPTION
[0016] As will be described in detail hereinafter, various embodiments of an exemplary system and method for representation learning of data on arbitrary domains for deep convolutional architectures are presented. In the effort to provide a concise description of these embodiments, all features of an actual implementation may not be described in the specification. It should be appreciated that in the development of any such actual implementation, as in any engineering or design project, numerous implementation-specific decisions must be made to achieve the
developer’s specific goals such as compliance with system-related and business-related constraints.
[0017] When describing elements of the various embodiments of the present specification, the articles “a”, “an”, “the” and “said” are intended to mean that there are one or more of the elements. The terms “comprising”, “including” and “having” are intended to be inclusive and mean that there may be additional elements other than the listed elements. Furthermore, the terms “build” and “construct” and their variations are intended to mean a mathematical determination or computation of mathematical constructs. The terms “data drawn on arbitrary domains” or “data on arbitrary domains” are intended to mean data corresponding to a domain for example, social media data, sensor data, enterprise data and the like.
[0018] An exemplary system 100 for representation learning of data on arbitrary domains for deep convolutional architectures, in accordance with aspects of the present specification, is illustrated in FIG. 1. The system 100 is configured to receive data from one or more data sources (not shown). In one embodiment, the data may include graph representations of data on different domains. By way of example, the data that is provided by the data sources to the system 100 may include graph representations of imaging modality data (IMD) 102, social media data (SMD) 104, text document corpora (TDC) 106, bioinformatic data (BD) 108, and the like. It may be noted that the terms “graph representations,” “graph data representations,” and “graph data” may be used interchangeably in conjunction with the term “data 102-108” and are intended to mean data on arbitrary domains represented in a graph form. Additionally, the system 100 is further configured to receive a pre-determined score corresponding to the graph representations of data on the arbitrary domains from the one or more data sources. The pre-determined scores will be described in greater detail hereinunder with references to FIG. 2.
[0019] The system 100 includes a representation learning subsystem 110 that is communicatively coupled to the one or more data sources and configured to receive the graph data 102-108. In addition, the system 100 includes a convolutional neural network (CNN) subsystem 124 that is communicatively coupled to the representation learning subsystem 110.
[0020] In a presently contemplated configuration of FIG. 1, the representation learning subsystem 110 includes a processor unit 112, a memory unit 114, a modeling unit 116, and an
ordering unit 118. The processor unit 112 is communicatively coupled to the memory unit 114. Also, the modeling unit 116 and the ordering unit 118 are operatively coupled to the processor unit 112 and the memory unit 114. It may be noted that the representation learning system 110 may include other components or hardware, and is not limited to the components shown in FIG. 1.
[0021] In certain embodiments, the modeling unit 116 and the ordering unit 118 may be implemented as software systems or computer instructions executable via one or more processing units 112 and stored in the memory unit 114. In other embodiments, the modeling unit 116 and ordering unit 118 may be implemented as a hardware system, for example, via FPGAs, custom chips, integrated circuits (ICs), and the like.
[0022] In one embodiment, the modeling unit 116 is configured to build one or more similarity matrices corresponding to the graph data 102-108 using one or more similarity functions or measures. A similarity matrix may be defined as a graphical representation of similar sequences in a data series. The similarity matrices and similarity functions and measures will be described in greater detail hereinunder with references to FIG. 2.
[0023] Further, the modeling unit 116 is configured to generate one or more adjacency matrices corresponding to the one or more similarity matrices associated with the graph data 102-108. More particularly, the modeling unit 116 is configured to generate the adjacency matrices based on corresponding similarity matrices and one or more graph kernel functions. The adjacency matrices and graph kernel functions will be described in greater detail hereinunder referring to FIG. 2.
[0024] Moreover, the ordering unit 118 is configured to reorder the one or more adjacency matrices based on order advisory data 120. In one embodiment, reordering the adjacency matrices entails re-arranging the rows and columns of the adjacency matrices. Also, the order advisory data 120 may be provided to the representation learning subsystem 110 via a user interface/input unit 126. In other embodiments, the order advisory data 120 may be obtained from a data repository 130. The order advisory data 120 will be described in greater detail hereinunder referring to FIG. 2.
[0025] Furthermore, the ordering unit 118 is configured to reorder the adjacency matrices based on the order advisory data 120 to generate an image grid or image grid data 122. The image grid
Commented [TL(GO1]: Just for consistency can we make it processor
122 is generally representative of a re-ordered adjacency matrix. It may be noted that the image grid 122 is the representation learning of the graph data 102-108.
[0026] The image grid 122 may serve as an input to the CNN subsystem 124 that is communicatively coupled to the representation learning system 110. In some non-limiting examples, the CNN subsystem 124 may be a stand-alone computer system, a distributed computer network, one or more cloud servers, and the like. Moreover, the CNN subsystem 124 is configured to run a CNN based on standard software packages, frameworks, or libraries, for example, Caffe, DeepLearning4j, Tensorflow, Theano, Keras, and the like. In other embodiments, the CNN subsystem 124 may be implemented as software instructions executable by one or more processors of a computer system, tablet or mobile phone. In some other embodiments, the CNN subsystem 124 may be implemented as a hardware system, for example, via FPGAs, custom chips, integrated circuits (ICs), and the like.
[0027] In certain embodiments, the system 100 is configured to obtain the graph data 102-108 on arbitrary domains and the corresponding pre-determined scores and derive a representation learning of the graph data 102-108 in the form of one or more image grids 122. In accordance with aspects of the present specification, these image grids 122 may be divided into training and test datasets. The training dataset and corresponding pre-determined scores may be used to train a CNN in a training phase.
[0028] Subsequently, the test dataset may be provided as input to the CNN in a testing phase. In the testing phase, the CNN may generate a predicted score for each image grid in the test dataset. Further, the predicted scores obtained for the image grids of the test dataset may be compared with corresponding pre-determined scores to assess or evaluate the performance of the CNN. In certain embodiments, a “close match” between the predicted scores and the corresponding pre-determined scores is representative of an accuracy and/or efficacy of the image grid 122 as the learning representation of the graph data on the arbitrary domains. By way of example, if a percentage error of the predicted score with respect to the corresponding pre-determined score lies within a range from about 0% to about 5%, then the predicted score may be determined to be a close match to the pre-determined score. Accordingly, the close match of the predicted score to the pre-determined score is generally representative of an accuracy and/or efficacy of the image grid 122
as the representation learning of the graph data on the arbitrary domains. The working of the system 100 and more particularly, the working of the representation learning subsystem 110 will be described in greater detail with references to FIG. 2 and FIGs. 3A-3B.
[0029] As will be appreciated, data structures are used for storing graphs in a computer system. These data structures depend on both the graph structure and the algorithm used for manipulating the graph. List structures are often preferred for sparse graphs as these structures have smaller memory requirements. Matrix structures on the other hand provide faster access for some applications, however, these matrix structures consume huge amounts of memory. Matrix structures that characterize graphs may include similarity matrices and adjacency matrices. In one embodiment, the adjacency matrix may be stored/represented as a matrix structure in which both the rows and columns are indexed by vertices, wherein a “1” indicates two adjacent nodes and a 0 indicates two non-adjacent nodes.
[0030] Furthermore, for data analysis of graphs, it is advantageous to generate one or more feature vector sequences of complex graphs having many nodes and edges. Feature vector sequences may be used to generate self-similarity or similarity matrices where similar segments of the feature vectors manifest as paths of high similarity along the diagonals of a similarity matrix. As previously noted, a similarity matrix may be defined as a graphical representation of similar sequences in a data series. To construct a similarity matrix, the data series is first transformed into an ordered sequence of feature vectors, where each vector describes relevant features of the data series for a given local interval. Further, the similarity matrix is formed by computing a similarity of pairs of feature vectors based on different similarity measures or functions. Some non-limiting examples of similarity measures or functions include spatial distance as expressed by distance matrices, correlation or a comparison of local histograms or spectral properties.
[0031] As previously noted, the graph data 102-108 including one or more graphs is obtained from data sources corresponding to arbitrary domains. The graph data 102-108 may include multi-channel graph data. A dataset with one or more entities may be generated from the graph data 102-108, where each entity corresponds to a graph. Additionally, the dataset may include a pre-determined score corresponding to each entity in the dataset. In one embodiment, the entities may be functional MRI (fMRI) connectivity matrices for patients or subjects and the pre-determined
scores may be obtained from the subjects, using questionnaires and other tests. The pre-determined score may be generally representative of a conditional probability of a response category corresponding to each entity of the dataset. In one example, the response category may be generally representative of a classification category corresponding to the entity of the dataset.
[0032] With the foregoing in mind, a flowchart 200 generally representative of a method for representation learning of data on an arbitrary domain for use in deep convolutional architectures, in accordance with aspects of the present specification, is presented in FIG. 2. The method 200 is described with references to the components of FIG. 1. In one embodiment, the method 200 entails generating a re-ordered adjacency matrix using the graph data 102-108 and presenting the re-ordered matrix to the CNN subsystem 124 as the image grid 122. It may be noted that the flowchart 200 illustrates the main steps of the method for representation learning of data on an arbitrary domain for use in deep convolutional architectures. In some embodiments, various steps of the method 200 of FIG. 2, more particularly, steps 202-206 may be performed by the processor unit 112 in conjunction with memory unit 114 and modeling unit 116 of the representation learning subsystem 110. Moreover, steps 208-210 may be performed by processor unit 112 in conjunction with memory unit 114 and ordering unit 118. Furthermore, steps 212-218 may be performed by the CNN subsystem 124.
[0033] The method 200 starts at step 202, where at least one dataset corresponding to an arbitrary domain is obtained. It may be noted that the dataset includes at least one entity and a corresponding pre-determined score. In FIG. 2, reference numerals 102-108 correspond to reference numerals 102-108 of FIG. 1, and are generally representative of graph data on arbitrary domains. In certain embodiments, each entity of the dataset may be a graph representation of a raw data entity in the arbitrary domain. In one example, a raw data entity of the SMD 104 may correspond to a user, and the corresponding entity in the dataset may be a graph representing the user and friends, activities, locations, and the like associated with that user. In another example, a raw data entity of the IMD 102 may correspond to fMRI data and the corresponding entity in the dataset may be a graph representation of the fMRI data.
[0034] At step 204, at least one similarity matrix corresponding to the at least one entity is built. In one embodiment, the similarity matrix may be built based on at least one similarity function.
Some non-limiting examples of similarity functions or measures include correlation functions, mutual information, spatial distance functions, functions related to spectral properties, and the like. In certain embodiments, the similarity matrix may be built or constructed in accordance with equation (1):
??(??,??)= ?? (???? ,????) (1)
where s is the similarity matrix, s (i, j) is a value corresponding to the ith row and the jth column of the similarity matrix s, xi and xj, are feature vectors, and ?? is a similarity function of the feature vectors such as an inner product of xi and xj.
[0035] Further, at step 206, at least one adjacency matrix corresponding to the at least one entity is generated. In one embodiment, the adjacency matrix is generated based at least on the corresponding similarity matrix and at least one graph kernel function. Some non-limiting examples of graph kernel functions or conversion functions may include a Gaussian kernel, a shortest path kernel, a random walk kernel, and the like. In certain embodiments, the adjacency matrix may be built in accordance with equation (2):
?? (??,??)= ??-||????-????||22??2 (2)
where a is an adjacency matrix, a (i, j) is a value of the ith row and the jth column of the adjacency matrix a, ??-||????-????||22??2 is a Gaussian kernel function, si and sj are the ith and jth feature vectors of the similarity matrix s from equation (1), and s is a width of the Gaussian kernel function.
[0036] With continuing reference to FIG. 2, at step 207, an order advisory 208 is obtained. The order advisory 208 may be representative of the order advisory data 120 of FIG. 1. In one example, the order advisory 208 may be obtained via user input. In other examples, the order advisory 208 may be obtained from the data repository 130 or from a remote computer. The order advisory 208 is generally representative of a processing function which, when applied to the adjacency matrix, generates a re-ordered matrix where the rows and/or columns of the matrix are re-ordered or rearranged. In one embodiment, the order advisory 208 may include an optimization function of the row and column order configuration of the adjacency matrix a. In one example, the
optimization function may include an entropy function. Further, in other embodiments, the order advisory 208 may also include representations of domain knowledge of one or more of the arbitrary domains corresponding to the graph data 102-108 that may be applied to the adjacency matrix a. Moreover, in some non-limiting examples, the representations of domain knowledge may include a mapping lookup table or a transfer function.
[0037] Once the order advisory 208 is obtained, control is passed to step 210. At step 210, the adjacency matrix is re-ordered based on the order advisory 208 to generate an image grid 212. In this example, the image grid 212 is generally representative of a re-ordered adjacency matrix. The image grid 212 may be representative of the image grid 122 of FIG. 1. Control is then passed to step 214. It may be noted that in certain situations, the order advisory 208 may not be available. In this example, step 210 may be bypassed. More particularly, the adjacency matrix is not re-ordered. Accordingly, in these situations, the image grid 212 is generally representative of the adjacency matrix in its original form and control is passed to step 214.
[0038] It may be noted that the image grid 212 is generally representative of a representation learning corresponding to an entity of the dataset obtained from the graph data 102-108. Moreover, as previously noted, each entity of the dataset is a graph representation of a raw data entity on the arbitrary domain. It may further be noted that steps 202-210 of the method 200 may be iterated for each entity corresponding to the dataset to obtain one or more image grids 212, where the image grids 212 include re-ordered or original adjacency matrices corresponding to the one or more entities of the dataset.
[0039] In accordance with further aspects of the present specification, the representation learning in the form of image grids 212 may be used to evaluate performance of a CNN. Accordingly, in one example, the image grids 212 corresponding to the one or more entities of the dataset are divided into a training set and a test set, as indicated by step 214.
[0040] As previously noted, each of the one or entities associated with the dataset has a corresponding pre-determined score. Accordingly, each entity corresponding to the training set and the test set has a corresponding pre-determined score. The training set and the test set may be used to train and test the CNN running on the CNN subsystem 124 to evaluate the performance of the CNN.
[0041] Subsequently, at step 216, the CNN running on the CNN subsystem 124 is trained with one or more image grids and pre-determined scores corresponding to the training set. In one embodiment, during a training phase of the CNN, one or more image grids and pre-determined scores corresponding the training set are provided as inputs to the CNN. The CNN may be trained to associate one or more entities of the training set with a corresponding pre-determined score.
[0042] Furthermore, following the training of the CNN via use of the training set, the performance of the CNN is evaluated using the test set, as indicated by step 218. More particularly, during a testing phase of the CNN, one or more image grids corresponding to the test set are provided as input to the CNN. In one embodiment, the CNN is configured to generate as output one or more predicted scores corresponding to the one or more image grids corresponding to the test set.
[0043] Additionally, at step 220, the performance of the CNN is evaluated. In one embodiment, the performance of the CNN is assessed based on a comparison of the predicted scores corresponding to the one or more image grids of the test set generated at step 218 and the pre-determined scores corresponding to the one or more image grids of the test set. As previously noted if a percentage error of the predicted score with respect to the predetermined score corresponding to an image grid lies within a range of from about 0% to about 5%, the predicted score may be determined to be a close match to the pre-determined score. Accordingly, the close match between the predicted score and corresponding pre-determined score of the image grid may be generally indicative of an accuracy of the image grid generated by the method 200 in characterizing the graph data 102-108. In one embodiment, the percentage error of the predicted score with respect to the pre-determined score may be determined in accordance with equation (3):
|?????????????????????????? ??????????-?????????????????? ??????????|?????????????????????????? ?????????? × 100 =5 (3)
where Predetermined score is the pre-determined score corresponding to the image grid, and Predicted score is the predicted score generated as an output by the CNN of the CNN subsystem 124.
[0044] In certain embodiments, fMRI data may be advantageously analyzed when a representation learning of the fMRI data in graph form is used as input to a CNN. As will be
appreciated, fMRI is a wide-spread, non-invasive imaging modality used for studying brain activity. fMRI images are volumetric images, composed of voxels. A voxel represents a value on a regular grid in three-dimensional space. The position of a voxel is inferred based upon its position relative to other voxels (i.e., its position in the data structure that makes up a single volumetric image). fMRI is employed with objectives of localizing brain regions participating in a specific task, determining brain connectivity networks, making predictions about disease states, and the like. Current approaches to feature learning or representation learning and analysis of fMRI data include feature selection by predefined regions of interest (ROIs) through prior knowledge, statistical methods, thresholding, and the like. Disadvantageously, these approaches often overlook or fail to adequately deconstruct the 3D structural characteristics of the brain. Moreover, these analytic approaches fail to capture potentially complex interactions between voxels in the fMRI image. However, if the fMRI data is represented as a graph, these complex interactions can be advantageously modeled and subsequently analyzed by suitable learning paradigms such as regression/predictive models, CNNs and the like.
[0045] With the foregoing in mind, FIGs. 3A and 3B are diagrammatic representations of exemplary re-ordered adjacency matrices or image grids such as the image grids 122, 212 (see FIGs. 1-2) that are generated by the system 100 using the method 200. More particularly, FIG. 3A is representative of a re-ordered adjacency matrix 300 corresponding to a training set, in accordance with aspects of the present specification. Also, FIG. 3B is representative of a re-ordered adjacency matrix 310 corresponding to a test set, in accordance with aspects of the present specification
[0046] These exemplary image grids or re-ordered adjacency matrices 300, 310 may be used for a regression task performed by a CNN running on the CNN subsystem 124 (see FIG. 1) that predicts clinical scores for mild traumatic brain injury severity using adjacency matrices built from resting state fMRI data represented as a graph. In FIG. 3A, the matrix 300 is generally representative of an image grid or re-ordered adjacency matrix generated via use of a graph representation of fMRI data and using the method 200 of FIG. 2. Moreover, the matrix 300 is built as a training set or sample from fMRI data using 90 ROIs. A region of interest (ROI) may be defined as a demarcated region of the brain based on structural or functional features. Reference numeral 302 is generally representative of a row including the 90 ROIs. In a similar fashion,
reference numeral 304 is generally representative of a column including the 90 ROIs. Thus, the matrix 300 is a 90x90 matrix of ROIs of resting state fMRI data. Each cell of the matrix 300, denoted by ??(??,??) is generally representative of a correlation between the ROIs of the ith row and the jth column. Reference numeral 306 is generally indicative of a clustering of strongly correlated ROIs around a diagonal 308 of the matrix 300. As previously noted, in accordance with step 202 of the method 200, the matrix 300 is assigned an exemplary pre-determined clinical score of 31. Further, in accordance with step 216 of the method 200, the CNN running on the CNN subsystem 124 is trained with the matrix 300 of FIG. 3A and the corresponding pre-determined score of 31.
[0047] In a similar fashion, in FIG. 3B, the matrix 310 is generally representative of an image grid or re-ordered adjacency matrix generated via use of a graph representation of the fMRI data and using the method 200 of FIG. 2. The matrix 310 is built as a testing set or sample using 90 ROIs. In accordance with step 202 of method 200, the matrix 310 is assigned an exemplary pre-determined clinical score of 21. Reference numeral 312 is generally representative of a row including 90 ROIs. In a similar fashion, reference numeral 314 is generally representative of a column including the 90 ROIs. Thus, the matrix 310 is a 90x90 matrix of ROIs of resting state fMRI data. Each cell of the matrix 310, denoted by ??(??,??) is generally representative of a correlation between the regions of interest of the ith row and the jth column of the matrix 310. Reference numeral 316 is generally indicative of the clustering of strongly correlated ROIs around a diagonal of the matrix 310. Further, in accordance with step 218 of the method 200, the CNN is tested with the matrix 310 of FIG. 3B. It may be noted that during the testing phase, only the matrix 310 is provided as input to the CNN. However, the corresponding pre-determined score of the matrix 310 is not provided as an input to the CNN.
[0048] Subsequently, the CNN is tested with the matrix 310 of FIG. 3B. In this example, the CNN generated an output score of 20.85 in response to the matrix 310 provided as input. As previously noted, the pre-determined clinical score corresponding to matrix 310 is 21. Application of equation (3) to determine the percentage error of the predicted score of 20.85 with respect to the pre-determined clinical score of 21 corresponding to matrix 310 yields a percentage error of about 0.71% which lies within the range from about 0% to about 5%. Therefore, the predicted score of 20.85 generated by the CNN during the testing phase is a close match to the pre-determined clinical score of 21 corresponding to matrix 310 of FIG. 3B. The close match is a
general indication that the matrices 300 and 310 of FIGs. 3A and 3B are accurate and efficient representation learnings of graph representations of the fMRI data.
[0049] The system and method for representation learning of data on arbitrary domains for deep convolutional architectures presented hereinabove provide an approach that leverages the existing compute infrastructure of existing CNNs. More particularly, the systems and methods of the present specification generate re-ordered adjacency matrices/image grids. The CNNs thus directly operate on the adjacency matrix representations, without invoking spectral representations. Moreover, this representation learning of data enables a fast implementation of convolution through fast Fourier transforms. More particularly, the ordering of the adjacency matrices enables the CNNs to optimally exploit the neighborhood and localization properties to perform tasks such as classification and regression. Additionally, for data analysis of graph representations of data on arbitrary domains, similarity matrices advantageously consolidate the complexity of the attributes of entities/nodes and relationships between entities/nodes of data represented by the graph. Furthermore, the amenability of CNNs to multichannel inputs (for example, RGB images) may be advantageously leveraged for graph data. By way of example, multiple adjacency matrices may be built or constructed for the same graph based on different similarity scores or kernels. Additionally, the output of the CNN may also include images or output graphs, which may be further analyzed by different comparison techniques. Also, in this framework, other representations of graphs, such as line graph representations (where the roles of edges and nodes are interchanged), or any other latent representation of adjacency matrices may be used in addition to or alternative to the current adjacency matrix representation.
[0050] It is to be understood that not necessarily all such objects or advantages described above may be achieved in accordance with any particular embodiment. Thus, for example, those skilled in the art will recognize that the systems and techniques described herein may be embodied or carried out in a manner that achieves or improves one advantage or group of advantages as taught herein without necessarily achieving other objects or advantages as may be taught or suggested herein.
[0051] While the technology has been described in detail in connection with only a limited number of embodiments, it should be readily understood that the specification is not limited to
such disclosed embodiments. Rather, the technology can be modified to incorporate any number of variations, alterations, substitutions or equivalent arrangements not heretofore described, but which are commensurate with the spirit and scope of the claims. Additionally, while various embodiments of the technology have been described, it is to be understood that aspects of the specification may include only some of the described embodiments. Accordingly, the specification is not to be limited by the foregoing description, but is only limited by the scope of the appended claims.