Description:FORM 2

THE PATENTS ACT, 1970
(39 of 1970)
&
THE PATENT RULES, 2003

COMPLETE SPECIFICATION
(See Section 10 and Rule 13)

Title of invention:

SYSTEMS AND METHODS FOR MATERIAL MICROSTRUCTURE DESIGNING USING VARIATIONAL AUTOENCODER-REGRESSION MODEL WITH A MULTI-MODAL PRIOR

Applicant

Tata Consultancy Services Limited
A company Incorporated in India under the Companies Act, 1956
Having address:
Nirmal Building, 9th floor,
Nariman point, Mumbai 400021,
Maharashtra, India

Preamble to the description:
The following specification particularly describes the invention and the manner in which it is to be performed.
TECHNICAL FIELD
The disclosure herein generally relates to the field of computational materials engineering and materials informatics, and, more particularly, to systems and methods for material microstructure designing using variational autoencoder-regression model with a multi-modal prior.

BACKGROUND
Material engineering is a field focusing on studying inherent properties of matter and designing new materials out of known materials by way of various technologies. Processes such as heating, tempering, and/or the like modify internal structure of a material which in turn alters the properties of the material such as tensile strength, ductility and/or the like. Understanding and modeling the relationships between processing, structure and properties (also known as the P-S-P linkages) is at the core of computational materials science. Materials scientists and engineers are often interested in inverse analysis that is, predicting the structure for target properties, and the processing route required to get the structure. This involves systematically exploring a huge design space consisting of initial compositions, parameters of all processes and the resulting structures. Also, the problem is often ill-posed. That is, more than one solution is possible, since multiple processing routes and structures can lead to the same properties. Traditionally, a combination of experimentation and physics-based numerical simulations are used for this purpose. However, experimental exploration has limitations because of the time and cost involved. Further, physics-based models, which are based on solving underlying differential equations, are only useful for predicting forward path which means predicting structure from composition and processing.
Conventional fast and accurate machine learning models of forward prediction are being seen as alternative to physics-based simulations and approximate response surfaces. However, these models are not capable of inverse inference themselves and still have to be used inside an optimization loop. Although, conventional data based models are faster than physics-based models, yet the design space is often so huge that optimization around these faster models also turns out to be infeasible. State-of-the-art deep generative models are also attempted to be used for inverse inference, but they have not been explored for the purpose of direct inverse inference.
SUMMARY
Embodiments of the present disclosure present technological improvements as solutions to one or more of the above-mentioned technical problems recognized by the inventors in conventional systems. For example, in one embodiment, a processor implemented method is provided. The processor implemented method, comprising receiving, via one or more hardware processors, (i) a plurality of microstructure images of a material under consideration, (ii) a plurality of properties associated with the plurality of microstructure images of the material under consideration, (iii) a neural network architecture, and (iv) a pre-determined loss function associated with the neural network architecture, wherein the neural network architecture represents a variational autoencoder model combined with a regression model through a multi-modal mixture-of-Gaussians prior conditioned on the plurality of properties associated with the plurality of microstructure images; and iteratively training, via the one or more hardware processors, the neural network architecture using the plurality of microstructure images and the plurality of properties associated with the plurality of microstructure images using a backpropagation algorithm, wherein the steps performed for training the neural network architecture at each of a plurality of iterations, comprises: obtaining, a plurality of latent representations of (i) the plurality of microstructure images and (ii) the plurality of properties associated with the plurality of microstructure images from trained neural network architecture; sampling, a latent representation from the plurality of latent representations, for a target property from among the plurality of properties associated with the plurality of microstructure images, based on the multi-modal mixture-of-Gaussians prior conditioned on the plurality of properties associated with the plurality of microstructure images; reconstructing, a set of microstructure images corresponding to the target property, from the plurality of microstructure images, using a decoder network of the trained neural network architecture; and computing, the pre-determined loss function for the trained neural network architecture, wherein the computed pre-defined loss function comprises a Kullback-Leibler (KL) loss function, a style loss function and a regression loss function.
In another aspect, a system is provided. The system comprising a memory storing instructions; one or more communication interfaces; and one or more hardware processors coupled to the memory via the one or more communication interfaces, wherein the one or more hardware processors are configured by the instructions to: receive, (i) a plurality of microstructure images of a material under consideration, (ii) a plurality of properties associated with the plurality of microstructure images of the material under consideration, (iii) a neural network architecture and (iv) a pre-determined loss function associated with the neural network architecture, wherein the neural network architecture represents a variational autoencoder model combined with a regression model through a multi-modal mixture-of-Gaussians prior conditioned on the plurality of properties associated with the plurality of microstructure images; and iteratively train, the neural network architecture using the plurality of microstructure images and the plurality of properties associated with the plurality of microstructure images using a backpropagation algorithm, wherein the steps performed for training the neural network architecture at each iteration, comprises: obtaining, a plurality of latent representations of (i) the plurality of microstructure images and (ii) the plurality of properties associated with the plurality of microstructure images from trained neural network architecture; sampling, a latent representation for a target property from the plurality of latent representations of the plurality of properties associated with the plurality of microstructure images based on the multi-modal mixture-of-Gaussians prior conditioned on the plurality of properties associated with the plurality of microstructure images; reconstructing, a set of microstructure images corresponding to the target property from the plurality of microstructure images using a decoder network of the trained neural network architecture; and computing, the pre-determined loss function for the trained neural network architecture, wherein the computed pre-defined loss function comprises a Kullback-Leibler (KL) loss function, a style loss function and a regression loss function.
In yet another aspect, a non-transitory computer readable medium is provided. The non-transitory computer readable medium are configured by instructions for receiving, (i) a plurality of microstructure images of a material under consideration, (ii) a plurality of properties associated with the plurality of microstructure images of the material under consideration, (iii) a neural network architecture, and (iv) a pre-determined loss function associated with the neural network architecture, wherein the neural network architecture represents a variational autoencoder model combined with a regression model through a multi-modal mixture-of-Gaussians prior conditioned on the plurality of properties associated with the plurality of microstructure images; and iteratively training, the neural network architecture using the plurality of microstructure images and the plurality of properties associated with the plurality of microstructure images using a backpropagation algorithm, wherein the steps performed for training the neural network architecture at each of a plurality of iterations, comprises: obtaining, a plurality of latent representations of (i) the plurality of microstructure images and (ii) the plurality of properties associated with the plurality of microstructure images from trained neural network architecture; sampling, a latent representation from the plurality of latent representations, for a target property from among the plurality of properties associated with the plurality of microstructure images, based on the multi-modal mixture-of-Gaussians prior conditioned on the plurality of properties associated with the plurality of microstructure images; reconstructing, a set of microstructure images corresponding to the target property, from the plurality of microstructure images, using a decoder network of the trained neural network architecture; and computing, the pre-determined loss function for the trained neural network architecture, wherein the computed pre-defined loss function comprises a Kullback-Leibler (KL) loss function, a style loss function and a regression loss function.
In accordance with an embodiment of the present disclosure, the processor implemented method further comprising receiving a set of incoming target properties associated with the plurality of microstructure images of the material under consideration; and obtaining, a plurality of reconstructed microstructure images corresponding to the set of incoming target properties for the plurality of incoming microstructure images of the material under consideration, using the trained neural network architecture.
In accordance with an embodiment of the present disclosure, the neural network architecture comprises a variational autoencoder model with an encoder network and the decoder network, the regression model, and a generator network.
In accordance with an embodiment of the present disclosure, the encoder network of the variational autoencoder comprises three blocks of convolution neural network including a convolution layer followed by batch normalization and leaky ReLU non-linearity with a slope of 0.3, and two blocks of two fully connected parallel output layers that predict mean and diagonal co-variance of posterior distribution of the plurality of microstructure images.
In accordance with an embodiment of the present disclosure, the decoder network of the variation autoencoder comprises three blocks of convolution neural network including a transposed convolution layer followed by batch normalization and leaky ReLU non-linearity with a slope of 0.3, and two blocks of two fully connected parallel output layers.
In accordance with an embodiment of the present disclosure, the generator network of the neural network architecture comprises an input layer configured to receive at least one property from the plurality of properties associated with the plurality of microstructure images as input, a fully connected hidden layer with tanh activation, and two parallel linear output layers that predict means and standard deviations of each component of the multi-modal mixture-of-Gaussians conditional prior.
In accordance with an embodiment of the present disclosure, the regression model of the neural network architecture comprises three blocks of convolution neural network shared by the encoding network, and two blocks of two fully connected parallel output layers that predict the mean and standard deviation of the plurality of properties associated with the plurality of microstructure images.
In accordance with an embodiment of the present disclosure, the style loss function determines a style loss between each microstructure image from the plurality of microstructure images and a corresponding reconstructed microstructure image using a style loss network.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS
The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate exemplary embodiments and, together with the description, serve to explain the disclosed principles:
FIG. 1 is a functional block diagram of the system for material microstructure designing using variational autoencoder-regression model with a multi-modal prior according to some embodiments of the present disclosure.
FIG. 2 illustrates an exemplary flow diagram illustrating a method for material microstructure designing using variational autoencoder-regression model with a multi-modal prior in accordance with some embodiments of the present disclosure.
FIG. 3 illustrates a block diagram of neural network architecture along with a loss function unit for material microstructure designing according to some embodiments of the present disclosure.
FIGS. 4A through 4D show some example microstructures obtained using different Gaussian filters for material microstructure designing using variational autoencoder-regression model with a multi-modal prior according to some embodiments of the present disclosure.
FIG. 5A though 5L through shows example reconstructions of test set microstructures obtained from the trained VAE model with different morphological features according to some embodiments of the present disclosure.
FIGS. 6A and 6B illustrates two microstructures that achieve the effective elastic stiffness property (C_(11,eff) ) ?25Gpa corresponding to two morphologies for material microstructure designing using variational autoencoder-regression model with a multi-modal prior according to some embodiments of the present disclosure.
FIG. 7 shows the effective elastic stiffness property (C_(11,eff) ) plotted against volume fraction of black phase in all data corresponding to two morphologies for material microstructure designing using variational autoencoder-regression model with a multi-modal prior according to some embodiments of the present disclosure.
FIGS. 8A and 8B provides a comparison of uni-modal and multi-modal prior in terms of inferred microstructures for a specific target effective elastic stiffness property under a corresponding morphology according to some embodiments of the present disclosure.
FIGS. 9A and 9B show plots of a specific target effective elastic stiffness property against volume fraction of black phase in inverse inferred microstructures using the uni-modal and the multi-modal prior respectively according to some embodiments of the present disclosure.
FIG. 10A and 10B show results of inverse inference using data from two different morphologies according to some embodiments of the present disclosure according to some embodiments of the present disclosure.
FIGS. 11A and 11B show a plot of the set of target properties values against volume fraction of black phase in the inverse inferred microstructures using the two priors according to some embodiments of the present disclosure.

DETAILED DESCRIPTION OF EMBODIMENTS
Exemplary embodiments are described with reference to the accompanying drawings. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. Wherever convenient, the same reference numbers are used throughout the drawings to refer to the same or like parts. While examples and features of disclosed principles are described herein, modifications, adaptations, and other implementations are possible without departing from the scope of the disclosed embodiments. It is intended that the following detailed description be considered as exemplary only, with the true scope being indicated by the following embodiments described herein.
Materials science and engineering involves the study of different materials, their processing and the resulting properties that govern the performance of the material in operation. Processes such as heating, tempering, and/or the like modify internal structure of the material which in turn alters material’s properties such as tensile strength, ductility and the like. Understanding and modelling relationships between processing, structure, and properties is at the core of computational materials engineering. Materials scientists and engineers are often interested in inverse analysis. Inverse analysis refers to predicting the structure for target properties, and processing route required to get the structure - With the advent of such models, one can set up manufacturing processes to achieve target properties and performance in the final product. This involves systematically exploring a huge design space consisting of initial compositions, parameters of all processes and the resulting structures. Also, the problem is often ill-posed. That is, more than one solution is possible, since multiple processing routes and structures can lead to the same properties
Typically, physics-based numerical simulations are used to infer the structure and processing required to achieve target properties. However, physics-based models, which are based on solving underlying differential equations, are only useful for predicting the forward path. The prediction of forward path refers to predicting structure from composition and processing conditions, and properties from structure. They cannot be used directly for the inverse problem. So, physics-based models are typically used inside an optimization loop to solve the design problem. However, in many cases the precise governing equations are not known, and so physics-based models are not available. Even when available, for a lot of applications, they are computationally too expensive to be useful for design space exploration. Due to recent advancements in machine learning, state of-the art deep learning models ae used for learning P-S-P linkages as well as various microstructure informatics tasks. For example, deep convolutional neural networks are used for structure-property linkages and deep learning methods are used for microstructure segmentation and classification. However, all of these works focus on the forward prediction. In a recent method, a variational autoencoder based model is used for generating synthetic microstructure images and these synthetic microstructure images are further used for improving downstream property prediction model using a separate CNN model. However, this method uses the variational autoencoder based model only for generating new images unconditionally and in isolation from regression.
There exist several approaches using deep generative modelling for inverse inference. For example, a method for inverse inference using Generative Adversarial Nets (GANs) is proposed. However, this method uses GAN for only generating new candidate solutions, which are then evaluated using a Gaussian Process regression model. They finally use optimization around this setup for inverse inference. The advantage is that the latent space of GANs can be used to generate subsequent candidate solutions, rather than operating in the intractable space of structures. However, the method of the present disclosure performs this optimization only once in an amortized manner. It learns a conditional prior which can then be used to directly get latent vectors (candidate solutions) for a target property. Further, few works have discussed the limitations of VAE due to a restrictive simple Gaussian prior and have motivated a stronger prior. These works are mainly aimed at improving the quality of generations using VAE. The most important one is the variational aggregate mixture of posteriors (VAMP) prior which is based on the idea of using an aggregate posterior (i.e., a mixture by definition) itself as a prior. However, in the present disclosure, a multi-modal prior is used to model many to-one relationship from structure to properties. So, a simple mixture-of-Gaussians prior is used with number of mixture components being a hyperparameter. However, multimodal prior based formulation is also used in an existing approach which results in a two-level prior. A model used in this existing approach is shown to be more useful for multimodal data generation. However, they assume that extra conditioning variable is discrete which leads to a mixture-of Gaussians prior, and that it is unobserved. It is expected to model different modalities of the data. In the method of present disclosure, the extra conditioning variable is continuous, observed and the number of mixture components is treated as a hyperparameter.
In an embodiment, most common data available about the structure is in the form of microscopic images, known as the microstructure. Microstructure images contain a lot of information about the micro-constituents (also known as phases), their geometry (shape, size etc.) and their orientations. These features are known to have a profound impact on the material properties. However, depending upon the material system and property under consideration, very different sets of features are relevant. Therefore, methods to obtain microstructure representations that encode all and only the relevant information (and discard all noise) are desirable.
Embodiments of the present disclosure provide systems and methods for material microstructure designing using variational autoencoder-regression model with a multi-modal prior. The present disclosure addresses unresolved problems of the conventional methods by building a fast and reasonably accurate model of the structure-property linkage supporting efficient inverse inference. The method of the present disclosure proposes a probabilistic generative model of structure-property linkage which can be directly used for inverse inference (i.e., inferring structures required to achieve target properties). The present disclosure proposes a neural network architecture that combines a variational autoencoder (VAE) with regression and derive a joint loss function for training the neural network architecture. While deriving the loss function, a Gaussian prior is replaced with a mixture-of-Gaussians prior. The VAE and the regression model are joined through the prior, which is conditioned on the property. After training, latent representations specific for a desired target property value are sampled from the conditional prior and passed through the decoder network to get microstructure(s) having the target property value. Since the prior is a mixture-of-Gaussians distributions, it has multiple modes and thus, for a target property value, more than one microstructure can be achieved as solutions.
In other words, the method of the present disclosure combines a variational autoencoder (VAE) with a regression model such that the resulting model learns microstructure features salient for reconstruction, and accuracy of property-prediction. The VAE and the regression model are joined through the VAE prior which is conditioned on the property. Latent representations for a target property value can be sampled from the conditional prior and decoded to get microstructures with that property value. Finally, Gaussian prior are replaced with a mixture-of-Gaussians so that multiple candidate microstructures for a target property value can be obtained, addressing the ill-posedness of inverse inference. The present disclosure shows that on a dataset of 3D synthetic microstructures, the resulting model can infer multiple solutions for a target property value. Finally, the resulting model can also be trained in a semi supervised setting by a straight-forward extension, replacing regression loss with an entropy term. Hence, the resulting model can be effectively used in situations where there are many unlabeled and a few labeled microstructures, which is a common case because of the high cost and time required for property-testing. More Specifically, the present disclosure describes the following:
A method for forward prediction of properties from structure, with quantification of standard deviation in the prediction.
A method for inverse prediction of structures for a target property.
Ability to handle ill-posedness of inverse problem by providing multiple solutions.
Combining advantages of unsupervised and supervised representation learning to learn salient features of microstructures for property prediction as well as reconstruction.
Referring now to the drawings, and more particularly to FIGS. 1 through 11B, where similar reference characters denote corresponding features consistently throughout the figures, there are shown preferred embodiments and these embodiments are described in the context of the following exemplary system and/or method.
FIG. 1 is a functional block diagram of a system 100 for material microstructure designing using variational autoencoder-regression model with a multi-modal prior according to some embodiments of the present disclosure. In an embodiment, the system 100 includes or is otherwise in communication with one or more hardware processors 104, communication interface device(s) or input/output (I/O) interface(s) 106, and one or more data storage devices or memory 102 operatively coupled to the one or more hardware processors 104. The one or more hardware processors 104, the memory 102, and the I/O interface(s) 106 may be coupled to a system bus 108 or a similar mechanism.
The I/O interface(s) 106 may include a variety of software and hardware interfaces, for example, a web interface, a graphical user interface, and the like. The I/O interface(s) 106 may include a variety of software and hardware interfaces, for example, interfaces for peripheral device(s), such as a keyboard, a mouse, an external memory, a plurality of sensor devices, a printer and the like. Further, the I/O interface(s) 106 may enable the system 100 to communicate with other devices, such as web servers and external databases.
The I/O interface(s) 106 can facilitate multiple communications within a wide variety of networks and protocol types, including wired networks, for example, local area network (LAN), cable, etc., and wireless networks, such as Wireless LAN (WLAN), cellular, or satellite. For the purpose, the I/O interface(s) 106 may include one or more ports for connecting a number of computing systems with one another or to another server computer. Further, the I/O interface(s) 106 may include one or more ports for connecting a number of devices to one another or to another server.
The one or more hardware processors 104 may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuitries, and/or any devices that manipulate signals based on operational instructions. Among other capabilities, the one or more hardware processors 104 are configured to fetch and execute computer-readable instructions stored in the memory 102. In the context of the present disclosure, the expressions ‘processors’ and ‘hardware processors’ may be used interchangeably. In an embodiment, the system 100 can be implemented in a variety of computing systems, such as laptop computers, portable computer, notebooks, hand-held devices, workstations, mainframe computers, servers, a network cloud and the like.
The memory 102 may include any computer-readable medium known in the art including, for example, volatile memory, such as static random access memory (SRAM) and dynamic random access memory (DRAM), and/or non-volatile memory, such as read only memory (ROM), erasable programmable ROM, flash memories, hard disks, optical disks, and magnetic tapes. In an embodiment, the memory 102 includes a plurality of modules 102a and a repository 102b for storing data processed, received, and generated by one or more of the plurality of modules 102a. The plurality of modules 102a may include routines, programs, objects, components, data structures, and so on, which perform particular tasks or implement particular abstract data types.
The plurality of modules 102a may include programs or computer-readable instructions or coded instructions that supplement applications or functions performed by the system 100. The plurality of modules 102a may also be used as, signal processor(s), state machine(s), logic circuitries, and/or any other device or component that manipulates signals based on operational instructions. Further, the plurality of modules 102a can be used by hardware, by computer-readable instructions executed by the one or more hardware processors 104, or by a combination thereof. In an embodiment, the plurality of modules 102a can include various sub-modules (not shown in FIG.1). Further, the memory 102 may include information pertaining to input(s)/output(s) of each step performed by the processor(s) 104 of the system 100 and methods of the present disclosure.
The repository 102b may include a database or a data engine. Further, the repository 102b amongst other things, may serve as a database or includes a plurality of databases for storing the data that is processed, received, or generated as a result of the execution of the plurality of modules 102a. Although the repository 102b is shown internal to the system 100, it will be noted that, in alternate embodiments, the repository 102b can also be implemented external to the system 100, where the repository 102b may be stored within an external database (not shown in FIG. 1) communicatively coupled to the system 100. The data contained within such external database may be periodically updated. For example, new data may be added into the external database and/or existing data may be modified and/or non-useful data may be deleted from the external database. In one example, the data may be stored in an external system, such as a Lightweight Directory Access Protocol (LDAP) directory and a Relational Database Management System (RDBMS). In another embodiment, the data stored in the repository 102b may be distributed between the system 100 and the external database.
FIG. 2, with reference to FIG. 1, illustrates an exemplary flow diagram illustrating a method for material microstructure designing using variational autoencoder-regression model with a multi-modal prior, using the system 100 of FIG. 1 in accordance with some embodiments of the present disclosure.
Referring to FIG. 2, in an embodiment, the system(s) 100 comprises one or more data storage devices or the memory 102 operatively coupled to the one or more hardware processors 104 and is configured to store instructions for execution of steps of the method by the one or more processors 104. The steps of the method 200 of the present disclosure will now be explained with reference to components of the system 100 of FIG. 1, the flow diagram as depicted in FIG. 2, the block diagram of FIG. 3, and one or more examples. Although steps of the method 200 including process steps, method steps, techniques or the like may be described in a sequential order, such processes, methods and techniques may be configured to work in alternate orders. In other words, any sequence or order of steps that may be described does not necessarily indicate a requirement that the steps be performed in that order. The steps of processes described herein may be performed in any practical order. Further, some steps may be performed simultaneously, or some steps may be performed alone or independently
In an embodiment, at step 202 of the present disclosure, the one or more hardware processors 104 are configured to (i) a plurality of microstructure images of a material under consideration, (ii) a plurality of properties associated with the plurality of microstructure images of the material under consideration, (iii) a neural network architecture, and (iv) a pre-determined loss function associated with the neural network architecture. The material under consideration may be a process material such as iron, whose microstructure images are required to be designed such that features that are salient for reconstructing the original input can be learnt and properties or quantities of interest are predicted.
In the context of the present disclosure, the terms ‘microstructure’ and ‘microstructure image’ may be interchangeably used, however, they refer to the microscopic images of the material under consideration. Each microstructure image of the plurality of microstructure images is of a predefined image size. The predefined image size represents an image resolution (number of pixels) of the microstructure image. In an embodiment, the predefined image size of each microstructure image of the plurality of microstructure images may be same. In another embodiment, the predefined image size of each microstructure image of the plurality of microstructure images may be different. Further some microstructure images of the plurality of microstructure images may be of same predefined image size, while the other microstructure images may be of different predefined image size. In an embodiment, the plurality of properties associated with the plurality of microstructure images of the material under consideration may include tensile strength, ductility, effective elastic stiffness, and /or the like. The plurality of properties are subjected to change when the internal structure of the material changes due to processes such as heating, tempering, and/or the like.
In an embodiment, the neural network architecture represents a variational autoencoder model combined with a regression model through a multi-modal mixture-of-Gaussians prior conditioned on the plurality of properties associated with the plurality of microstructure images. FIG. 3 illustrates a block diagram of neural network architecture along with a style loss function unit for material microstructure designing according to some embodiments of the present disclosure. As shown in FIG. 3, the neural network architecture comprises a variational autoencoder model with an encoder network and the decoder network, the regression model, and a generator network. In the context of the present disclosure, the expressions ‘regression model’ and ‘regressor’ and ‘regressor network’ may be interchangeably used. Also, the expressions ‘mixture-of-Gaussians’ and ‘Gaussian mixture’ may be interchangeably used in the context of present disclosure. The encoder network of the variational autoencoder comprises three blocks of convolution neural network including a convolution layer followed by batch normalization and leaky ReLU non-linearity with a slope of 0.3, and two blocks of two fully connected parallel output layers that predict mean and diagonal co-variance of posterior distribution of the plurality of microstructure images. The decoder network of the variation autoencoder is a mirror image of the encoder network and comprises three blocks of convolution neural network including a transposed convolution layer followed by batch normalization and leaky ReLU non-linearity with a slope of 0.3, and two blocks of two fully connected parallel output layers. The regression model of the neural network architecture comprises three blocks of convolution neural network shared by the encoding network, and two blocks of two fully connected parallel output layers that predict the mean and standard deviation of the plurality of properties associated with the plurality of microstructure images. The generator network of the neural network architecture comprises an input layer configured to receive at least one property from the plurality of properties associated with the plurality of microstructure images as input, a fully connected hidden layer with tanh activation, and two parallel linear output layers that predict means and standard deviations of each component of the multi-modal mixture-of-Gaussians conditional prior.
In an embodiment, at step 204 of the present disclosure, the one or more hardware processors 204 are configured to iteratively train the neural network architecture using the plurality of microstructure images and the plurality of properties associated with the plurality of microstructure images using a backpropagation algorithm. In an embodiment, complete data is split into 80% training, 20% validation and 20% test data. Validation data is used only to decide when to stop training. Training is stopped when a validation loss in the validation data does not decrease for 10 consecutive epochs. The validation loss is a configurable hyperparameter of and best learned value of the validation loss achieved by the neural network architecture was 10 consecutive epochs. In an embodiment, for training, at each iteration, first a mini batch is formed using (i) one or more microstructure images out of the plurality of microstructure images and (ii) one or more properties out of the plurality of properties associated with the plurality of microstructure images of the material under consideration, based on a predefined mini batch size, to obtain one or more mini batches from the plurality of microstructure images and the plurality of properties. The predefined mini batch size defines the number of the microstructure images and the number of properties to be present in each mini batch. If the predefined mini batch size is 128, then each mini batch includes 128 microstructure images and 128 properties. Each mini batch includes a plurality of unique microstructure images and unique properties i.e., each microstructure image and each property are present in only one mini batch. Further, the one or more microstructure images and the one or more properties present in each mini batch, at a time, out of the one or more mini batches, are passed to the variational autoencoder and regression model respectively for the training. In an embodiment, each iteration refers to each mini batch, at a time, until the one or more mini batches are completed. In an embodiment, the steps performed for training the neural network architecture at each iteration, comprises first obtaining, a plurality of latent representations of (i) the plurality of microstructure images and (ii) the plurality of properties associated with the plurality of microstructure images from the trained neural network architecture. Further, a latent representation for a target property is sampled from the plurality of latent representations of the plurality of properties associated with the plurality of microstructure images based on the multi-modal mixture-of-Gaussians prior conditioned on the plurality of properties associated with the plurality of microstructure images. In case of the multi-modal mixture-of-Gaussians prior, first a mixture component is sampled using mixture weights, followed by sampling from the corresponding Gaussian. Furthermore, a set of microstructure images corresponding to the target property from the plurality of microstructure images are reconstructed using a decoder network of the trained neural network architecture. As shown in FIG. 3, a plurality of microstructure images are provided as an input to the variational autoencoder model and the plurality of microstructure image with a target property are provided as an input to the regression model of the neural network architecture. The variational autoencoder model provides a latent representation of the plurality of input microstructure images and the regression model predicts the target property for the input microstructure image. The latent representation of the predicted target property is obtained using the generator network. The latent representation of the plurality of input microstructure images and the latent representation of the predicted target property are fed to the decoder network of the trained neural network architecture to reconstruct those microstructure images from the plurality of input microstructure images that exhibit the predicted target property. Upon reconstruction, the pre-determined loss function for the trained neural network architecture is computed to determine how close the reconstructed images are to the original input image. The computed pre-defined loss function comprises a Kullback-Leibler (KL) loss function, a style loss function and a regression loss function. The style loss function determines a style loss between each microstructure image from the plurality of microstructure images and a corresponding reconstructed microstructure image using the style loss network.
The steps 202 and 204 are better understood by way of the following description provided as exemplary explanation.
Variational autoencoders are commonly used for unsupervised representation learning. The problem of representation learning is posed as a probabilistic inference problem with the underlying generative model p_? (x,z) =p_? (x|z)p_? (z), where x is input and z is a latent representation which is to be inferred. Variational autoencoders perform variational inference which means they find an approximate posterior q_f (z|x) from a family of parameterized distributions by minimizing KL-Divergence from true posterior. Minimizing the KL-Divergence is shown to be equivalent to maximizing another derived quantity known as Evidence Lower Bound (ELBO), which can be optimized efficiently as shown in equation (1) below:
L = -D_KL (q_f (z|x)?p_? (z)¦) + E_(q_f (z|x)) [log p_? (x|z)] (1)
The distributions q_f (z|x) and p_? (x|z) are parameterized by neural networks, which can be seen as “encoder” and “decoder”, respectively. First term of the ELBO provided in equation (1) acts like a regularizer, penalizing posterior distributions which are very different from prior p(z) while second term quantifies how well an input x is reconstructed under current encoder and decoder distributions.
Variational autoencoders (VAE) have been used successfully for unsupervised learning of high-quality latent representations in a wide variety of applications ranging from medical imaging to program synthesis. However, use of VAE in semi-supervised or supervised regression settings (e.g., predicting a scalar or vector of real numbers from an image) has been rarely explored. In the present disclosure, the input image is a material microstructure and real numbers to be predicted from the input image are mechanical properties of the material. Although variational autoencoders have been used in such problems as well, the usage has been mostly limited to scenarios where latent representations obtained from the VAE are used as input to a downstream regression model such as SVR, logistic regression and/or the like. In state of the art works, the two models (i.e., VAE and the regression model) are learned in isolation, with no shared feature learning. Instead, if the two models are systematically combined and trained end-to-end, then a joint single model can learn features that are salient for reconstructing the original input as well as for predicting the quantities of interest. This means advantages of unsupervised and supervised representation learning can be combined. In an existing approach, a ‘VAE for regression’ model was proposed in and used for predicting age of a subject from their structural Magnetic Resonance (MR) images. They start by assuming that the latent representation z of an MR image x is dependent on a quantity c to be predicted. Here, the quantity c is assumed to be subject’s age. This changes the generative model as: p(x,z,c) = p(c)p(z|c)p(x|z), leading to a two-level prior. Here, p(c) is the prior on age whereas p(z|c) is the prior on latent representation, conditioned on age. Variational inference amounts to finding an approximate posterior q(z,c|x) that minimizes KL-Divergence from the true posterior p(z,c|x). Assuming that z and c are marginally independent for a given x (the mean-field assumption), the ELBO derived in this case is provided in equation (2) below as:
L = -D_KL (q(c+|x¦)?p(c)¦)+E_(q(z¦x) ) [log p(x¦z)]-E_(q(c¦x) ) [D_KL (q(z|x)?p(z|c)¦)] ------ (2)
It is assumed that in a supervised setting, the first term of the equation (1) can be replaced by log q(c|x) since c values are known. Also, it is assumed that q(c|x) is a univariate Gaussian whose parameters are output by a regressor model. Thus, the first term of the derived ELBO in equation (2) is a regression loss (under Gaussian assumption, log q(c|x) is proportional to mean squared error). The second term is a reconstruction loss, which is same as in ELBO from equation (1). The last term of equation (2) is a counterpart of the regularizer from equation (1), except, the prior is now conditioned on c. Another difference is that the KL divergence is now in expectation. This is the term that connects the regressor and the VAE. The expectation is with respect to the regressor distribution q(c|x). Thus, for a given x, the encoder posterior q(z|x) and the conditional prior p(z|c) obtained from the c predicted by the regressor model were encouraged to be similar. It was shown that i) the VAE and regressor model regularize each other and hence the predictions from the joint model were more accurate than an isolated regression model and ii) the joint model allowed synthesizing age-specific brains, that facilitated the identification of brain-aging pattern.
In the present discourse, the ‘VAE for regression’ model is used for linking microstructures with properties. It is observed that the reconstruction loss from vanilla VAE objective function typically leads to pixel by-pixel comparison between input and reconstruction, which is not suitable for microstructure images. Microstructure images are repeating patterns with certain distributions. In computer vision community, these types of images are also known as textures. Two textures are considered the same as long as they contain the same pattern, even if pixel-wise there is a difference (e.g., a shift of the pattern). Hence, only a comparison between statistics/spatial correlations rather than pixel-by pixel comparison is suited to quantify the difference between two textures. Even in materials science literature, it has been observed that the most important information to quantify the microstructures is the n^th-order statistics or spatial correlations. Thus, the pixel-wise reconstruction loss is replaced with a style loss which captures desired statistics/spatial correlations in the microstructure. The style loss is based on such comparison between spatial correlations. It is the sum of squared differences between Gram matrices of input and reconstruction as computed from various layers of a deep pre-trained network such as Visual Geometry Group 19 (VGG19). In the present disclosure, the dataset used contains 3D microstructures, so the style loss is required to be computed between two 3D inputs. However, the pre-trained VGG19 network takes 2D images as input. One way to compute the style loss directly on 3D inputs is to use a 3D version of VGG19. To achieve this, weights of the 2D version can be replicated in a third dimension to initialize the 3D version. However, such an extension may not capture the true 3D statistics of the inputs. However, in the present disclosure, original pre-trained VGG19 network is used to compute style loss between 2D slices taken from 3D inputs. Slices are taken from all three directions to make sure the 3D statistics are captured. Input size in the data set used in the present disclosure is 51×51×51. Slices of size 51×51 are taken from each direction (X, Y and Z). So, 51*3 = 153 slices are taken from each input, effectively multiplying mini batch size by that factor.
The Gram matrix of an image at layer l is the matrix of correlations among features of the images at layer l. If layer l has C_l feature maps of size W_l× H_l then, the gram matrix and the style loss are computed as provided in equation (3) and (4) respectively as:
G_ij^l=?_k¦F_ik^l F_jk^l (3)
L_style (x,x ^ )=?_(l=0)^L¦?w_l [1/(4C_l^2 M_l^2 ) ?_(i,j)¦(G_ij^l-G ^_(i,j)^l )^2 ] ? (4)
Here, M_l = W_l* H_l and F l is the C_l × M_l matrix. In the present disclosure, the original reconstruction loss from the ELBO as provided in equation (2) is replaced with the style loss. Further, structure to property is a many-to-one relationship, so the conditional prior on property needs to be multi-modal. So instead of a standard, uni-modal Gaussian prior, a mixture-of-Gaussians prior is used and a new loss function is derived. In an embodiment, the derived new loss function is referred as the pre-determined loss function associated with the with the neural network architecture. Further, the ‘VAE for regression’ model can be used in a semi-supervised setting too. This means that many unlabeled and a few labeled microstructures (which is often the case due to high time and cost of property-testing as compared to only imaging) can be utilized to learn better representations, addressing data sparsity to some extent. The first term from equation (2) is -D_KL (q(c+|x¦)?p(c)¦). It is known that when c is completely observed, this term can be replaced with log q(c|x). However, in semi-supervised settings, it can be replaced with H(q) = ?¦?q(c|x) log q(c|x)?. Here, H(q) represents entropy of q(c|x) for the unlabeled inputs. So, for unlabeled cases, the first term in equation (2) is replaced with its expectation under the distribution learned so far using labeled cases.
In an embodiment, the conditional prior in the VAE-regression model links the latent representations with the property. Hence, after training, the latent representations can be used for inverse inference, where latent representations specific to the target/desired property value is sampled from the conditional prior p(z|c) and decoded to get the required microstructures to achieve the target/desired property. However, a major challenge in direct inverse inference is that the problem is often ill-posed. This means that multiple structures can lead to same properties, so there could be many solutions to the inverse problem. So, for a model that is conditioned on a property value, there may be more than one likely latent representations. The standard Gaussian prior is a uni-modal distribution and is not suitable for this purpose. Hence a mixture-of-Gaussians prior is used in the present disclosure. The mixture-of-Gaussians prior with K components is provided in equation (5) below:
p(z|c) ~ ?_(k=1)^K¦?p_k N_k (µ_k,? s?_k^2)? --- (5)
Here, p represents probabilities of components. A diagonal co-variance matrix for all mixture components is assumed. In case of uni-modal Gaussian priors, it has been observed that individual latent dimensions are encouraged to be independent. This leads to a factored representation which is expected to learn true factors of variation in the data, and so interpretable by humans. This distribution is parameterized by the neural network architecture that outputs all the parameters (i.e., K pairs of µ and s and the K component probabilities). Since the posterior q_f (z|x) is still a Gaussian, a reparameterization trick from original VAE works. The only difficulty is in computing the KL-Divergence of the posterior from the conditional prior, that is the third term from equation (2). Thus, the KL-Divergence of a Gaussian from a mixture-of-Gaussians is required to be computed. This is intractable and can’t be computed in closed form. In an existing method, a variational approximation exists which for Gaussian mixtures f and g is provided in equation (6) below:
D_KL (f?g¦)˜?_k¦??_i log??(?_i¦??_i exp(-D_KL (f_i ?f_i^' ¦)) ?)/(?_j¦?p_j exp(-D_KL (f_i ?g_j ¦)) ?)? ? --- (6)
Here, f_i, f_i^', and g_jdenote the component Gaussians of f and g and ? and p are their component weights respectively. Since in the present disclosure, the first distribution f which is the posterior, is a uni-modal Gaussian, the numerator of equation (6) reduces to 1 (since D_KL (f?f¦)=0). So, the equation (5) is modified and expressed as equation (7) provided below as:
D_KL (f?g¦)˜log??1/(?_j¦?p_j exp(-D_KL (f?g_j ¦)) ?)? ---- (7)
Further, the conditional prior term from equation (2) is replaced to obtain a final loss function provided in equation (8) below as:
?L ?_(VAE-REG)= -D_KL (q(c+|x¦)?p(c)¦)+?L ?_Style-log??1/(?_j¦?p_j exp(-D_KL (q_f (z|x)?p_j (z|c)¦)) ?)? ---- (8)

A similar formulation as provided in equation (8) with a mixture-of-Gaussians prior, except the style loss is used in a recent existing approach for a conditional VAE for video prediction. While they use it for conditional generation of videos, context of the present disclosure is that of supervised regression. Interpretation of the present disclosure leads to a graphical model in which x is not directly dependent on c but through an ancestral relationship via z. Further, motivation for multi-modal prior in the present disclosure comes from the ill-posed nature of the inverse inference.
In an embodiment, the one or more hardware processors 104 are further configured to receive a set of incoming target properties associated with the plurality of microstructure images of the material under consideration. In an embodiment, the set of incoming target properties refers to unseen test inputs. In an embodiment, the set of incoming target properties could be among the plurality of properties that are classified during the interpretation of the trained variational autoencoder or a set of new properties that are used as test data. Further, a plurality of reconstructed microstructure images corresponding to the set of incoming target properties for the plurality of incoming microstructure images of the material under consideration are obtained using the trained neural network architecture. The neural network architecture can be qualitatively evaluated by assessing a similarity between the unseen test inputs and their reconstructions obtained through the neural network architecture.
Experimental Results:
In the present disclosure, results obtained on a dataset of synthetic 3-D microstructures and associated elastic stiffness property are described. It is shown that the property prediction accuracy is comparable to a state-of-the-art 3-D convolutional neural network (CNN) regression model with almost double the number of parameters. Further, it is shown in the present disclosure that i) the set of reconstructed microstructure images are statistically equivalent to the received plurality of microstructure images (i.e., input microstructures) and ii) multiple microstructures for a target property value can be obtained using the conditional prior and the decoder network.
Dataset
In the present disclosure, a dataset created as a part of work of a known in the art approach (refer, ‘Patxi Fernandez-Zelaia, Yuksel C. Yabansu, and Surya R. Kalidindi. A comparative study of the efficacy of local/global and parametric/nonparametric machine learning methods for establishing structure-property linkages in high-contrast 3d elastic composites. Integrating Materials and Manufacturing Innovation, 8(2), 6 2019’) on an efficient Finite Element Method (FEM) for micromechanics simulation to estimate the effective elastic stiffness (C_(11,eff)) is used. The dataset consists of 8900 3D microstructures and the corresponding values of effective elastic stiffness. In the known in the art approach, first a large ensemble of voxelized 3D microstructures with diverse morphological features is synthetically generated. This was done by starting with a 3D array of random numbers in the range [0,1], followed by convolution with a variety of 3D Gaussian filters (with zero mean and diagonal co-variance) and thresholding to obtain a binary microstructure containing two phases namely black and white. The thresholding was performed in such a way that a pre-determined volume fraction of the black phase is obtained. The diagonal of the co-variance matrix of the 3D Gaussian filters was of the form s = [i,j,k],i,j,k ? {1,3,5,7}. Thus, there are 64 different Gaussian filters. These were applied to 150 random initial volumes to get ~8900 3D microstructures. These Gaussian filters affect the morphological features, resulting in a wide variety of morphologies.
FIGS. 4A through 4D show some example microstructures obtained using different Gaussian filters for material microstructure designing using variational autoencoder-regression model with a multi-modal prior according to some embodiments of the present disclosure. The example microstructures are obtained by convolution with different 3D Gaussian filters of the form N(0,sI). It is observed from FIGS. 4A through 4D that higher numbers in s result in larger gain size whereas asymmetry (i.e., larger/smaller numbers) in a particular axis result in anisotropy or elongation in that direction. For example, s = [1,1,1] results in small, isotropic grains whereas s = [7,7,7] results in very large grains. Other asymmetric choices such as s = [7,1,1] or [1,1,7] result in anisotropic grains with elongation in the corresponding directions. In the above-mentioned known in the art approach, further the effective elastic stiffness property denoted by (C_(11,eff)) for these microstructures are estimated through finite element (FE) simulations of elastic response. Here, few assumptions are considered, which are as follows: i) The black color in the example microstructures corresponds to a hard phase and white color to a soft phase ii) Young’s moduli of the hard and soft phases are 120Gpa and 2.4Gpa, respectively, and iii) Poisson’s ratio for both the phases is 0.3. A high difference between the young’s moduli of the constituent phases means that the material under consideration is a high-contrast composite, leading to longer range complex and nonlinear interactions at the microscale. The elastic stiffness relates stress with strain. Since stress and strain are second rank tensors (matrices), a fourth rank tensor is required to relate them as shown in equation (9) provided below:
s = C? --- (10)
Or using an index form as provided in equation (10) below:
s_ij = C_ijkl ?_kl --- (11)
However, due to symmetry of stress and strain tensors, only 6 of the values are independent. Thus, a 6 × 6 matrix (instead of a 3 × 3 × 3 × 3 tensor) is enough to represent C . Further, this 6 × 6 matrix is also symmetric. So only 15 elements out of 36 are independent. The effective elastic stiffness parameter estimated above is the element <1,1,1,1> of the stiffness tensor C, and is denoted as (C_(11,eff) ) for short.
In the present disclosure, following the Finite Element based simulation method performed in the above-mentioned known in the art approach, additional simulations are performed to estimate all the elements of the stiffness tensor. It is assumed that inverse inference should be performed for the entire tensor instead of just one element that captures an aggregate response. This is because the stiffness tensor as a whole is a complete measure of a material’s elastic behavior, while C_11 is like an average measure that depends upon features in a particular direction only. Using this additional data, it is shown that the method of the present disclosure is capable of linking the structure with a vector of properties rather than just a single scalar property.
FIG. 5A though 5L through shows example reconstructions of test set microstructures obtained from the trained VAE model with different morphological features according to some embodiments of the present disclosure. FIGS. 5A through 5F show original inputs while FIGS. 5G through 5L shows corresponding reconstruction of the original inputs. For example, FIG. 5G is the reconstruction of FIG. 5A. Similarly, FIG. 5H is the reconstruction of FIG. 5B. It is observed from FIG. 5A through 5L that the reconstructions are not exactly a replica of the original inputs, rather they are statistically equivalent. This means that average grain sizes, elongation, and volume fraction of black phase are quite similar in the input and the reconstruction. This shows that the latent representation captures these physically significant attributes of the microstructure.
Predicting Properties - Forward Inference
The present disclosure discusses about accuracy of the neural network architecture in forward prediction of properties for a given microstructure. Table 1 provides a comparison of the prior art with a conventional approach using a state-of-the-art 3D CNN in terms of forward prediction accuracy.
Method MAE MAPE R2 #Params
VAE-Reg 1.76 5.13 0.99 ~3.5M
3D CNN 1.04 3.10 0.99 ~7.7M
Table 1
First row in Table I shows the prediction accuracy of the neural network architecture of the present disclosure on the test data. Second row shows the accuracy obtained in the conventional approach using the state-of-the-art 3D CNN which is trained solely on a regression objective and contains almost double the number of parameters as compared to the regression model used in the present disclosure ( i.e., ~7.7M as compared to ~3.5M in the present disclosure). It is observed form Tale 1 that the neural network architecture of the present disclosure achieves comparable accuracy with much smaller number of parameters. This is desirable, because a simpler model means less chances of overfitting and better generalization to unseen regimes of data. Additionally, the neural network architecture of the present disclosure can also be used for direct inverse inference, by sampling latent vectors required for a target property value from the learned conditional prior. In the context of present disclosure, the direct inverse inference refers to inverse inference without involving an optimization step.
Predicting Structures - Inverse Inference
In an embodiment, the effective elastic stiffness property(C_(11,eff) ) is largely a function of volume fraction. However, this function changes with the morphology. FIGS. 6A and 6B illustrates two microstructures that achieve the effective elastic stiffness property (C_(11,eff) ) ?25Gpa corresponding to two morphologies for material microstructure designing using variational autoencoder-regression model with a multi-modal prior according to some embodiments of the present disclosure. The two microstructures shown in FIG. 6A and 6B, both have (C_(11,eff) ) ?25Gpa. The microstructure shown in FIG. 6A is from the morphology resulting from s = [1,7,7], with 74.49% volume fraction of black phase, while the microstructure shown in FIG. 6B is from the morphology resulting from s = [7,1,1] with 26.02% volume fraction of black phase. FIG. 7 shows the effective elastic stiffness property (C_(11,eff) ) plotted against volume fraction of black phase in all data corresponding to two morphologies for material microstructure designing using variational autoencoder-regression model with a multi-modal prior according to some embodiments of the present disclosure. Thus, a given value of (C_(11,eff) ) can be achieved by multiple microstructures, possibly each coming from a different morphology, with a different volume fraction of black-phase. This motivates the use of a multi-modal conditional prior p(z|c), so that there are multiple expected latent vectors z, for a target c. To demonstrate multi-modal inverse inference, a subset of the entire data, namely, the microstructures coming from the Gaussian filters with s = [1,1,7] and s = [7,1,1], is chosen which has remarkably different morphologies and are well-separated in the property space too as shown in FIG. 7. Inverse inference is performed for five target values of (C_(11,eff) ) spread across a complete range observed in the data. For each target (C_(11,eff) ) value, a latent vector is sampled multiple times from the conditional prior. In case of a Gaussian mixture prior, first a mixture component is sampled using the mixture weights, followed by sampling from the corresponding Gaussian.
FIGS. 8A and 8B provides a comparison of uni-modal and multi-modal prior in terms of inferred microstructures for a specific target effective elastic stiffness property under a corresponding morphology according to some embodiments of the present disclosure. FIG. 8A shows the inferred microstructures using a Gaussian (uni-modal) prior, whereas FIG. 8B shows the inferred microstructures using a Gaussian mixture prior with 2 components. It is observed that for (C_(11,eff) ) values which are possible under both the morphologies, the uni-modal prior ends up inferring an average of the two, making the microstructure look like a mix of the two morphologies. I is observed that growths tend to be in both directions rather than being clearly in one direction. These types of structures are difficult to realize in reality, so are less probable. Even if they occur, they will not have the same (C_(11,eff) ) value. Whereas the Gaussian mixture model learns the solutions separately under different mixture components, with suitable component weights that reflect a probability of achieving the target (C_(11,eff) ) under the corresponding morphology. Here, the number of mixture components is a hyperparameter which needs to be set empirically. FIGS. 9A and 9B show plots of a specific target effective elastic stiffness property against volume fraction of black phase in inverse inferred microstructures using the uni-modal and the multi-modal prior respectively according to some embodiments of the present disclosure. The points corresponding to inverse inference are super-imposed on the earlier plot of complete training data from FIG. 7. It is observed from FIGS. 9A and 9B that the points tend to lie in between two curves corresponding to training data (i.e., learning an average of the two solutions) in case of uni-modal Gaussian prior. Whereas, in the case of Gaussian mixture prior, the inverse inferred points tend to lie near one of the curves rather than in the middle (separating the two solutions).
Extension to multiple properties
In an embodiment, the neural network architecture of the present disclosure does not assume anything about conditioning variable c and so it is straightforward to extend the neural network architecture of the present disclosure to a case where c is a vector of properties rather a single property. In such a case, the neural network architecture of the present disclosure is modified by changing size of the output layer of regression model and input layer of generator network appropriately. Further, it is more useful to perform inverse inference on a set of properties rather than a single property, since a set of properties form a clearer target in terms of performance requirement specification. For example, a material with a target ‘tensile strength’ and ‘hardness’ may be desired for an automobile application. With this extension, it is shown that the neural network architecture of the present disclosure is capable of linking the microstructures with a vector of properties too, using a larger data set. While all stiffness parameters (i.e., C_11...C_66) are estimated as part of the augmentation, in the experiments conducted, but only diagonal elements C_11, C_22and C_33 are used.
For forward inference, entire data set (all 8900) is used, whereas for inverse inference, microstructures from only two morphologies are selected such that they are remarkably different and well-separated in property space too. Table 2 shows the forward prediction accuracy on the test set, using a Gaussian prior.
Metric C_11 C_22 C_33
MAE 2.75 2.81 2.91
MAPE 7.91 7.59 7.73
R2 97.63 97.76 97.46
Table 2
In case of vector properties, inverse inference cannot be easily validated for any set of target values spread across a range, like in the scalar case. Because, to validate inverse inference, instances from observed data with property values close to the targets for inverse inference are looked upon. But in the vector case, it is not easy to find instances from observed data that are near the targets due to high dimensionality. Instead, inverse inference is performed for vectors [C_11,C_22,C_33] from the test data. These are observed vectors and corresponding microstructure from the test data that achieved these vectors of properties are obtained. Thus, at least partial validation can be done with these microstructures. FIG. 10A and 10B show results of inverse inference using data from two different morphologies according to some embodiments of the present disclosure. The two different morphologies correspond to s = [1,7,7] and s = [7,1,1]. FIG. 10A shows inverse inference for a few property vectors using uni-modal Gaussian prior, whereas FIG. 10B shows inferred microstructures for the same property vectors using the multi-modal mixture of-Gaussians prior. As before, the solutions obtained using a Gaussian prior tend to be a mix of the two morphologies. FIGS. 11A and 11B show a plot of the set of target properties C_11, C_22and C_33 values against volume fraction of black phase in the inverse inferred microstructures using the two priors according to some embodiments of the present disclosure. In other words, FIGS. 11A and 11B depict quantitatively a difference between quality of solutions obtained using uni-modal and multimodal priors respectively. The points corresponding to inverse inference are superimposed on the points corresponding to training data. As shown in FIG. 11A, in case of uni-modal Gaussian prior, the points tend to lie scattered in the space between the two curves corresponding to training data. This indicates that the inferred solutions are average of the two actual solutions. Whereas, as shown in FIG. 11B, in the case of Gaussian mixture prior, the scatter of inverse inferred points is much less, with all new points clustered near one of the curves.
The written description describes the subject matter herein to enable any person skilled in the art to make and use the embodiments. The scope of the subject matter embodiments is defined herein and may include other modifications that occur to those skilled in the art. Such other modifications are intended to be within the scope of the present disclosure if they have similar elements that do not differ from the literal language of the embodiments or if they include equivalent elements with insubstantial differences from the literal language of the embodiments described herein.
The embodiments of present disclosure provide systems and methods to navigate in unknown scenes based only on egocentric RGB perception of a wheeled service robot. The present disclosure provides a neural network architecture combining a VAE model with a regression model for structure-property linkages in materials science. The neural network architecture can be used for inverse inference directly, avoiding an optimization step. With a multi-modal prior formulation, the neural network architecture is capable of handling ill-posed inverse inference and suggest multiple feasible solutions. With the help of simulation data, it has been demonstrated that the inferred solutions indeed fit the target requirements. Also, it is shown that the neural network architecture extends to multiple properties. Hence, it is believed that the neural network architecture can be used to provide a materials scientist with a few good initial guesses (microstructures) for a set of target properties. Detailed analysis using either physics based simulations or experiments can then be carried out more effectively in this drastically reduced design space. The method of the present disclosure has a potential to replace traditional methods of exploration in materials science.
It is to be understood that the scope of the protection is extended to such a program and in addition to a computer-readable means having a message therein; such computer-readable storage means contain program-code means for implementation of one or more steps of the method, when the program runs on a server or mobile device or any suitable programmable device. The hardware device can be any kind of device which can be programmed including e.g., any kind of computer like a server or a personal computer, or the like, or any combination thereof. The device may also include means which could be e.g., hardware means like e.g., an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a combination of hardware and software means, e.g., an ASIC and an FPGA, or at least one microprocessor and at least one memory with software processing components located therein. Thus, the means can include both hardware means and software means. The method embodiments described herein could be implemented in hardware and software. The device may also include software means. Alternatively, the embodiments may be implemented on different hardware devices, e.g., using a plurality of CPUs.
The embodiments herein can comprise hardware and software elements. The embodiments that are implemented in software include but are not limited to, firmware, resident software, microcode, etc. The functions performed by various components described herein may be implemented in other components or combinations of other components. For the purposes of this description, a computer-usable or computer readable medium can be any apparatus that can comprise, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
The illustrated steps are set out to explain the exemplary embodiments shown, and it should be anticipated that ongoing technological development will change the manner in which particular functions are performed. These examples are presented herein for purposes of illustration, and not limitation. Further, the boundaries of the functional building blocks have been arbitrarily defined herein for the convenience of the description. Alternative boundaries can be defined so long as the specified functions and relationships thereof are appropriately performed. Alternatives (including equivalents, extensions, variations, deviations, etc., of those described herein) will be apparent to persons skilled in the relevant art(s) based on the teachings contained herein. Such alternatives fall within the scope of the disclosed embodiments. Also, the words “comprising,” “having,” “containing,” and “including,” and other similar forms are intended to be equivalent in meaning and be open ended in that an item or items following any one of these words is not meant to be an exhaustive listing of such item or items, or meant to be limited to only the listed item or items. It must also be noted that as used herein, the singular forms “a,” “an,” and “the” include plural references unless the context clearly dictates otherwise.
Furthermore, one or more computer-readable storage media may be utilized in implementing embodiments consistent with the present disclosure. A computer-readable storage medium refers to any type of physical memory on which information or data readable by a processor may be stored. Thus, a computer-readable storage medium may store instructions for execution by one or more processors, including instructions for causing the processor(s) to perform steps or stages consistent with the embodiments described herein. The term “computer-readable medium” should be understood to include tangible items and exclude carrier waves and transient signals, i.e., be non-transitory. Examples include random access memory (RAM), read-only memory (ROM), volatile memory, nonvolatile memory, hard drives, CD ROMs, DVDs, flash drives, disks, and any other known physical storage media.
It is intended that the disclosure and examples be considered as exemplary only, with a true scope of disclosed embodiments being indicated herein.
, Claims:We Claim:
A processor implemented method (200), comprising:
receiving (202), via one or more hardware processors, (i) a plurality of microstructure images of a material under consideration, (ii) a plurality of properties associated with the plurality of microstructure images of the material under consideration, (iii) a neural network architecture, and (iv) a pre-determined loss function associated with the neural network architecture, wherein the neural network architecture represents a variational autoencoder model combined with a regression model through a multi-modal mixture-of-Gaussians prior conditioned on the plurality of properties associated with the plurality of microstructure images; and
iteratively training (204), via the one or more hardware processors, the neural network architecture using the plurality of microstructure images and the plurality of properties associated with the plurality of microstructure images using a backpropagation algorithm, wherein the steps performed for training the neural network architecture at each of a plurality of iterations, comprises:
obtaining, a plurality of latent representations of (i) the plurality of microstructure images and (ii) the plurality of properties associated with the plurality of microstructure images from trained neural network architecture;
sampling, a latent representation from the plurality of latent representations, for a target property from among the plurality of properties associated with the plurality of microstructure images, based on the multi-modal mixture-of-Gaussians prior conditioned on the plurality of properties associated with the plurality of microstructure images;
reconstructing, a set of microstructure images corresponding to the target property, from the plurality of microstructure images, using a decoder network of the trained neural network architecture; and
computing, the pre-determined loss function for the trained neural network architecture, wherein the computed pre-defined loss function comprises a Kullback-Leibler (KL) loss function, a style loss function and a regression loss function.

The processor implemented method of claim 1, further comprising:
receiving a set of incoming target properties associated with the plurality of microstructure images of the material under consideration; and
obtaining, a plurality of reconstructed microstructure images corresponding to the set of incoming target properties for the plurality of incoming microstructure images of the material under consideration, using the trained neural network architecture.

The processor implemented method of claim 1, wherein the neural network architecture comprises a variational autoencoder model with an encoder network and the decoder network, the regression model, and a generator network.

The processor implemented method of claim 1, wherein the encoder network of the variational autoencoder comprises three blocks of convolution neural network including a convolution layer followed by batch normalization and leaky ReLU non-linearity with a slope of 0.3, and two blocks of two fully connected parallel output layers that predict mean and diagonal co-variance of posterior distribution of the plurality of microstructure images.

The processor implemented method of claim 1, wherein the decoder network of the variation autoencoder comprises three blocks of convolution neural network including a transposed convolution layer followed by batch normalization and leaky ReLU non-linearity with a slope of 0.3, and two blocks of two fully connected parallel output layers.

The processor implemented method of claim 1, wherein the generator network of the neural network architecture comprises an input layer configured to receive at least one property from the plurality of properties associated with the plurality of microstructure images as input, a fully connected hidden layer with tanh activation, and two parallel linear output layers that predict means and standard deviations of each component of the multi-modal mixture-of-Gaussians conditional prior.

The processor implemented method of claim 1, wherein the regression model of the neural network architecture comprises three blocks of convolution neural network shared by the encoding network, and two blocks of two fully connected parallel output layers that predict the mean and standard deviation of the plurality of properties associated with the plurality of microstructure images.

The processor implemented method of claim 1, wherein the style loss function determines a style loss between each microstructure image from the plurality of microstructure images and a corresponding reconstructed microstructure image using a style loss network.

A system (100), comprising:
a memory (102) storing instructions;
one or more communication interfaces (106); and
one or more hardware processors (104) coupled to the memory (102) via the one or more communication interfaces (106), wherein the one or more hardware processors (104) are configured by the instructions to:
receive, (i) a plurality of microstructure images of a material under consideration, (ii) a plurality of properties associated with the plurality of microstructure images of the material under consideration, (iii) a neural network architecture and (iv) a pre-determined loss function associated with the neural network architecture, wherein the neural network architecture represents a variational autoencoder model combined with a regression model through a multi-modal mixture-of-Gaussians prior conditioned on the plurality of properties associated with the plurality of microstructure images; and
iteratively train, the neural network architecture using the plurality of microstructure images and the plurality of properties associated with the plurality of microstructure images using a backpropagation algorithm, wherein the steps performed for training the neural network architecture at each iteration, comprises:
obtaining, a plurality of latent representations of (i) the plurality of microstructure images and (ii) the plurality of properties associated with the plurality of microstructure images from trained neural network architecture;
sampling, a latent representation for a target property from the plurality of latent representations of the plurality of properties associated with the plurality of microstructure images based on the multi-modal mixture-of-Gaussians prior conditioned on the plurality of properties associated with the plurality of microstructure images;
reconstructing, a set of microstructure images corresponding to the target property from the plurality of microstructure images using a decoder network of the trained neural network architecture; and
computing, the pre-determined loss function for the trained neural network architecture, wherein the computed pre-defined loss function comprises a Kullback-Leibler (KL) loss function, a style loss function and a regression loss function.

The system of claim 9, further comprising:
receiving a set of incoming target properties associated with the plurality of microstructure images of the material under consideration; and
obtaining, a plurality of reconstructed microstructure images corresponding to the set of incoming target properties for the plurality of incoming microstructure images of the material under consideration, using the trained neural network architecture.

The system of claim 9, wherein the neural network architecture comprises a variational autoencoder model with an encoder network and the decoder network, the regression model, and a generator network.

The system of claim 9, wherein encoder network of the variational autoencoder comprises three blocks of convolution neural network including a convolution layer followed by batch normalization and leaky ReLU non-linearity with a slope of 0.3, and two blocks of two fully connected parallel output layers that predict mean and diagonal co-variance of posterior distribution of the plurality of microstructure images.

The system of claim 9, wherein the decoder network of the variation autoencoder comprises three blocks of convolution neural network including a transposed convolution layer followed by batch normalization and leaky ReLU non-linearity with a slope of 0.3, and two blocks of two fully connected parallel output layers.

The system of claim 9, wherein the generator network of the neural network architecture comprises an input layer configured to receive at least one property from the plurality of properties associated with the plurality of microstructure images as input, a fully connected hidden layer with tanh activation, and two parallel linear output layers that predict means and standard deviations of each component of the multi-modal mixture-of-Gaussians conditional prior.

The system of claim 9, wherein the regression model of the neural network architecture comprises three blocks of convolution neural network shared by the encoding network, and two blocks of two fully connected parallel output layers that predict the mean and standard deviation of the plurality of properties associated with the plurality of microstructure images.

The system of claim 9, wherein the style loss function determines a style loss between each microstructure image from the plurality of microstructure images and a corresponding reconstructed microstructure image using a style loss network.

Dated this 28th Day of September 2022

Tata Consultancy Services Limited
By their Agent & Attorney

(Adheesh Nargolkar)
of Khaitan & Co
Reg No IN-PA-1086