Sign In to Follow Application
View All Documents & Correspondence

Method And System For Creating Tiny Deep Neural Network Model

Abstract: State-of-the-art methods for creating tiny deep neural network model provide a reasonable reduction in size. However, while deploying such tiny deep neural network model considerable performance is not attained. A method and system for creating tiny deep neural network model suitable to be deployed on hardware constraint devices is disclosed. The method provides an intermediate student model obtained from sparsification of an uncompressed deep neural network model followed by employing sparsity distribution-based edits to the intermediate student model. Further, the method provides model reduction within a defined search space utilizing NAS architecture framework. The resulting model with optimal size reduction of the uncompressed deep neural network model is suitable to be deployed on hardware constraint edge devices. The method results into a desired performance of tiny deep neural network model within short span of time due to lesser number of iterations required in the defined search space.

Get Free WhatsApp Updates!
Notices, Deadlines & Correspondence

Patent Information

Application #
Filing Date
05 April 2023
Publication Number
41/2024
Publication Type
INA
Invention Field
COMPUTER SCIENCE
Status
Email
Parent Application

Applicants

Tata Consultancy Services Limited
Nirmal Building, 9th Floor, Nariman Point, Mumbai 400021, Maharashtra, India

Inventors

1. DEY, Swarnava
Tata Consultancy Services Limited, Global Development Centre (GDC), Plot C, Block EP, Sector V, Salt Lake Electronic Complex, Kolkata - 700091, West Bengal, India
2. UKIL, Arijit
Tata Consultancy Services Limited, Eden Building, Plot B-1, Block EP & GP, Sector 5, Salt Lake Electronics Complex, Kolkata - 700091, West Bengal, India
3. GHOSE, Avik
Tata Consultancy Services Limited, Global Development Centre (GDC), Plot C, Block EP, Sector V, Salt Lake Electronic Complex, Kolkata - 700091, West Bengal, India
4. MUKHERJEE, Arijit
Tata Consultancy Services Limited, Global Development Centre (GDC), Plot C, Block EP, Sector V, Salt Lake Electronic Complex, Kolkata - 700091, West Bengal, India
5. PAL, Arpan
Tata Consultancy Services Limited, Global Development Centre (GDC), Plot C, Block EP, Sector V, Salt Lake Electronic Complex, Kolkata - 700091, West Bengal, India
6. MUKHOPADHYAY, Shalini
Tata Consultancy Services Limited, Eden Building, Plot B-1, Block EP & GP, Sector 5, Salt Lake Electronics Complex, Kolkata - 700091, West Bengal, India
7. SAHU, Ishan
Tata Consultancy Services Limited, Eden Building, Plot B-1, Block EP & GP, Sector 5, Salt Lake Electronics Complex, Kolkata - 700091, West Bengal, India
8. KULKARNI, Gitesh
Tata Consultancy Services Limited, Unit-III, No 18, SJM Towers, Seshadri Road, Gandhinagar, Bangalore – 560009, Karnataka, India
9. BASAK, Barnali
Tata Consultancy Services Limited, Tata Consultancy Services, SJM Towers Annexe, No. 18, Seshadri Road, 6th Cross, Gandhinagar, BANGALORE – 560009, Karnataka, India
10. DASGUPTA, Pallab
IIT Kharagpur, Kharagpur – 721602, West Bengal, India

Specification

DESC:FORM 2 THE PATENTS ACT, 1970 (39 of 1970) & THE PATENT RULES, 2003 COMPLETE SPECIFICATION (See Section 10 and Rule 13) Title of invention: METHOD AND SYSTEM FOR CREATING TINY DEEP NEURAL NETWORK MODEL Applicant: Tata Consultancy Services Limited A company Incorporated in India under the Companies Act, 1956 Having address: Nirmal Building, 9th Floor, Nariman Point, Mumbai 400021, Maharashtra, India The following specification particularly describes the invention and the manner in which it is to be performed. CROSS-REFERENCE TO RELATED APPLICATIONS AND PRIORITY The present application claims priority from Indian provisional patent application no. 202321025696, filed on April, 05, 2023. The entire contents of the aforementioned application are incorporated herein by reference. TECHNICAL FIELD The disclosure herein generally relates to the field of neural network compression, and, more particularly, to a method and a system of creating tiny deep neural network (MTiny) model based on pruning technique coupled with neural architecture search (NAS). BACKGROUND The prevalent success of Artificial Intelligence and Machine Learning (AI and ML) models have catalyzed their use on edge devices close to the sensors, so that such devices can address necessary requirements for low latency, data privacy, and be free from concerns of unreliable connectivity. It leverages analytical computation resources very close to the end-users, resulting in higher throughput and better responsiveness in the applications. Deep neural network (DNN) architectures have been widely used in AI and computer-vision fields to produce state-of-the-art results for tasks such as visual object recognition, detection and segmentation. Deploying DNN on edge platforms for model automation requires automated learning of tiny neural architectures through search, and automated model reduction techniques capable of migrating pre-trained DNNs to tiny neural architectures. Both approaches need to meet platform specific constraints and various objectives, such as low latency, low memory footprint and low power consumption. While working in edge computing environment, edge devices have limited capacity to process the data thus implementing DNN in edge computing becomes challenging. Neural Architecture Search (NAS) aims to explore alternative DNN architectures until it finds a suitably optimal variant satisfying all the design constraints. NAS has shown great success in automating the design of neural networks, but the prohibitive number of computations behind current NAS methods discourage its implementation when DNN are to be deployed on edge devices. SUMMARY Embodiments of the present disclosure present technological improvements as solutions to one or more of the above-mentioned technical problems recognized by the inventors in conventional systems. For example, in one embodiment a processor implemented method for creating a tiny deep neural network (MTiny) model suitable hardware constraint device is provided. The method includes initializing an uncompressed deep neural network model by randomly assigning a plurality of weight to a plurality of nodes of the plurality of deep neural network layers of an uncompressed DNN model. The method further includes training the initialized uncompressed deep neural network model with a training dataset. Then from among the plurality of weights assigned to the plurality of nodes, one or more weights having least magnitude from among the plurality of weights of the trained uncompressed deep neural network model are pruned. The sparsified neural network model is generated by pruning of the weights of least magnitude from the plurality of weights. The method further includes computing distribution of sparsity across the sparsified neural network model and applying edits based on sparsity distribution to obtain an intermediate student model. The intermediate student model based on sparsity distribution has reduced architecture with smaller number of deep neural network layers compared to the sparsified neural network model and a different set of hyperparameters. The method further includes defining a search space, for the intermediate student model, in terms of domain constraints using a search space generation model. The domain constraints include one or more operation types, one or more deep neural network layer parameters for defining a deep neural network architecture, verifying the one or more deep neural network layer parameters with a target hardware device having a plurality hardware constraints. Finally, the method includes applying by one or more hardware processors, a Neural Architecture Search (NAS) framework on the intermediate student model within the defined search space to create a tiny deep neural network model deployable on the target hardware having the set of hardware constraints, a plurality of performance metrics comprising an accuracy, a latency, a runtime memory usage, and a size of the tiny deep neural network model. In another aspect, a system for creating a tiny deep neural network (MTiny) model suitable for hardware constraint device is provided. The system comprises memory storing instructions; one or more communication interfaces; and one or more hardware processors coupled to the memory via the one or more communication interfaces, wherein the one or more hardware processors are configured by the instructions to initialize an uncompressed deep neural network model by randomly assigning a plurality of weight to a plurality of nodes of the plurality of deep neural network layer of an uncompressed DNN model. The plurality of deep neural network layers comprises of convolutional layers. Further, the system includes to train the initialized uncompressed deep neural network model with a training dataset. Then from among the plurality of weights assigned to the plurality of nodes, one or more weights having least magnitude from among the plurality of weights of the trained uncompressed deep neural network model are pruned. The sparsified neural network model is generated by pruning of the weights of least magnitude from the plurality of weights. Further, the system includes to compute distribution of sparsity across the sparsified neural network model and applying edits based on sparsity distribution to obtain an intermediate student model. The intermediate student model based on sparsity distribution has reduced architecture with smaller number of deep neural network layers compared to the sparsified neural network model and a different set of hyperparameters. Furthermore, the system includes to define a search space, for the intermediate student model, in terms of domain constraints using a search space generation model. The domain constraints include one or more operation types, one or more deep neural network layer parameters for defining a deep neural network architecture, verifying the one or more deep neural network layer parameters with a target hardware device having a plurality hardware constraints. Finally, the system includes to apply by one or more hardware processors, a Neural Architecture Search (NAS) framework on the intermediate student model within the defined search space to create a tiny deep neural network model deployable on the target hardware having the set of hardware constraints, a plurality of performance metrics comprising an accuracy, a latency, a runtime memory usage, and a size of the tiny deep neural network model. The uncompressed neural network is subjected to Lottery Ticket Hypothesis (LTH) based iterative pruning of layers to obtain sparsified neural network model, wherein the layer-wise pruning is executed by adjusting a pruning ratio obtained by identifying number of neurons to be pruned in each layer of the uncompressed neural network. The intermediate student model obtained by leveraging sparsity information of the sparsified neural network has a reduced model architecture, wherein reduction in the model architecture is obtained by reducing number of layers, modifying number of kernels, sizes and stride lengths of the deep neural network layers, and modifying the number of neurons in the final dense layer. The layers of the intermediate student model are verified for compatibility with the target hardware device and subsequently, a smaller subset of this search space is used by the NAS framework. In yet another aspect, there is provided a computer program product comprising a non-transitory computer readable medium having a computer readable program embodied therein, wherein the computer readable program, when executed on a computing device causes the computing device with hardware constraints create a tiny deep neural network (MTiny) model by initializing an uncompressed deep neural network model by randomly assigning a plurality of weight to a plurality of nodes of the plurality of deep neural network layers of an uncompressed DNN model. Further the computer readable program includes training the initialized uncompressed deep neural network model with a training dataset. Then from among the plurality of weights assigned to the plurality of nodes, one or more weights having least magnitude from among the plurality of weights of the trained uncompressed deep neural network model are pruned. The sparsified neural network model is generated by pruning of the weights of least magnitude from the plurality of weights. Further the computer readable program includes computing distribution of sparsity across the sparsified neural network model and applying edits based on sparsity distribution to obtain an intermediate student model. The intermediate student model based on sparsity distribution has reduced architecture with smaller number of deep neural network layers compared to the sparsified neural network model and a different set of hyperparameters. Further the computer readable program includes defining a search space, for the intermediate student model, in terms of domain constraints using a search space generation model. The domain constraints include one or more operation types, one or more deep neural network layer parameters for defining a deep neural network architecture, verifying the one or more deep neural network parameters with a target hardware device having a plurality hardware constraints. Finally, the computer readable program includes applying by one or more hardware processors, a Neural Architecture Search (NAS) framework on the intermediate student model within the defined search space to create a tiny deep neural network model deployable on the target hardware having the set of hardware constraints, a plurality of performance metrics comprising an accuracy, a latency, a runtime memory usage, and a size of the tiny deep neural network model. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed. BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate exemplary embodiments and, together with the description, serve to explain the disclosed principles: FIG. 1 is a functional block diagram of a system for automated creation of tiny deep neural network (MTiny) model derived from pruning technique coupled with neural architecture search (NAS), in accordance with some embodiments of the present disclosure. FIG. 2 illustrates a generalized workflow for deployment of the tiny deep neural network model (MTiny) on an edge device, in accordance with some embodiments of the present disclosure. FIG. 3 is a flow diagram illustrating a method for creation of sparsified neural network (a student model, (MStudent)) based on Lottery Ticket Hypothesis (LTH)-based pruning coupled by performance edits, using the system of FIG. 1, in accordance with some embodiments of the present disclosure. FIG. 4 is a flow diagram illustrating a method for automated creation of tiny deep neural network (MTiny) from the intermediate student model (MStudent) through Neural Architecture Search (NAS), in accordance with some embodiments of the present disclosure. FIG. 5 illustrates a flow diagram for a method for creating the MTiny model, in accordance with some embodiments of the present disclosure. FIGS. 6A and 6B illustrate comparative flow diagrams depicting steps of creating MTiny model, in accordance with some embodiments of the present disclosure. FIGS. 7A, 7B and 7C illustrate architectures of an example uncompressed deep neural network (DNN) model, the sparsified neural network model (MLTH) corresponding to the example uncompressed DNN model; and the tiny deep neural network model (MTiny) which corresponds to the example of uncompressed DNN model obtained using method of FIG. 3, respectively, in accordance with some embodiments of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION Exemplary embodiments are described with reference to the accompanying drawings. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. Wherever convenient, the same reference numbers are used throughout the drawings to refer to the same or like parts. While examples and features of disclosed principles are described herein, modifications, adaptations, and other implementations are possible without departing from the scope of the disclosed embodiments. Machine learning models, especially deep neural networks (DNNs), have recently enabled unprecedented levels of intelligence on numerous systems and found successful applications in a broad spectrum of domains, including computing vision, healthcare, autonomous driving, machine translation, among many others. To achieve accuracies comparable to or even exceeding human levels, today’s machine learning models are becoming increasingly more complex, as evidenced by DNNs that can contain millions of or even more parameters. Major hindrances behind running such massive deep neural networks on edge devices includes computing requirement, energy budget, privacy and security. Such aforementioned issues can be addressed simultaneously if the DNNs could somehow be made smaller and more efficient. DNN compression is of utmost importance to bring the capability of the DNNs into the edge devices for performing different intelligent tasks. Neural network pruning is a commonly used technique for compressing the neural networks and it can reduce the parameter counts of trained networks by over 90%, decreasing storage requirements and improving computational performance of inference without compromising accuracy. However, sparse architectures produced by conventional neural network pruning techniques are difficult to train from the start, which would similarly improve training performance. The level of pruning achieved by an algorithm, say Lottery Ticket Hypothesis (LTH), is an indicative of the existence of tiny ML models having equivalent accuracy. Neural Architecture Search (NAS) has emerged as a leading tool for building customized models for tiny machine learning (TinyML) platforms. NAS aims to explore alternative DNN architectures until it finds a suitably optimal variant satisfying all the design constraints. Model reduction approaches are capable of sparsifying the large model, but tiny ML platforms do not provide support for automatically leveraging such sparsity. On the other hand, the opacity of DNN models rule out automated gleaning of knowledge that can potentially accelerate NAS. At present, despite the efficacy of NAS in finding tiny multi-objective (that is, high accuracy, low memory footprint, etc.) models, the time taken to search the models from scratch is still prohibitive. Therefore, conventional NAS method can be accelerated by narrowing its search space using a sparse model. In the present disclosure, a reference uncompressed DNN model trained using a given dataset is taken as a base model, which is accurate but does not fit a targeted tiny micro-controller or any such edge device with constrained resources. Further, the uncompressed DNN model is subjected to Lottery Ticket Hypothesis (LTH sparse LTH model) based pruning to obtain a sparsified neural network model (also called as (MLTH)). The MLTH is not small but many of the weights of the uncompressed DNN model are zeroed down. Further, sparsity information of MLTH model is leveraged to obtain an intermediate student model (MStudent) by applying various modifications (or editing) along with sparsity distribution. The intermediate student model thus obtained is found to be smaller and dense, compared to the larger, sparsified neural network model (MLTH), but it is not necessarily optimal in size and performance. Further, with the purpose of performance optimization, the student model is subjected to NAS. The intermediate student model acts as a reference for a set of NAS constraints, which narrows down the search space for NAS, thereby accelerating the discovery of an efficient tiny ML model (MTiny) suitable for the target hardware (i.e. edge device like smartwatch, smartphone, portable health monitoring devices etc.). Referring now to the drawings, and more particularly to FIGS. 1 through 7C, where similar reference characters denote corresponding features consistently throughout the figures, there are shown preferred embodiments and these embodiments are described in the context of the following exemplary system and/or method. FIG. 1 is a functional block diagram of a system 100 for automated creation of MTiny model derived from pruning technique coupled with neural architecture search (NAS), in accordance with some embodiments of the present disclosure. In an embodiment, the system 100 includes a processor(s) 104, communication interface device(s), alternatively referred as input/output (I/O) interface(s) 106, and one or more data storage devices or a memory 102 operatively coupled to the processor(s) 104. The system 100 with one or more hardware processors is configured to execute functions of one or more functional blocks of the system 100. Referring to the components of system 100, in an embodiment, the processor(s) 104, can be one or more hardware processors 104. In an embodiment, the one or more hardware processors 104 can be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuitries, and/or any devices that manipulate signals based on operational instructions. Among other capabilities, the one or more hardware processors 104 are configured to fetch and execute computer readable instructions stored in the memory 102. In an embodiment, the system 100 can be implemented in a variety of computing systems including laptop computers, notebooks, hand-held devices such as mobile phones, workstations, mainframe computers, servers, and the like having hardware constraints such that it can support only to smaller size neural network models. The I/O interface(s) 106 can include a variety of software and hardware interfaces, for example, a web interface, a graphical user interface to display the generated target images and the like and can facilitate multiple communications within a wide variety of networks N/W and protocol types, including wired networks, for example, LAN, cable, etc., and wireless networks, such as WLAN, cellular and the like. In an embodiment, the I/O interface (s) 106 can include one or more ports for connecting to a number of external devices or to another server or devices. The memory 102 may include any computer-readable medium known in the art including, for example, volatile memory, such as static random-access memory (SRAM) and dynamic random access memory (DRAM), and/or non-volatile memory, such as read only memory (ROM), erasable programmable ROM, flash memories, hard disks, optical disks, and magnetic tapes. In an embodiment, the memory 102 includes a plurality of models 110A-110D comprising an uncompressed DNN model 110A received by the system for optimization, a sparsified model (MLTH) 110B, an intermediate student model (MStudent) 110C and a tiny deep neural network model (MTiny) 110D generated by the system 100. The models are explained further in conjunction with architecture of the system 100 as depicted in FIG. 2 and flow diagram of the method executed by the system 100 for creating tiny models as depicted in FIG. 5. The plurality of models 110A-110C are used by the system 100 for executing different steps involved in the process of automated creation of tiny deep neural network (MTiny) model 110D derived from intermediate MStudent model, wherein the intermediate MStudent model undergo pruning, sparsification and further transformed within defined space of Neural Architecture Search (NAS) framework. Further, the memory 102 includes a dataset 108. The dataset (or repository) 108 may include a plurality of abstracted piece of code for refinement and data that is processed, received, or generated as a result of the execution of the method 200. Although the dataset 108 is shown internal to the system 100, it will be noted that, in alternate embodiments, the dataset 108 can also be implemented external to the system 100, and communicatively coupled to the system 100. The data contained within such external database may be periodically updated. For example, new data may be added into the dataset (not shown in FIG. 1) and/or existing data may be modified and/or non-useful data may be deleted from the dataset. Functions of the components of the system 100 are now explained with reference to flow diagrams of FIGS. 3-6B and example illustrated in FIGS. 7A, 7B and 7C. Major hindrances behind running massive deep neural networks at edge devices includes computing requirement and energy budget. Such aforementioned issues may be addressed simultaneously if the size of DNNs could somehow be reduced and creation of such reduced DNNs could be automated. DNN size reduction as disclosed by the method herein through pruning followed by obtaining optimal DNN architecture is of utmost importance to bring the capability of the DNNs into the edge devices for performing different intelligent tasks like ECG monitoring. FIG. 2 illustrates a generalized workflow for deployment of the tiny deep neural network model (MTiny) on an edge device, in accordance with some embodiments of the present disclosure. As is illustrated, MTiny 110D is a size optimized as well as trained deep neural network model. Along with training dataset 108, the uncompressed DNN model (M) 110A undergo intermediate student model (MStudent) model 110C creation as per the method of the present disclosure. The creation of MStudent model 110C involved size reduction through pruning of an uncompressed DNN model (M) 110A followed by sparsification based on sparsity distribution. The NAS framework is applied on the intermediate student model (MStudent) 110C within the defined search space to create a tiny deep neural network model (MTiny) 110D deployable on the target hardware such as edge devices having the set of hardware constraints. FIG. 3 is a flow diagram illustrating a method 200 for creation of sparsified neural network (a student model, (MStudent)) based on LTH-based pruning coupled by performance edits, using the system of FIG. 1, in accordance with some embodiments of the present disclosure. It should be appreciated that the process depicted in FIG. 3 illustrates one possible iteration of the process flow, which repeats for each uncompressed neural network received by the system 100. At step 202 of the method 200, the one or more hardware processors 104 are configured to receive an uncompressed DNN model. The uncompressed DNN model M comprises m different layers ((?f_i)?_(i=1)^m ) with corresponding weight matrices (W_i )_(i=1)^m and hyperparameters (h_i )_(i=1)^m. Further, at step 204 of the method 200, the one or more hardware processors 104 are configured to execute Lottery Ticket Hypothesis (LTH) based model size reduction technique. As understood by a person skilled in the art, LTH states that given a randomly initialized dense neural network a smaller subnetwork called the "winning ticket" can be found which when suitably initialized and trained can match the test performance of the original network. Usually, sparser the network, more difficult it is to train, i.e., it trains slowly, and the final performance is also lower. However winning ticket subnetworks have been shown to learn from at most the same number of training iterations as in the case of their corresponding larger network and achieve similar test performance. Identification of winning tickets can aid in designing better networks with improved training performance. Many studies have followed the original work, among which Frankle et al. study displayed some stability issues with such subnetworks and Malach et al. tried to provide some theoretical evidence of the empirical results and the existence of good subnetworks. The existence and application of such subnetworks have been reported in different domains. Girish et al. in studying the phenomenon with respect to object recognition tasks. Chen et al. proposed a unified lottery ticket hypothesis for graph neural networks. The LTH based pruning of an uncompressed DNN reduces the size by zeroing down some of the associated weights to provide light weighted DNN model as MLTH (the sparsified neural network model) using the iterative pruning as per the algorithm 1 presented below. Algorithm 1 to find reduced winning ticket by iterative pruning: Input: Training data, deep neural network M, target sparsity ?target Output: Sparse Model M*LTH Procedure: 1: Set per iteration pruning ratio p. 2: Randomly initialize the given network M and save these initialization weights. 3: Train M with training data 4: Prune p percent of least magnitude weights to get M’ 5: Compute sparsity of M’. 6: if sparsity > ?target then 7: stop and output the sparse model. 8: else 9: Reset the remaining parameters to the ones saved in step 2. 10: M ? M’ 11: Goto step 3. 12: end if Thus, the step 204 employs algorithm 1 for finding reduced winning ticket by iterative pruning to generate a smaller DNN. The LTH based model size reduction is executed on M to get sparse model MLTH which has sparsified weight matrices ?(W_(i,sparse))?_(i=1)^m. The smaller DNN (MLTH) employs the sparsity information that LTH derives from the uncompressed DNN model to empirically find a winning ticket subnetwork. At step 206, the one or more hardware processors 104 are configured to calculate distribution of sparsity at each layer and at each component of the MLTH. Sparsity of a layer f_i is defined according to equation 1 and sparsity of a component c_j is calculated by equation 2. In equation 1, W_i is a weight matrix of ith layer in the parent uncompressed DNN model and W_(i,sparse) is its corresponding sparsified weight matrix of the ith layer in MLTH (model output of LTH process). ?Sparsity?_fi = (|W_(i,sparse) |)/(?|W?_i |) .....(1) ?Sparsity?_cj= (?_(i;f_(i ) ?_cj)¦?|W_(i,sparse) |?)/(?_(i;f_(i ) ?_cj)¦?|W_i |?) .....(2) At step 208 of the method 200, the one or more hardware processors 104 are configured to reduce architecture of the sparsified model MLTH by first removing its layers based on the calculated sparsity distribution according to equation 3. Remove the f_i ?_j where ?Sparsity?_cj > s (3) After removing the layers, for each remaining convolutional layer f_k, number of kernels/filters, and their sizes are modified from their initial values in h_k in proportion to their ?Sparsity?_fk while maintaining the overall input-output invariant to yield h_(k,new). Different permutations of number of filters and filter sizes can effectively result in the same proportional reduction in size as ?Sparsity?_fk. In such a case, the permutation which results in least degradation in performance is chosen. If h_(k,new) violates input-output structure constraints, it is adjusted accordingly. For each dense layer f_q, number of neurons d is modified proportional to ?Sparsity?_fq. If there is a high degradation in classification performance, the number of neurons d are set to increase stepwise as it determines the feature space dimensionality. At step 210, the resultant student model architecture MStudent is given as an output of the method 200. The said MStudent model has now reduced number of layers p where p s. 4: For each remaining convolutional layer f_k, number of kernels/filters, and their sizes are modified from their initial values in h_k in proportion to their ?Sparsity?_fk while maintaining the overall input-output invariant to yield h_(k,new). Different permutations of number of filters and filter sizes can effectively result in the same proportional reduction in size as ?Sparsity?_fk. In such a case, the permutation which results in least degradation in performance is chosen. If h_(k,new) violates input-output structure constraints, it is adjusted accordingly. 5: For each dense layer f_q, number of neurons d is modified proportional to ?Sparsity?_fq. If there is a high degradation in classification performance, d is increased stepwise as it determines the feature space dimensionality. 6: Output the resultant student model architecture, MStudent which has now reduced number of layers p where p

Documents

Application Documents

# Name Date
1 202321025696-STATEMENT OF UNDERTAKING (FORM 3) [05-04-2023(online)].pdf 2023-04-05
2 202321025696-PROVISIONAL SPECIFICATION [05-04-2023(online)].pdf 2023-04-05
3 202321025696-FORM 1 [05-04-2023(online)].pdf 2023-04-05
4 202321025696-DRAWINGS [05-04-2023(online)].pdf 2023-04-05
5 202321025696-DECLARATION OF INVENTORSHIP (FORM 5) [05-04-2023(online)].pdf 2023-04-05
6 202321025696-FORM-26 [31-05-2023(online)].pdf 2023-05-31
7 202321025696-Proof of Right [01-12-2023(online)].pdf 2023-12-01
8 202321025696-FORM 3 [05-04-2024(online)].pdf 2024-04-05
9 202321025696-FORM 18 [05-04-2024(online)].pdf 2024-04-05
10 202321025696-ENDORSEMENT BY INVENTORS [05-04-2024(online)].pdf 2024-04-05
11 202321025696-DRAWING [05-04-2024(online)].pdf 2024-04-05
12 202321025696-COMPLETE SPECIFICATION [05-04-2024(online)].pdf 2024-04-05
13 Abstract1.jpg 2024-06-06