DESC:
FORM 2
THE PATENTS ACT, 1970
(39 of 1970)
&
THE PATENTS RULES, 2003

COMPLETE SPECIFICATION
(See section 10 and rule 13)
1. TITLE OF THE INVENTION
SYSTEM AND METHOD FOR OPTIMIZING A RATIO OF TRAINING DATASET TO EVALUATION DATASET
2. APPLICANT(S)
NAME NATIONALITY ADDRESS
JIO PLATFORMS LIMITED INDIAN OFFICE-101, SAFFRON, NR. CENTRE POINT, PANCHWATI 5 RASTA, AMBAWADI, AHMEDABAD 380006, GUJARAT, INDIA
3.PREAMBLE TO THE DESCRIPTION

THE FOLLOWING SPECIFICATION PARTICULARLY DESCRIBES THE NATURE OF THIS INVENTION AND THE MANNER IN WHICH IT IS TO BE PERFORMED.

FIELD OF THE INVENTION
[0001] The present invention relates to the field of wireless communication systems, more particularly relates to a method and a system for optimizing a ratio of training dataset to evaluation dataset.
BACKGROUND OF THE INVENTION
[0002] In general, an Artificial Intelligence (AI) model or a Machine Learning (ML) model have gained significant attention due to their ability to analyze vast amounts of data and provide valuable insights and predictions. Training these models requires a substantial amount of labeled data, which is split into a training dataset and an evaluation dataset. The training dataset is used to teach the model, while the evaluation dataset is employed to evaluate the model's performance.
[0003] Traditionally, the ratio of the training dataset to the evaluation dataset has been determined based on arbitrary or heuristic methods. For instance, a common practice is to allocate 70% of the available data to the training dataset and the remaining 30% to the evaluation dataset. However, this approach does not consider the unique characteristics and complexities of the data being used or the specific requirements of the AI/ML model. Consequently, suboptimal ratios may be chosen, leading to less accurate models, longer training times, and inefficient resource utilization.
[0004] Moreover, choosing an inappropriate ratio can result in issues such as overfitting or underfitting. Overfitting occurs when the model learns the training dataset too well, causing it to perform poorly on unseen data. Underfitting, on the other hand, happens when the model fails to capture the underlying patterns in the data, leading to a lack of generalization.
[0005] There is, therefore, a need for a system and method to optimize the ratio of the training dataset to the evaluation dataset for an AI/ML model. By determining the ideal ratio based on the specific characteristics of the data and the requirements of the model, the performance and accuracy of the AI/ML model can be significantly improved.
SUMMARY OF THE INVENTION
[0006] One or more embodiments of the present disclosure provide a method and a system for optimizing a ratio of a training dataset to an evaluation dataset.
[0007] In one aspect of the present invention, a method for optimizing a ratio of the training dataset to the evaluation dataset is disclosed. The method includes the step of receiving the training dataset and the evaluation dataset from one or more sources. The method further includes the step of feeding the training dataset to a model. The method further includes the step of training the model utilizing at least one of, the fed training dataset and one or more parameters. The method further includes the step of feeding the evaluation dataset to the trained model. The method further includes the step of generating, utilizing the trained model, results based on the fed evaluation dataset. The method further includes the step of feeding the generated results as a feedback to the trained model. The method further includes the step of dynamically autotuning the one or more parameters pertaining to the model and splitting ratio of training dataset to evaluation datasets based on the feedback fed to the trained model.
[0008] In one embodiment, the one or more parameters includes at least one of, a learning rate, a number of epochs, and weight decay.
[0009] In another embodiment, the feedback includes at least one of, a positive feedback, and a negative feedback.
[0010] In yet another embodiment, if the trained model generates a more accurate result, then the positive feedback is fed to the model.
[0011] In yet another embodiment, if the trained model generates a lesser accurate result, then the negative feedback is fed to the model.
[0012] In yet another embodiment, the one or more processors continuously monitor the model's performance during training.
[0013] In yet another embodiment, the splitting ratio is the ratio of the training dataset to evaluation dataset.
[0014] In another aspect of the present invention, a system for optimizing a ratio of a training dataset to an evaluation dataset is disclosed. The system includes a transceiver configured to receive the training dataset and the evaluation dataset from one or more sources. The system further includes a feeding unit configured to feed the training dataset to a model. The system further includes a training unit configured to train the model utilizing at least one of, the fed training dataset and one or more parameters. The system further includes the feeding unit configured to feed the evaluation dataset to the trained model. The system further includes a generating unit configured to generate, utilizing the trained model, results based on the fed evaluation dataset. The system further includes a feedback unit configured to feed the generated results as a feedback to the trained model. The system further includes a tuner configured to dynamically autotuning the one or more parameters pertaining to the model and splitting ratio of training dataset to evaluation datasets based on the feedback fed to the trained model.
[0015] In yet another aspect of the present invention, a non-transitory computer-readable medium having stored thereon computer-readable instructions that, when executed by a processor. The processor is configured to receive a training dataset and an evaluation dataset from one or more sources. The processor is further configured to feed the training dataset to a model. The processor is further configured to train the model utilizing at least one of, the fed training dataset and one or more parameters. The processor is further configured to feed the evaluation dataset to the trained model. The processor is further configured to generate, utilizing the trained model, results based on the fed evaluation dataset. The processor is further configured to feed the generated results as a feedback to the trained model. The processor is further configured to dynamically autotune the one or more parameters pertaining to the model and splitting ratio of training dataset to evaluation datasets based on the feedback fed to the trained model.
[0016] Other features and aspects of this invention will be apparent from the following description and the accompanying drawings. The features and advantages described in this summary and in the following detailed description are not all-inclusive, and particularly, many additional features and advantages will be apparent to one of ordinary skill in the relevant art, in view of the drawings, specification, and claims hereof. Moreover, it should be noted that the language used in the specification has been principally selected for readability and instructional purposes and may not have been selected to delineate or circumscribe the inventive subject matter, resort to the claims being necessary to determine such inventive subject matter.
BRIEF DESCRIPTION OF THE DRAWINGS
[0017] The accompanying drawings, which are incorporated herein, and constitute a part of this disclosure, illustrate exemplary embodiments of the disclosed methods and systems in which like reference numerals refer to the same parts throughout the different drawings. Components in the drawings are not necessarily to scale, emphasis instead being placed upon clearly illustrating the principles of the present disclosure. Some drawings may indicate the components using block diagrams and may not represent the internal circuitry of each component. It will be appreciated by those skilled in the art that disclosure of such drawings includes disclosure of electrical components, electronic components or circuitry commonly used to implement such components.
[0018] FIG. 1 is an exemplary block diagram of an environment for optimizing a ratio of training dataset to evaluation dataset, according to one or more embodiments of the present invention;
[0019] FIG. 2 is an exemplary block diagram of a system for optimizing the ratio of training dataset to evaluation dataset, according to one or more embodiments of the present invention;
[0020] FIG. 3 is an exemplary block diagram of architecture for optimizing the ratio of training dataset to evaluation dataset, according to one or more embodiments of the present invention;
[0021] FIG. 4 is an exemplary flow diagram illustrating the flow for optimizing the ratio of training dataset to evaluation dataset, according to one or more embodiments of the present disclosure; and
[0022] FIG. 5 is a flow diagram of a method for optimizing the ratio of training dataset to evaluation dataset, according to one or more embodiments of the present invention.
[0023] The foregoing shall be more apparent from the following detailed description of the invention.
DETAILED DESCRIPTION OF THE INVENTION
[0024] Some embodiments of the present disclosure, illustrating all its features, will now be discussed in detail. It must also be noted that as used herein and in the appended claims, the singular forms "a", "an" and "the" include plural references unless the context clearly dictates otherwise.
[0025] Various modifications to the embodiment will be readily apparent to those skilled in the art and the generic principles herein may be applied to other embodiments. However, one of ordinary skill in the art will readily recognize that the present disclosure including the definitions listed here below are not intended to be limited to the embodiments illustrated but is to be accorded the widest scope consistent with the principles and features described herein.
[0026] A person of ordinary skill in the art will readily ascertain that the illustrated steps detailed in the figures and here below are set out to explain the exemplary embodiments shown, and it should be anticipated that ongoing technological development will change the manner in which particular functions are performed. These examples are presented herein for purposes of illustration, and not limitation. Further, the boundaries of the functional building blocks have been arbitrarily defined herein for the convenience of the description. Alternative boundaries can be defined so long as the specified functions and relationships thereof are appropriately performed. Alternatives (including equivalents, extensions, variations, deviations, etc., of those described herein) will be apparent to persons skilled in the relevant art(s) based on the teachings contained herein. Such alternatives fall within the scope and spirit of the disclosed embodiments.
[0027] The present disclosure describes optimizing a ratio of a training dataset to an evaluation dataset. In accordance with the exemplary embodiment, a feedback-mechanism may continuously monitor a model's performance during training. Further dynamically adjusting the ratio of the training dataset to the evaluation dataset based on real-time feedback, the system adapts to changes in data patterns and ensures optimal model performance throughout the training process.
[0028] Referring to FIG. 1, FIG. 1 illustrates an exemplary block diagram of an environment 100 for optimizing a ratio of the training dataset to the evaluation dataset, according to one or more embodiments of the present invention. The environment 100 includes, a User Equipment (UE) 102, a server 104, a communication network 106, and a system 108. The UE 102 aids the user to interact with the system 108 for providing the training dataset and the evaluation dataset to the system 108. In an embodiment, the user includes, at least one of, a network operator.
[0029] For the purpose of description and explanation, the description will be explained with respect to one or more user equipment’s (UEs) 102, or to be more specific will be explained with respect to a first UE 102a, a second UE 102b, and a third UE 102c, and should nowhere be construed as limiting the scope of the present disclosure. Each of the at least one UE 102 namely the first UE 102a, the second UE 102b, and the third UE 102c is configured to connect to the server 104 via the communication network 106.
[0030] In an embodiment, each of the first UE 102a, the second UE 102b, and the third UE 102c is one of, but not limited to, any electrical, electronic, electro-mechanical or an equipment and a combination of one or more of the above devices such as smartphones, Virtual Reality (VR) devices, Augmented Reality (AR) devices, laptop, a general-purpose computer, desktop, personal digital assistant, tablet computer, mainframe computer, or any other computing device.
[0031] The communication network 106 includes, by way of example but not limitation, one or more of a wireless network, a wired network, an internet, an intranet, a public network, a private network, a packet-switched network, a circuit-switched network, an ad hoc network, an infrastructure network, a Public-Switched Telephone Network (PSTN), a cable network, a cellular network, a satellite network, a fiber optic network, or some combination thereof. The communication network 106 may include, but is not limited to, a Third Generation (3G), a Fourth Generation (4G), a Fifth Generation (5G), a Sixth Generation (6G), a New Radio (NR), a Narrow Band Internet of Things (NB-IoT), an Open Radio Access Network (O-RAN), and the like.
[0032] The communication network 106 may also include, by way of example but not limitation, at least a portion of one or more networks having one or more nodes that transmit, receive, forward, generate, buffer, store, route, switch, process, or a combination thereof, etc. one or more messages, packets, signals, waves, voltage or current levels, some combination thereof, or so forth.
[0033] The environment 100 includes the server 104 accessible via the communication network 106. The server 104 may include by way of example but not limitation, one or more of a standalone server, a server blade, a server rack, a bank of servers, a server farm, hardware supporting a part of a cloud service or system, a home server, hardware running a virtualized server, a processor executing code to function as a server, one or more machines performing server-side functionality as described herein, at least a portion of any of the above, some combination thereof. In an embodiment, the entity may include, but is not limited to, a vendor, a network operator, a company, an organization, a university, a lab facility, a business enterprise side, a defense facility side, or any other facility that provides service.
[0034] The environment 100 further includes the system 108 communicably coupled to the server 104, and the UE 102 via the communication network 106. The system 108 is adapted to be embedded within the server 104 or is embedded as the individual entity.
[0035] Operational and construction features of the system 108 will be explained in detail with respect to the following figures.
[0036] FIG. 2 is an exemplary block diagram of the system 108 for optimizing the ratio of a training dataset to an evaluation dataset, according to one or more embodiments of the present invention.
[0037] As per the illustrated and preferred embodiment, the system 108 for optimizing the ratio of the training dataset to the evaluation dataset, the system 108 includes one or more components such as one or more processors 202, a memory 204, and a storage unit 206. The one or more processors 202, hereinafter referred to as the processor 202, may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuitries, single board computers, and/or any devices that manipulate signals based on operational instructions. However, it is to be noted that the system 108 may include multiple processors as per the requirement and without deviating from the scope of the present disclosure. Among other capabilities, the processor 202 is configured to fetch and execute computer-readable instructions stored in the memory 204.
[0038] As per the illustrated embodiment, the processor 202 is configured to fetch and execute computer-readable instructions stored in the memory 204 as the memory 204 is communicably connected to the processor 202. The memory 204 is configured to store one or more computer-readable instructions or routines in a non-transitory computer-readable storage medium, which may be fetched and executed for optimizing the ratio of the training dataset to the evaluation dataset. The memory 204 may include any non-transitory storage device including, for example, volatile memory such as RAM, or non-volatile memory such as disk memory, EPROMs, FLASH memory, unalterable memory, and the like.
[0039] As per the illustrated embodiment, the storage unit 206 is configured to store data pertaining to the training dataset and the evaluation dataset. The storage unit 206 is one of, but not limited to, a centralized database, a cloud-based database, a commercial database, an open-source database, a distributed database, an end-user database, a graphical database, a No-Structured Query Language (NoSQL) database, an object-oriented database, a personal database, an in-memory database, a document-based database, a time series database, a wide column database, a key value database, a search database, a cache databases, and so forth. The foregoing examples of the storage unit 206 types are non-limiting and may not be mutually exclusive e.g., the database can be both commercial and cloud-based, or both relational and open-source, etc.
[0040] In order for the system 108 to optimize the ratio of the training dataset to the evaluation dataset, the processor 202 includes a transceiver 208, a feeding unit 210, a training unit 212, a generating unit 214, a feedback unit 216, a tuner 218, and an Artificial Intelligence/Machine Learning (AI/ML) model 220. The processor 202 is communicably coupled to the one or more components of the system 108 such as the storage unit 206, and the memory 204. In an embodiment, operations and functionalities of the transceiver 208, the feeding unit 210, the training unit 212, the generating unit 214, the feedback unit 216, the tuner 218, the AI/ML model 220 and the one or more components of the system 108 can be used in combination or interchangeably.
[0041] In an embodiment, the transceiver 208 of the processor 202 is configured to receive the training dataset and the evaluation dataset from one or more sources. The one or more sources includes at least one of, but not limited to, a user such as a network operator which provides input via the UE 102. In particular, a user may have a dataset from which the user may segregate the training dataset and the evaluation dataset. For example, user may provide some part of the data from the dataset as the training dataset to the processor 202 and remaining part of the data from the dataset as the evaluation dataset to the processor 202. In an alternate embodiment, the user may provide historical data pertaining to the training dataset and the evaluation dataset to the processor 202.
[0042] In one embodiment, the training data is the subset of original data i.e. the dataset that is used to train the model. Hereinafter, the model referred to as the AI/ML model 220. The training dataset is one of the inputs provided to the AI/ML model 220 by the one or more sources. For example, the training data facilitates the AI/ML model 220 to learn patterns in order to execute a particular task. In one embodiment, the evaluation dataset is a testing dataset which is used to check the accuracy of the AI/ML model 220 after the training of the AI/ML model 220. The testing dataset is used to evaluate the AI/ML model 220 performance.
[0043] In an embodiment, upon reception of the training dataset and the evaluation dataset from the one or more sources, the feeding unit 210 of the processor 202 is configured to feed the training dataset to the AI/ML model 220. In other words, the training dataset is provided as an input to the AI/ML model 220.
[0044] Upon feeding the training dataset to the AI/ML model 220, the training unit 212 is configured to train the AI/ML model 220 utilizing at least one of the fed training dataset. In particular, based on providing the training dataset to the AI/ML model 220, the training unit 212 trains the AI/ML model 220. While training, the AI/ML model 220 learns the trends/patterns pertaining to the training dataset. In one embodiment, the the AI/ML model 220 is trained by the training unit 212 based on one or more parameters. The one or more parameters are also known as training parameters or hyperparameters. In one embodiment, one or more parameters are predefined by the one or more sources such as the user. For example, the AI/ML model 220 accepts the one or more parameters from the user, that allow user to control at least one of, but not limited to, a learning rate of the AI/ML model 220.
[0045] In one embodiment, the one or more parameters includes at least one of, but not limited to, the learning rate, a number of epochs, and weight decay. In particular, the learning rate is a hyper-parameter used to govern the pace at which an AI/ML model 220 updates or learns estimated values of the one or more parameters. In particular, the number of epochs is a hyperparameter that defines the number of times that the learning AI/ML model 220 will work through the entire training dataset. The number of epochs is traditionally large, often hundreds or thousands, allowing one or more logics to run until the error from the AI/ML model 220 has been sufficiently minimized. In particular, the weight decay is a technique that reduces overfitting by adding a penalty term to a loss function of the AI/ML model 220. The overfitting is an undesirable machine learning behavior that occurs when the AI/ML model 220 gives accurate predictions for training data but not for new data. The loss function is a measurement of how good the AI/ML model 220 is in terms of predicting the expected result/output.
[0046] In an embodiment, an ideal ratio of the training dataset to the evaluation dataset ensures optimal performance of the AI/ML model 220 throughout the training process. Further to obtain the ideal ratio of the training dataset to the evaluation dataset, the one or more parameters are automatically adjusted by the AI/ML model 220.
[0047] In one embodiment, subsequent to training the AI/ML model 220, the feeding unit 210 of the processor 202 is further configured to feed the evaluation dataset to the AI/ML model 220. In other words, the evaluation dataset is provided as another input to the AI/ML model 220 in order to understand the AI/ML model 220 performance. The feeding of the evaluation dataset to the AI/ML model 220 is inferred as the model evaluation process which facilitates in assessing the accuracy and efficiency of the AI/ML model 220, and also plays a role in the AI/ML model 220 monitoring. In one embodiment, the feeding unit 210 continuously monitors the performance of the AI/ML model 220. For example, the AI/ML model 220 monitoring helps the user to track performance shifts. In other words, the user can determine how well the AI/ML model 220 performs. In one scenario, if one or more issues related to the AI/ML model 220 are detected by the processor 202 then the feeding unit 210 facilitates user to understand how to debug the detected one or more issues effectively. In particular, the feeding unit 210 enables users to identify and eliminate the one or more issues, including at least one of, but not limited to, bad quality predictions and a poor technical performance.
[0048] In one embodiment, the generating unit 214 of the processor 202 is further configured to generate results based on the fed evaluation dataset utilizing the trained AI/ML model 220. In particular, the generated results facilitate the user to determine the performance of the AI/ML model 220. In one embodiment, the generating unit 214 generates a more accurate result or a lesser accurate result.
[0049] Furthermore, the feedback unit 216 of the processor 202 is configured to feed the generated results as a feedback to the trained AI/ML model 220. In one embodiment, the feedback includes at least one of, but not limited to, a positive feedback, and a negative feedback. In particular, when the trained AI/ML model 220 generates the more accurate result then the positive feedback is provided to the AI/ML model 220. Similarly, when the trained AI/ML model 220 generates the lesser accurate result then the negative feedback is provided to the AI/ML model 220. In one embodiment, the feedback unit 216 is further configured to continuously monitor the AI/ML model 220 performance during training without deviating from the scope of the invention.
[0050] Thereafter, the tuner 218 of the processor 202 is configured to dynamically autotune the one or more parameters pertaining to the AI/ML model 220 and a splitting ratio of training dataset to evaluation datasets based on the feedback fed to the trained AI/ML model 220 by the feedback unit 216. In particular, the generated results are fed to the tuner 218 based on which the tuner 218 modifies/tunes the splitting ratio and the one or more parameters. Thereafter, the tuned splitting ratio and the tuned one or more parameters are provided to the AI/ML model 220 by at least one of the tuner 218 and the feedback unit 216. In particular, the ratio of the training dataset and the evaluation dataset is considered as the splitting ratio. For example, the splitting ratio may be 70% training dataset and the remaining 30% is considered as the evaluation dataset. The optimized splitting ratio facilitates the AI/ML model 220 to generate the result/outputs with high accuracy. By autotuning the splitting ratio based on the one or more parameters the performance and accuracy of the AI/ML model 220 is significantly improved. In other words, the optimal splitting ratio leads to improved output in future.
[0051] The transceiver 208, the feeding unit 210, the training unit 212, the generating unit 214, the feedback unit 216, the tuner 218, and the AI/ML model 220 in an exemplary embodiment, are implemented as a combination of hardware and programming (for example, programmable instructions) to implement one or more functionalities of the processor 202. In the examples described herein, such combinations of hardware and programming may be implemented in several different ways. For example, the programming for the processor 202 may be processor-executable instructions stored on a non-transitory machine-readable storage medium and the hardware for the processor may comprise a processing resource (for example, one or more processors), to execute such instructions. In the present examples, the memory 204 may store instructions that, when executed by the processing resource, implement the processor 202. In such examples, the system 108 may comprise the memory 204 storing the instructions and the processing resource to execute the instructions, or the memory 204 may be separate but accessible to the system 108 and the processing resource. In other examples, the processor 202 may be implemented by electronic circuitry.
[0052] FIG. 3 illustrates an exemplary block diagram of an architecture for optimizing the ratio of the training dataset to the evaluation dataset, according to one or more embodiments of the present invention. More specifically, FIG. 3 illustrates the system 108 for optimizing the ratio of the training dataset to the evaluation dataset. It is to be noted that the embodiment with respect to FIG. 3 will be explained with respect to the UE 102 for the purpose of description and illustration and should nowhere be construed as limited to the scope of the present disclosure.
[0053] FIG. 3 shows communication between the UE 102, and the system 108. For the purpose of description of the exemplary embodiment as illustrated in FIG. 3, the UE 102 uses a network protocol connection to communicate with the system 108. In an embodiment, the network protocol connection is the establishment and management of communication between the UE 102 and the system 108 over the communication network 106 (as shown in FIG. 1) using a specific protocol or set of protocols. The network protocol connection includes, but not limited to, Session Initiation Protocol (SIP), System Information Block (SIB) protocol, Transmission Control Protocol (TCP), User Datagram Protocol (UDP), File Transfer Protocol (FTP), Hypertext Transfer Protocol (HTTP), Simple Network Management Protocol (SNMP), Internet Control Message Protocol (ICMP), Hypertext Transfer Protocol Secure (HTTPS) and Terminal Network (TELNET).
[0054] In an embodiment, the UE 102 includes a primary processor 302, and a memory 304 and a User Interface 306. In alternate embodiments, the UE 102 may include more than one primary processor 302 as per the requirement of the communication network 106. The primary processor 302, may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuitries, single board computers, and/or any devices that manipulate signals based on operational instructions.
[0055] In one embodiment, the User Interface (UI) 306 includes a variety of interfaces, for example, interfaces for a Graphical User Interface (GUI), a web user interface, a Command Line Interface (CLI), and the like. The UI 306 facilitates the user to communicate with the system 108. In one embodiment, the UI 306 provides a communication pathway between the user and the one or more components of the system 108. In one embodiment, the UI 306 may be integrated within the UE 102 or may be integrated within the system 108.
[0056] In an embodiment, the primary processor 302 is configured to fetch and execute computer-readable instructions stored in the memory 304. The memory 304 may be configured to store one or more computer-readable instructions or routines in a non-transitory computer-readable storage medium, which may be fetched and executed to transmits the training dataset and the evaluation dataset via the UE 102 to the one or more processors 202 for training the AI/ML model 220. The memory 304 may include any non-transitory storage device including, for example, volatile memory such as RAM, or non-volatile memory such as disk memory, EPROMs, FLASH memory, unalterable memory, and the like.
[0057] For example, whenever the user interacts with the system 108 via the UI 306 of the UE 102, the user may select at least one AI/ML model 220 from a list of plurality of models. Further, the user provides the training data to the training unit 212 in order to train the AI/ML model 220. Thereafter, the user provides the evaluation data to the AI/ML model 220 to execute a particular task. Based on training, the AI/ML model 220 performs the particular task and generates an output which is fed as the feedback to the AI/ML model 220. Based on the positive or negative feedback, the AI/ML model 220 auto tunes the at least one of, the one or more parameters and the splitting ratio utilizing the tuner 218 in order to improve the performance of the AI/ML model 220 which leads to improved output in future.
[0058] As mentioned earlier in FIG.2, the system 108 includes the processors 202, the memory 204, the storage unit 206, for optimizing the ratio of the training dataset to the evaluation dataset are already explained in FIG. 2. For the sake of brevity, a similar description related to the working and operation of the system 108 as illustrated in FIG. 2 has been omitted to avoid repetition.
[0059] Further, the processor 202 includes the transceiver 208, the feeding unit 210, the training unit 212, the generating unit 214, the feedback unit 216, the tuner 218, and the AI/ML model 220 which are already explained in FIG. 2. Hence, for the sake of brevity, a similar description related to the working and operation of the system 108 as illustrated in FIG. 2 has been omitted to avoid repetition. The limited description provided for the system 108 in FIG. 3, should be read with the description provided for the system 108 in the FIG. 2 above, and should not be construed as limiting the scope of the present disclosure.
[0060] FIG. 4 is an exemplary architecture illustrating the flow for optimizing the ratio of the training dataset to the evaluation dataset, according to one or more embodiments of the present disclosure.
[0061] Initially, one or more parameters for training the AI/ML model 220 are defined by the user via the UI 306 of the UE 102. Then a training dataset is fed by the user to the AI/ML model 220 in order to train the AI/ML model 220. Before feeding the training dataset to the AI/ML model 220, the training dataset is preprocessed by the processor 202. In particular, the preprocessing of the training dataset includes at least one of, but not limited to, data normalization. The data normalization is the process of at least one of, but not limited to, reorganizing training dataset, removing the redundant data from the training dataset, and removing null values from the training data. The main goal of data normalization is to achieve a standardized data format across the entire system 108. Thereafter, the normalized training dataset is provided to the training unit 212 in order to train the AI/ML model 220. Herin, the training unit 212 is integrated within the AI/ML model 220. While training, the AI/ML model 220 learns trends/patterns pertaining to the training dataset.
[0062] Further, the evaluation dataset is fed by the user via the UI 306 to the AI/ML model 220 in order to evaluate the performance of the AI/ML model 220. Before feeding the evaluation dataset to the AI/ML model 220, the evaluation dataset is preprocessed by the processor 202 similar to the preprocessing of the training dataset. Based on the fed evaluation dataset, the generating unit 214 generates the output/results utilizing the trained AI/ML model 220. Herein, the generating unit 214 is integrated within the AI/ML model 220. Thereafter, the feedback unit 216 receives the generated output/results and the tuner 218 tunes the one or more parameter and the splitting ratio based on the generated output/results, in such a way that performance of the AI/ML model 220 is improved when the tuned one or more parameter and the splitting ratio is provided to the AI/ML model 220. Advantageously, by continuously training the AI/ML model 220 utilizing the generated output/results, an optimal splitting ratio is achieved.
[0063] FIG. 5 is a flow diagram of a method 500 for optimizing the ratio of the training dataset to the evaluation dataset, according to one or more embodiments of the present invention. For the purpose of description, the method 500 is described with the embodiments as illustrated in FIG. 2 and should nowhere be construed as limiting the scope of the present disclosure.
[0064] At step 502, the method 500 includes the step of receiving the training dataset and the evaluation dataset from one or more sources. In one embodiment, the transceiver 208 of the processor 202 is configured to receive the training dataset and the evaluation dataset from one or more sources from the user via the UE 102. For example, the user may had stored a dataset in a particular storage unit and thereafter the user may provide 75% of the data among the dataset as the training dataset to the processor 202 and 25% of the data among the dataset as the evaluation dataset to the processor 202.
[0065] At step 504, the method 500 includes the step of feeding, the training dataset to the AI/ML model 220. In one embodiment, the the training dataset is preprocessed before feeding to the AI/ML model 220. In particular, the feeding unit 210 of the processor 202 is configured to feed the training dataset to the AI/ML model 220. For example, the 75% of the data among the dataset provided by the user is fed by the feeding unit 210 as the training dataset to the AI/ML model 220.
[0066] At step 506, the method 500 includes the step of training, the AI/ML model 220 utilizing at least one of, the fed training dataset and one or more parameters. In one embodiment, the feeding unit 210 of the processor 202 feeds the training dataset to the training unit 212 and further the training unit 212 trains the AI/ML model 220 utilizing at least one of, the fed training dataset and the one or more parameters. For example, initially the user may predefine the one or more parameters in order to train the AI/ML model 220. Thereafter, the AI/ML model 220 is trained on the fed training dataset i.e. 75% of the data provided by the user and the one or more parameters predefined by the user via the UI 306.
[0067] At step 508, the method 500 includes the step of feeding, the evaluation dataset to the trained AI/ML model 220. In one embodiment, the feeding unit 210 of the processor 202 is further configured to feed the evaluation dataset to the the AI/ML model 220. For example, the 25% of the data among the dataset is fed by the feeding unit 210 as the evaluation dataset to the trained AI/ML model 220 in order to evaluate the performance of the the trained AI/ML model 220.
[0068] At step 510, the method 500 includes the step of generating, utilizing the trained model, results based on the fed evaluation dataset. In one embodiment, the generating unit 214 of the processor 202 is configured to generate results utilizing the trained AI/ML model 220 based on the fed evaluation dataset. In particular, the evaluation dataset is the testing dataset which is fed to AI/ML model 220 to check the performance of the trained AI/ML model 220. In one embodiment, the generation unit 214 generates the more accurate result or the lesser accurate result. The user may compare the result with a predefined result or a predicted result in order to determine whether the generated result is the more accurate result or the lesser accurate result. In alternate embodiment, the generation unit 214 may compare the result with the predefined result or the predicted result in order to determine whether the generated result is the more accurate result or the lesser accurate result.
[0069] At step 512, the method 500 includes the step of feeding, the generated results as a feedback to the trained AI/ML model 220. In one embodiment, the feedback unit 216 of the processor 202 is configured to feed the generated results as the feedback to the trained AI/ML model 220. In particular, the feedback unit 216 may feed the generated results as the feedback to the tuner 218 which may be the integrated with the trained AI/ML model 220. In particular, when the generated result is the more accurate result, then the positive feedback is provided to the trained AI/ML model 220. Similarly, when the generated result is the less accurate result, then negative feedback is provided to the trained AI/ML model 220. For example, the positive feedback infers that the trained AI/ML model 220 can used for further usage and the negative feedback infers that the trained AI/ML model 220 needs more training.
[0070] At step 514, the method 500 includes the step of dynamically autotuning, the one or more parameters pertaining to the model and the splitting ratio of training dataset to evaluation datasets based on the feedback fed to the trained AI/ML model 220. In one embodiment, the tuner 218 of the processor 202 is configured to dynamically autotune the one or more parameters pertaining to the AI/ML model 220 and splitting ratio of training dataset to evaluation datasets based on the feedback fed to the trained AI/ML model 220. In particular, the feedback unit 216 may feed the generated results as the feedback to the tuner 218 which may be the integrated with the trained AI/ML model 220.
[0071] Based on the positive feedback or the negative feedback, the tuner 218 tunes the splitting ratio and the one or more parameters defined by the user. For example, if the positive feedback is received from the AI/ML model 220, then the tuner 218 may infer that the one or more parameters and the splitting ratio provided by the user such as the 75% of the training dataset to 25% of the evaluation dataset is appropriate. Alternatively, if the negative feedback is received from the AI/ML model 220, then the tuner 218 tunes the one or more parameters and the splitting ratio provided by the user to 65% training dataset and 35% evaluation dataset, so when the training of the AI/ML model 220 is performed utilizing the 65% training dataset and evaluating the trained AI/ML model 220 utilizing the 35% evaluation dataset, the AI/ML model 220 generates more accurate results.
[0072] In alternate embodiment, if the positive feedback is received from the AI/ML model 220, then also the tuner 218 may tune the splitting ratio and the one or more parameters in order to get more accurate results from the AI/ML model 220.
[0073] In one embodiment, the tuner 218 continuously tunes at least one of, the one or more parameters and the splitting ratio until the optimum splitting ratio is achieved, which may be utilized by the AI/ML model 220 to generate more accurate results. Advantageously, due to the autotuning of the one or more parameters and the splitting ratio, the time required for training the AI/ML model 220 is reduced.
[0074] The present invention further discloses a non-transitory computer-readable medium having stored thereon computer-readable instructions. The computer-readable instructions are executed by the processor 202. The processor 202 is configured to receive a training dataset and an evaluation dataset from one or more sources. The processor 202 is further configured to feed the training dataset to a model 220. The processor 202 is further configured to train the model 220 utilizing at least one of, the fed training dataset and one or more parameters. The processor 202 is further configured to feed the evaluation dataset to the trained model 220. The processor 202 is further configured to generate, utilizing the trained model 220, results based on the fed evaluation dataset. The processor 202 is further configured to feed the generated results as a feedback to the trained model 220. The processor 202 is further configured to dynamically autotune the one or more parameters pertaining to the model 220 and splitting ratio of training dataset to evaluation datasets based on the feedback fed to the trained model 220.
[0075] A person of ordinary skill in the art will readily ascertain that the illustrated embodiments and steps in description and drawings (FIG.1-5) are set out to explain the exemplary embodiments shown, and it should be anticipated that ongoing technological development will change the manner in which particular functions are performed. These examples are presented herein for purposes of illustration, and not limitation. Further, the boundaries of the functional building blocks have been arbitrarily defined herein for the convenience of the description. Alternative boundaries can be defined so long as the specified functions and relationships thereof are appropriately performed. Alternatives (including equivalents, extensions, variations, deviations, etc., of those described herein) will be apparent to persons skilled in the relevant art(s) based on the teachings contained herein. Such alternatives fall within the scope and spirit of the disclosed embodiments.
[0076] The present disclosure provides technical advancement such as optimal ratio of training dataset to evaluation dataset. Further, the user’s time and efforts are reduced as the system automatically adjusts the training parameters and the ratio of training dataset to evaluation dataset thereby optimum splitting ratio is achieved. Due to which the time required for training the model is reduced. Further the best results are obtained in terms of accuracy and resource utilization is most efficient without any extra effort.
[0077] The present invention offers multiple advantages over the prior art and the above listed are a few examples to emphasize on some of the advantageous features. The listed advantages are to be read in a non-limiting manner.

REFERENCE NUMERALS

[0078] Environment - 100;
[0079] User Equipment (UE) - 102;
[0080] Server - 104;
[0081] Communication network- 106;
[0082] System -108;
[0083] Processor - 202;
[0084] Memory - 204;
[0085] Storage unit – 206;
[0086] Transceiver – 208;
[0087] Feeding unit– 210;
[0088] Training unit – 212;
[0089] Generating unit - 214;
[0090] Feedback unit -216;
[0091] Tuner – 218;
[0092] AI/ML model – 220;
[0093] Primary processor – 302;
[0094] Memory – 304;
[0095] User Interface –306.

,CLAIMS:CLAIMS
We Claim:
1. A method (500) for optimizing a ratio of training dataset to evaluation dataset, the method (500) comprising the steps of:
receiving, by one or more processors (202), the training dataset and the evaluation dataset from one or more sources;
feeding, by the one or more processors (202), the training dataset to a model (220);
training, by the one or more processors (202), the model (220) utilizing at least one of, the fed training dataset and one or more parameters;
feeding, by the one or more processors (202), the evaluation dataset to the trained model (220);
generating, by the one or more processors (202), utilizing the trained model (220), results based on the fed evaluation dataset;
feeding, by the one or more processors (202), the generated results as a feedback to the trained model (220); and
dynamically autotuning, by the one or more processors (202), the one or more parameters pertaining to the model (220) and splitting ratio of training dataset to evaluation datasets based on the feedback fed to the trained model (220).

2. The method (500) as claimed in claim 1, wherein the one or more parameters includes at least one of, a learning rate, a number of epochs, weight decay.

3. The method (500) as claimed in claim 1, wherein the feedback includes at least one of, a positive feedback, and a negative feedback.

4. The method (500) as claimed in claim 1, wherein if the trained model (220) generates a more accurate result then the positive feedback is fed to the model (220).

5. The method (500) as claimed in claim 1, wherein if the trained model (220) generates a lesser accurate result then the negative feedback is fed to the model.

6. The method (500) as claimed in claim 1, wherein the one or more processors (202) continuously monitor the model's (220) performance during training.

7. The method (500) as claimed in claim 1, wherein the splitting ratio is the ratio of the training dataset to the evaluation dataset.

8. A system (108) for optimizing a ratio of training dataset to evaluation dataset, the system (108) comprises:
a transceiver (208), configured to, receive, the training dataset and the evaluation dataset from one or more sources;
a feeding unit (210), configured to, feed, the training dataset to a model (220);
a training unit (212), configured to, train, the model utilizing at least one of, the fed training dataset and one or more parameters;
the feeding unit (210), configured to, feed, the evaluation dataset to the trained model (220);
a generating unit (214), configured to, generate, utilizing the trained model (220), results based on the fed evaluation dataset;
a feedback unit (216), configured to, feed, the generated results as a feedback to the trained model (220); and
a tuner (218), configured to, dynamically autotune, the one or more parameters pertaining to the model (220) and splitting ratio of training dataset to evaluation datasets based on the feedback fed to the trained model (220).

9. The system (108) as claimed in claim 8, wherein the one or more parameters includes at least one of, a learning rate, a number of epochs, weight decay.

10. The system (108) as claimed in claim 8, wherein the feedback includes at least one of, a positive feedback, and a negative feedback.

11. The system (108) as claimed in claim 8, wherein if the trained model (220) generates a more accurate result then the positive feedback is fed to the model.

12. The system (108) as claimed in claim 8, wherein if the trained model (220) generates a lesser accurate result then the negative feedback is fed to the model.

13. The system (108) as claimed in claim 8, wherein the feedback unit (216) continuously monitor the model's (220) performance during training.

14. The system (108) as claimed in claim 8, wherein the splitting ratio is the ratio of the training dataset to the evaluation dataset.