1. A method for prediction of automatic software bug using ensemble learning technique, comprising the steps of: separating suggested framework into three phases, wherein each of which is described as preprocessing, ensemble machine learning-based bug prediction and evaluation of performance measure and comparative analysis on different evaluation metrics; wherein the two ensemble machine learning (ML) models based on the bagging (random forests (RFs)) and boosting (AdaBoost) techniques are used to predict the bugs; and wherein the performance of two ensemble learning models has been evaluated based on different quantitative parameters, like F-measure, MCC, AUC etc.

2. The method as claimed in the claim 1, wherein Random forests (RFs) are one of the most successful ensemble algorithms, exhibiting boosting and support vector machine-like performance.

3. The method as claimed in the claim 1, wherein the random forests (RFs) ensemble learning model are a set of ensembles classifiers (trees) and are widely used classification technique in various applications.

4. The method as claimed in the claim 1, wherein AdaBoost is an ensemble learning approach based on boosting i.e., combining weak and inaccurate rules for creating accurate prediction rules.

5. The method as claimed in the claim 1, wherein the traditional machine learning models, such ANN, SVM etc., are performing inefficiently due to their inability in reducing bias, variance and noise. Dated this 27th day of April, 2023 POOJA AGENT FOR THE APPLICANT IN/PA/1838

Description:FIELD OF INVENTION
[0001] The present disclosure relates to a method for prediction of automatic software bug using ensemble learning technique.
BACKGROUND OF INVENTION
[0002] Software Bug prediction (SBP) helps in improving the quality of software to be developed. Bugs can be of different kinds such as functional, usability, security, etc, which leads to irrelevant features, data redundancy and missing samples. SBP is a very popular area in the IT world as this helps in improving the software product.SBP is a complicated task that can be performed both manually (semi-automated) or automatically. The semi-automated task requires extensive knowledge about the product to be developed and manpower. On the other hand, automatic SBP eliminated such issues and improves the bug prediction cost significantly.
[0003] In the present world, the evolution of computational devices and technologies has transformed human life in every aspect and they are more dependent on computing systems. To maximize the utilization of computational capability of high computing machine for reducing human effort there is a need to develop software products at a faster rate, economically and reduce maintainability. While designing large systems it is necessary to find the bugs at the early stages as the cost of fixing the bug is less as compared to later stages and machine learning and its variations have the potential to ease the quality of software product development and improve businesses by making predictions and assisting them in decision-making. It is vital to understand the pros and downsides of machine learning (ML) applicability. MLis a part of computing technologieswidely used to solve complex problems which require intelligenceequivalent to human by learning from their experiences and improve on their own.
[0004] Typically, the ML helps software researchers to keep their focus on how to improve the quality of software products, reduce maintainability and human efforts which lead to more efficient and effective software development. Predicting software flaws early in the software life cycle can help decrease product development and maintenance expenses(Khuat and Le, 2019).The primary objective of SBP techniques is to discovermodules with and without bugs;the developer will then allocate appropriate testing sources and priorities testing of multiple software modules to improve software quality.The study of SBP is categorized into four classes (a) prediction of bugs present in software objects like functions, files, modules etc., by using machine learning and probability-based methods; (b) the second category involves the prediction of number of faults by using regression analysis etc. (c) In the third type of study the relationship between various types of software metrics has been covered to deduces the associations rules and finally the software products are prioritized based on the number of defects presents.
[0005] The machine learning (ML) techniques can play a key role in designing bug-free software by predicting bugs early in the software development processwhich in turn reduces overall development cost. In the Software bug prediction technique researchers studied many machine learning techniques like supervised(Moosavian et al., 2014; Singh et al., 2014a; Arar and Ayan, 2015; Sharma, Sharma and Gujral, 2015; Wang et al., 2021) andunsupervised learning(Zhang, Wang and Wang, 2016; Feng et al., 2021; Goyal, 2022).A fuzzy-based bug prediction model at each phase of SDLC has been proposed that minimizes the difference between predicted and actual(Yadav and Yadav, 2015). Immune theory-based fault prediction framework has been presented that uses a Naïve Bayes classifier for prediction (Soleimani and Asdaghi, 2014). In the Software bug prediction technique, software metrics can be trained by finding out the software efficiency and also checking for the software quality (Ghotra, McIntosh and Hassan, 2015).The exiting techniques for SBP suffers from issues such as The exiting techniques for SBP suffers from issues such as Class imbalance problems (Tong, Liu and Wang, 2018), overfitting issues(Zhu et al., 2021), other problems with SBP are when many attributes are added to the SBP model also Facing an issues of heterogenousity.
[0006] In order to over the above mention limitations, a multistage automatic SBP model is designed to predict the bugs from given software metrics. The proposed invention is to develop a unique method for prediction of automatic software bug using ensemble learning technique, and to identify defective and non-defective modules, which will be provided by the developer as suitable testing sources as well as the testing of numerous software modules in order to enhance the quality of software.
[0007] Thus, it is desired to address the above-mentioned disadvantages or other shortcomings or at least provide a useful alternative.
OBJECTIVE OF INVENTION
[0008] The principal object of the embodiments herein is to identify defective and non-defective modules, which can be provided by a developer as suitable testing sources as well as the testing of numerous software modules in order to enhance the quality of software.
[0009] Another object of the embodiments herein is to improve the quality of software to be developed.
[0010] Another object of the embodiments herein is simple and user friendly.
SUMMARYOF INVENTION
[0011] The following presents a simplified summary in order to provide a basic understanding of some aspects of the disclosed invention. This summary is not an extensive overview, and it is not intended to identify key/critical elements or to delineate the scope thereof. Its sole purpose is to present some concepts in a simplified form as a prelude to the more detailed description that is presented later.
[0012] Accordingly, there is provided to a method for prediction of automatic software bug using ensemble learning technique.
[0013] In accordance with an embodiment of the present subject matter relates to a method for prediction of automatic software bug using ensemble learning technique. The method involves separating suggested framework into three phases where each of which is described as preprocessing, ensemble learning-based bug prediction and evaluation of performance measure and comparative analysis on different evaluation metrics. The two ensemble machine learning models based on the bagging (random forest) and boosting (AdaBoost) techniques are used to predict the bugs. In addition, the performance of two ensemble learning models has been evaluated based on different quantitative parameters, like F-measure, MCC, AUC etc. While in the case of the dataset having less defective percentage, the performance of both classifiers is almost identical.
[0014] The proposed method performance is measured using different parameters obtained from error metrics, like precision, recall, receiver operating characteristic curve (ROC) (Robin et al., 2011), area under the ROC curve (AUC), F-measure (Luque et al., 2019) and Mathews co-relation Coefficient (MCC) (Matthews, 1975) based on the ensemble learning technique.
[0015] These and other aspects of the embodiments herein will be better appreciated and understood when considered in conjunction with the following description and the accompanying drawings. It should be understood, however, that the following descriptions, while indicating preferred embodiments and numerous specific details thereof, are given by way of illustration and not of limitation. Many changes and modifications may be made within the scope of the embodiments herein without departing from the scope thereof, and the embodiments herein include all such modifications.
BRIEF DESCRIPTION OF FIGURES
[0016] These and other features, aspects and advantages of the present invention will be better understood if the following detailed description is read with reference to the accompanying drawings, in which like characters represent like parts in the drawings, where:
[0017] Figure 1 illustrates flow diagram of a method for prediction of automatic software bug using ensemble learning technique, according to the present invention.
[0018] Figure 2 illustrates graphical representation of comparative analysis on accuracy, according to the present invention.
[0019] Figure 3 illustrates graphical representation of comparative analysis on F-Measure on F-measure, according to the present invention.
[0020] Figure 4 illustrates graphical representation of a comparative analysis on MCC, according to the present invention.
[0021] Figure 5 illustrates graphical representation of a comparative analysis on ROC, according to the present invention.
[0022] It will be understood by those skilled in the art that the elements in the drawings are shown for convenience and have not necessarily been drawn to scale. For example, the flowcharts illustrate the process using key steps to enhance understanding of aspects of the present invention. In addition, one or more components of the device may be represented in the drawings by conventional symbols, and the drawings may show only the specific details relevant to an understanding of embodiments of the present invention, so as not to clutter the drawings with details that are readily apparent to those skilled in the art who are familiar with the present description.
DETAILED DESCRIPTION OF INVENTION
[0023] The embodiments herein and the various features and advantageous details thereof are explained more fully with reference to the non-limiting embodiments that are illustrated in the accompanying drawings and detailed in the following description. Descriptions of well-known components and processing techniques are omitted so as to not unnecessarily obscure the embodiments herein. Also, the various embodiments described herein are not necessarily mutually exclusive, as some embodiments can be combined with one or more other embodiments to form new embodiments. The term “or” as used herein, refers to a non-exclusive or, unless otherwise indicated. The examples used herein are intended merely to facilitate an understanding of ways in which the embodiments herein can be practiced and to further enable those skilled in the art to practice the embodiments herein. Accordingly, the examples should not be construed as limiting the scope of the embodiments herein.
[0024] The accompanying drawings are used to help easily understand various technical features and it should be understood that the embodiments presented herein are not limited by the accompanying drawings. As such, the present disclosure should be construed to extend to any alterations, equivalents and substitutes in addition to those which are particularly set out in the accompanying drawings. Although the terms first, second, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are generally only used to distinguish one element from another.
[0025] Accordingly, embodiments here are provided to a method for prediction of automatic software bug using ensemble learning technique.
[0026] Referring Figure 1 illustrates flow diagram of a method for prediction of automatic software bug using ensemble learning technique. The method involves separating suggested framework into three phases where each of which is described as preprocessing, ensemble learning-based bug prediction and evaluation of performance measure and comparative analysis on different evaluation metrics. The two ensemble machine learning models based on the bagging (random forest) and boosting (AdaBoost) techniques are used to predict the bugs. In addition, the performance of two ensemble learning models has been evaluated based on different quantitative parameters, like F-measure, MCC, AUC etc. While in the case of the dataset having less defective percentage, the performance of both classifiers is almost identical.
[0027] Preprocessing: most real-world data has a skewed and asymmetric-distribution of instances available for each class. Due to the problem of asymmetric class distributions, algorithms may train unnecessarily complex models that over fit the data and are of limited use. Despite their superior performance, computational intelligence techniques are directed towards majority class instances, learning more about the majority class while learning very little or nothing about the minority class(Ball, Reay and David, 2002).The most challenging work in nowadays is with imbalanced datasets in the machine learning techniques and can be handled by synthetic minority oversampling technique (SMOTE) in which the minority class is augmented. The SMOTE approach is efficiently employed in combination with the supervised methodsfor bug detection and the proposed method has low computation cost and satisfactory performance(Chen et al., 2019).
[0028] In the domain of bug prediction, the traditional machine learning models, such ANN, SVM etc. are performs inefficiently due to their inability in reducing bias, variance and noise. All these issues are resolved by Ensemble learning models which combine the effort of various individual learning algorithms in predicting outcomes. The general framework for ensemble learning for datasets having samples and features shown in Eq. 1can are expressed and represented in Eq. 2:
(1)
(2)
Where the is ensemble learning model has an aggregation function that combines learning classifiers to generate the aggregate output .
[0029] In the proposed method has two ensemble learning models which are used to predict the bug. The first the theory rests on the bagging (RF) and the other one is based on the boosting approach (Adaboost).The best model has been selected on the basis the different performance evaluation metrics. Before employing the Ensemble learning model in the classification step of proposed method, the data must be divided into training and testing classes, and this information is used to assess the accuracy of model.
[0030] The fold generation approach is used to pick the training and testing datawhich is relatively new and widely used cross-validation technique in ML and has the following steps: Shuffle the data at random and partition into to k groups; for each individualgroup; and consider the some group to be a test data set. All the other groups are for training set of data. On the training set, build the model, then test it on the test set. Analyze the model and, based on the evaluation score, eliminate it; summarize the model's fitness using the sample model validation scores (Immaculate, Begam and Floramary, 2019).
[0031] On the defect prediction models, we utilized 10-fold cross-validation to train the dataset; the input dataset was subdivided into 10 equal-sized folds. Nine out of ten folds (90%) are assigned to the training corpus, with one fold set aside for testing (10 %).The testing sample is used to evaluate model performance while the training corpus is utilized to train models using various classification strategies. This method is performed 10 times, with each fold only acting as the testing corpus once. To confirm our findings, the10-fold process ten times (Ghotra, McIntosh and Hassan, 2015).
[0032] The random forests (RFs) are one of the most successful ensemble algorithms, exhibiting boosting and support vector machine-like performance. The method is quick, noise-resistant, does not over fit, and allows for explanation and visualization of its results. We look into different strategies for increasing or decreasing the strength of particular forest trees' correlations(Robnik-Šikonja, 2004).Whereas, a powerful ensemble of classifiers is generated by successively refitting a weak classifier to different weighted realizations of a data set in the "boosting" technique to machine learning. This simple approach has shown to be extremely effective.
[0033] The performance of the RF model depends on suitable feature selection technique which maximizes the randomness between various classes. The most common methods for feature selection in decision trees are gain-ratio (Quinlan, 1993), Gini-index(Breiman, 2001) and Chi-square (Mingers, 1989).The RF model uses Gini-index for best split selection that measures the impurities of a given sample to the rest of the classes(Rodriguez-Galiano et al., 2012). When developing an RF model, two important parameters to consider are the number of trees (Ntree) and the number of randomly picked features (Nfeatures)(Sheykhmousa et al., 2020). The fewer Nfeature speed up computation but lowers the correlation between any two trees and affects the classification accuracy (Sahin, Colkesen and Kavzoglu, 2020). Ntree depends on the requirement of the specific domain but it has been shown that 500 trees are optimum.
[0034] AdaBoost is an ensemble learning approach based on boosting i.e., combining weak and inaccurate rules for creating accurate prediction rules (Schapire, 2013).It has a strong theoretical foundation which is also known as the rule of thumb and this led to the introduction of a new method and design idea for learning algorithm creations(CAO et al., 2013) by combining numerous weak learners whose accuracies are marginally superior to conventional guessing to create an arbitrarily accurate strong predictor(Saon and Soltau, 2012). AdaBoost and related evolution techniques are applied in a multiple of applications, including text classification (Kowsari et al., 2019), text filtering (Carreras and Marquez, 2001)speech recognition(Park et al., 2011; Saon and Soltau, 2012), human action recognition (Yan and Luo, 2012), object detection (Chen et al., 2011)and remote sensing(McIver and Friedl, 2001). The boosting approach in machine learning is equivalent to logistic regression andis used to minimize the error of any "weak" learning algorithm.
[0035] The AdaBoost is designed specifically for binary class problems and in the work presented in the paper deals with the detection of bugs which is a primarily belong to an above mentioned category and explained in Algorithm 1:
(1) Given a set of training samples ,where and initialize the weight vector as
Repeat step 2 to 6 for T iterations t=1…..T
(2) Set
(3) Obtain weak hypothesis over
(4) Calculate the hypothesis error( ) as,
(5) Set
(6) Update weight vector as,
(7) Finally, AdaBoost Model is developed as given below:

[0036] The bagging based techniques are based on the random selection of rules while in the boosting approach weak rules based on the rule of thumb are used. It has been observed that the classification performance of ensembles learners based on the weak rule is better as compared to random. This improvement in the performance of boosting approach is due to the reduced training error as compared to bagging based techniques which can be expressed as,
(3)
[0037] The proposed method performance is measured using different parameters obtained from error metrics, like precision, recall, receiver operating characteristic curve (ROC) (Robin et al., 2011), area under the ROC curve (AUC), F-measure (Luque et al., 2019) and Mathews co-relation Coefficient (MCC) (Matthews, 1975) based on the ensemble learning technique.
[0038] Precision is a measure that can be determined as the number of correct positive predictions made separatedas the number of correct positive predictions made.
(4)
Where TP stands for True positive and FP stands for false positive respectively. The recall is a measure of how many right positive predictions were made out of all possible positive predictions and can be calculated as,
(5)
The total number of successfully identified bugs divided by the total number of bugs is the definition of accuracy, which is calculated as,
(6)
[0039] The performance of a classification model is depicted by a ROC curve, which shows true positive rate (TPR) vs. false positive rate (FPR) at various classification thresholds. As the categorization threshold is lowered, more objects are classified as positive, increasing both FP and TP. The AUC is classification-threshold-invariant, meaning it shows how well predictions are ordered rather than their absolute values, and it assesses the quality of the model's predictions regardless of the classification threshold used.
[0040] The F-measure is calculated by weighing precision and recall equally in the harmonic mean. It allows a model to also be evaluated using a specific value that provides for both precision and recall, which would be valuable for comparing models and discussing their performance.
(7)
MCC is also an ensemble estimation methodology that takes into account all TP and FP, as well as all TN and FN. The MCC ranges from -1 to 1, with higher values indicating greater efficiency in predicting asymmetric datasets. MCC is calculated as,
(8)

[0041] Referring Figure 2 illustrating graphical representation of comparative analysis on accuracy. The performance of ABPET is discussed in detail on different performance on different quantitative parameters as discussed in section 4.4.1.The comparative analysis ofprecision, recall and accuracy is presented in Table 3. It is being observed that when the number of defects is more in a dataset then the accuracy of AdaBoost is better as compared to the random forest as shown in Table 3 databases like Lucene 2.0, velocity 1.6, log4j1.1, synpose1.2, jedit3.2 etc. In case, when the number of the defective sample is less in a dataset the performance of AdaBoost on accuracy is marginally less as compared to the random forest.
[0042] Referring Figure 3 illustrating graphical representation of comparative analysis on F-Measure on F-measure. In terms of F Measure, it depicts taking into account both precision and recall using a single score to represent the model's performance. The performance of AdaBoost is better as compared to the random forest in terms of F_Measurewhen the defective % there'll be more as described in Table 4 having datasets like log4j-1.1, lucene-2.0, synapse-1.2, velocity-1.6 etc.

Table 3: Comparative analysis on Precision, Recall and Accuracy

Table 4: Dataset comparative analysis on F_Measure, MCC and AUC
[0043] Referring Figure 4 illustrating graphical representation of a comparative analysis on MCC. Whereas, MCC is considered to be the best performance measure for binary setting class prediction for imbalance class.AdaBoost comes out to be the better algorithm as compared to the random forest as shown in Table 4 in a different dataset like log4j-1.1, lucene-2.0, synapse-1.2, velocity-1.6 etc. The comparative analysis of both ensembles learning techniques on MCC on whole bug datasets of promise repository.
[0044] Referring Figure 5 illustrates graphical representation of a comparative analysis on ROC. Whereas, AUC is the performance measure whose ability is to classify and distinguish between classes and also gives the summary of the ROC curve. So after using AUC it is being found that AdaBoost is comes out to be the better algorithm as compared to the random forest as shown in Table 4.
[0045] The foregoing description of the specific embodiments will so fully reveal the general nature of the embodiments herein that others can, by applying current knowledge, readily modify and/or adapt for various applications such specific embodiments without departing from the generic concept, and, therefore, such adaptations and modifications should and are intended to be comprehended within the meaning and range of equivalents of the disclosed embodiments. It is to be understood that the phraseology or terminology employed herein is for the purpose of description and not of limitation. Therefore, while the embodiments herein have been described in terms of preferred embodiments, those skilled in the art will recognize that the embodiments herein can be practiced with modification within the spirit and scope of the embodiments as described herein.
, Claims:We claim:
1. A method for prediction of automatic software bug using ensemble learning technique, comprising the steps of:
separating suggested framework into three phases, wherein each of which is described as preprocessing, ensemble machine learning-based bug prediction and evaluation of performance measure and comparative analysis on different evaluation metrics;
wherein the two ensemble machine learning (ML) models based on the bagging (random forests (RFs)) and boosting (AdaBoost) techniques are used to predict the bugs; and
wherein the performance of two ensemble learning models has been evaluated based on different quantitative parameters, like F-measure, MCC, AUC etc.
2. The method as claimed in the claim 1, wherein Random forests (RFs) are one of the most successful ensemble algorithms, exhibiting boosting and support vector machine-like performance.
3. The method as claimed in the claim 1, wherein the random forests (RFs) ensemble learning model are a set of ensembles classifiers (trees) and are widely used classification technique in various applications.
4. The method as claimed in the claim 1, wherein AdaBoost is an ensemble learning approach based on boosting i.e., combining weak and inaccurate rules for creating accurate prediction rules.
5. The method as claimed in the claim 1, wherein the traditional machine learning models, such ANN, SVM etc., are performing inefficiently due to their inability in reducing bias, variance and noise.
Dated this 27th day of April, 2023

POOJA
AGENT FOR THE APPLICANT IN/PA/1838