Description:Cross-Reference to Related Applications
[0001] The present application claims priority to Indian Provisional Patent Application No. 202141044385, filed on 30 September 2021 and titled “Systems For Utilizing Self-Confidence Of A Multi-Label Classification Pipeline To Refocus Training” the entire disclosure of which is hereby incorporated by reference.
[0002] The present application claims priority to U.S. Non-Provisional Patent Application No. 17/534,558 filed on 24 November 2021 and titled “Applying Self-Confidence In Multi-Label Classification To Model Training” the entire disclosure of which is hereby incorporated by reference.

Technical Field
[0003] This disclosure relates generally to computer modeling, and particularly to improvement of multi-label classification training using model-determined label confidence.

Background
[0004] Multi-label classification is a common task in various research fields such as image classification, video classification, audio auto-tagging and text categorization. In multi-label classification, a computer model attempts to label a particular input with one of several different classes.
[0005] Additionally, there is a lack of visibility (or interpretability) in assessing the misclassifications of a network, inhibiting effective learning from such misclassifications.
[0006] Previous works have the following problems: (i) they do not differentiate between various members of a class, which could further result in propagating improper errors, and (ii) the weights (of the loss, such as the classwise weights) do not evolve over time based on how confident the model is in predicting the sample.
[0007] As discussed below, the present disclosure provides an approach using the computer model’s “self-confidence” for identifying additional training for improving network performance.
Brief Description of the Drawings
[0008] Embodiments will be readily understood by the following detailed description in ‎conjunction with the accompanying drawings. To facilitate this description, like reference ‎numerals designate like structural elements. Embodiments are illustrated by way of ‎example, and not by way of limitation, in the figures of the accompanying drawings.
[0009] FIG. 1 is an example overview flow for applying a model’s self-confidence to improve training of the model.
[0010] FIG. 2 shows an example class prediction for a region of a space by a multi-label computer model, according to one embodiment.
[0011] FIG. 3 shows one example for determining confidence scores for an input space according to one embodiment.
[0012] FIG. 4 shows an example of modifying training for the computer model based on confidence scores for a training space, according to one embodiment.
[0013] FIG. 5 shows an additional flow for generating a modified training space based on confidence scores, according to one embodiment.
[0014] FIG. 6 shows an example flow for using model self-confidence to modify the loss function during training.
[0015] FIG. 7 shows an example flow for training a multi-label classifier using self-confidence according to one embodiment.
[0016] FIG. 8 shows example computer model inference and computer model training.
[0017] FIG. 9 illustrates an example neural network architecture.
[0018] FIG. 10 is a block diagram of an example computing device 1000 that may include one or more components used for training, analyzing, or implementing a computer model in accordance with any of the embodiments disclosed herein.

Detailed Description
Overview
[0019] The systems, methods and devices of this disclosure each have several innovative aspects, no single one of which is solely responsible for all desirable attributes disclosed herein. Details of one or more implementations of the subject matter described in this specification are set forth in the description below and the accompanying drawings.
[0020] Described herein are approaches for using a computer model’s confidence in multi-label classification to further refine the computer model’s accuracy. The computer model generates a plurality of classification predictions, representing the respective prediction that a portion of an input belongs to each of the particular classes. The input to the model as discussed herein is generally referred to as a “space” and may be a two or three-dimensional area represented by individual regions within the space, such as points or discrete volumes making up the space (e.g., a pixel or voxel). The model’s “confidence” may be determined based on the similarity of the class having the highest prediction (i.e., the most-likely class) and other classes that were also highly predicted, such as the next-most-likely class. The confidence score may thus be generated on a per-region basis, as each portion or region of the input (e.g., a two-dimensional or three-dimensional space) may be evaluated by the computer model to generate a set of predictions. The per-region confidence score may then be used to modify subsequent training of the model to focus the training on these areas of “low-confidence.” Since the model may have already been trained with an initial loss function (which may also be termed an error or cost function), the model may be expected to have directly gained whatever benefit that labeled data may provide. By focusing on “low-confidence” regions, subsequent training iterations may focus on further refining parameters based on more difficult (according to the model’s own predictions) problems and provide a way to further refine the model’s accuracy.
, Claims:1. A method, comprising:
training a computer model for an initial training period with an initial training set, the computer model trained to predict, for a region of a space, a plurality of class predictions;
identifying a training space having a plurality of regions;
for each region in the plurality of regions, applying the computer model to the region to generate a plurality of class predictions; and determining a confidence score for the region based on the plurality of class predictions for the region; and
training the computer model for a further training period based on the confidence scores for the plurality of regions.