1. A method for balancing sparsity in a deep neural network (DNN), the
method including:
obtaining a weight vector that represents at least part of a weight matrix
of a layer in a sequence of layers in the DNN, the weight vector comprising a first
sequence of weights having different values;
generating a bitmap based at least on the weight vector, the bitmap
comprising a second sequence of bitmap elements, each of the bitmap elements
in the second sequence corresponding to a different weight in the first sequence,
wherein a value of a bitmap element is determined based at least on a value of
the corresponding weight and an index of the bitmap element in the second
sequence matches an index of the corresponding weight in the first sequence;
generating a new bitmap from the bitmap by rearranging the bitmap
elements in the second sequence based on the values of the bitmap elements; and
assigning the weight vector to a plurality of processing elements in the
layer of the DNN based on the new bitmap, each processing element configured
to use a different subset of the weight vector to perform a multiply-accumulate
operation.
2. The method of claim 1, wherein the layer is associated with an activation
vector including a third sequence of activations, further comprising:
rearranging the activations in the third sequence based on the new bitmap.
3. The method of claim 1, wherein generating the bitmap based at least on
the weight vector comprises:
generating a weight bitmap based on the weight vector, the weight bitmap
comprising a sequence of weight bitmap elements;
46 INTL-8087-IN
generating an activation bitmap based on an activation associated with the
layer, the activation bitmap comprising a sequence of activation bitmap elements;
and
generating the bitmap based on the weight bitmap and the activation
bitmap.
4. The method of claim 3, wherein each respective bitmap element in the
bitmap corresponds to a different weight bitmap element in the weight bitmap
and a different activation bitmap element in the activation bitmap, the respective
bitmap element has a value equal to a product of a value of the corresponding
weight bitmap element and a value of the corresponding activation bitmap
element.
5. The method of claim 1, wherein the DNN further comprises a preceding
layer that precedes the layer in the sequence, the preceding layer is associated
with a third sequence of weights, further comprising:
rearranging the weights in the third sequence based on the new bitmap.
6. The method of claim 1, wherein:
the first sequence of weights includes non-zero valued weights and zero
valued weights;
the bitmap elements corresponding to the non-zero valued weights have
values of one; and
the bitmap elements corresponding to the zero valued weights have values
of zero.
7. The method of claim 6, wherein rearranging the bitmap elements in the
second sequence based on the values of the bitmap elements comprises:
47 INTL-8087-IN
switching an index of a one valued bitmap element and an index of a zero
valued bitmap element.
8. The method of claim 6, wherein rearranging the bitmap elements in the
second sequence based on the values of the bitmap elements comprises:
determining a first number of the one valued bitmap elements;
determining a second number of the zero valued bitmap elements, the
second number larger than the first number;
determining an interval number based on the first and second numbers;
and
rearranging the bitmap elements in the second sequence so that every
interval number of bitmap elements in the new bitmap includes a one valued
bitmap element.
9. The method of claim 6, wherein rearranging the bitmap elements in the
second sequence based on the values of the bitmap elements comprises:
determining a first number of the one valued bitmap elements;
determining a second number of the zero valued bitmap elements, the
second number smaller than the first number;
determining an interval number based on the first and second numbers;
and
rearranging the bitmap elements in the second sequence so that every
interval number of bitmap elements in the new bitmap includes a zero valued
bitmap element.
10. The method of claim 1, wherein assigning the weight vector to the
plurality of processing elements in the layer of the DNN based on the new bitmap
comprises:
48 INTL-8087-IN
generating a new weight vector by repositioning one or more weights in
the first sequence so that an index of each weight in the new weight vector
matches an index of the corresponding bitmap element in the new bitmap.