Title of invention: Neural network device
Technical field
[0001]
　The present technology relates to a neural network device, and more particularly to a neural network device capable of improving recognition performance.
Background technology
[0002]
　For example, consider automatic recognition technology (identification, detection, etc.) of various signals such as images and audio. Here, it is assumed that a neural network is considered as a method for recognition (see, for example, Non-Patent Document 1).
[0003]
　A neural network processing device that takes a certain signal as an input and outputs the result of recognition processing for that signal is, for example, a convolution layer processing unit, an activation layer processing unit, a pooling layer processing unit, a convolution layer processing unit, and an activation layer. The processing unit, the pooling layer processing unit, the convolution layer processing unit, and the activation layer processing unit are provided in order from the input side to the output side.
[0004]
　Such a neural network processing device takes data of a certain signal as an input, performs data conversion with eight components from the first convolution layer processing unit to the last activation layer processing unit, and obtains a recognition result for the input data. Output.
[0005]
　Generally, it is said that the larger the scale of the neural network (the number of components and the number of coefficients), the more complicated the input / output relationship can be realized.
Prior art literature
Non-patent literature
[0006]
Non-Patent Document 1: Ian Goodfellow, Yoshua Bengio, and Aaron Courville, Deep Learning, The MIT Press, 2016
Outline of the invention
Problems to be solved by the invention
[0007]
　However, the magnitude of the signal input to the neural network may be biased.
[0008]
　For example, consider using a neural network to identify whether or not an office sound is an office sound from various environmental sounds.
[0009]
　A very loud signal such as the environmental sound in a train or an aircraft may be input to this neural network, but on the other hand, the signal of the environmental sound in the office to be identified is usually small.
[0010]
　Small signals need to be analyzed in more detail in order to accurately identify office environmental sounds. As described above, in order to obtain high recognition performance, it is necessary to construct and learn a neural network so as to correspond to the bias of the magnitude of the input signal.
[0011]
　In addition, when the signal input to the microphone is detected by the neural network due to the user interface using the microphone (striking the microphone, blocking the microphone, etc.), the size of the input signal is biased.
[0012]
　For example, when the microphone is hit, a signal that is much louder than other environmental sounds is input to the microphone. Also, when the microphone is closed, a very small signal is input to the microphone compared to other environmental sounds. Even when these are detected individually or simultaneously, it is required to construct and learn a neural network so as to correspond to the bias of the magnitude of the input signal.
[0013]
　However, at present, there is no neural network having components corresponding to the bias of the magnitude of the input signal. In addition, in order to deal with the bias of the size of the input signal, it is necessary to increase the scale of the neural network so that a complicated input / output relationship can be realized. When there are restrictions, it is difficult to obtain high performance.
[0014]
　This technology was made in view of such a situation, and makes it possible to improve the recognition performance.
Means to solve problems
[0015]
　The neural network device of one aspect of the present technology includes a non-linear conversion layer processing unit that performs conversion by a non-linear function having learnable parameters.
[0016]
　In one aspect of the present technology, conversion is performed by a non-linear conversion layer processing unit that performs conversion by a non-linear function having learnable parameters.
A brief description of the drawing
[0017]
[Fig. 1] Fig. 1 is a diagram showing a configuration example of a neural network processing device.
[Fig. 2] Fig. 2 is a flowchart illustrating a recognition process.
[Fig. 3] Fig. 3 is a diagram showing a configuration example of a neural network learning device.
[Fig. 4] Fig. 4 is a flowchart illustrating a learning process.
[Fig. 5] Fig. 5 is a diagram showing an input and output relational expression and a graph of a logarithmic layer.
[Fig. 6] Fig. 6 is a diagram illustrating an operation by a user.
[Fig. 7] Fig. 7 is a diagram illustrating a detection success rate of each operation.
[Fig. 8] Fig. 8 is a diagram showing an input and output relational expression and a graph of an inverse proportional layer.
[Fig. 9] Fig. 9 is a diagram showing a relational expression and a graph of input and output of a power layer.
[Fig. 10] Fig. 10 is a diagram showing a configuration example of a computer.
Mode for carrying out the invention
[0018]
　Hereinafter, embodiments to which the present technology is applied will be described with reference to the drawings.
[0019]

　This technology can improve recognition performance by constructing a neural network having non-linear transformation by learnable parameters as a component. It is something to do. That is, even if the scale of the neural network is limited, high performance can be obtained.
[0020]
　The above non-linear conversion is, for example, one or more of logarithmic functions, 冪 functions, exponential functions, trigonometric functions, hyperbolic functions, other linear or non-linear functions, and functions obtained by four-rule operations or synthesis thereof. Can be done using.
[0021]
　The neural network of the present technology is designed to have components corresponding to the bias of the magnitude of the input signal. This neural network has a non-linear transformation with learnable parameters as a component.
[0022]
　The components of such a non-linear transformation perform the optimum scale transformation for the bias of the input signal size, so that the neural network can analyze the part where the input signal size is concentrated in more detail. become.
[0023]
　As a result, even a small-scale neural network can cope with a bias in the size of the input signal, and high recognition performance can be obtained.
[0024]
　In the following, as an example of a neural network to which this technology is applied, a neural network having a "logarithmic layer" that performs nonlinear transformation using a logarithmic function as a component is given as an example, and the features of the neural network to which this technology is applied will be described. ..
[0025]
　FIG. 1 is a diagram showing a configuration example of an embodiment of a neural network processing device to which the present technology is applied.
[0026]
　The neural network processing device 11 shown in FIG. 1 is composed of a neural network, and is a convolution layer processing unit 21, an activation layer processing unit 22, a pooling layer processing unit 23, a logarithmic layer processing unit 24, and a convolution layer processing unit. 25, it has an activation layer processing unit 26, a pooling layer processing unit 27, a convolution layer processing unit 28, and an activation layer processing unit 29.
[0027]
　In particular, the neural network processing device 11 is a neural network in which a logarithmic layer processing unit 24, that is, a logarithmic layer is introduced in addition to the general configuration.
[0028]
　The neural network processing device 11 performs processing of each layer (hierarchy) of the neural network on the input data which is the input data, and outputs the recognition result of the predetermined recognition target for the input data. Here, the convolution layer processing unit 21 to the activation layer processing unit 29 are each layer of the neural network.
[0029]
　The convolution layer processing unit 21 performs convolution layer processing on the supplied input data, and supplies the processing result to the activation layer processing unit 22.
[0030]
　The activation layer processing unit 22 performs activation layer processing on the processing result supplied from the convolution layer processing unit 21, and supplies the processing result to the pooling layer processing unit 23.
[0031]
　The pooling layer processing unit 23 performs pooling layer processing on the processing result supplied from the activation layer processing unit 22, and supplies the processing result to the logarithmic layer processing unit 24.
[0032]
　The logarithmic layer processing unit 24 performs a non-linear conversion process using a logarithmic function on the processing result supplied from the pooling layer processing unit 23 as a logarithmic layer processing, and supplies the processing result to the convolution layer processing unit 25.
[0033]
　The convolution layer processing unit 25 performs convolution layer processing on the processing result supplied from the logarithmic layer processing unit 24, and supplies the processing result to the activation layer processing unit 26.
[0034]
　The activation layer processing unit 26 performs activation layer processing on the processing result supplied from the convolution layer processing unit 25, and supplies the processing result to the pooling layer processing unit 27.
[0035]
　The pooling layer processing unit 27 performs pooling layer processing on the processing result supplied from the activation layer processing unit 26, and supplies the processing result to the convolution layer processing unit 28.
[0036]
　The convolution layer processing unit 28 performs convolution layer processing on the processing result supplied from the pooling layer processing unit 27, and supplies the processing result to the activation layer processing unit 29.
[0037]
　The activation layer processing unit 29 performs activation layer processing on the processing result supplied from the convolution layer processing unit 28, and outputs the processing result as a recognition result of the recognition target for the input data.
[0038]

　Next, the operation of the neural network processing device 11 shown in FIG. 1 will be described.
[0039]
　That is, the recognition process by the neural network processing device 11 will be described below with reference to the flowchart of FIG.
[0040]
　In step S11, the convolution layer processing unit 21 performs convolution layer processing on the supplied input data, and supplies the processing result to the activation layer processing unit 22.
[0041]
　In step S12, the activation layer processing unit 22 performs activation layer processing on the processing result supplied from the convolution layer processing unit 21, and supplies the processing result to the pooling layer processing unit 23.
[0042]
　In step S13, the pooling layer processing unit 23 performs pooling layer processing on the processing result supplied from the activation layer processing unit 22, and supplies the processing result to the logarithmic layer processing unit 24.
[0043]
　In step S14, the logarithmic layer processing unit 24 performs logarithmic layer processing on the processing result supplied from the pooling layer processing unit 23, and supplies the processing result to the convolution layer processing unit 25.
[0044]
　In step S15, the convolution layer processing unit 25 performs the convolution layer processing on the processing result supplied from the logarithmic layer processing unit 24, and supplies the processing result to the activation layer processing unit 26.
[0045]
　In step S16, the activation layer processing unit 26 performs the activation layer processing on the processing result supplied from the convolution layer processing unit 25, and supplies the processing result to the pooling layer processing unit 27.
[0046]
　In step S17, the pooling layer processing unit 27 performs pooling layer processing on the processing result supplied from the activation layer processing unit 26, and supplies the processing result to the convolution layer processing unit 28.
[0047]
　In step S18, the convolution layer processing unit 28 performs the convolution layer processing on the processing result supplied from the pooling layer processing unit 27, and supplies the processing result to the activation layer processing unit 29.
[0048]
　In step S19, the activation layer processing unit 29 performs activation layer processing on the processing result supplied from the convolution layer processing unit 28, outputs the processing result as a recognition result of the recognition target for the input data, and outputs the processing result. The recognition process ends.
[0049]
　As described above, the neural network processing device 11 performs the processing of converting the data input in each layer of the neural network, and outputs the processing result as the recognition result of the recognition target. At this time, by performing the non-linear conversion processing in at least one layer, high recognition performance can be obtained even in a small-scale neural network. That is, the recognition performance can be improved.
[0050]
Further
　, the neural network learning device that generates the neural network processing device 11 by learning is configured as shown in FIG. 3, for example. In FIG. 3, the same reference numerals are given to the parts corresponding to the cases in FIG. 1, and the description thereof will be omitted as appropriate.
[0051]
　In the example shown in FIG. 3, the neural network learning device 51 generates (constructs) the neural network processing device 11 by learning based on the signal data input from the database 52.
[0052]
　The neural network learning device 51 has an input data selection unit 61 and a coefficient update unit 62.
[0053]
　The input data selection unit 61 selects the signal data to be used for learning from the signal data recorded in the database 52, and supplies the data to the coefficient update unit 62 and the neural network processing device 11.
[0054]
　The coefficient updating unit 62 performs processing on the neural network coefficient, that is, the layer of the neural network processing device 11, according to the supply of data from the input data selection unit 61 and the supply of the recognition result from the neural network processing device 11. The coefficients (parameters) used are updated and supplied to the neural network processing device 11.
[0055]
　In FIG. 3, a learning system for learning the neural network processing device 11 is configured by the neural network processing device 11, the neural network learning device 51, and the database 52.
[0056]

　Next, the learning process performed by the learning system shown in FIG. 3 will be described. That is, the learning process performed by the learning system will be described below with reference to the flowchart of FIG.
[0057]
　In step S41, the input data selection unit 61 selects the input data for selecting the input data to be used for learning from the signal data recorded in the database 52, and the input data selected as a result is used as the coefficient update unit 62. , And the convolution layer processing unit 21 of the neural network processing device 11.
[0058]
　When the input data is supplied to the convolution layer processing unit 21 of the neural network processing device 11 in this way, the processes of steps S42 to S50 are performed, and these processes are performed in steps S11 to S19 of FIG. Since it is the same as the process, the description thereof will be omitted.
[0059]
　That is, in steps S42 to S50, from the convolution layer processing unit 21 on the leftmost side, that is, the input side in FIG. 1 in the neural network processing device 11, the activation layer processing on the rightmost side, that is, the output side in FIG. Up to unit 29, conversion processing (data conversion) for data is performed by nine components (layers).
[0060]
　Then, the data obtained by the processing in the activation layer processing unit 29 is supplied to the coefficient updating unit 62 as the recognition result of the recognition target for the input data.
[0061]
　In the neural network processing device 11, the convolution layer and the logarithmic layer, that is, the convolution layer processing unit 21, the logarithmic layer processing unit 24, the convolution layer processing unit 25, and the convolution layer processing unit 28 were supplied from the coefficient updating unit 62. It is assumed that the convolution layer processing and the logarithmic layer processing, that is, the data conversion processing (conversion processing) are performed using the coefficient.
[0062]
　In step S51, the coefficient updating unit 62 updates the coefficient based on the input data supplied from the input data selection unit 61 and the recognition result supplied from the activation layer processing unit 29 of the neural network processing device 11.
[0063]
　In step S51, the coefficient updating unit 62 updates the coefficient of the neural network so that the input data and the recognition result have a desired relationship, that is, a desired input / output relationship is realized. Here, the coefficients used in the three convolution layers, that is, the coefficients used in the convolution layer processing in the convolution layer processing unit 21, the convolution layer processing unit 25, and the convolution layer processing unit 28, and the logarithm in the logarithm layer processing unit 24. The coefficients (parameters) used in the layer processing are updated. The coefficients can be updated, for example, by the backpropagation method.
[0064]
　When the coefficient is updated, the coefficient updating unit 62 supplies the updated coefficient to each unit of the neural network processing device 11. The convolution layer processing unit 21, the logarithmic layer processing unit 24, the convolution layer processing unit 25, and the convolution layer processing unit 28 replace the retained coefficients with the coefficients newly supplied from the coefficient updating unit 62, and replace the coefficients with the coefficients newly supplied. Update.
[0065]
　In step S52, the coefficient updating unit 62 determines whether or not the condition for completing learning is satisfied.
[0066]
　For example, when the processes of steps S41 to S51 are repeated a predetermined number of times, it is considered that the condition for ending learning is satisfied. The learning end condition may be any condition such that the error between the desired input / output relationship and the actual input / output relationship is equal to or less than the threshold value.
[0067]
　If it is determined in step S52 that the condition for ending learning is not satisfied, the process returns to step S41, and the above-described process is repeated.
[0068]
　On the other hand, if it is determined in step S52 that the condition for ending learning is satisfied, the learning process ends.
[0069]
　In this case, the final neural network processing device 11 is obtained by learning, and the neural network processing device 11 is supplied from the coefficient updating unit 62 and finally holds the coefficient for the input data. Perform recognition processing.
[0070]
　By using the neural network processing device 11 obtained by such learning, it is possible to output a correct recognition result even for unknown input data not included in the input data stored in the database 52. Will be.
[0071]
　As described above, the learning system learns the neural network processing device 11 by updating the coefficients used in the neural network processing device 11.
[0072]
　In particular, by learning one or more coefficients including the coefficients of a layer that performs non-linear conversion processing such as a logarithmic layer, high recognition performance can be obtained even in a small-scale neural network. That is, the recognition performance of the neural network processing device 11 obtained by learning can be improved.
[0073]

　Here, the improvement of recognition performance by introducing the logarithmic layer into the neural network will be described.
[0074]
　FIG. 5 shows the input and output relational expressions and graphs of the logarithmic layer (logarithmic layer processing unit 24). In FIG. 5, the horizontal axis represents the input x of the logarithmic layer, and the vertical axis represents the output y of the logarithmic layer.
[0075]
　In this example, the logarithmic layer (logarithmic layer processing unit 24) outputs 0 as the output y when the input x is negative, that is, when x <0.
[0076]
　On the other hand, in the logarithmic layer (logarithmic layer processing unit 24), when the input x is positive, that is, when x ≧ 0, the rate of change of the output y with respect to the input x increases as the input x increases. The value of the function that becomes smaller is output as the output y.
[0077]
　Here, the output y is the output y = (log (x + e -p ) + p) / (log (1 + e -p ) + p), where p is a predetermined coefficient (parameter) . At the time of learning, this coefficient p is updated (learned) by the coefficient updating unit 62.
[0078]
　In this example, the rate of change in the output y is very large for a particularly small positive input x. Further, it has a coefficient (parameter) p, and by changing this coefficient p, the relationship between the input x and the output y can be changed as shown in FIG.
[0079]
　In particular, here, the polygonal line L11 shows the relationship between the input x and the output y when the coefficient p = -4, and the curve L12 shows the relationship between the input x and the output y when the coefficient p = -2. The curve L13 shows the relationship between the input x and the output y when the coefficient p = 0.
[0080]
　Similarly, the curve L14 shows the relationship between the input x and the output y when the coefficient p = 2, and the curve L15 shows the relationship between the input x and the output y when the coefficient p = 4.
[0081]
　As the coefficient p is increased in this way, the ratio of the change in the output y to the small positive input x is increased, and the curvature of the graph (curve) is increased. On the other hand, when the coefficient p is reduced, the curvature of the graph becomes smaller, and the graph showing the relationship between the input x and the output y approaches a straight line in the range where the input x is positive. Furthermore, the value of the coefficient p can be learned, and the shape of the graph more suitable for the bias of the size of the input signal (input data) is automatically obtained by learning rather than being determined by humans in some way. Can be done.
[0082]
　In the logarithmic layer (logarithmic layer processing unit 24), the ratio of the change in the output y to the small positive input x is large, so that the neural network having the logarithmic layer as a component, that is, the neural network processing device 11, has a small input. The signal (input data) can be analyzed in more detail.
[0083]
　Therefore, this neural network (neural network processing device 11) is particularly effective when the magnitude of the input signal (input data) is biased, such as when identifying the above-mentioned office environmental sound or the closing sound of the microphone. Is.
[0084]
　While large signals such as environmental sounds in trains and aircraft are also input, it was necessary to increase the scale of the neural network in order to accurately identify small input signals.
[0085]
　However, in the present technology, by introducing a logarithmic layer, a small input signal can be analyzed in more detail, so that high discrimination performance (recognition performance) can be realized even in a small-scale neural network.
[0086]
　For example, as a user interface that actually uses a microphone, four types of "direct tap", "rubbing", "close", and "close tap" as shown in FIG. 6 are considered, and the signal of each microphone input is input. , Detected using a neural network.
[0087]
　In FIG. 6, the portion indicated by the arrow Q11 shows a user interface, that is, an operation “direct tap” in which the user directly taps the microphone portion with a finger as an operation by the user. Further, the portion indicated by the arrow Q12 indicates "rubbing", which is an operation in which the user rubs the microphone portion with a finger as a user interface.
[0088]
　The part indicated by the arrow Q13 indicates "close", which is an operation in which the user closes the microphone part with a finger as a user interface. Further, the portion indicated by the arrow Q14 indicates, as a user interface, an operation "close tap" in which the user taps (tap while closing) the microphone portion with a finger.
[0089]
　For these four types of operations, recognition processing that recognizes each operation by a neural network using the acoustic data obtained by collecting sound with a microphone as input data, that is, the sound generated when each operation is performed. The result of the recognition process is shown in FIG.
[0090]
　In FIG. 7, four types of operations, "direct tap", "rubbing", "close", and "close tap", were successfully detected when each operation was detected using a general neural network (DNN). The rate and the detection success rate when each operation is detected by the neural network processing device 11 in which the logarithmic layer is introduced are shown. That is, in FIG. 7, the vertical axis shows the detection success rate when each operation is detected (recognized).
[0091]
　In particular, in FIG. 7, the part indicated by the arrow Q21 shows the detection success rate of the operation “direct tap”, and the part indicated by the arrow Q22 shows the detection success rate of the operation “rubbing”. In addition, the part indicated by the arrow Q23 shows the detection success rate of the operation "close", and the part indicated by the arrow Q24 shows the detection success rate of the operation "close tap".
[0092]
　In the parts indicated by arrows Q21 to Q24, the left side in the figure shows the detection success rate when a general neural network is used, and the right side in the figure uses the neural network processing device 11. It shows the detection success rate of the case.
[0093]
　Further, in FIG. 7, it is the detection success rate of the sound to be detected, that is, the operation of the recognition target when the threshold value is set so that the excess detection rate is 0.01%.
[0094]
　In FIG. 7, it can be seen that the identification performance (recognition performance) is improved by the introduction of the logarithmic layer for the three types of operations of "direct tap", "rubbing", and "close". In particular, the identification performance is significantly improved in the operation "close".
[0095]
　The value of the coefficient (parameter) p of the logarithmic layer learned in the operation "close" is 4.25, and the coefficient p learned in the other three types of operations "direct tap", "rubbing", and "close tap". Was greater than the value of (2.34, 1.29, and 1.06, respectively).
[0096]
　It is said that the logarithmic layer was trained to analyze smaller signals in detail in order to detect the sound obtained when performing the operation "block", which is a minute signal, that is, the operation "block". It means that.
[0097]
　Furthermore, the effective range of the logarithmic layer is not limited to identifying office environmental sounds and microphone blockage sounds, but is general for audio signals that often convert the signal magnitude to a logarithmic scale (decibel value, etc.). It is effective for.
[0098]
　In addition, this technique may be effective for other signals such as images. Further, this technique is similarly effective not only in a small-scale neural network but also in a large-scale neural network.
[0099]
　The neural network described with reference to FIGS. 1 to 4 is an example of a neural network having components that perform non-linear conversion by learnable coefficients (parameters), and various other modifications can be considered. Be done. First, various examples other than the logarithmic layer can be considered as this component.
[0100]
　For example, as an example of a component (layer) that performs a non-linear transformation, an input and output relational expression and a graph of an inverse proportional layer using an inverse proportional function and a power layer using a power function are shown in FIGS. 8 and 9. Shown in. In FIGS. 8 and 9, the horizontal axis represents the input x and the vertical axis represents the output y.
[0101]
　FIG. 8 shows the relationship between the input x and the output y in the inverse proportional layer. The inverse proportional layer outputs 0 as the output y when the input x is negative, that is, when x <0.
[0102]
　On the other hand, in the inverse proportional layer, when the input x is positive, that is, when x ≧ 0, the coefficient (parameter) is p, and the output y is the output y = (1 + p) x / (x +). p). At the time of learning, this coefficient p is updated (learned) by the coefficient updating unit 62.
[0103]
　Further, in FIG. 8, the polygonal line L21 shows the relationship between the input x and the output y when the coefficient p = 16, and the curve L22 shows the relationship between the input x and the output y when the coefficient p = 4. .. Similarly, the curve L23 shows the relationship between the input x and the output y when the coefficient p = 0, and the curve L24 shows the relationship between the input x and the output y when the coefficient p = 1/4.
[0104]
　On the other hand, FIG. 9 shows the relationship between the input x and the output y in the exponentiation layer. The exponentiation layer outputs 0 as the output y when the input x is negative, that is, when x <0.
[0105]
　On the other hand, in the power layer, when the input x is positive, that is, when x ≧ 0, the coefficient (parameter) is set to p and the output y is set to output y = x p . At the time of learning, this coefficient p is updated (learned) by the coefficient updating unit 62.
[0106]
　In FIG. 9, the curve L31 shows the relationship between the input x and the output y when the coefficient p = 2, and the polygonal line L32 shows the relationship between the input x and the output y when the coefficient p = 1. Similarly, the curve L33 shows the relationship between the input x and the output y when the coefficient p = 5/8, and the curve L34 shows the relationship between the input x and the output y when the coefficient p = 3/8. There is.
[0107]
　In the inverse proportional layer shown in FIG. 8, the ratio of the change in the output y to the small positive input x is larger than that in the logarithmic layer.
[0108]
　Further, in the power layer shown in FIG. 9, when the coefficient p is smaller than 1, the ratio of the change in the output y is larger than that of the small positive input x, while when the coefficient p is larger than 1, the ratio of the change is larger. The ratio of the change in the output y to the large positive input x becomes larger. That is, a large input signal can be analyzed in more detail.
[0109]
　In both the inverse proportional layer and the power layer, the relationship between the input x and the output y can be changed by changing the coefficient, that is, the parameter p, and the parameter can be learned. Also, non-linear transformations include not only logarithmic functions, 冪 functions (including inversely proportional functions), but also exponential functions, trigonometric functions, hyperbolic functions, one or more of other linear or non-linear functions, and their four rules. It may be performed using a function obtained by synthesis or the like. There may be two or more parameters (coefficients) for changing the relationship between the input x and the output y.
[0110]
　Further, the present component, that is, a component (layer) that performs non-linear transformation can be introduced at an arbitrary position in the neural network in an arbitrary form.
[0111]
　For example, it may be introduced as an activation function for the output of the convolution layer, or it may be introduced for the coefficient of the convolution layer. Further, the present component may be introduced at a plurality of places in the neural network.
[0112]
　Further, the coefficients (parameters) of the present components may be applied in common to all the dimensions of the input x, or may be applied differently for each dimension.
[0113]
　For example, in the example shown in FIG. 1, when the number of filter types of the leftmost convolution layer (convolution layer processing unit 21) is 16, the logarithmic layer (logarithmic layer processing unit 24) has 16 types of input channels. , Different parameters (coefficients) may be applied to each.
[0114]
　The parameters (coefficients) of this component may not be included in the learning target and fixed values ​​may be used. The fixed value may be determined by humans in some way. For example, a fixed value may be determined based on a certain rule determined by a human being from a statistical value of the distribution of the magnitude of an input signal.
[0115]
　Further, the initial value at the time of learning the parameter (coefficient) of the present component may be determined based on the value thus determined by a human being. The parameters of this component and the coefficients of other components of the neural network (convolution layer, etc.) may be learned at the same time, or one may be fixed and the other may be learned.
[0116]
　According to the present technology as described above, the recognition performance of the neural network can be improved. Moreover, according to this technology, high recognition performance can be obtained even with a small-scale neural network.
[0117]

　By the way, the series of processes described above can be executed by hardware or software. When a series of processes are executed by software, the programs that make up the software are installed on the computer. Here, the computer includes a computer embedded in dedicated hardware and, for example, a general-purpose personal computer capable of executing various functions by installing various programs.
[0118]
　FIG. 10 is a block diagram showing a configuration example of hardware of a computer that executes the above-mentioned series of processes programmatically.
[0119]
　In a computer, a CPU (Central Processing Unit) 501, a ROM (Read Only Memory) 502, and a RAM (Random Access Memory) 503 are connected to each other by a bus 504.
[0120]
　An input / output interface 505 is further connected to the bus 504. An input unit 506, an output unit 507, a recording unit 508, a communication unit 509, and a drive 510 are connected to the input / output interface 505.
[0121]
　The input unit 506 includes a keyboard, a mouse, a microphone, an image sensor, and the like. The output unit 507 includes a display, a speaker, and the like. The recording unit 508 includes a hard disk, a non-volatile memory, and the like. The communication unit 509 includes a network interface and the like. The drive 510 drives a removable recording medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.
[0122]
　In the computer configured as described above, the CPU 501 loads the program recorded in the recording unit 508 into the RAM 503 via the input / output interface 505 and the bus 504 and executes the above-described series. Is processed.
[0123]
　The program executed by the computer (CPU 501) can be recorded and provided on a removable recording medium 511 as a package medium or the like, for example. Programs can also be provided via wired or wireless transmission media such as local area networks, the Internet, and digital satellite broadcasting.
[0124]
　In the computer, the program can be installed in the recording unit 508 via the input / output interface 505 by mounting the removable recording medium 511 in the drive 510. Further, the program can be received by the communication unit 509 and installed in the recording unit 508 via a wired or wireless transmission medium. In addition, the program can be pre-installed in the ROM 502 or the recording unit 508.
[0125]
　The program executed by the computer may be a program that is processed in chronological order according to the order described in this specification, or may be a program that is processed in parallel or at a necessary timing such as when a call is made. It may be a program in which processing is performed.
[0126]
　Further, the embodiment of the present technology is not limited to the above-described embodiment, and various changes can be made without departing from the gist of the present technology.
[0127]
　For example, the present technology can have a cloud computing configuration in which one function is shared by a plurality of devices via a network and jointly processed.
[0128]
　Further, each step described in the above-mentioned flowchart can be executed by one device or can be shared and executed by a plurality of devices.
[0129]
　Further, when a plurality of processes are included in one step, the plurality of processes included in the one step can be executed by one device or shared by a plurality of devices.
[0130]
　Further, the present technology can also have the following configurations.
[0131]
(1) A 　neural network device including
　a non-linear conversion layer processing unit that performs conversion by a non-linear function having learnable parameters
.
(2) 　The neural network apparatus according to (1),
　wherein the conversion by the non-linear function of the non-linear conversion layer processing unit is a conversion by a logarithmic function
.
(3) 　The neural network apparatus according to (1),
　wherein the non-linear function conversion of the non-linear conversion layer processing unit is a conversion by a combination of a plurality of non-linear functions
.
(4) The 　item according to any one of (1) to (3),
　further
　comprising an input unit into which an input signal is input, and the input signal having a biased signal magnitude is input to the input unit.
Neural network device.
(5) 　The neural network apparatus according to any one of (1) to (4),
　further including a pooling layer processing unit,
　wherein the processing of the nonlinear conversion layer processing unit is performed after the processing by the pooling layer processing unit.
..
(6)
　Further equipped with a convolution layer processing unit

　The neural network apparatus according to any one of (1) to (5) 　, wherein the processing of the non-linear transformation layer processing unit is performed before the processing by the convolution layer processing unit .
(7) 　The neural network apparatus according to any one of (1) to (6),
　wherein the non-linear function conversion of the non-linear conversion layer processing unit is performed as an activation function
.
Code description
[0132]
　11 Neural network processing device, 21 Convolution layer processing unit, 24 Logarithmic layer processing unit, 25 Convolution layer processing unit, 28 Convolution layer processing unit, 51 Neural network learning device, 61 Input data selection unit, 62 Coefficient update unit
The scope of the claims
[Claim 1]
　A
　neural network device including a non-linear conversion layer processing unit that performs conversion by a non-linear function having learnable parameters .
[Claim 2]

　The neural network device according to claim 1, 　wherein the transformation by the nonlinear function of the nonlinear transformation layer processing unit is a transformation by a logarithmic function .
[Claim 3]

　The neural network device according to claim 1, 　wherein the conversion by the non-linear function of the non-linear conversion layer processing unit is a conversion by a combination of a plurality of non-linear functions .
[Claim 4]
　The neural network device
　according to
claim 1, 　further comprising an input unit into which an input signal is input, and the input signal having a bias in the magnitude of the signal is input to the input unit.
[Claim 5]
　Further comprising a pooling layer processing unit,
　the processing of the non-linear transformation layer processing unit is performed after the processing by the pooling layer processing unit
　neural network system according to claim 1.
[Claim 6]
　Convolution further comprising a layer processing unit,
　the processing of the non-linear transformation layer processing unit is performed prior to treatment with the convolution layer processing unit
　neural network system according to claim 1.
[Claim 7]

　The neural network device according 　to claim 1, wherein the conversion by the non-linear function of the non-linear conversion layer processing unit is performed as an activation function .