DESCRIPTION
TITLE OF INVENTION: NETWORK CONSTRUCTION APPARATUS AND
NETWORK CONSTRUCTION METHOD
TECHNICAL FIELD
[0001] The present invention relates to a network construction apparatus and a network construction method for constructing a neural network.
BACKGROUND ART
[0002] A neural network is a network in which an input layer, an intermediate layer, and an output layer are connected in cascade.
The neural network is a kind of approximation function for predicting output data corresponding to arbitrary input data when arbitrary input data is given, by learning a correlation between input data and output data in advance.
[0003] A structure of the neural network such as the number of elements included in the intermediate layer is often determined manually by a designer of the neural network, but it is difficult for a designer who is not familiar with the neural network to properly determine the structure of the neural network.
In the following Non-Patent Literature 1, a network construction method is disclosed for automatically determining a structure of a neural network by using a technique called Add if Silent (AiS).
This neural network is a neural network imitating visual information processing of organisms called neocognitron, and an element included in the intermediate layer of this neural network is an element whose input/output response is determined by a normalized linear function.

CITATION LIST NON-PATENT LITERATURE
[0004] Non-Patent Literature 1: Fukushima, K.: "Artificial vision by multi-layered neural networks: Neocognitron and its advances", Neural Networks, vol. 37, pp. 103-119(2013).
SUMMARY OF INVENTION TECHNICAL PROBLEM
[0005] Since the conventional network construction method is configured as described above, when the element included in the intermediate layer is an element whose input/output response is determined by a normalized linear function, the number of elements included in the intermediate layer and the like can be automatically determined. However, there has been a problem that, in a case where the element included in the intermediate layer is an element whose input/output response is determined by a Gaussian function, even when the technique called AiS is used, the number of elements included in the intermediate layer and the like cannot be automatically determined.
[0006] The present invention has been made to solve the above problem, and it is an object to obtain a network construction apparatus and a network construction method capable of automatically determining the number of elements included in the intermediate layer even when the element included in the intermediate layer is an element whose input/output response is determined by a Gaussian function.
SOLUTION TO PROBLEM

[0007] A network construction apparatus according to the present invention is provided with an output value calculating unit for calculating output values of one or more elements included in an intermediate layer of a neural network in accordance with an output value of an element included in an input layer of the neural network when constructing the neural network including the intermediate layer including an element whose input/output response is determined by a Gaussian function, and an element construction unit compares the output values of the one or more elements calculated by the output value calculating unit with a threshold value, and maintains a number of elements included in the intermediate layer when an output value of any of the elements out of the output values of the one or more elements included in the intermediate layer is greater than the threshold value, and performs element construction processing of increasing the number of elements included in the intermediate layer when all of the output values of the one or more elements included in the intermediate layer are equal to or less than the threshold value.
ADVANTAGEOUS EFFECTS OF INVENTION
[0008] According to the present invention, the element construction unit compares the output values of the one or more elements calculated by the output value calculating unit with a threshold value, and maintains a number of elements included in the intermediate layer when an output value of any of the elements out of the output values of the one or more elements included in the intermediate layer is greater than the threshold value, and performs element construction processing of increasing the number of elements included in the intermediate layer when all of the output values of the one or more elements included in the intermediate layer are equal to or less than the threshold value, so that there is an effect that the number of elements included in the

intermediate layer can be automatically determined even when the element included in the intermediate layer is an element whose input/output response is determined by a Gaussian function.
BRIEF DESCRIPTION OF DRAWINGS
[0009] FIG. 1 is a configuration diagram illustrating a network construction apparatus
according to a first embodiment of the present invention.
FIG. 2 is a hardware configuration diagram of the network construction apparatus according to the first embodiment of the present invention.
FIG. 3 is an explanatory diagram illustrating an example of a neural network to which the network construction apparatus according to the first embodiment of the present invention is applied.
FIG. 4 is a hardware configuration diagram of a computer in a case where the network construction apparatus is implemented by software, firmware, or the like.
FIG. 5 is a flowchart illustrating a network construction method that is a processing procedure in the case where the network construction apparatus is implemented by software, firmware, or the like.
FIG. 6 is a flowchart illustrating a network construction method that is a processing procedure in the case where the network construction apparatus is implemented by software, firmware, or the like.
FIG. 7 is an explanatory diagram illustrating an example of a neural network to which a network construction apparatus according to a second embodiment of the present invention is applied.
FIG. 8 is a flowchart illustrating a network construction method that is a processing procedure in the case where the network construction apparatus is

implemented by software, firmware, or the like.
FIG. 9 is a configuration diagram illustrating a network construction apparatus according to a third embodiment of the present invention.
FIG. 10 is a hardware configuration diagram of the network construction apparatus according to the third embodiment of the present invention.
DESCRIPTION OF EMBODFMENTS
[0010] Hereinafter, to explain the present invention in more detail, embodiments for carrying out the present invention will be described with reference to the accompanying drawings.
[0011] In a case where the input/output response of an element included in an intermediate layer of a neural network is a normalized linear function, for example, when positive data deviating largely from an assumed range is input to the intermediate layer, it is assumed that large positive data is output from the intermediate layer. As a result, it is assumed that large positive data is also output from an output layer.
Thus, for example, when data deviating largely from the assumed range is input to the intermediate layer, large positive data is output from the output layer of the neural network, so that an apparatus on the output side of the neural network may be largely affected.
In a case where the input/output response of the element included in the intermediate layer of the neural network is a Gaussian function, for example, when positive or negative data deviating largely from the assumed range is input to the intermediate layer, data close to zero is output from the intermediate layer. As a result, data close to zero is also output from the output layer.
Thus, for example, even when the data deviating largely from the assumed

range is input to the intermediate layer, the data close to zero is output from the output layer of the neural network, so that the apparatus on the output side of the neural network can avoid a large influence. [0012] First Embodiment.
FIG. 1 is a configuration diagram illustrating a network construction apparatus according to a first embodiment of the present invention, and FIG. 2 is a hardware configuration diagram of the network construction apparatus according to the first embodiment of the present invention.
FIG. 3 is an explanatory diagram illustrating an example of a neural network to which the network construction apparatus according to the first embodiment of the present invention is applied.
In FIGS. 1 to 3, the neural network includes an input layer 1, an intermediate layer 2, and an output layer 3.
In the first embodiment, an example in which the number of intermediate layers 2 is one will be described, and an example in which the number of intermediate layers 2 is two or more will be described in a second embodiment. [0013] The input layer 1 includes I (I is an integer equal to or greater than 1) elements ai (i = 1, ..., I), and, for example, M (M is an integer equal to or greater than 1) pieces of learning data xm = (xim, X2m, ..., xim) are sequentially given from a sensor or the like. The superscript m represents m = 1, 2, ..., M.
The number of dimensions of the learning data xm is I, and x;m (i = 1, ..., I) included in the learning data xm is referred to as component data.
When the i-th component data x;m (i = 1, ..., I) of the learning data x;m is given to the i-th element ai (i = 1, ..., I), the input layer 1 outputs yi = x;m as an output value of the i-th element ai for the j-th element bj (j = 1, ..., J) included in the intermediate layer

2.
[0014] The intermediate layer 2 includes J (J is an integer equal to or greater than 1) elements bj (j = 1, ..., J), and the j-th element bj is an element whose input/output response is determined by a Gaussian function.
However, in the first embodiment, for convenience of description, it is assumed that the number of elements included in the intermediate layer 2 is zero before construction of the network by the network construction apparatus. This is merely an example, and the intermediate layer 2 may include one or more elements even before the construction of the network by the network construction apparatus.
When the output value yi (i = 1, ..., I) of the I elements ai included in the input layer 1 is given to the j-th element bj, the intermediate layer 2 calculates an output value ZJ (j = 1, ..., J) of the j-th element bj from the output value yi of the I elements ai, and outputs the output value ZJ of the element bj to the output layer 3. [0015] The output layer 3 includes an element c, and the element c calculates, for example, a sum of products of the output values ZJ (j = 1, ..., J) of the J elements bj included in the intermediate layer 2 and a weight VJ (j = 1, ..., J) between the intermediate layer 2 and the output layer 3, and outputs the sum.
FIG. 3 illustrates an example in which the number of the elements c included in the output layer 3 is one; however, a plurality of the elements c may be included. [0016] An initial setting unit 11 is implemented by, for example, an initial setting circuit 21 in FIG. 2.
The initial setting unit 11 performs processing of initializing parameters of the Gaussian function related to the element bj for each element included in the intermediate layer 2.
The initial setting unit 11 initializes, as the parameters of the Gaussian

function, for example, a standard deviation value oy of a Gaussian distribution, a center coordinate uy of the Gaussian function, and a weight Wy between the i-th element ai included in the input layer 1 and the j-th element bj included in the intermediate layer 2.
The standard deviation value oy of the Gaussian distribution is a standard deviation value of, for example, the output values yi (i = 1, ..., I) of the I elements ai for the j-th element bj included in the intermediate layer 2.
The center coordinate uy of the Gaussian function is a component of the center of the j-th element bj included in the intermediate layer 2.
[0017] An output value calculating unit 12 is implemented by, for example, an output value calculating circuit 22 in FIG. 2.
As initial values of the parameters of the Gaussian function, the standard deviation value oy of the Gaussian distribution, the center coordinate [j,y of the Gaussian function, and the weight Wy are given from the initial setting unit 11 to the output value calculating unit 12.
The output value calculating unit 12 performs processing of calculating the output value ZJ (J = 1, ..., J) of the j-th element bj, by substituting, for example, the output values yi (i = 1, ..., I) of the I elements ai included in the input layer 1, to the Gaussian function having the initial values of the parameters given from the initial setting unit 11.
[0018] An element construction unit 13 is implemented by, for example, an element construction circuit 23 in FIG. 2.
The element construction unit 13 compares the output values ZJ (j = 1, ..., J) of the J elements bj calculated by the output value calculating unit 12 with a preset threshold value Th, and maintains the number of elements bj included in the intermediate layer 2 when the output value ZJ of any of the elements bj is greater than

the threshold value Th, out of the output values ZJ of the J elements bj included in the intermediate layer 2.
In addition, when all of the output values ZJ of the J elements bj are equal to or less than the threshold value Th, the element construction unit 13 performs element construction processing of increasing the number of elements bj included in the intermediate layer 2.
[0019] A data storage unit 14 is implemented by, for example, a data storage circuit 24 in FIG. 2.
The data storage unit 14 stores the parameters of the Gaussian function related to the element bj initialized by the initial setting unit 11 and the output value ZJ of the element bj calculated by the output value calculating unit 12 in addition to the number J of the elements bj included in the intermediate layer 2.
A parameter updating unit 15 is implemented by, for example, a parameter updating circuit 25 in FIG. 2.
The parameter updating unit 15 updates the weight VJ between the j-th element bj included in the intermediate layer 2 and the element c included in the output layer 3, and the parameters of the Gaussian function stored in the data storage unit 14, by performing supervised learning for learning the parameters of the Gaussian function when the learning data is given to the input layer 1, after the element construction processing is performed by the element construction unit 13.
In addition, the parameter updating unit 15 performs processing of updating the weight Wij between the i-th element ai included in the input layer 1 and the j-th element bj included in the intermediate layer 2 by performing the supervised learning. [0020] In FIG. 1, a network construction apparatus is assumed that the initial setting unit 11, the output value calculating unit 12, the element construction unit 13, the data

storage unit 14, and the parameter updating unit 15 that are components of the network construction apparatus are respectively implemented by dedicated hardware circuits, which are the initial setting circuit 21, the output value calculating circuit 22, the element construction circuit 23, the data storage circuit 24, and the parameter updating circuit 25, as illustrated in FIG. 2.
Here, examples of the data storage circuit 24 include a nonvolatile or volatile semiconductor memory such as random access memory (RAM), read only memory (ROM), flash memory, erasable programmable read only memory (EPROM), and electrically erasable programmable read only memory (EEPROM); a magnetic disk, a flexible disk, an optical disc, a compact disc, a mini disc, a digital versatile disc (DVD), and the like.
In addition, examples of the initial setting circuit 21, the output value calculating circuit 22, the element construction circuit 23, and the parameter updating circuit 25 include a single circuit, a composite circuit, a programmed processor, a parallel-programmed processor, an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a combination thereof. [0021] However, the components of the network construction apparatus are not limited to those implemented by dedicated hardware, and the network construction apparatus may be implemented by software, firmware, or a combination of software and firmware.
Software and firmware are stored as programs in a memory of a computer. The computer means hardware for executing a program, and its examples include a central processing unit (CPU), a central processing device, a processing device, an arithmetic device, a microprocessor, a microcomputer, a processor, a digital signal processor (DSP), and the like. [0022] FIG. 4 is a hardware configuration diagram of a computer in a case where the

network construction apparatus is implemented by software, firmware, or the like.
In the case where the network construction apparatus is implemented by software, firmware, or the like, it is sufficient that the data storage unit 14 is configured on a memory 31 of the computer, and a program for causing the computer to execute processing procedures of the initial setting unit 11, the output value calculating unit 12, the element construction unit 13, and the parameter updating unit 15 is stored in the memory 31, and a processor 32 of the computer executes the program stored in the memory 31.
FIGS. 5 and 6 are flowcharts each illustrating a network construction method that is a processing procedure in the case where the network construction apparatus is implemented by software, firmware, or the like.
[0023] In addition, FIG. 2 illustrates an example in which each of the components of the network construction apparatus is implemented by dedicated hardware, and FIG. 4 illustrates an example in which the network construction apparatus is implemented by software, firmware, or the like; however, some components of the construction apparatus may be implemented by dedicated hardware and the remaining components may be implemented by software, firmware, or the like. [0024] Next, the operation will be described.
Processing contents of the network construction apparatus in the first embodiment are roughly divided into two processing contents.
A first processing content is a processing content for determining a structure of the intermediate layer 2, that is, a processing content for determining the number J of the elements bj included in the intermediate layer 2 and initializing the parameters of the Gaussian function related to the element bj (step ST1 in FIG. 5).
A second processing content is a processing content for updating the initialized

parameters of the Gaussian function and the weight Wy between the i-th element ai included in the input layer 1 and the j-th element bj included in the intermediate layer 2 by performing the supervised learning (step ST2 in FIG. 5). [0025] Hereinafter, with reference to FIG. 6, the first processing content will be specifically described.
In the first embodiment, it is assumed that an initial value of the number J of the elements bj included in the intermediate layer 2 is zero.
In executing the first processing content, the element construction unit 13 determines whether or not the element bj is included in the intermediate layer 2.
That is, the element construction unit 13 determines whether or not the number J of the elements bj included in the intermediate layer 2 is zero (step ST11 in FIG. 6). [0026] When it is determined that the number J of the elements bj included in the intermediate layer 2 is zero (step ST11 in FIG. 6: YES), the element construction unit 13 newly adds one element bj in the intermediate layer 2. That is, the element construction unit 13 creates an element bi in the intermediate layer 2 (step ST12 in FIG.
6).
When it is determined that the number J of the elements bj included in the intermediate layer 2 is not zero (step ST11 of FIG. 6: NO), at this stage, the element construction unit 13 does not perform processing of adding the new element bj in the intermediate layer 2. In the first embodiment, since the initial value of the number J of the elements bj included in the intermediate layer 2 is set to zero, here, the element construction unit 13 creates the element bi in the intermediate layer 2. [0027] When the element construction unit 13 creates the element bi in the intermediate layer 2, the initial setting unit 11 determines whether or not all of the M pieces of learning data xm = (xim, X2m, ..., xim) have already been acquired (step ST 13 in

FIG. 6).
When the M pieces of learning data xm have already been acquired (step ST13 in FIG. 6: YES), the initial setting unit 11 ends the first processing content for determining the structure of the intermediate layer 2.
When the M pieces of learning data xm have not yet been acquired (step ST 13 of FIG. 6: NO), the initial setting unit 11 acquires the learning data xm that has not yet been acquired (step ST 14 of FIG. 6).
In the first embodiment, since the M pieces of learning data xm have not been acquired at this stage, the initial setting unit 11 acquires first learning data x1 = (xi1, X21, ..., xi1) as the learning data xm that has not yet been acquired.
When the first learning data x1 = (xi1, X21, ..., xi1) is acquired, the initial setting unit 11 initializes parameters of the Gaussian function related to the element bi included in the intermediate layer 2 from an output value yi = x;1 of the i-th element ai included in the input layer 1 (step ST 15 in FIG. 6).
[0028] That is, the initial setting unit 11 initializes a standard deviation value cy (i = 1, ..., I) of the Gaussian distribution as one of the parameters of the Gaussian function related to the element bi included in the intermediate layer 2.
The standard deviation value Gi,i of the Gaussian distribution is a standard deviation value of the output values yi = x;1 of the I elements ai for the element bi included in the intermediate layer 2, and is calculated from the output values yi = x;1 of the I elements ai included in the input layer 1. Since processing of calculating the standard deviation value GU itself is a known technique, a detailed description thereof will be omitted.
In addition, the initial setting unit 11 initializes a center coordinate ui,i (i = !,...,!) of the Gaussian function as one of the parameters of the Gaussian function

related to the element bi included in the intermediate layer 2.
The center coordinate uy of the Gaussian function is the output value yi = x;1 of the i-th element ai included in the input layer 1, as indicated in the following expression (1).
Uy = Xi* (1)
Further, the initial setting unit 11 initializes a weight Wy (i = 1, ..., I) between the i-th element ai included in the input layer 1 and the element b i included in the intermediate layer 2, as one of the parameters of the Gaussian function related to the element b i included in the intermediate layer 2, as indicated in the following expression
(2).
Wy = 1 (2)
Here, an example is described where the weight Wy is set to 1; however, this is merely an example, and a value other than 1 may be set. [0029] When the parameters of the Gaussian function related to the element bi included in the intermediate layer 2 are initialized, the initial setting unit 11 outputs the initial values of the parameters to the output value calculating unit 12, and stores the initial values of the parameters in the data storage unit 14.
[0030] The output value calculating unit 12 acquires from the initial setting unit 11 the initial values of the parameters of the Gaussian function related to the element bi included in the intermediate layer 2.
In addition, the output value calculating unit 12 acquires the first learning data
X1 = (X11, X21, ..., XI1).
[0031] When the first learning data x1 is acquired, the output value calculating unit 12 calculates an output value zi of the element bi included in the intermediate layer 2 (step ST16inFIG. 6).

That is, the output value calculating unit 12 calculates the output value zi of the element bi included in the intermediate layer 2, by substituting the initial values of the parameters of the Gaussian function related to the element bi included in the intermediate layer 2 and the output value yi = x;1 of the I elements ai included in the input layer 1, to the Gaussian function indicated in the following expression (3).
z^exXL 55 j (3)
When the output value zi of the element bi included in the intermediate layer 2 is calculated, the output value calculating unit 12 outputs the output value zi of the element bi to the element construction unit 13, and stores the output value zi of the element bi in the data storage unit 14.
[0032] When the output value zi of the element bi is received from the output value calculating unit 12, the element construction unit 13 compares the output value zi of the element bi with the preset threshold value Th (step ST17 in FIG. 6).
Since the output value zi of the element bi is 1 at the maximum, as the threshold value Th, a positive real number equal to or less than 1, or a positive real number equal to or greater than e"°25 and equal to or less than 1 is conceivable. [0033] When the output value zi of the element bi included in the intermediate layer 2 is greater than the threshold value Th (step ST17 in FIG. 6: NO), the element construction unit 13 maintains the number J (J = 1) of the elements included in the intermediate layer 2. When the output value zi of the element bi included in the intermediate layer 2 is greater than the threshold value Th, it can be said that the element bi included in the intermediate layer 2 is an element corresponding to an output value y i of an element ai included in the input layer 1. That is, it can be said that the element bi included in the intermediate layer 2 is an element representing the center

coordinate of the Gaussian distribution in the Gaussian function. For this reason, there is no need to add a new element bi to the intermediate layer 2, so that the element construction unit 13 maintains the number J (J = 1) of the elements included in the intermediate layer 2.
When the output value zi of the element bi included in the intermediate layer 2 is equal to or less than the threshold value Th (step ST17 in FIG. 6: YES), the element construction unit 13 adds the new element bi to the intermediate layer 2 so that the number of elements included in the intermediate layer 2 is increased (step ST18 in FIG. 6). When the output value zi of the element b i included in the intermediate layer 2 is equal to or less than the threshold value Th, it cannot be said that the element bi included in the intermediate layer 2 is the element corresponding to the output value yi of the element ai included in the input layer 1. For this reason, the element construction unit 13 adds the new element bi to the intermediate layer 2.
In the first embodiment, for convenience of description, the output value zi of the element bi included in the intermediate layer 2 is equal to or less than the threshold value Th, so that the new element bi is added to the intermediate layer 2. [0034] When the element construction unit 13 adds the new element bi to the intermediate layer 2 or maintains the number J of the elements included in the intermediate layer 2, the initial setting unit 11 determines whether or not all of the M pieces of learning data xm = (xim, X2m, ..., xim) have already been acquired (step ST 13 in FIG. 6).
When the M pieces of learning data xm have already been acquired (step ST13 in FIG. 6: YES), the initial setting unit 11 ends the first processing content for determining the structure of the intermediate layer 2.
When the M pieces of learning data xm have not yet been acquired (step ST 13

of FIG. 6: NO), the initial setting unit 11 acquires the learning data xm that has not yet been acquired (step ST 14 of FIG. 6).
In the first embodiment, since the M pieces of learning data xm have not been acquired at this stage, the initial setting unit 11 acquires second learning data x2 = (xi2, X22, ..., xi2) as the learning data xm that has not yet been acquired.
When the second learning data x2 = (xi2, X22, ..., xi2) is acquired, the initial setting unit 11 initializes the parameters of the Gaussian function related to the elements bi and b2 included in the intermediate layer 2 from the output value yi = x;2 of the i-th element ai included in the input layer 1 (step ST 15 in FIG. 6).
[0035] That is, the initial setting unit 11 calculates the standard deviation value oy (i = 1, ..., I: j = 1, 2) of the Gaussian distribution from the output values yi = x;2 of the I elements ai included in the input layer 1, as one of the parameters of the Gaussian function related to the elements bi and b2 included in the intermediate layer 2.
In addition, the initial setting unit 11 initializes the output value yi = x;2 of the i-th element ai included in the input layer 1 to the center coordinate uij (i = 1, ..., I: j = 1, 2) of the Gaussian function, as one of the parameters of the Gaussian function related to the elements bi and b2 included in the intermediate layer 2, as indicated in the following expression (4).
M-iJ = X!2 (4)
Further, the initial setting unit 11 initializes the weight Wy between the i-th element ai (i = 1, ..., I) included in the input layer 1 and the j-th element bj (j = 1, 2) included in the intermediate layer 2, as one of the parameters of the Gaussian function related to the elements bi and b2 included in the intermediate layer 2, as indicated in the following expression (5).
Wij = 1 (5)

Here, an example is described in which the weight Wy is set to 1; however, this is merely an example, and a value other than 1 may be set.
[0036] When the parameters of the Gaussian function related to the elements b i and hi included in the intermediate layer 2 are initialized, the initial setting unit 11 outputs the initial values of the parameters to the output value calculating unit 12, and stores the initial values of the parameters in the data storage unit 14.
[0037] The output value calculating unit 12 acquires from the initial setting unit 11 the initial values of the parameters of the Gaussian function related to the elements bi and b2 included in the intermediate layer 2.
In addition, the output value calculating unit 12 acquires the second learning datax2 = (xi2, X22, ..., xi2).
[0038] When the second learning data x2 is acquired, the output value calculating unit 12 calculates the output value zi of the element bi included in the intermediate layer 2 (stepST16inFIG. 6).
That is, the output value calculating unit 12 calculates the output value zi of the element bi included in the intermediate layer 2, by substituting the initial values of the parameters of the Gaussian function related to the element bi included in the intermediate layer 2, and the output values yi = x;2 of the I elements ai included in the input layer 1, to the Gaussian function indicated in the following expression (6).
The initial values of the parameters of the Gaussian function related to the element bi included in the intermediate layer 2 are the standard deviation value oi,i (i = 1, ..., I) of the Gaussian distribution, the center coordinate ui,i (i = 1, ..., I), and the weight Wu (i = l, ..., I).
In addition, the output value calculating unit 12 calculates an output value Z2 of the element b2 included in the intermediate layer 2, by substituting the initial values of

the parameters of the Gaussian function related to the element b2 included in the intermediate layer 2, and the output values yi = x;2 of the I elements ai included in the input layer 1, to the Gaussian function indicated in the following expression (6).
The initial values of the parameters of the Gaussian function related to the element b2 included in the intermediate layer 2 are a standard deviation value oi,2 (i = 1, ..., I) of the Gaussian distribution, a center coordinate [a,2 (i = 1, ..., I) of the Gaussian function, and a weight Wi,2 (i = 1, ..., I). [0039]
zi = exXL—^—J (6)
Here, in the expression (6), I = 1, ..., I, and j = 1,2.
When the output value zi of the element bi and the output value Z2 of the element b2 included in the intermediate layer 2 are calculated, the output value calculating unit 12 outputs the output values zi and Z2 of the elements bi and b2 to the element construction unit 13, and stores the output values zi and Z2 of the elements bi and b2 in the data storage unit 14.
[0040] When the output values zi and Z2 of the elements bi and b2 are received from the output value calculating unit 12, the element construction unit 13 compares the output values zi and Z2 of the elements bi and b2 with the threshold value Th (step ST17inFIG. 6).
When any of the output values zi and Z2 of the elements bi and b2 included in the intermediate layer 2 is greater than the threshold value Th (step ST 17 in FIG. 6: NO), the element construction unit 13 maintains the number J (J = 2) of the elements included in the intermediate layer 2. When any of the output values zi and Z2 of the elements bi and b2 included in the intermediate layer 2 is greater than the threshold

value Th, it can be said that an element whose output value is greater than the threshold value Th is an element corresponding to the output value of an element included in the input layer 1. For this reason, there is no need to add a new element b3 to the intermediate layer 2, so that the element construction unit 13 maintains the number J (J = 2) of the elements included in the intermediate layer 2.
When all of the output values zi and Z2 of the elements bi and hi included in the intermediate layer 2 are equal to or less than the threshold value Th (step ST 17 in FIG. 6: YES), the element construction unit 13 adds the new element b3 to the intermediate layer 2 so that the number of elements included in the intermediate layer 2 is increased (step ST 18 in FIG. 6). When all of the output values zi and Z2 of the elements bi and hi included in the intermediate layer 2 are equal to or less than the threshold value Th, it cannot be said that any element included in the intermediate layer 2 is an element corresponding to the output value of an element included in the input layer 1. For this reason, the element construction unit 13 adds the new element b3 to the intermediate layer 2.
In the first embodiment, for convenience of description, all of the output values zi and Z2 of the elements bi and b2 included in the intermediate layer 2 are equal to or less than the threshold value Th, so that the new element b3 is added to the intermediate layer 2.
[0041] Hereinafter, description will be made assuming that the number of elements currently included in the intermediate layer 2 is J (J ^ 3).
When the element construction unit 13 adds a new element to the intermediate layer 2 or maintains the number J of the elements included in the intermediate layer 2, the initial setting unit 11 determines whether or not all of the M pieces of learning data xm = (xim, X2m, ..., xim) have already been acquired (step ST13 in FIG. 6).

When the M pieces of learning data xm have already been acquired (step ST13 in FIG. 6: YES), the initial setting unit 11 ends the first processing content for determining the structure of the intermediate layer 2.
When the M pieces of learning data xm have not yet been acquired (step ST 13 of FIG. 6: NO), the initial setting unit 11 acquires the learning data xm that has not yet been acquired (step ST 14 of FIG. 6).
For example, when the m-th learning data xm of the M pieces of learning data xm has not yet been acquired, the initial setting unit 11 acquires the m-th learning data xm.
When the m-th learning data xm is acquired, the initial setting unit 11 initializes the parameters of the Gaussian function related to the elements bi to bj included in the intermediate layer 2, from the output value yi = x;m of the i-th element ai included in the input layer 1 (step ST 15 in FIG. 6).
[0042] That is, the initial setting unit 11 calculates the standard deviation value oy (i = 1, ..., I: j = 1, ..., J) of the Gaussian distribution from the output values yi = x;m of the I elements ai included in the input layer 1, as one of the parameters of the Gaussian function related to the elements bi to bj included in the intermediate layer 2.
In addition, the initial setting unit 11 initializes the output value yi = x;m of the i-th element ai included in the input layer 1 to the center coordinate uij of the Gaussian function, as one of the parameters of the Gaussian function related to the elements bi to bj included in the intermediate layer 2, as indicated in the expression (4).
Further, the initial setting unit 11 initializes the weight Wy between the i-th element ai (i = 1, ..., J) included in the input layer 1 and the j-th element bj (j = 1, ..., J) included in the intermediate layer 2, as one of the parameters of the Gaussian function related to the elements bi to bj included in the intermediate layer 2, as indicated in the

expression (5).
Here, an example is described in which the weight Wy is set to 1; however, this is merely an example, and a value other than 1 may be set.
[0043] When the parameters of the Gaussian function related to the elements b i to bj included in the intermediate layer 2 are initialized, the initial setting unit 11 outputs the initial values of the parameters to the output value calculating unit 12, and stores the initial values of the parameters in the data storage unit 14.
[0044] The output value calculating unit 12 acquires from the initial setting unit 11 the initial values of the parameters of the Gaussian function related to the elements bi to bj included in the intermediate layer 2.
In addition, the output value calculating unit 12 acquires the m-th learning data
Xm = (xim,X2m, ...,Xim).
[0045] When the m-th learning data xm = (xim, X2m, ..., xim) is acquired, the output value calculating unit 12 calculates the output values zi to zj of the elements bi to bj included in the intermediate layer 2 (step ST 16 in FIG. 6).
That is, the output value calculating unit 12 calculates the output value ZJ (j = 1, ..., J) of the element bj included in the intermediate layer 2, by substituting the initial values of the parameters of the Gaussian function related to the element bj (j = 1, ..., J) included in the intermediate layer 2, and the output values yi = x;m of the I elements ai included in the input layer 1, to the Gaussian function indicated in the expression (6).
The initial values of the parameters of the Gaussian function related to the element bj included in the intermediate layer 2 are the standard deviation value oy (i = 1, ..., I: j = 1, ..., J) of the Gaussian distribution, the center coordinate (j,y (i = 1, ..., I: j = 1, ..., J) of the Gaussian function, and the weight Wy (i = 1, ..., I: j = 1, ..., J). [0046] When the output values zi to zj of the elements bi to bj included in the

intermediate layer 2 are calculated, the output value calculating unit 12 outputs the output values zi to zj of the elements bi to bj to the element construction unit 13, and stores the output values zi to zj of the elements bi to bj in the data storage unit 14. [0047] When the output values zi to zj of the elements bi to bj are received from the output value calculating unit 12, the element construction unit 13 compares the output values zi to zj of the elements bi to bj with the threshold value Th (step ST17 in FIG.
6).
When any of the output values zi to zj of the elements bi to bj included in the intermediate layer 2 is greater than the threshold value Th (step ST17 in FIG. 6: NO), the element construction unit 13 maintains the number j of the elements included in the intermediate layer 2.
When all of the output values zi to zj of the elements bi to bj included in the intermediate layer 2 are equal to or less than the threshold value Th (step ST 17 in FIG. 6: YES), the element construction unit 13 adds a new element to the intermediate layer 2 so that the number of elements included in the intermediate layer 2 is increased (step ST18inFIG 6).
The processing of steps ST13 to ST18 is repeatedly performed until all of the M pieces of learning data xm are acquired, and when it is determined to be "YES" in the determination processing of step ST13, the first processing content is ended. [0048] After the first processing content is ended, the parameter updating unit 15 performs the second processing content.
That is, the parameter updating unit 15 updates the parameters of the Gaussian function stored in the data storage unit 14 and the weight VJ between the j-th element bj included in the intermediate layer 2 and the element c included in the output layer 3, by performing supervised learning for learning the parameters of the Gaussian function

stored in the storage unit 14, that is, the parameters of the Gaussian function related to the J elements bj (j = 1, ..., J) included in the intermediate layer 2, each time the M pieces of learning data xm = (xim, X2m, ..., xim) are sequentially given, after the first processing content is ended.
In addition, the parameter updating unit 15 updates the weight Wy between the i-th element ai included in the input layer 1 and the j-th element bj included in the intermediate layer 2 by performing the supervised learning.
Since supervised learning itself is a known technique, a detailed description will be omitted, but for example, supervised learning can be performed by using a known back propagation method.
Note that, the parameter updating unit 15 may update all the parameters in the Gaussian function, but may update only some parameters in the Gaussian function.
For example, a method A can be considered in which out of the standard deviation value oy of the Gaussian distribution, the center coordinate uy of the Gaussian function, and the weight Wy, the center coordinate uy of the Gaussian function is fixed and the standard deviation value oy and the weight Wy are updated.
In addition, a method B in which the standard deviation value oy of the Gaussian distribution is fixed and the center coordinate [j,y of the Gaussian function and the weight Wy are updated, and a method C in which the weight Wy is fixed and the standard deviation value oy of the Gaussian distribution and the center coordinate [j,y of the Gaussian function are updated, can be considered.
[0049] As is apparent from the above description, according to the first embodiment, the element construction unit 13 compares the output value ZJ of one or more elements bj included in the intermediate layer 2 calculated by the output value calculating unit 12 and the threshold value Th, and maintains the number of elements bj included in the

intermediate layer 2 when any of the output values ZJ out of the output values ZJ of the one or more elements bj included in the intermediate layer 2 is greater than the threshold value Th, and increases the number of elements bj included in the intermediate layer 2 when all of the output values ZJ of the one or more elements bj included in the intermediate layer 2 are equal to or less than the threshold value Th, so that there is an effect that the number of elements bj included in the intermediate layer 2 of the neural network can be automatically determined even when the element bj included in the intermediate layer 2 of the neural network is an element whose input/output response is determined by the Gaussian function. [0050] Second Embodiment.
In the first embodiment, an example in which the number of intermediate layers 2 included in the neural network is one has been described.
In this second embodiment, an example in which the number of intermediate layers 2 included in the neural network is two or more will be described. [0051] FIG. 7 is an explanatory diagram illustrating an example of a neural network to which a network construction apparatus according to the second embodiment of the present invention is applied. In FIG. 7, since the same reference numerals as those in FIG. 3 denote the same or corresponding portions, the description thereof will be omitted.
Intermediate layers 2-1 to 2-G are connected in cascade between the input layer 1 and the output layer 3.
The g-th intermediate layer 2-g (g = 1, 2, ..., G) includes J (J is an integer equal to or greater than 1) elements bjg (j = 1, 2, ..., J), and the j-th element bjg is an element whose input/output response is determined by a Gaussian function. [0052] When the output values yi = x;m of the I elements ai included in the input layer

1 are given to the j-th element bj1, similarly to the intermediate layer 2 in FIG. 3, the first intermediate layer 2-1 calculates an output value zjm of the j-th element bjm from the output values yi of the I elements ai, and outputs the output value zjm of the element bjm to the second intermediate layer 2-2.
When the output values zjg_1 of all the elements bjg_1 included in the (g - l)-th intermediate layer 2-(g - 1) are given to the j-th element bjg, the g-th intermediate layer 2-g (g = 2, 3, ..., G-l) calculates the output value zjg of the j-th element bjg from the output values zjg_1 of all the elements bjg_1, and outputs the output value zjg of the element bjg to the G-th intermediate layer 2-G.
When the output values ZJG_1 of all the elements bjG_1 included in the (G - l)-th intermediate layer 2-(G - 1) are given to the j-th element bjG, the G-th intermediate layer 2-G calculates the output value ZJG of the j-th element bjG from the output values ZJG_1 of all the elements bj0"1, and outputs the output value ZJG of the element bjG to the output layer 3.
In FIG. 7, although the numbers of elements included in the intermediate layers 2-1 to 2-G are all illustrated to be the same as each other, it is needless to say that the numbers of elements included in the intermediate layers 2-1 to 2-G may be determined to be different from each other depending on processing of the element construction unit 13 described later. [0053] Next, the operation will be described.
In the first embodiment, an example has been described in which the structure of the intermediate layer 2 is determined before the supervised learning is performed; however, in the second embodiment, an example will be described in which structures of the intermediate layers 2-1 to 2-G are determined before the supervised learning is performed.

In the second embodiment, the structure of the intermediate layer 2-g is determined in order from the intermediate layer 2-g (g = 1, 2, ..., G) on the input layer 1 side, out of the intermediate layers 2-1 to 2-G.
[0054] FIG. 8 is a flowchart illustrating a network construction method that is a processing procedure in the case where the network construction apparatus is implemented by software, firmware, or the like.
The processing of determining the structure of the intermediate layer 2-1 on the side closest to the input layer 1 out of the intermediate layers 2-1 to 2-G is the same as the processing of determining the structure of the intermediate layer 2 in FIG. 3, so that the description thereof will be omitted.
Hereinafter, with reference to FIG. 8, processing contents of determining the structure of the intermediate layer 2-g (g = 2, 3, ..., G) will be described.
In the second embodiment, it is assumed that an initial value of the number J of the elements bjg included in the intermediate layer 2-g is zero.
In addition, it is assumed that the number J of the elements bjg_1 included in the intermediate layer 2-(g - 1) has already been determined to be K, and the output value zkg_1 of the k-th element bkg_1 (k = 1, ..., K) included in the intermediate layer 2-(g - 1) has been calculated.
[0055] In executing the first processing content, the element construction unit 13 determines whether or not the element bjg is included in the intermediate layer 2-g.
That is, the element construction unit 13 determines whether or not the number J of the elements bjg included in the intermediate layer 2-g is zero (step ST21 in FIG. 8). [0056] When it is determined that the number J of the elements bjg included in the intermediate layer 2-g is zero (step ST21 in FIG. 8: YES), the element construction unit 13 newly adds one element bjg in the intermediate layer 2-g. That is, the element

construction unit 13 creates an element big in the intermediate layer 2-g (step ST22 in FIG. 8).
When it is determined that the number J of the elements bjg included in the intermediate layer 2-g is not zero (step ST21 in FIG. 8: NO), at this stage, the element construction unit 13 does not perform processing of adding the new element bjg in the intermediate layer 2-g. In the second embodiment, since the initial value of the number J of the elements bjg included in the intermediate layer 2-g is set to zero, here, the element construction unit 13 creates the element big in the intermediate layer 2-g. [0057] When the element construction unit 13 creates the element big in the intermediate layer 2-g, the initial setting unit 11 determines whether or not the M pieces of learning data xm = (xim, X2m, ..., xim) (m = 1, ..., M) have been given to the input layer 1, and output values zkg_1 of the K elements bkg_1 (k = 1, ..., K) included in the immediately preceding intermediate layer 2-(g - 1) based on the M-th learning data xM have been acquired (step ST23 in FIG. 8).
When the output values zkg_1 of the K elements bkg_1 (k = 1, ..., K) included in the immediately preceding intermediate layer 2-(g - 1) based on the M-th learning data xM have been acquired (step ST23 in FIG. 8: YES), the initial setting unit 11 ends the first processing content for determining the structure of the intermediate layer 2-g.
When the output values zkg_1 of the K elements bkg_1 included in the immediately preceding intermediate layer 2-(g - 1) based on the M-th learning data xM have not yet been acquired (step ST23 in FIG. 8: NO), the initial setting unit 11 acquires the output values zkg_1 of the K elements bkg_1 included in the immediately preceding intermediate layer 2-(g - 1) that have not yet been acquired (step ST24 in FIG. 8).
In the second embodiment, since the output values zkg_1 of the K elements bkg_1 included in the immediately preceding intermediate layer 2-(g - 1) based on the first

learning data x have not been acquired at this stage, the initial setting unit 11 acquires the output values zkg_1 of the K elements bkg_1 included in the immediately preceding intermediate layer 2-(g - 1) based on the first learning data x1.
When the output values zkg_1 of the K elements bkg_1 included in the immediately preceding intermediate layer 2-(g - 1) based on the first learning data x1 are acquired, the initial setting unit 11 initializes the parameters of the Gaussian function related to the element big included in the intermediate layer 2-g from the output values zkg_1 of the K elements bkg_1 included in the intermediate layer 2-(g - 1) (step ST25 in FIG. 8).
[0058] That is, the initial setting unit 11 calculates a standard deviation value ok,ig (k = 1, ..., K) of the Gaussian distribution from the output values zkg_1 of the K elements bkg_1 included in the intermediate layer 2-(g - 1), as one of the parameters of the Gaussian function related to the element big included in the intermediate layer 2-g.
In addition, the initial setting unit 11 initializes a center coordinate [j,k,ig (k = 1, ..., K) of the Gaussian function, as one of the parameters of the Gaussian function related to the element big included in the intermediate layer 2-g.
The center coordinate [j,k,ig of the Gaussian function is the output value zkg_1 of the k-th element bkg_1 included in the intermediate layer 2-(g - 1), as indicated in the following expression (7).
UMg = Zk8"1 (7)
Further, the initial setting unit 11 initializes a weight Wk,ig between the k-th element bkg_1 included in the intermediate layer 2-(g - 1) and the element big included in the intermediate layer 2-g, as one of the parameters of the Gaussian function related to the element big included in the intermediate layer 2-g, as indicated in the following expression (8).

Wk,ig = 1 (8)
Here, an example is described in which the weight Wk,ig is set to 1; however, this is merely an example, and a value other than 1 may be set. [0059] When the parameters of the Gaussian function related to the element big included in the intermediate layer 2-g are initialized, the initial setting unit 11 outputs the initial values of the parameters to the output value calculating unit 12, and stores the initial values of the parameters in the data storage unit 14.
[0060] The output value calculating unit 12 acquires from the initial setting unit 11 the initial values of the parameters of the Gaussian function related to the element big included in the intermediate layer 2-g.
In addition, the output value calculating unit 12 acquires the output values zkg_1 of the K elements bkg_1 included in the immediately preceding intermediate layer 2-(g -1) based on the first learning data x1.
[0061] When the output values zkg_1 of the K elements bkg_1 included in the immediately preceding intermediate layer 2-(g - 1) based on the first learning data x1 are acquired, the output value calculating unit 12 calculates an output value zig of the element big included in the intermediate layer 2-g (step ST26 in FIG. 8).
That is, the output value calculating unit 12 calculates the output value zig of the element big included in the intermediate layer 2-g, by substituting the initial values of the parameters of the Gaussian function related to the element big included in the intermediate layer 2-g and the output values zkg_1 of the K elements bkg_1 included in the intermediate layer 2-(g -1), to the Gaussian function indicated in the following expression (9).

When the output value zig of the element big included in the intermediate layer 2-g is calculated, the output value calculating unit 12 outputs the output value zig of the element big to the element construction unit 13, and stores the output value zig of the element big in the data storage unit 14.
[0062] When the output value zig of the element big is received from the output value calculating unit 12, the element construction unit 13 compares the output value zig of the element big with the preset threshold value Th (step ST27 in FIG. 8).
Since the output value zig of the element big is 1 at the maximum, as the threshold value Th, a positive real number equal to or less than 1, or a positive real number equal to or greater than e"°25 and equal to or less than 1 is conceivable. [0063] When the output value zig of the element big included in the intermediate layer 2-g is greater than the threshold value Th (step ST27 of FIG. 8: NO), the element construction unit 13 maintains the number J (J = 1) of the elements included in the intermediate layer 2-g.
When the output value zig of the element big included in the intermediate layer 2-g is equal to or less than the threshold value Th (step ST27 in FIG. 8: YES), the element construction unit 13 adds a new element b2g to the intermediate layer 2-g so that the number of elements included in the intermediate layer 2-g is increased (step ST28 in FIG. 8).
In the second embodiment, for convenience of description, the output value zig of the element big included in the intermediate layer 2-g is equal to or less than the threshold value Th, so that the new element b2g is added to the intermediate layer 2-g.

[0064] When the element construction unit 13 adds the new element b2g to the intermediate layer 2-g or maintains the number J of the elements included in the intermediate layer 2-g, the initial setting unit 11 determines whether or not the output values zkg_1 of the K elements bkg_1 (k = 1, ..., K) included in the immediately preceding intermediate layer 2-(g - 1) based on the M-th learning data xM have already been acquired (step ST23 in FIG. 8).
When the output values zkg_1 of the K elements bkg_1 (k = 1, ..., K) included in the immediately preceding intermediate layer 2-(g - 1) based on the M-th learning data xm have already been acquired (step ST23 in FIG. 8: YES), the initial setting unit 11 ends the first processing content for determining the structure of the intermediate layer 2.
When the output values zkg_1 of the K elements bkg_1 (k = 1, ..., K) included in the immediately preceding intermediate layer 2-(g - 1) based on the M-th learning data xM have not yet been acquired (step ST23 of FIG. 8: NO), the initial setting unit 11 acquires the output values zkg_1 of the K elements bkg_1 (k = 1, ..., K) included in the immediately preceding intermediate layer 2-(g - 1) that have not yet been acquired (step ST24 in FIG. 8).
In the second embodiment, since the output values zkg_1 of the K elements bkg_1 (k = 1, ..., K) included in the intermediately preceding layer 2-(g - 1) based on the second learning data x2have not been acquired at this stage, the initial setting unit 11 acquires the output values zkg_1 of the K elements bkg_1 (k = 1, ..., K) included in the intermediately preceding intermediate layer 2-(g - 1) based on the second learning data x2.
When the output values zkg_1 of the K elements bkg_1 (k = 1, ..., K) included in the intermediately preceding layer 2-(g - 1) based on the second learning data x2 are

acquired, the initial setting unit 11 initializes the parameters of the Gaussian function related to the elements big and b2g included in the intermediate layer 2-g from the output values zkg_1 of the K elements bkg_1 (k = 1, ..., K) included in the intermediately preceding layer 2-(g - 1) based on the second learning data x2 (step ST25 in FIG. 8). [0065] That is, the initial setting unit 11 calculates a standard deviation value ok,jg (k = 1, ..., K: j = 1, 2) of the Gaussian distribution from the output values zkg_1 of the K elements bkg_1 included in the intermediate layer 2-(g - 1), as one of the parameters of the Gaussian function related to the elements big and b2g included in the intermediate layer 2-g.
In addition, the initial setting unit 11 initializes the output value zkg_1 of the k-th element bkg_1 included in the intermediate layer 2-(g - 1) to a center coordinate [j,k,jg (k = 1, ..., K: j = 1, 2) of the Gaussian function, as one of the parameters of the Gaussian function related to the elements big and b2g included in the intermediate layer 2-g, as indicated in the following expression (10).
^kog = zkg-1 (10)
Further, the initial setting unit 11 initializes a weight Wkjg (k = 1, ..., K: j = 1, 2) between the k-th element bkg_1 included in the intermediate layer 2-(g -1) and the j-th element bjg (j = 1, 2) included in the intermediate layer 2-g, as one of the parameters of the Gaussian function related to the elements big and b2g included in the intermediate layer 2-g, as indicated in the following expression (11).
Wk/=1 (11)
Here, an example is described in which the Wkjg is set to 1; however, this is merely an example, and a value other than 1 may be set.
[0066] When the parameters of the Gaussian function related to the elements b ig and b2g included in the intermediate layer 2-g are initialized, the initial setting unit 11

outputs the initial values of the parameters to the output value calculating unit 12, and stores the initial values of the parameters in the data storage unit 14. [0067] The output value calculating unit 12 acquires from the initial setting unit 11 the initial values of the parameters of the Gaussian function related to the elements big and b2g included in the intermediate layer 2-g.
In addition, the output value calculating unit 12 acquires the output values zkg_1 of the K elements bkg_1 included in the immediately preceding intermediate layer 2-(g -1) based on the second learning data x2.
[0068] When the output values zkg_1 of the K elements bkg_1 included in the immediately preceding intermediate layer 2-(g - 1) based on the second learning data x2 are acquired, the output value calculating unit 12 calculates the output value zig of the element big and an output value Z2g of the element b2g included in the intermediate layer 2-g (step ST26 in FIG. 8).
That is, the output value calculating unit 12 calculates the output value zig of the element big included in the intermediate layer 2-g, by substituting the initial values of the parameters of the Gaussian function related to the element big included in the intermediate layer 2-g and the output values zkg_1 of the K elements bkg_1 included in the intermediate layer 2-(g - 1), to the Gaussian function indicated in the following expression (12).
The initial values of the parameters of the Gaussian function related to the element big included in the intermediate layer 2-g are the standard deviation value ok,ig (k = 1, ..., K) of the Gaussian distribution, the center coordinate [j,k,ig (k = 1, ..., K) of the Gaussian function, and the weight Wk,ig (k = 1, ..., K).
In addition, the output value calculating unit 12 calculates the output value Z2g of the element b2g included in the intermediate layer 2-g, by substituting the initial

values of the parameters of the Gaussian function related to the element b2g included in the intermediate layer 2-g and the output values zkg_1 of the K elements bkg_1 included in the intermediate layer 2-(g - 1), to the Gaussian function indicated in the following expression (12).
The initial values of the parameters of the Gaussian function related to the element b2g included in the intermediate layer 2-g are a standard deviation value ok,2g (k = 1, ..., K) of the Gaussian distribution, a center coordinate [j,k,2g (k = 1, ..., K) of the Gaussian function, and a weight Wk,2g (k = 1, ..., K). [0069]
Here, j = 1, 2 in the expression (12).
When the output value zig of the element big and the output value Z2g of the element b2g included in the intermediate layer 2-g are calculated, the output value calculating unit 12 outputs the output values zig and Z2g of the elements big and b2g to the element construction unit 13, and stores the output values zig and Z2g of the elements big and b2g in the data storage unit 14.
[0070] When the output values zig and Z2g of the elements b ig and b2g are received from the output value calculating unit 12, the element construction unit 13 compares the output values zig and Z2g of the elements big and b2g with the threshold value Th (step ST27 in FIG. 8).
When any of the output values zig and Z2g of the elements big and b2g included in the intermediate layer 2-g is greater than the threshold value Th (step ST27 in FIG. 8: NO), the element construction unit 13 maintains the number J (J = 2) of the elements

included in the intermediate layer 2-g.
When all of the output values zig and Z2g of the elements big and b2g included in the intermediate layer 2-g are equal to or less than the threshold value Th (step ST27 in FIG. 8: YES), the element construction unit 13 adds a new element b3g to the intermediate layer 2-g so that the number of elements included in the intermediate layer 2-g is increased (step ST28 in FIG. 8).
In the second embodiment, for convenience of description, all of the output values zig and Z2g of the elements big and b2g included in the intermediate layer 2-g are equal to or less than the threshold value Th, so that the new element b3g is added to the intermediate layer 2.
[0071] Hereinafter, description will be made assuming that the number of elements currently included in the intermediate layer 2-g is J (J ^ 3).
When the element construction unit 13 adds a new element to the intermediate layer 2-g or maintains the number J of the elements included in the intermediate layer 2-g, the initial setting unit 11 determines whether or not the output values zkg_1 of the K elements bkg_1 (k = 1, ..., K) included in the immediately preceding intermediate layer 2-(g - 1) based on the M-th learning data xM have already been acquired (step ST23 in FIG. 8).
When the output values zkg_1 of the K elements bkg_1 (k = 1, ..., K) included in the immediately preceding intermediate layer 2-(g - 1) based on the M-th learning data xM have already been acquired (step ST23 in FIG. 8: YES), the initial setting unit 11 ends the first processing content for determining the structure of the intermediate layer 2.
When the output values zkg_1 of the K elements bkg_1 (k = 1, ..., K) included in the immediately preceding intermediate layer 2-(g - 1) based on the M-th learning data

xm have not yet been acquired (step ST23 of FIG. 8: NO), the initial setting unit 11 acquires the output value zkg_1 of the K elements bkg_1 (k = 1, ..., K) included in the immediately preceding intermediate layer 2-(g - 1) that have not yet been acquired (step ST24 in FIG. 8).
In the second embodiment, since the output values zkg_1 of the K elements bkg_1 (k = 1, ..., K) included in the immediately preceding intermediate layer 2-(g - 1) based on the m-th learning data xm out of the M pieces of learning data have not been acquired at this stage, the initial setting unit 11 acquires the output value zkg_1 of the K elements bkg_1 (k = 1, ..., K) included in the immediately preceding intermediate layer 2-(g - 1) based on the m-th learning data xm.
When the output values zkg_1 of the K elements bkg_1 (k = 1, ..., K) included in the immediately preceding intermediate layer 2-(g - 1) based on the m-th learning data xm are acquired, the initial setting unit 11 initializes the parameters of the Gaussian function related to the elements big to bjg included in the intermediate layer 2-g from the output values zkg_1 of the K elements bkg_1 (k = 1, ..., K) included in the immediately preceding intermediate layer 2-(g - 1) based on the m-th learning data xm (step ST25 in FIG. 8).
[0072] That is, the initial setting unit 11 calculates the standard deviation value okjg (k = 1, ..., K: J: j = 1, ..., J) of the Gaussian distribution from the output values zkg_1 of the K elements bkg_1 included in the intermediate layer 2-(g - 1), as one of the parameters of the Gaussian function related to the elements big to bjg included in the intermediate layer 2-g.
In addition, the initial setting unit 11 initializes the output value zkg_1 of the k-th element bkg_1 included in the intermediate layer 2-(g - 1) to the center coordinate (j,kjg (k = 1,..., K: j = 1, ..., J) of the Gaussian function, as one of the parameters of the Gaussian

function related to the elements big to bjg included in the intermediate layer 2-g, as indicated in the expression (10).
Further, the initial setting unit 11 initializes the weight Wk,jg (k = 1, ..., K: J: j = 1, ..., J) between the k-th element bkg_1 included in the intermediate layer 2-(g - 1) and the j-th element bjg (j = 1, ..., J) included in the intermediate layer 2-g, as one of the parameters of the Gaussian function related to the elements big to bjg included in the intermediate layer 2-g, as indicated in the expression (11).
Here, an example is described in which the Wkjg is set to 1; however, this is merely an example, and a value other than 1 may be set.
[0073] When the parameters of the Gaussian function related to the elements b ig to bjg included in the intermediate layer 2-g are initialized, the initial setting unit 11 outputs the initial values of the parameters to the output value calculating unit 12, and stores the initial values of the parameters in the data storage unit 14.
[0074] The output value calculating unit 12 acquires from the initial setting unit 11 the initial values of the parameters of the Gaussian function related to the elements big to bjg included in the intermediate layer 2.
In addition, the output value calculating unit 12 acquires the output values zkg_1 of the K elements bkg_1 included in the immediately preceding intermediate layer 2-(g -1) based on the m-th learning data xm.
[0075] When the output values zkg_1 of the K elements bkg_1 included in the immediately preceding intermediate layer 2-(g - 1) based on the m-th learning data xm are acquired, the output value calculating unit 12 calculates the output values zig to zjg of the elements big to bjg included in the intermediate layer 2-g (step ST26 in FIG. 8).
That is, the output value calculating unit 12 calculates the output value zjg (j = 1, ..., J) of the element bjg included in the intermediate layer 2-g, by substituting the

initial values of the parameters of the Gaussian function related to the element bjg (j = 1, ..., J) included in the intermediate layer 2-g and the output values zkg_1 of the K elements bkg_1 included in the intermediate layer 2-(g - 1), to the Gaussian function indicated in the expression (12).
The initial values of the parameters of the Gaussian function related to the element bjg included in the intermediate layer 2-g are the standard deviation value okjg (k = 1, ..., K: j = 1, ..., J) of the Gaussian distribution, the center coordinate (j,kjg (k = 1, ..., K: j = 1, ..., J) of the Gaussian function, and the weight Wkjg (k = 1, ..., K: j = 1,...,J).
[0076] When the output values zig to zjg of the elements b ig to bjg included in the intermediate layer 2-g are calculated, the output value calculating unit 12 outputs the output values zig to zjg of the elements big to bjg to the element construction unit 13, and stores the output values zig to zjg of the elements big to bjg in the data storage unit 14.
[0077] When the output values zig to zjg of the elements b ig to bjg are received from the output value calculating unit 12, the element construction unit 13 compares the output values zig to zjg of the elements big to bjg with the threshold value Th (step ST27 in FIG. 8).
When any of the output values zig to zjg of the elements big to bjg included in the intermediate layer 2-g is greater than the threshold value Th (step ST27 in FIG. 8: NO), the element construction unit 13 maintains the number J of the elements included in the intermediate layer 2-g.
When all of the output values zig to zjg of the elements big to bjg included in the intermediate layer 2-g are equal to or less than the threshold value Th (step ST27 in FIG. 8: YES), the element construction unit 13 adds a new element to the intermediate

layer 2-g so that the number of elements included in the intermediate layer 2-g is increased (step ST28 in FIG. 8).
The processing of steps ST23 to ST28 is repeatedly performed until the output values zkg_1 of the K element bkg_1 (k = 1, ..., K) included in the immediately preceding intermediate layer 2-(g - 1) based on the M pieces of learning data xm (m = 1, ..., M) are acquired , and when it is determined to be "YES" in the determination processing of step ST23, the first processing content is ended.
[0078] After the first processing content is ended, the parameter updating unit 15 performs the second processing content.
That is, as in the first embodiment, the parameter updating unit 15 updates the parameters of the Gaussian function stored in the data storage unit 14, and updates the weight VJ between the j-th element bjg included in the intermediate layer 2-G and the element c included in the output layer 3, by performing supervised learning for learning the parameters of the Gaussian function related to the elements included in the intermediate layers 2-1 to 2-G, each time the M pieces of learning data xm = (xim, X2m, ..., xim) are sequentially given, after the structures of the intermediate layers 2-1 to 2-G are determined.
In addition, the parameter updating unit 15 updates a weight Wy1 between the i-th element ai included in the input layer 1 and the j-th element bj1 included in the intermediate layer 2-1, and the weight Wkjg between the k-th element bkg_1 (k = 1, ..., K) included in the intermediate layer 2-(g - 1) and the j-th element bjg 0 = 1, ..., J) included in the intermediate layer 2-g, by performing the supervised learning.
That is, by performing the supervised learning for learning the parameters of the Gaussian function related to the elements included in the intermediate layers 2-1 to 2-G, the parameters of the Gaussian function stored in the data storage unit 14 and the

weights are updated.
[0079] As is apparent from the above description, according to the second
embodiment, there is an effect that the number of elements bjg included in the
intermediate layer 2-g of the neural network can be automatically determined even
when the neural network includes two or more intermediate layers 2-g.
[0080] Third Embodiment.
In the second embodiment, an example has been described in which the number of intermediate layers 2-g included in the neural network is fixed to G.
In this third embodiment, an example in which the number of intermediate layers 2-g included in the neural network is determined as needed will be described. [0081] FIG. 9 is a configuration diagram illustrating a network construction apparatus according to the third embodiment of the present invention, and FIG. 10 is a hardware configuration diagram of the network construction apparatus according to the third embodiment of the present invention.
In FIGS. 9 and 10, since the same reference numerals as those in FIGS. 1, and 2 denote the same or corresponding portions, the description thereof will be omitted.
A number-of-layers determining unit 16 is implemented by, for example, a number-of-layers determining circuit 26 in FIG. 10, and performs processing of determining the number G of the intermediate layers 2-g included in the neural network. [0082] In FIG. 9, a network construction apparatus is assumed that the initial setting unit 11, the output value calculating unit 12, the element construction unit 13, the data storage unit 14, the parameter updating unit 15, and the number-of-layers determining unit 16 that are components of the network construction apparatus are respectively implemented by dedicated hardware circuits, which are the initial setting circuit 21, the output value calculating circuit 22, the element construction circuit 23, the data storage

circuit 24, the parameter updating circuit 25, and the number-of-layers determining circuit 26, as illustrated in FIG. 10.
Here, examples of the data storage circuit 24 include a nonvolatile or volatile semiconductor memory such as RAM, ROM, flash memory, EPROM, and EEPROM; a magnetic disk, a flexible disk, an optical disc, a compact disc, a mini disc, a DVD, and the like.
In addition, examples of the initial setting circuit 21, the output value calculating circuit 22, the element construction circuit 23, the parameter updating circuit 25, and the number-of-layers determining circuit 26 include a single circuit, a composite circuit, a programmed processor, a parallel-programmed processor, an ASIC, an FPGA, or a combination thereof.
[0083] However, the components of the network construction apparatus are not limited to those implemented by dedicated hardware, and the network construction apparatus may be implemented by software, firmware, or a combination of software and firmware.
In the case where the network construction apparatus is implemented by software, firmware, or the like, it is sufficient that the data storage unit 14 is configured on the memory 31 of the computer illustrated in FIG. 4, and a program for causing the computer to execute processing procedures of the initial setting unit 11, the output value calculating unit 12, the element construction unit 13, the parameter updating unit 15, and the number-of-layers determining unit 16 is stored in the memory 31, and the processor 32 of the computer illustrated in FIG. 4 executes the program stored in the memory 31.
[0084] In addition, FIG. 10 illustrates an example in which each of the components of the network construction apparatus is implemented by dedicated hardware, and FIG. 4 illustrates an example in which the network construction apparatus is implemented by

software, firmware, or the like; however, some components of the construction apparatus may be implemented by dedicated hardware and the remaining components may be implemented by software, firmware, or the like. [0085] Next, the operation will be described.
Since the operation is the same as that of the second embodiment except that the number-of-layers determining unit 16 is implemented, only a processing content of the number-of-layers determining unit 16 will be described here.
When the initial setting unit 11, the output value calculating unit 12, and the element construction unit 13 determine a structure of one intermediate layer 2-g in the same manner as in the second embodiment, the number-of-layers determining unit 16 connects the intermediate layer 2-g whose structure has been determined between the input layer 1 and the output layer 3, at each time of the determination.
For example, at a stage where the determination is completed of structures of three intermediate layers 2-1 to 2-3, the three intermediate layers 2-1 to 2-3 are connected between the input layer 1 and the output layer 3.
In addition, at a stage where the determination is completed of structures of four intermediate layers 2-1 to 2-4, the four intermediate layers 2-1 to 2-4 are connected between the input layer 1 and the output layer 3.
[0086] When an intermediate layer 2-g whose structure is newly determined is connected between the input layer 1 and the output layer 3, the number-of-layers determining unit 16 gives arbitrary data to the input layer 1.
Then, the number-of-layers determining unit 16 measures a time from when data is given to the input layer 1 until data is output from the output layer 3, as an input/output time Ta of data in the neural network.
Ta = TOUT-TIN (13)

In the expression (13), TIN is a time when data is input to the input layer 1, and TOUT is a time when data is output from the output layer 3.
[0087] When the input/output time Ta of the data in the neural network is measured, the number-of-layers determining unit 16 compares the input/output time Ta with an allowable time Tb of the input/output time.
The allowable time Tb of the input/output time is a time allowed by the network construction apparatus, and is a time set in advance.
In a case where the input/output time Ta of the data in the neural network is shorter than the allowable time Tb of the input/output time, the number-of-layers determining unit 16 calculates, from the input/output time Ta of the data, an input/output time Ta/E of the intermediate layer 2-g per layer connected between the input layer 1 and the output layer 3.
E is the number of the intermediate layers 2-g connected between the input layer 1 and the output layer 3 at the present moment.
[0088] In a case where the following expression (14) is satisfied, it is highly likely that the input/output time Ta of the data in the neural network will be within the allowable time Tb even when the number of intermediate layers 2-g connected between the input layer 1 and the output layer 3 is increased by one, so that the number-of-layers determining unit 16 permits to increase by one the number of the intermediate layers 2-g connected between the input layer 1 and the output layer 3.
Thus, the initial setting unit 11, the output value calculating unit 12, and the element construction unit 13 perform processing of determining a structure of a newly added intermediate layer 2-g in the same manner as in the second embodiment.
Tb > Ta + Ta/E (14)
[0089] In a case where the expression (14) is not satisfied, when the number of

intermediate layers 2-g connected between the input layer 1 and the output layer 3 is increased by one, it is highly likely that the input/output time Ta of the data in the neural network will exceed the allowable time Tb, so that the number-of-layers determining unit 16 refuses to increase the number of intermediate layers 2-g connected between the input layer 1 and the output layer 3.
Thus, the number E of the intermediate layers 2-g connected between the input layer 1 and the output layer 3 is determined as the number G of the intermediate layers 2-g of the neural network at the present moment.
[0090] As is apparent from the above description, according to the third embodiment, the number-of-layers determining unit 16 determines the number G of the intermediate layer 2-g from the input/output time Ta of the data in the neural network and the allowable time Tb of the input/output time, so that there is an effect that the input/output time Ta of the data in the neural network can be set within the allowable time Tb. [0091] In the third embodiment, an example has been described in which the number-of-layers determining unit 16 measures the input/output time Ta of the data in the neural network and determines the number G of the intermediate layers 2 from the input/output time Ta and the allowable time Tb; however, this is not a limitation.
For example, the number-of-layers determining unit 16 may measure a learning time Tc of the neural network, and determine the number G of the intermediate layers 2 from the learning time Tc and an allowable time Td of the learning time.
Specifically, it is as follows. [0092] When the intermediate layer 2-g whose structure is newly determined is connected between the input layer 1 and the output layer 3, the number-of-layers determining unit 16 causes the neural network to perform learning by giving learning data to the input layer 1.

Then, for example, the number-of-layers determining unit 16 measures a time from when learning data is given to the input layer 1 until data is output from the output layer 3, as the learning time Tc of the neural network.
Tc = TOUT-TIN (15)
In the expression (15), TIN is a time when learning data is input to the input layer 1, and TOUT is a time when data is output from the output layer 3. [0093] When the learning time Tc of the neural network is measured, the number-of-layers determining unit 16 compares the learning time Tc with the allowable time Td of the learning time.
The allowable time Td of the learning time is a time allowed by the network construction apparatus, and is a time set in advance.
In a case where the learning time Tc of the neural network is shorter than the allowable time Td of the learning time, the number-of-layers determining unit 16 calculates, from the learning time Tc of the neural network, a learning time Tc/E of the intermediate layer 2-g per layer connected between the input layer 1 and the output layer 3.
E is the number of the intermediate layers 2-g connected between the input layer 1 and the output layer 3 at the present moment.
[0094] In a case where the following expression (16) is satisfied, it is highly likely that the learning time Tc of the neural network will be within the allowable time Td even when the number of intermediate layers 2-g connected between the input layer 1 and the output layer 3 is increased by one, so that the number-of-layers determining unit 16 permits to increase by one the number of the intermediate layers 2-g connected between the input layer 1 and the output layer 3.
Thus, the initial setting unit 11, the output value calculating unit 12, and the

element construction unit 13 perform processing of determining a structure of a newly added intermediate layer 2-g in the same manner as in the second embodiment.
Td>Tc+Tc/E (16)
[0095] In a case where the expression (16) is not satisfied, when the number of intermediate layers 2-g connected between the input layer 1 and the output layer 3 is increased by one, it is highly likely that the learning time Tc of the neural network will exceed the allowable time Td, so that the number-of-layers determining unit 16 refuses to increase the number of intermediate layers 2-g connected between the input layer 1 and the output layer 3.
Thus, the number E of the intermediate layers 2-g connected between the input layer 1 and the output layer 3 is determined as the number G of the intermediate layers 2-g of the neural network at the present moment.
In a case where the number-of-layers determining unit 16 measures the learning time Tc of the neural network and determines the number G of the intermediate layers 2 from the learning time Tc and the allowable time Td, there is an effect that the learning time Tc of the neural network can be set within the allowable time Td. [0096] Note that, in the invention of the present application, within the scope of the invention, free combination of each embodiment, a modification of an arbitrary component of each embodiment, or omission of an arbitrary component in each embodiment is possible.
INDUSTRIAL APPLICABILITY
[0097] The present invention is suitable for a network construction apparatus and a network construction method for constructing a neural network.

REFERENCE SIGNS LIST
[0098] 1 Input layer
2, 2-1 to 2-G Intermediate layer
3 Output layer
11 Initial setting unit
12 Output value calculating unit
13 Element construction unit
14 Data storage unit
15 Parameter updating unit
16 Number-of-layers determining unit

21 Initial setting circuit
22 Output value calculating circuit
23 Element construction circuit
24 Data storage circuit
25 Parameter updating circuit
26 Number-of-layers determining circuit

31 Memory
32 Processor

We Claim:
1. A network construction apparatus comprising:
an output value calculating unit for calculating output values of one or more elements included in an intermediate layer of a neural network in accordance with an output value of an element included in an input layer of the neural network when the neural network is constructed, the neural network including the intermediate layer including an element whose input/output response is determined by a Gaussian function; and
an element construction unit for comparing the output values of the one or more elements calculated by the output value calculating unit with a threshold value, and maintaining a number of elements included in the intermediate layer when an output value of any of the elements out of the output values of the one or more elements included in the intermediate layer is greater than the threshold value, and performing element construction processing of increasing the number of elements included in the intermediate layer when all of the output values of the one or more elements included in the intermediate layer are equal to or less than the threshold value.
2. The network construction apparatus according to claim 1, further comprising
an initial setting unit for initializing, for each element included in the
intermediate layer, parameters of the Gaussian function related to the element, wherein
the output value calculating unit calculates an output value of each element included in the intermediate layer by substituting an output value of an element included in the input layer to the Gaussian function having parameters initialized by the initial setting unit.

3. The network construction apparatus according to claim 2, further comprising a parameter updating unit for updating the parameters of the Gaussian function and updating a weight between an element included in the intermediate layer and an element included in an output layer, by performing supervised learning for learning the parameters of the Gaussian function when learning data is given to the input layer after the element construction processing is performed by the element construction unit.
4. The network construction apparatus according to claim 3, wherein the parameter updating unit updates a weight between an element included in the input layer and an element included in the intermediate layer by performing the supervised learning.
5. The network construction apparatus according to claim 2, wherein the output value calculating unit uses a standard deviation value of a Gaussian distribution in the Gaussian function for individual elements included in the intermediate layer, as one of the parameters of the Gaussian function.
6. The network construction apparatus according to claim 5, wherein the output value calculating unit uses a standard deviation value of output values of a plurality of the elements included in the input layer, as the standard deviation value of the Gaussian distribution for the individual elements included in the intermediate layer.
7. The network construction apparatus according to claim 5, wherein the output value calculating unit uses a positive real number as the standard deviation value of the Gaussian distribution for the individual elements included in the intermediate layer.

8. The network construction apparatus according to claim 1, wherein the element construction unit uses a positive real number equal to or less than 1, as the threshold value.
9. The network construction apparatus according to claim 1, wherein the element construction unit uses a positive real number equal to or greater than e"°25 and equal to or less than 1, as the threshold value.
10. The network construction apparatus according to claim 3, wherein the parameter updating unit updates any one or more parameters out of a parameter indicating the standard deviation value of the Gaussian distribution in the Gaussian function and a parameter indicating a center coordinate of the Gaussian function, as the parameters of the Gaussian function.
11. The network construction apparatus according to claim 1, wherein
the neural network includes G intermediate layers, where G is an integer equal to or greater than 2, and the G intermediate layers are connected in cascade between the input layer and an output layer,
the output value calculating unit calculates output values of one or more elements included in a first intermediate layer in accordance with an output value of an element included in the input layer when the output value of the element included in the input layer is given to the one or more elements included in the first intermediate layer, and calculates output values of one or more elements included in a g-th intermediate layer, where g = 2, ..., G, in accordance with an output value of an element included in a

(g - l)-th intermediate layer, where g = 2, ..., G, when the output value of the element included in the (g - l)-th intermediate layer, where g = 2, ..., G, is given to the one or more elements included in the g-th intermediate layer, where g = 2, ..., G, and
the element construction unit compares the output values of the one or more elements included in the g-th intermediate layer, where g = 1, ..., G, calculated by the output value calculating unit with the threshold value, and maintains a number of elements included in the g-th intermediate layer, where g = 1, ..., G, when an output value of any of the elements out of the output values of the one or more elements included in the g-th intermediate layer, where g = 1, ..., G, is greater than the threshold value, and performs element construction processing of increasing the number of elements included in the g-th intermediate layer, where g = 1, ..., G, when all of the output values of the one or more elements included in the g-th intermediate layer, where g = 1, ..., G, are equal to or less than the threshold value.
12. The network construction apparatus according to claim 11, further comprising a parameter updating unit for updating parameters of the Gaussian function related to an element included in a G-th intermediate layer, where G is an integer equal to or greater than 2, and updating a weight between an element included in the G-th intermediate layer and an element included in the output layer, by performing supervised learning for learning the parameters of the Gaussian function when learning data is given to the input layer after the element construction processing is performed by the element construction unit.
13. The network construction apparatus according to claim 12, wherein the parameter updating unit updates a weight between an element included in the (g - l)-th

intermediate layer, where g = 2, ..., G, and an element included in g-th intermediate layer, where g = 2, ..., G, by performing the supervised learning.
14. The network construction apparatus according to claim 12, wherein the parameter updating unit updates a weight between an element included in the input layer and an element included in the first intermediate layer, by performing the supervised learning.
15. The network construction apparatus according to claim 11, further comprising a numb er-of-1 ay ers determining unit for determining a number of the intermediate layers included in the neural network.
16. The network construction apparatus according to claim 15, wherein the numb er-of-1 ay ers determining unit determines the number of the intermediate layers from an input/output time of data in the neural network and an allowable time of the input/output time.
17. The network construction apparatus according to claim 15, wherein the numb er-of-1 ay ers determining unit determines the number of the intermediate layers from a learning time of the neural network and an allowable time of the learning time.
18. A network construction method comprising:
calculating, by an output value calculating unit, output values of one or more elements included in an intermediate layer of a neural network in accordance with an output value of an element included in an input layer of the neural network when the

neural network is constructed, the neural network including the intermediate layer including an element whose input/output response is determined by a Gaussian function; and
comparing the output values of the one or more elements calculated by the output value calculating unit with a threshold value, and maintaining a number of elements included in the intermediate layer when an output value of any of the elements out of the output values of the one or more elements included in the intermediate layer is greater than the threshold value, and performing element construction processing of increasing the number of elements included in the intermediate layer when all of the output values of the one or more elements included in the intermediate layer are equal to or less than the threshold value, by an element construction unit.
19. The network construction method according to claim 18, wherein
an initial setting unit initializes, for each element included in the intermediate layer, parameters of the Gaussian function related to the element, and
the output value calculating unit calculates an output value of each element included in the intermediate layer by substituting an output value of an element included in the input layer to the Gaussian function having parameters initialized by the initial setting unit.
20. The network construction method according to claim 19, wherein a parameter
updating unit updates the parameters of the Gaussian function and updates a weight
between an element included in the intermediate layer and an element included in an
output layer, by performing supervised learning for learning the parameters of the
Gaussian function when learning data is given to the input layer after the element

construction processing is performed by the element construction unit.