Specification
[Description]
[Title of Invention]
INFORMATION PROCESSING DEVICE
[Technical Field]
5 [OOOl]
The present invention relates to an information processing
apparatus, and more particularly, to an information processing apparatus
for performing distributed parallel processing on predetermined data.
10 [Background Art]
[0002]
A technique of a distributed processing system has been known, in
which large-scale data are divided into multiple data units in a case where
large-scale data are processed, and multiple work units respectively
15 perform processing on the divided multiple data units.
[0003]
For example, first, a distributed processing system divides, into
data units, a processing request (job) which is given by the client and
which is targeted 011 all or some of data unit groups divided large-scale
20 data. Subsequently, the distributed processing system generates a
processing request (task) of a data unit for a work unit performing
predetermined processing on each of the divided data units.
Subsequently, in the divided processing system, each of n~ultiplew ork
units performs predetermined processing on the data unit in accordance
25 with this processing request, and outputs a processing result. Then, the
distributed processing system collects the processing results of the task,
thus perforniing processing on the data units for the entire or a part of the
large-scale data.
[0004]
2
In the tecl~niqueo f such distributed processing system, it is desired
to have a distributed processing system of large-scale data having a high
degree of reliability capable of returning a processing result even in a case
where a failure and the like occurs in a snlall number of work units (for
5 example, physical servers), and the work units goes down. In the
technique of the distributed processing system, many work units are
required, and therefore, it is desired to reduce the overhead due to the tasks
by reducing the arnount of co~nmunicationa nd the number of times
communication is perfornled between the work unit and the controller for
10 conlmanding the work unit to perfor111 the processing.
[OOOS]
In this case, in the distributed processing system, a technique for
distributing the processing load to ~nultiplew ork units is known (for
example, see PTL 1). In the technique of PTL 1, in a case where an
15 access request is given to nlultiple processors to access various kinds of
data arranged in a distributed manner, a distributed-type data base
management system in whicl~p rocessing according to the access request is
performed by a particular processor arranged with desired data includes a
processing load deviation detector and a data arrangenient change unit.
20 Then, this processing load deviation detector detects tlle processing load
deviation on the basis of the system load statistics infor~nationa nd the
access inforlnation for accessing the data unit, and changes the
arrangement configuration of the data in accordance with the load
deviation. As a result, the load of the task can be distributed.
25 [0006]
In the technique of the distributed processing system, a redundant
arrangenlent technique of data for arranging the same data unit in a
distributed manner to each of the inultiple work units so as not to lose the
data unit is known (for example, see PTL 2). The technique of PTL 2
3
includes means for classifying the physical node of the storage into groups,
and means for allocating data so that distributed data and copied data of
the distributed data do not exist in the group. The distributed processing
system having such configuration arranges copied data to multiple
5 different groups, thus capable of maintaining redundancy of the data.
[Citation List]
[Patent Document]
[0007]
10 PTL 1: Japanese Patent Laid-Open No. H9-218858
PTL 2: International Publication No. 2008/114441
[Summary of Invention]
[Technical Proble~n]
1s [0008]
However, in the above redundant arrangenlent technique, the data
units are arranged in a distributed manner only in view of the redundancy,
and therefore, in a case where each of ~nultipiew ork units performs
processing on the data unit, it takes a long time to complete all the
20 processing.
[0009]
For example, in a case wl~eree ach of the two work units includes
many identical data units, and a failure and the like occurs in one of the
work units, one of the work units does not perform processing on the data
25 unit, and instead, the other of the work units performs processing on the
data unit. Therefore, the number of data units processed by the other of
the work units increases. Other work units that do not hold the same data
unit as the data unit held by the one of the work units in which a failure
and the like occors cannot perform processing on the data unit instead of
4
the one of the work units. As a result, the nuniber of work units
performing processing at a time decreases in the entire distributed
processing system, and therefore, there is a problenl in that it takes a long
time to complete the processing on all of the data units, and the throughput
5 decreases.
[OO lo]
Therefore, it is an object of the present invention to provide an
information processing apparatus capable of solving the problem of the
reduction of the throughput which is the problem explained above.
10
[Solution to Problem]
[OOll]
An information processing apparatus according to an aspect of the
present invention for solving the above object, includes,
15 data unit output control means for outputting, to a plurality of data
unit processors, copied data units obtained by respectively copying a
plurality of data units for a number that is set in advance, .
the data unit output control Ineans being configured to respectively
allocate output destinations of a plurality of identical copied data units to
20 different data tunit processors,
for each combination of at least two data unit processors that is set
in advance, obtain a number of the copied data unit groups in which the
identical copied data units are respectively allocated to the data unit
processors made into a combination, and
25 respectively allocate the output destinations of the plurality of
copied data units to the data unit processors, so that the degree of variation
of the number of copied data unit groups obtained for each co~nbinationo f
the data unit processors becomes smaller.
[OO 121
j, 5
An information processing method according to another aspect of
the present invention, includes,
while outputting, to a plurality of data miit processors, copied data
units obtained by respectively copying a plurality of data units for a
5 number that is set in advance,
allocati~lgo utput destinations of a plurality of identical copied data
units to different data unit processors,
for each combination of at least two data unit processors that is set
in advance, obtaining a number of the copied data unit groups in which the
10 identical copied data units are respectively allocated to the data unit
processors made into a combination, and
respectively allocati~lgth e output destinations of the plurality of
copied data units to the data unit processors, so that the degree of variation
of the number of copied data unit groups obtained for each c o ~ n b i ~ l a t ioof~ i
15 the data unit processors becomes smaller.
[00 131
A program, according to another aspect of the present invention, for
causing an i~lformationp rocessing apparatus to realize data unit output
control mealis for outputting, to a plurality of data unit processors, copied
20 data units obtained by respectively copying a plurality of data units for a
lumber that is set in advance,
wherein the data unit output co~ltrolm eans respectively allocates
output destinations of a plurality of identical copied data units to different
data unit processors, and for each combinatio~lo f at least two data unit
25 processors that is set in advance, the data unit output control means obtains
a number of the copied data unit groups in which the identical copied data
units are respectively allocated to the data unit processors made into a
combination, and the data unit output co~ltroml eans respectively allocates
the output desti~latio~oisf the plurality of copied data units to the data unit
6
processors, so that the degree of variation of the number of copied data
unit groups obtained for each combination of the data unit processors
becomes smaller.
[Advantageous Effects of Invention]
5 [0014]
The present invention is configured as described above, and can
irnprove the throughput of the processing performed on the data unit.
[Brief Description of Drawings]
[00 151
10 [Fig. I] Fig. 1 is a figure represe~iting a configuration of a
distributed processing system accordiilg to a first exemplary embodiment
of the present invention.
[Fig. 21 Fig. 2 is a figure for explaining an exa~npleo f a work unit
storage.
15 [Fig. 31 Fig. 3 is a figure for explaining an exa~npleo f a master data
storage.
[Fig. 41 Fig. 4 is a figure for explaining allocation of data units.
[Fig. 51 Fig. 5 is a figure for explaining allocatio~ol f data units.
[Fig. 61 Fig. 6 is a figure for explaining a mirror table value.
20 [Fig. 71 Fig. 7 is a figure for explaining an example of a data
arrangelnellt storage.
[Fig. 81 Fig. 8 is a figure illustrating an example of a data
arrangement ~natrix.
[Fig. 91 Fig. 9 is a figure illustrating an exaniple of a standard
25 deviation of a mirror table value.
[Fig. 101 Fig. 10 is a figure illustrating an example of a data
arrangelnent matrix.
[Fig. 111 Fig. 11 is a figure illustrating an example of a standard
deviation of a mirror table value.
7
[Fig. 121 Fig. 12 is a figure for explairiing an example of a total
summation of a mirror table value.
[Fig. 131 Fig. 13 is a figure for explaining an example of a job
storage.
[Fig. 141 Fig. 14 is a figure for explaining an example of a task
storage.
[Fig. 151 Fig. 15 is a figure for explaining an example of a copied
data storage.
[Fig. 161 Fig. 16 is a figure for explaining an exanlple of a task
10 storage.
[Fig. 171 Fig. 17 is a figure for explaining an example of a task
storage.
[Fig. 181 Fig. 18 is a figure for explaining an example of a task
storage.
1.5 [Fig. 191 Fig. 19 is a figure for explaining an example of a job
storage.
[Fig. 201 Fig. 20 is a figure for explaining timing up to a job
completion.
[Fig. 211 Fig. 21 is a figure for explaining tinling up to a job
20 completion.
[Fig. 221 Fig. 22 is a figure for explaining an operation of an
infornlatio~pl rocessing apparatus.
[Fig. 231 Fig. 23 is a figure for explaining an operation of a data
arrangement management module.
25 [Fig. 241 Fig. 24 is a figure for explaining an operation of a data
arrangement planning module.
[Fig. 251 Fig. 25 is a figure for explaining an operation of a task
management module.
[Fig. 261 Fig. 26 is a figure for explaining an operation of a work
8
unit module.
[Fig. 271 Fig. 27 is a figure for explaitling an operati011 of a task
scheduler.
[Fig. 281 Fig. 28 is a figure for explaiuing an operation of a task
5 collector.
[Fig. 291 Fig. 29 is a block diagram illustrati~lgf unctio~lso f an
information processor according to a second exemplary embodiment of the
present invention.
10 [Description of Embodiments]
[00 161
Hereinafter, an exemplary embodiment of a distributed processing
system according to the present invention will be explained with reference
to Figs. 1 to 28. Figs. 1 to 21 are figures for explaining a configuration
15 of a distributed processing system. Figs. 22 to 28 are a figure for
explai~linga n operation of the distributed processing system.
[00 171
(Configuration)
20 As shown in Fig. 1, a distributed processing syste~tl1 according to a
first exenlplary embodi~llento f the present i~lve~ltioi~nl cludesa n
infornlation processing apparatus 11, a job client 12, and a systenl
managetnent client 13. The job client 12 feeds a job (processing request)
which is to be executed by the information processing apparatus 11. The
25 systenl management client 13 lnallages each of multiple work units (work
unit modules 25 explained later) executing a task (divided processiug
request) generated on the basis of a job. Then, the information
processing apparatus 11 generates multiple tasks on the basis of the job fed
fro111 the job client 12, and causes the ~liultiplew ork units to execute the
9
generated ~nultipleta sks. For example, the job client 12 outputs any
given character string, as the content of the fed job. Then, the
information processing apparatus 11 compares any given character string,
which is output from the job client 12, and the character string stored in
5 advance, and executes processing for outputting the nuniber of matching
characters.
[0018]
First, the configuration of the information processing apparatus 11
will be explained. The inforniation processing apparatus 11 includes a
10 work unit managenlent tnodule 21, a task managenlent module 22, a task
collector 23, a data arrangement managenlent module 24, multiple work
unit niodules 25A to 25N (hereinafter described as work unit niodules 25
when each of the multiple work unit modules 25A to 25N are not
distinguished from each other). For exanlple, the function parts
15 constituting the infor~nationp rocessing apparatus 11 are arranged on
different physical server. In this case, each function part constituting the
infor~nationp rocessing apparatus 11 includes a central processing unit
(CPU), a RAM (Random Access Memory), storage, and a network interface
card (NIC), and can operate with a control of the operating system (0s).
20 Then, each server constituting the work unit management nodule 21, the
task management nodule 22, the task collector 23, the data arrangelnent
~nanagementm odule 24, and multiple work unit nlodules 25 can be
connected with each other via a network and the like (for example, TCP
(Transmission Control Protocol)/IP (Internet Protocol)).
25 [0019]
Each of data unit allocation processing, job registration processing,
and job execution processing in the distributed processing systeni 1 will be
hereinafter explained in order. Tlte data unit is divided predeter~nined
data (for example, large-scale data) into multiple pieces. The data unit
10
allocation processing is processing for respectively allocating data units to
~nultiplew ork unit modules 25 (data unit processors). The job
registration processing is processing for generating a task divided the job
fed from the job client 12 in association with the data units. Then, in the
5 job execution processing, the work unit n~odule2 5 executes predetermined
processing on the allocated data unit on the basis of the task.
[0020]
It slioold be noted that the data unit allocation processing is
perforilled once at tlie first and the job registration processing and the job
10 execution processing are executed repeatedly. When the configuration of
the distributed processing system 1 is changed, e.g., there is a change in
the number of work unit nlodules 25 to which tlie data units are allocated,
the data unit allocation processing is perforined again.
[0021]
15 Explained below is a case where the number of work unit modules
25 (tlie number of work units) w is three, the number of data units d is four,
and a redundancy degree number r of data unit is two. It is to be
understood that the number of work units w, the number of data units d,
and the redundancy degree number r are not limited thereto, and each of
20 them may be an integer equal to or more than two. The redundancy
degree number r is a value that does not change for each data unit. The
identical tnultiple copied data units that obtained by copying a data unit are
respectively arranged in (allocated to) multiple different work unit
modules 25. At this occasion, in the case where the number of work units
25 w is less than tlie redundancy degree number r holds, the degree of
redundancy cannot be satisfied, and therefore, l
Subsequently, a distributed processing system according to the
second exemplary enlbodiment o f the present invention will be explained
with reference to Fig. 29.
A distributed processing system 201 according to the second
exemplary embodiment includes a data unit output controller 21 1 (data unit
output control means) for outputting, to ~nultipled ata unit processors 212,
copied data units obtained by respectively copying multiple data units for a
number that is set in advance,
10 wherein the data unit output controller 21 1 respectively allocates
the output destinations of the identical multiple copied data units to
different data unit processors 212, and
for each conlbination o f at least two data unit processors 212 that is
set in advance, the number of the identical copied data unit groups in
15 which the identical copied data units are allocated to the data unit
processors 212 made into a conlbination is obtained, and the output
destinations o f the multiple copied data units are respectively allocated to
the data unit processors 212, so that the degree of variation of the number
of copied data unit groups obtained for each combination o f the data unit
20 processors 212 becomes smaller.
[0 1091
According to the above configuration, in a case where the data unit
output controller 211 outputs the copied data units, which are obtained by
copying each o f multiple data units for the number of times that is set in
25 advance, to ~nultipled ata unit processors 212, for exatnple, the multiple
copied data units of which identification information of the data units is
the same are respectively allocated to different data unit processors 212.
For each combination o f at least two data unit processors 212 that is set in
advance, the data unit output controller 21 1 obtains the number of the
47
identical copied data unit groups (mirror table values) in which the
identical copied data units are allocated to the data unit processors made
into the combination. Further, the data unit output controller 21 1
respectively allocates the output destinations of multiple copied data units
5 to the data unit processors 212 so that the degree of variation of the nlirror
table values obtained for each co~ubinationo f the data unit processors 212
beconles smaller. Therefore, the data unit processors 212 holding copies
of the identical data unit are allocated in a distributed manner without
being concentrated on a single data unit processor 212. Therefore, even
10 in a case where a failure and the like occurs in the predetermined data unit
processor, each data unit processor holding the same copied data unit as
the copied data unit held by the data unit processor 212 in which a failure
and the like occurs can perform predetermined processing on the copied
data units on behalf of the data unit processor 212. As a result, the
15 processing on all the data units can be done quickly, and the throughput
can be improved.
[ O l lo]
The invention of the present application has been hereinabove
explained with reference to the above exemplary embodiments, but the
20 invention of the present application is not limited to the above exe~nplary
e~nboditnents. Various changes that can be understood by a person
skilled in the art within the scope of the invention of the present
application can be made in the configuration and the details of the
invention of the present application.
25 [ O l l l ]
Some or all of the above exemplary embodiments may be described
as shown in the following Supplemental note s, but are not limited thereto.
[0112]
48
(Supplemental note 1)
An information processing apparatus includes,
data unit output control means for outputting, to a plurality of data
unit processors, copied data units obtained by respectively copying a
5 plurality of data units for a nunlber that is set in advance,
the data unit output control means being configured to respectively
allocate output destinations of a plurality of identical copied data units to
different data unit processors,
for each combination of at least two data unit processors that is set
10 in advance, obtain a nunlber of the copied data unit groups in which the
identical copied data units are respectively allocated to the data unit
processors nlade into a combination, and
respectively allocate the output destinations of the plurality of
copied data units to the data unit processors, so that the degree of variation
15 of the nunlber of copied data unit groups obtained for each colnbination of
the data unit processors beconles smaller.
[0113]
According to the above configuration, in a case where the data unit
output control means outputs, to the plurality of data unit processors, the
20 copied data units obtained by respectively copying the plurality of data
units for the number that is set in advance, for example, the data unit
output controller respectively allocates a plurality of copied data units of
which identification information of data units is the salne to different data
unit processors. For each combination of at least two data unit
25 processors that is set in advance, a number of the copied data unit groups
in which the identical copied data units are respectively allocated to the
data unit processors made into a conibination (mirror table value) is
obtained. The data unit output controller respectively allocates the
output destinations of the plurality of copied data units to the data unit
49
processors, so that the degree of variation of the number of the mirror table
values obtained for each co:nbinatioli of the data unit processors becomes
sn~aller. Therefore, the data unit processors holding copies of the
identical data unit are allocated in a distributed manner without being
5 concentrated on a single data unit processor. Therefore, even in a case
where a failure and the like occurs in the predetermined data unit processor,
each data unit processor holding the same copied data uuit as the copied
data unit held by the data unit processor in which a failure and the like
occurs can perforrn predetermined processing on the copied data units on
10 behalf of the data unit processor. As a result, the processing on all the
data units can be done quickly, and the throughput can be improved.
[0114]
(Supplelnental note 2)
The inforlnation processing apparatus according to Supplelnental
15 note 1, wherein, further, the data unit output control ~neansre spectively
allocates the output destinations of the plurality of copied data ullits to the
different data unit processors, so that the degree of variation of the nuntber
of copied data units allocated for each data unit processor beconles
smaller.
20 [0115]
According to the above configuration, the data unit output control
tneans respectively allocates the output destinations of the plurality of
copied data units to the different data unit processors, so that the degree of
variation of the nu~nbero f copied data units allocated for each data unit
25 processor beconles smaller. Therefore, the copied data units are not
output in such a lnanner that the copied data units are concentrated on a
single data unit processor, and the copied data units are respectively output
in a distributed nlanller to the plurality of data unit processors. As a
result, even in a case where a failure and the like occurs in the
50
predetermined data unit processor, another data unit processor can perform
predetermined processi~lgo n behalf of the original data unit processor.
As a result, the processing 011 all the data units can be done quickly, and
the througllput can be improved.
5 [0116]
(Supplemental note 3)
The information processing apparatus according to Supplemental
note 1 or 2, wherein the data unit output control means calculates the
degree of variation, with respect to a predetern~inedr eference value, of the
10 number of copied data unit groups allocated with the identical copied data
units obtained for each combination of the data unit processors, and
respectively allocates the output destinations of the plurality of copied
data units to the data unit processors, so that the degree of variation of the
number of copied data unit groups thus calculated becomes smaller.
15 [0117]
According to the above configuration, the data unit output control
means calculates the degree of variatiou (for example, a standard
deviation), with respect to a predetermined reference value, of the number
of mirror table values obtained for each combination of the data unit
20 processors. The data unit output control means respectively allocates the
atitput destinations of the plurality of copied data units to the data unit
processors, so that the standard deviation thus calculated becomes smaller.
As described above, by reducing the degree of variation of the number of
identical copied data units, the number of identical copied data units for
25 each colnbiuation of the data unit processors can be equalized. As a
result, the multiple copied data units can be more quickly and reliably
distributed to n~ultipled ata unit processors.
[OllS]
(Supplemental note 4)
5 1
The information processing apparatus according to any one of
Supplenlental notes 1 to 3, wherein the data unit output cor~trolm eans
respectively allocates the output destinations of the plurality of copied
data units to the data unit processors, so that the degree of variation of the
5 number of copied data unit groups allocated with the identical copied data
units obtained for each co~nbinationo f the data unit processors becomes
smaller, and a total su~n~natioonf the nuniber of copied data unit groups
obtained for each combination of the data unit processors is minimized.
[0119]
10 According to the above configuration, the data unit output control
nleans respectively allocates the output destinations of the plurality of
copied data units to the data unit processors, so that the degree of variation
of the mirror table values obtained for each conlbination of the data unit
processors beconles smaller, and a total summation of the nlirror table
15 values obtained for each conlbination of the data unit processors is
minimized. As described above, in a case where there is an allocation
pattern of data units in which the degree of variation of the nunlber of
copied data unit groups is the same, the number of redundant data units
(copied data units) for the combination of the data unit processors can be
20 reduced. Therefore, the processing requests for the data units are not
output in snch a lnanner that the processing requests are concentrated 011 a
single data unit processor, and the processing requests are respectively
output in a distributed manner to the plurality of data unit processors. As
a result, the processing on all the data units can be done quickly, and the
25 throughput can be improved.
[0120]
(Supplemental note 5)
The information processing apparatus according to any one of
Supplenlental note s 1 to 4, wherein the data unit output control means
52
generates a data unit output table in which the data unit processor serving
as the output destination and the copied data nnit allocated to the data unit
processor are associated with each other, and outputs the plurality of
copied data units to the plurality o f data unit processors on the basis o f the
5 generated data unit output table.
[O121]
According to the above configuration, the data unit output control
means generates a data unit output table (for example, a data arrangement
matrix) in which the data unit processor and the data unit which is output
10 to the data unit processor are associated with each other in order to
allocate the copied data unit and the data unit processor. Therefore, the
matlagenlent and the control of the copied data units beco~nee asier, and the
~nultiplec opied data units call be more quickly and reliably output to data
unit processors.
15 [O122]
(Supplemental note 6 )
The information processing apparatus according to any one o f
Supplemental notes 1 to 5, further includes, processing request control
lllealls for obtaining a processing request for the plurality o f data units, and
20 generating a divided processing request divided the obtained processing
request in association with each o f the data units, and
in a case where the processing request co~ltroll nealls obtains
identificatioll inforlllatio~flo r identifying the copied data unit allocated to
the data unit processor from the data unit processor, the processing request
25 control lnealls outputs, to the data unit processor, the divided processing
request corresponding to the copied data unit identified by the obtained
identification information.
[0 1231
According to the above configuration, the processing request
53
control means obtains a processing request (for example, a job) for the
plurality of data units, and generates a divided processing request (a task
for each data unit) divided the obtained processing request in association
with each of the data units. Then, in a case where the processing request
5 control means obtains identification information for identifying the copied
data unit allocated to the data unit processor from the data unit processor,
the processing request control means obtains the task associated with the
copied data unit allocated to the data unit processor, and outputs the task to
the data unit processor. Therefore, for example, a processing request
10 (job) for large-scale data is efficiently divided, and can be output to the
data unit processors. As a result, the processing on the large-scale data
can be done quickly, and the throughput can be improved.
[0 1241
(Snpplen~entaln ote 7)
15 The infor~nationp rocessing apparatus according to Suppleniental
note 6,
wherein the processing request control nleans outputs divided
processing requests to the data unit processor,
the divided processing requests having not yet output to another
20 data unit processor chosen from anlong the divided processing requests
corresponding to the copied data unit identified by the obtained
identification information, and being as many divided processing requests
as a number equal to or less than an average value obtained by calculating
a total summation of the nu~nberso f copied data units allocated to the data
25 unit processors and dividing the calculated total summation by the nunlber
of data unit processors.
[0125]
According to the above configuration, the processing request
control nieans outputs, to the data unit processor, divided processing
54
requests that has not yet output to another data unit processor, wherein the
divided processing requests are as many as the number equal to or less than
an average value obtained by calculating a total summation of the numbers
o f copied data units allocated to the data unit processors and dividing the
5 calculated total sunlnlation by the number o f data unit processors.
Therefore, the number of copied data units treated by the data unit
processor at a time is reduced, so that the number o f divided processing
requests that have not yet output to another data unit processor is increased.
Therefore, all o f the divided processing requests for the data units held by
10 the predetermined data unit processor are less likely to be output to another
data unit processor, and the nunlber o f data unit processors that cannot
perform processing on the divided processing request is reduced. As a
result, the processing for all the divided processing requests (data units)
can be done quickly, and the throughput can be improved.
15 [0126]
(Supplemental note 8)
An information processing method includes,
while outputting, to a plurality o f data unit processors, copied data units
obtained by respectively copying a plurality of data units for a nu~nberth at
20 is set in advance,
allocating output destinations o f a plurality o f identical copied data
units to different data unit processors,
for each conlbination o f at least two data unit processors that is set
in advance, obtaining a number o f the copied data unit groups in which the
25 identical copied data units are respectively allocated to the data unit
processors made into a combination, and
respectively allocating the output destinations of the plurality of
copied data units to the data unit processors, so that the degree o f variation
of the number o f copied data unit groups obtained for each co~libinationo f
55
the data unit processors becomes smaller.
[0 1 271
(Supplen~entaln ote 9)
An infornlation processing nlethod according to claim 8, further
5 includes, respectively allocating the output destinations of the plurality of
copied data units to the data unit processors, so that the degree of variation
of the number of copied data units allocated for each data unit processor
becomes smaller.
[0128]
10 (Supplemental note 10)
A program for causing an information processing apparatus to
realize data unit output control means for outputting, to a plurality of data
unit processors, copied data units obtained by respectively copying a
plurality of data units for a number that is set in advance,
15 wherein the data unit output control means respectively allocates
output destinations of a plurality of identical copied data units to different
data unit processors, and for each combination of at least two data unit
processors that is set in advance, the data unit output control nleans obtains
a number of the copied data unit groups in which the identical copied data
20 units are respectively allocated to the data unit processors made into a
combination, and the data unit output control means respectively allocates
the output destinations of the plurality of copied data units to the data unit
processors, so that the degree of variation of the nulnber of copied data
unit groups obtained for each combination of the data unit processors
25 becomes s~naller.
[0129]
It should.be noted that the progranls described in each of the above
exenlplary e~nbodinlentsa nd Suppleniental notes are stored in a storage
device, or recorded in a computer-readable recording medium. For
56
example, the recording mediurn is a medium having portability such as a
flexible disk, an optical disk, a magneto-optical disk, a sen~iconductor
memory, and the like.
[0130]
5 The invention of the present application has been hereinabove
explained with reference to each of the above exemplary embodiments, but
the invention of the present application is not limited to the exelnplary
embodinlents described above. Various changes that can be understood
by a person skilled in the art within the scope of the invention of the
10 present application can be made in the configuration and the details of the
invention of the present application.
[0131]
It should be noted that the present invention enjoys the benefits of
the priority claim based on the patent application of Japanese Patent
15 Application No. 2013-0273 12 filed on February 15, 20 13, in Japan, and all
the contents described in the patent application is deemed to be included in
this Description.
[Reference Signs List]
[0132]
20 1 distributed processing system
11 infornlation processing apparatus
12 job client
13 system management client
21 work unit nlanagenient module
25 22 task lnanagement module
23 task collector
24 data arrangement management nlodule
25 work unit module
3 1 work unit storage
41 job receiver
42 task generator
43 task scheduler
44 job storage
5 45 task storage
5 1 task result receiver
52 job result output module
61 data arrangement planning nodule
62 data arrangement execution tnodule
10 63 master data storage
64 data arratigelneut storage
71 task request module
72 task executior~~ llodule
73 copied data storage
15 201 distributed processing system
21 1 data unit output co~ltroller
212 data unit processor
58
[CLAIMS]
[Claim 11
An illformation processing apparatus comprising:
data unit output co~ltroln leatls for outputting, to a plurality of data
5 unit processors, copied data units obtained by respectively copying a
plurality of data units for a number that is set in advance,
the data unit output control nleans being configured to respectively
allocate output destinations of a plurality of identical copied data units to
different data unit processors,
10 for each combi~latio~ofl at least two data unit processors that is set
in advance, obtain a number of the copied data unit groups in which the
identical copied data units are respectively allocated to the data unit
processors made into a combination, and
respectively allocate the output destinations of the plurality of
15 copied data units to the data unit processors, so that the degree of variation
of the nulnber of copied data uuit groups obtained for each colllbitlation of
the data unit processors becomes smaller.
[Claim 21
20 The infor~nationp rocessing apparatus accordi~lgto clailll 1, wherein,
further,
the data unit output coutrol means respectively allocates tile output
destinations of the plurality of copied data units to the different data unit
processors, so that the degree of variation of the uu~nbero f copied data
25 units allocated for each data unit processor becomes s~llaller.
[Claim 31
The infornlation processi~lga pparatus according to clai~n1 or 2,
wherein
59
the data unit output control means calculates the degree of variation,
with respect to a predetermined reference value, of the number of copied
data unit groups allocated with the identical copied data units obtained for
each colnbination of the data unit processors, and respectively allocates the
5 output destinations of the plurality of copied data units to the data unit
processors, so that the degree of variation of the number of copied data
unit groups thus calculated becollies smaller.
[Clainl 41
10 The information processing apparatus according to any one of
claims 1 to 3, wherein
the data unit output control means respectively allocates the output
destinations of the plurality of copied data units to the data unit processors.
so that the degree of variation of the number of copied data unit groups
15 allocated with the identical copied data units obtained for each
combination of the data unit processors beco~i~esms aller, and a total
sulnniatioii of the nulnber of copied data unit groups obtained for each
con~binationo f the data unit processors is minimized.
20 [Claim 51
The inforn~ationp rocessing apparatus according to any one of
claims 1 to 4, wherein
the data unit output control nleans genel.ates a data unit output table
in which the data unit processor serving as the output destination and the
25 copied data unit allocated to the data unit processor are associated with
each other, and outputs the plurality of copied data units to the plurality of
data unit processors on the basis of the generated data unit output table.
[Claim 61
60
The infor~nationp rocessing apparatus according to any one of
claims 1 to 5, further comprising,
processing request control means for obtaining a processing request
for the plurality of data units, and generating a divided processing request
5 divided the obtained processing request in association with each of the data
units, and
in a case where the processing request control means obtains
identification information for identifying the copied data unit allocated to
the data unit processor from the data unit processor, the processing request
10 control means outputs, to the data unit processor, the divided processing
request corresponding to the copied data unit identified by the obtained
identification information.
[Claim 71
15 The infor~nationp rocessing apparatus according to claim 6,
wherein the processing request control means outputs divided
processing requests to the data unit processor,
the divided processing requests having not yet output to another
data unit processor chosen from among the divided processing requests
20 corresponding to the copied data unit identified by the obtained
identification information, and being as many divided processing requests
as a nunlber equal to or less than an average value obtained by calculating
a total summation of the nu~llberso f copied data units allocated to the data
unit processors and dividing the calculated total sumnlation by the nulrtber
25 of data unit processors.
[Claim 81
An information processing method comprising,
while outputting, to a plurality of data unit processors, copied data units
6 1
obtained by respectively copying a plurality o f data units for a number that
is set in advance,
allocating output destinations o f a plurality o f identical copied data
units to different data unit processors,
5 for each conlbi~~atioonf at least two data unit processors that is set
in advance, obtaining a number o f the copied data unit groups in which the
identical copied data units are respectively allocated to the data unit
processors made into a combination, and
respectively allocating the output destinations of the plurality o f
10 copied data units to the data unit processors, so that the degree o f variation
o f the number o f copied data unit groups obtained for each combination o f
the data unit processors becomes smaller.
[Claim 91
15 An information processing method according to claim 8, further
comprising,
respectively allocating the output destinations o f the plurality o f
copied data lullits to the data lullit processors, so that the degree o f variation
o f the number o f copied data units allocated for each data unit processor
20 becomes smaller.
[Claim 101
A program for causing an information processing apparatus to
realize data unit output control means for outputting, to a plurality o f data
25 unit processors, copied data units obtained by respectively copying a
plurality o f data units for a number that is set in advance,
wherein the data unit output control means respectively allocates
output destinations o f a plurality o f identical copied data units to different
data unit processors, and for each con~binationo f at least two data unit
62
processors that is set in advance, the data unit output control means obtains
a number of the copied data unit groups in which the identical copied data
units are respectively allocated to the data unit processors made into a
combination, and the data unit output colitrol means respectively allocates
5 the output destinations of the plurality of copied data units to the data unit
processors, so that the degree of variation of the number of copied data
unit groups obtained for each combination of the data unit processors
becomes smaller.