•
SP342632XXOO
1/43
SIGNAL PROCESSING APPARATUS, SIGNAL PROCESSING METHOD,
PROGRAM, ELECTRONIC DEVICE, SIGNAL PROCESSING SYSTEMAND
SIGNAL PROCESSING METHOD THEREOF
5
10
BACKGROUND
[0001]
The present disclosure relates to a signal processmg apparatus, signal
processing method, program, electronic device, signal processing system and signal
processing method thereof. In particular, the present disclosure relates to a signal
processing apparatus, signal processing method, program, electronic device, signal
processing system and signal processing method thereof that can generate
synchronization information of contents in a robust manner.
[0002]
In the case of regenerating or editing, in synchronization with time, contents
15 acquired by recording images or sound of the identical event by a plurality of
devices, it is requested to find temporal synchronization between the contents. This
is because, even in the case of recording images or sound of the identical event, the
recording start time varies between the devices and a time delay occurs since the
internal clock frequency slightly varies between the devices. Here, the contents
20 denote acoustic data, image data and acoustic data corresponding to image data, and
so on.
[0003]
As a method of generating synchronization information used at the time of
synchronizing contents in a temporal manner, there is a method of using time
25 information attached to content files at the time oftaking pictures. However, it is not
limited that the time information attached to the files at the time of taking pictures is
accurate.
[0004]
Also, as a method of generating synchronization information, there is a
30 method of using the common element of acoustic data included in contents.
However, recorded acoustic data includes acoustic data of a wind sound, microphone
•
SP342632XXOO
2/43
rubbing sound and other various noise sounds, and therefore there are many cases
where the common element is very little.
[0005]
For example, in a case where only acoustic data recorded in one device
5 includes acoustic data of noise sound or where the acoustic data recorded in each
device includes different kinds of noise sounds, the common element is little. Also,
although acoustic data recorded in a party location or the like includes acoustic data
of BGM (background music) as a common element, since different kinds of
conversations are conducted near each device, even in the case of recording the
10 conversations and the BGM in an overlapping manner, the common element
becomes little. Especially, in a case where the devices are separated by a distance,
the common element is significantly reduced.
[0006]
Therefore, a method of generating synchronization information in a robust
15 manner with respect to noise sounds is desired.
[0007]
However, in a method of generating synchronization information using level
information of acoustic data, which is disclosed in Japanese Patent Laid-Open No.
2009-10548, it is not possible to generate synchronization information in a robust
20 manner with respect to noise sounds. Also, even in a method of generating
synchronization information using a correlation of acoustic data, which is disclosed
in Japanese Patent Laid-Open No. 2010-171625, it is not possible to generate
synchronization information in a robust manner with respect to noise sounds. Also,
in the disclosure of Japanese Patent Laid-Open No. 2010-171625, since a correlation
25 is calculated focusing on only a partial interval of acoustic data, it is not possible to
correct a synchronization difference caused over time due to a slight difference of
internal clock frequency between devices.
[0008]
Meanwhile, a method of modeling human's pitch perception and realizing it
30 on a calculator is disclosed in "A unitary model of pitch perception", J Acoust. Soc.
Am. Volume 102, Issue 3, pp.1811-1820 (1997), Ray Meddis and Lowel O'Mard.
•
SP342632XXOO
3/43
SUMMARY
[0011]
As described above, although a method of generating synchronization
5 information in a robust manner with respect to noise sounds is desired, such a
method has not been considered.
[0012]
The present disclosure is made in view of such a state and can generate
content synchronization information in a robust manner.
10 [0013]
According to a first aspect ofthe present disclosure, there is provided a signal
processing apparatus which includes: a fIrst periodicity detecting section detecting
periodicity information of an acoustic signal included in a fIrst content, as fIrst
periodicity information; a second periodicity detecting section detecting the
15 periodicity information of an acoustic signal included in a second content, as second
.periodicity information; a similarity calculating section calculating a similarity
between the fIrst periodicity information detected by the fIrst periodicity detecting
section and the second periodicity information detected by the second periodicity
detecting section; and a synchronization information generating section generating
20 synchronization information used at a time of synchronizing the fIrst content and the
second content, based on the similarity calculated by the similarity calculating
section.
[0014]
A signal processing method, program and electronic device according to the
25 fIrst aspect of the present disclosure correspond to the signal processing apparatus
according to the fIrst aspect ofthe present disclosure.
[0015]
According to the fIrst aspect of the present disclosure, periodicity information
of an acoustic signal included in a fIrst content is detected as fIrst periodicity
30 information, the periodicity information of an acoustic signal included in a second
content is detected as second periodicity information, a similarity between the fIrst
SP342632XXOO
4/43
• periodicity information and the second periodicity information is calculated, and,
based on the similarity, synchronization is generated which is used at the time of
synchronizing the fIrst content and the second content.
[0016]
5 According to a second aspect of the present disclosure, there is provided a
signal processing apparatus including a receiving section receiving fIrst periodicity
information corresponding to periodicity information of an acoustic signal included
in a fIrst content, and second periodicity information corresponding to the periodicity
information of an acoustic signal included in a second content, a similarity
10 calculating section calculating a similarity between the fIrst periodicity information
and the second periodicity information received by the receiving section, and a
synchronization information generating section generating synchronization
information used at a time of synchronizing the fIrst content and the second content,
based on the similarity calculated by the similarity calculating section.
15 [0017]
A signal processing method, program and electronic device according to the
second aspect of the present disclosure correspond to the signal processing apparatus
according to the second aspect ofthe present disclosure.
[0018]
20 According to the second aspect of the present disclosure, fIrst periodicity
information corresponding to periodicity information of an acoustic signal included
in a fIrst content and second periodicity information corresponding to the periodicity
information of an acoustic signal included in a second content are received, a
similarity between the fIrst periodicity information and the second periodicity
25 information is calculated, and, based on the similarity, synchronization information
used at the time of synchronizing the fIrst content and the second content is
generated.
[0019]
According to a third aspect of the present disclosure, there is provided a
30 signal processing system including a fIrst periodicity detecting section detecting
periodicity information of an acoustic signal included in a fIrst content, as fIrst
SP342632XXOO
5/43
• periodicity information, a second periodicity detecting section detecting the
periodicity information of an acoustic signal included in a second content, as second
periodicity information, a similarity calculating section calculating a similarity
between the first periodicity information detected by the first periodicity detecting
5 section and the second periodicity information detected by the second periodicity
detecting section, and a synchronization information generating section generating
synchronization information used at a time of synchronizing the first content and the
second content, based on the similarity calculated by the similarity calculating
section.
10 [0020]
A signal processing method according to an embodiment of the present
disclosure corresponds to the signal processing system according to an embodiment
ofthe present disclosure.
[0021]
15 According to the third aspect of the present disclosure, periodicity
information of an acoustic signal included in a first content is detected as first
periodicity information, the periodicity information of an acoustic signal included in
a second content is detected as second periodicity information, a similarity between
the first periodicity information and the second periodicity information is calculated,
20 and, based on the similarity, synchronization information used at the time of
synchronizing the first content and the second content is generated.
[0022]
According to a fourth aspect of the present disclosure, there is provided a
signal processing apparatus including a first band dividing section performing a band
25 division on an acoustic signal included in a first content, a first periodicity detecting
section detecting periodicity information of the acoustic signal subjected to the band
division by the first band dividing section, as first periodicity information, a first
periodicity intensity detecting section detecting an intensity of the first periodicity
information detected by the first periodicity detecting section, a first periodicity
30 information integrating section integrating the first periodicity information of all
bands, based on the intensity of the first periodicity information detected by the first
SP342632XXOO
6/43
• periodicity intensity detecting section, a second band dividing section performing a
band division on an acoustic signal included in a second content, a second periodicity
detecting section deteCting the periodicity information of the acoustic signal
subjected to the band division by the second band dividing section, as second
5 periodicity information, a second periodicity intensity detecting section detecting an
intensity of the second periodicity information detected by the second periodicity
detecting section, a second periodicity information integrating section integrating the
second periodicity information of all bands, based on the intensity of the second
periodicity information detected by the second periodicity intensity detecting section,
lOa similarity calculating section calculating a similarity between the first periodicity
information integrated by the first periodicity information integrating section and the
second periodicity information integrated by the second periodicity information
integrating section, and a synchronization information generating section generating
synchronization information used at a time of synchronizing the first content and the
15 second content, based on the similarity calculated by the similarity calculating
section.
[0023]
According to the fourth aspect of the present disclosure, an acoustic signal
included in a first content is subjected to band division, periodicity information of the
20 acoustic signal subjected to the band division is detected as first periodicity
information, an intensity of the first periodicity information is detected, the first
periodicity information of all bands are integrated based on the intensity of the first
periodicity information, an acoustic signal included in a second content is subjected
to band division, the periodicity information of the acoustic signal subjected to the
25 band division is detected as second periodicity information, an intensity of the
second periodicity information is detected, the second periodicity information of all
bands are integrated based on the intensity of the second periodicity information, a
similarity between the integrated first periodicity information and the integrated
second periodicity information is calculated, and, based on the similarity,
30 synchronization information used at the time of synchronizing the first content and
the second content is generated.
•
SP342632XXOO
7/43
[0024]
According to the present disclosure, it IS possible to generate content
synchronization infonnation in a robust manner.
5 BRIEF DESCRIPTION OFTHE DRAWINGS
[0025]
FIG. 1 is a block diagram illustrating a configuration example of a fIrst
embodiment of a signal processing system to which the present disclosure is applied;
FIG. 2 is a block diagram illustrating a configuration example of a feature
10 amount calculating section in FIG. 1;
FIG. 3 is a view illustrating an example of feature amount time series data;
FIG. 4 is a block diagram illustrating a configuration example of a
synchronization infonnation generating section in FIG. 1;
FIG. 5 is a view for explaining acoustic data to which noise sounds of
15 different frequency characteristics are attached;
FIG. 6 is a view for explaining blocks for which the similarity is calculated;
FIG. 7 is a view for explaining a similarity calculation method;
FIG. 8 is a view illustrating an example ofa similarity matrix;
FIG. 9 is a view illustrating an example of synchronization infonnation;
20 FIG. lOis a flowchart for explaining synchronization infonnation generation
processing in the signal processing apparatus in FIG. 1;
FIG. 11 is a view for explaining an outline of a second embodiment of a
signal processing system to which the present disclosure is applied;
FIG. 12 is a view for explaining an outline of a third embodiment of a signal
25 processing system to which the present disclosure is applied;
FIG. 13 is a block diagram illustrating another configuration example of a
feature amount calculating section; and
FIG. 14 is a block diagram illustrating a configuration example of hardware
ofa computer.
30
DETAILED DESCRIPTION OF THE EMBODIMENT(S)
SP342632XXOO
8/43
• [0000]
Hereinafter, preferred embodiments of the present disclosure will be
described in detail with reference to the appended drawings. Note that, in this
specification and the appended drawings, structural elements that have substantially
5 the same function and structure are denoted with the same reference numerals, and
repeated explanation ofthese structural elements is omitted.
[0026]

[Explanation of outline of first embodiment of signal processing system]
10 FIG. 1 is a block diagram illustrating a configuration example of a first
embodiment of the signal processing system to which the present disclosure is
applied.
[0027]
As illustrated in FIG. 1, a signal processing system 10 includes imaging
15 apparatuses 11-1 and 11-2 and a signal processing apparatus 12.
[0028]
The imaging apparatus 11-1 includes a video camera or the like. The imaging
apparatus 11-1 acquires dynamic images and sounds of the identical event, which are
different from those in the imaging apparatus 11-2. The imaging apparatus 11-1
20 transmits a resulting content A of the dynamic images and sounds to the signal
processing apparatus 12.
[0029]
Similar to the imaging apparatus 11-1, the imaging apparatus 11-2 includes a
video camera or the like. The imaging apparatus 11-2 acquires dynamic images and
25 sounds ofthe identical event, which are different from those in the imaging apparatus
11-1. The imaging apparatus 11-1 transmits a resulting content B of the dynamic
images and sounds to the signal processing apparatus 12:
[0030]
Also, in the following, in a case where the imaging apparatus 11-1 and the
30 imaging apparatus 11-2 are not especially requested to be distinguished from each
other, they are collectively referred to as "imaging apparatus 11.
SP342632XXOO
9/43
• [0031]
The signal processmg apparatus 12 includes feature amount calculating
sections 21-1 and 21-2, feature amount storing sections 22-1 and 22-2 and a
synchronization infonnation generating section 23.
5 [0032]
The feature amount calculating section 21-1 of the signal processing
apparatus 12 calculates feature amount time series data based on the acoustic data of
the content A supplied from the imaging apparatus 11-1, and supplies it to the feature
amount storing section 22-1. Also, the feature amount calculating section 21-2
10 calculates feature amount time series data based on the acoustic data ofthe content B
supplied from the imaging apparatus 11-2, and supplies it to the feature amount
storing section 22-2.
[0033]
The feature amount storing section 22-1 includes a memory and a hard disk,
15 and so on, and stores the feature amount time series data of the content A, which is
supplied from the feature amount calculating section 21-1. Also, the feature amount
storing section 22-2 includes a memory and a hard disk, and so on, and stores the
feature amount time series data of the content B, which is supplied from the feature
amount calculating section 21-2.
20 [0034]
The synchronization infonnation generating section 23 reads the feature
amount time series data ofthe content A from the feature amount storing section 22-1
and reads the feature amount time series data of the content B from the feature
amount storing section 22-2. The synchronization infonnation generating section 23
25 generates synchronization infonnation used at the time of synchronizing the contents
A and B, based on the feature amount time series data of the content A and the
feature amount time series data of the content B. The synchronization infonnation
generating section 23 outputs the synchronization infonnation.
[0035]
30 Also, in the following, in a case where the feature amount calculating section
21-1 and the feature amount calculating section 21-2 are not especially requested to
SP342632XXOO
10/43
• be distinguished from each other, they are collectively referred to as "feature amount
calculating section 21." Also, in a case where the feature amount storing section 221
and the feature amount storing section 22-2 are not especially requested to be
distinguished from each other, they are collectively referred to as "feature amount
5 storing section 22."
[0036]
[Configuration example offeature amount calculating section]
FIG. 2 is a block diagram illustrating a configuration example of the feature
amount calculating section 21 in FIG. 1.
10 [0037]
The feature amount calculating 21 in FIG. 2 includes a frequency band
dividing section 101, Periodicity detecting sections 102-1 to 102-4, periodicity
intensity detecting sections 103-1 to 103-4, a periodicity information integrating
section 104 and a peak detecting section 105.
15 [0038]
The frequency band dividing section 101 divides the acoustic data of the
contents supplied from the imaging apparatus 11 in FIG. 1, into time intervals of
around several tens of milliseconds to a hundred milliseconds, using a window
function. The frequency band dividing section 101 divides the acoustic data of the
20 time intervals into four frequency bands using a plurality of band-pass filters. The
frequency band dividing section 101 supplies the acoustic data of the frequency
bands to the periodicity detecting sections 102-1 to 102-4, respectively.
[0039]
Also, as a band-pass filter, it is effective to use a filter in which the passing
25 frequency bandwidth becomes wider in a higher frequency, such as an octave-band
filter.
[0040]
The periodicity detecting sections 102-1 to 102-4 each extract periodicity
information indicating the periodicity per time interval by· calculating an
30 autocorrelation function of the acoustic data per time interval in a predetermined
frequency band, which is supplied from the frequency band dividing section 101.
SP342632XXOO
11/43
• [0041]
Here, as periodicity information, although an autocorrelation function x(b;t)
of time-shifted acoustic data with an index 't in a frequency band with an index b is
used as is, it is possible to use a value dividing the autocorrelation function x(b,'t) by
5 x(b,O). Also, as a calculation method of the autocorrelation function x(b,'t), for
example, it is possible to adopt a method using a peak of a spectrum acquired by
performing discrete Fourier transform on the acoustic data in a predetermined
frequency band.
[0042]
10 The periodicity detecting sections 102-1 to 102-4 each supply the periodicity
information per extracted time interval to the periodicity intensity detecting sections
103-1 to 103-4 and the periodicity information integrating section 104. Also, in the
following, in a case where the periodicity detecting sections 102-1 to 102-4 are not
especially necessary to be distinguished from each other, they are collectively
15 referred to as "periodicity detecting section 102."
[0043]
The periodicity intensity detecting sections 103-1 to 103-4 each calculate the
periodicity intensity per time interval, based on the periodicity information per time
interval supplied from the periodicity detecting sections 102-1 to 102-4. To be more
20 specific, the maximum value of the autocorrelation function x(b,'t) corresponding to
periodicity information in 't different from around 't=0, is calculated as the
periodicity intensity. As this periodicity intensity is larger, the periodicity of acoustic
data of the processing target is more intense, and; as the periodicity intensity is
smaller, the periodicity ofthe acoustic data ofthe processing target is closer to noise.
25 [0044]
Subsequently, the periodicity intensity detecting sections 103-1 to 103-4 each
binarize the periodicity intensity per time interval depending on whether it is over a
threshold, and use it as periodicity intensity information per time interval. That is, in
each time interval, in a case where the periodicity intensity is over a predetermined
30 threshold, the periodicity intensity information is processed as 1, and, in a case the
periodicity intensity is equal to or below the predetermined threshold, the periodicity
•
SP342632XXOO
12/43
intensity infonnation is processed as O. The periodicity intensity detecting sections
103-1 to 103-4 each supply the periodicity intensity infonnation per time interval to
the periodicity infonnation integrating section 104.
[0045]
5 Also, in a case where the periodicity intensity detecting sections 103-1 to
103-4 are not especially requested to be distinguished from each other, they are
collectively referred to as "periodicity intensity detecting section 103."
[0046]
The periodicity infonnation integrating section 104 perfonns periodicity
10 integration processing of integrating the periodicity infonnation of time intervals,
based on the periodicity infonnation per time interval supplied from the periodicity
detecting section 102 and the periodicity intensity infonnation per time interval
supplied from the periodicity intensity detecting section 103. To be more specific,
the periodicity infonnation integrating section 104 finds a total sum of the
15 autocorrelation function x(b,'t) corresponding to the periodicity infonnation per time
interval, using following Equation 1.
[0047]
1 Nb
s(r)=-N ~ x(b, r)·p(b)
P b=l
.• • (1)
[0048]
20 Here, in Equation 1, Nb stands for the total number of frequency bands and
p(b) stands for periodicity intensity infonnation. Also, Np stands for the number of
frequency bands in which the periodicity intensity infonnation p(b) is 1.
[0049]
The periodicity infonnation integrating section 104 supplies a total sum S('t)
25 of periodicity infonnation of time intervals, which is acquired as a result of the
periodicity integration processing, to the peak: detecting section 105.
[0050]
The peak: detecting section 105 perfonns peak: detection on the total sum S('t)
of periodicity infonnation supplied in each time interval from the periodicity
SP342632XXOO
13/43
• information integrating section 104 and generates peak information pet) in which a
value of peak position 'tp is 1 and values other than in the peak position 'tp are O. As a
peak detection method, there is a method of detecting, as the peak position 'tp, an
index't at the time a differential value ofthe total sum S('t) of periodicity information
5 changes from a positive value to a negative value.
[0051]
Also, in a case where the total sum S('t) of periodicity information in the peak
position 'tp is less than a predetermined threshold, the peak detecting section 105 may
set peak information P('tp) in the peak position 'tp to O. By this means, it is possible
10 to reduce a noise of the peak information P('tp). Also, the peak information may be
the total sum S('t) ofperiodicity information itself.
[0052]
The peak detecting section 105 supplies the peak information P('t) of time
intervals as feature amount time series data of time intervals to the feature amount
15 storing section 22 in FIG. 1.
[0053]
[Example of feature amount time series data]
FIG. 3 is a view illustrating an example of feature amount time series data.
[0054]
20 Here, in the graph in FIG. 3, the horizontal axis represents the time interval
number indicating the number of time intervals from the head, and the vertical axis
represents the index 'to
[0055]
The graph in FIG. 3 shows each position of the index 't and the time interval
25 number, in which the time-series data P('t) of feature amounts is 1, by a black spot.
Therefore, in the time-series data P('t) of feature amounts of acoustic data in a
predetermined time interval, in the graph in FIG. 3, the time-series data P('t) of't
corresponding to points arranged on a line vertical to a position on the horizontal axis
corresponding to the time interval number in the time interval is 1 and the time-series
30 data P('t) of other 't is O.
[0056]
SP342632XXOO
14/43
• [Configuration example of synchronization information generating section]
FIG. 4 is a block diagram illustrating a configuration example of the
synchronization information generating section 23 in FIG. 1.
[0057]
5 The synchronization information generating section 23 in FIG. 4 includes
block integrating sections 121 and 122, a similarity calculating section 123 and an
optimal path searching section 124.
[0058]
The block integrating section 121 of the synchronization information
10 generating section 23 reads feature amount time series data of time intervals of the
content A from the feature amount storing section 22-1 in FIG. 1, and, using
consecutive (for example, 64) time intervals as one block, integrates them in units of
blocks. The block integrating section 121 supplies the time-series data of block-unit
feature amounts to the similarity calculating section 123.
15 [0059]
The block integrating section 122 reads feature amount time series data of
time intervals of the content B from the feature amount storing section 22-2 in FIG.
1, and, using consecutive (for example, 64) time intervals as one block, integrates
them in units of blocks. The block integrating section 122 supplies the time-series
20 data of block-units feature amounts to the similarity calculating section 123.
[0060]
Here, multiple time intervals forming a block may not be consecutive. For
example, it is possible to use multiple even-numbered time intervals as one block or
use multiple odd-numbered time intervals as one block. In this case, since it is
25 possible to perform thinning processing on the feature amount time series data of
time intervals, it is possible to reduce the computation amount.
[0061]
Also, the block integrating section 121 and the block integrating section 122
may not integrate all feature amount time series data of the index 1: in a time interval,
30 but may partially integrate feature amount time series data of the index 1: in the time
interval.
SP342632XXOO
15/43
• [0062]
For example, in a case where the acoustic data of the contents A and B is
acoustic data of sound whose fundamental frequency is between 100 Hz and 500 Hz,
the block integrating sections 121 and 122 can integrate only feature amount time
5 series data of the index 't in a time interval, where the index 't corresponds to a time
delay of2 milliseconds to 10 milliseconds.
[0063]
Also, in a case where the acoustic data of the contents A and B is acoustic
data of male voice whose fundamental frequency is between 100 Hz and 250 Hz, the
10 block integrating sections 121 and 122 can integrate only feature amount time series
data of the index 't in a time interval, where the index 't corresponds to a time delay
of 4 milliseconds to 10 milliseconds. Meanwhile, in a case where the acoustic data
of the contents A and B is acoustic data of female voice whose fundamental
frequency is between 200 Hz and 500 Hz, the block integrating sections 121 and 122
15 can integrate only feature amount time series data of the index 't in a time interval,
where the index 't corresponds to a time delay of2 milliseconds to 5 milliseconds.
[0064]
The similarity calculating section 123 calculates the similarities between the
time-series data of block-units feature amounts supplied from the block integrating
20 section 121 and the time-series data of block-units feature amounts supplied from the
block integrating section 122, and generates a similarity matrix representing the
similarities between these blocks. The similarity calculating section 123 supplies the
similarity matrix to the optimal path searching section 124.
[0065]
25 The optimal path searching section 124 functions as a synchronization
information generating section, and searches an optimal similarity path from the
similarity matrix supplied from the similarity calculating section 123 and generates,
as synchronization information, information indicating a time difference between two
blocks corresponding to the similarity on the path. Subsequently, the optimal path
30 searching section 124 outputs the synchronization information.
[0066]
SP342632XXOO
16/43
• As described above, using the peak information P('t) per time interval as
feature amount time series data, the synchronization information generating section
23 generates synchronization information to synchronize content blocks in which the
feature amount time series data is similar. That is, the synchronization information
5 generating section 23 generates synchronization information base on pitch
information of acoustic data. Therefore, even in a case where noises included in
acoustic data are different between imaging apparatuses, it is possible to generate
synchronization information in a robust manner.
[0067]
10 That IS, III the case of hearing a plurality of sounds having frequency
characteristics, by perceiving sounds having the identical fundamental frequency as
the common element, that is, by perceiving sounds of the identical pitch as the
common element, humans can easily perceive the common element even in a case
where noise sounds are included. In view of the above, the present disclosure
15 generates synchronization information in a robust manner with respect to noise
sounds by generating the synchronization information based on pitch information.
[0068]
By contrast with this, as a method of generating synchronization information,
there is a method of generating synchronization information using the similarity
20 between spectrums of acoustic data. However, in this method, for example, in two
imaging apparatuses, in the case of acquiring acoustic data obtained by attaching
noise sounds of different frequency characteristics as illustrated in FIG's. 5B and 5C
to the fundamental tone illustrated in FIG. 5A, the common element of spectrums
between acoustic data is reduced by the noise sounds. Therefore, it is not possible to
25 accurately calculate the similarity and correctly generate synchronization
information. Therefore, in such a method, it is not possible to generate
synchronization information in a robust manner with respect to noise sounds.
[0069]
Here, although synchronization information is generated in units of blocks,
30 synchronization information may be generated with respect to the whole feature
amount time series data. However, in a case where synchronization information is
SP342632XXOO
17/43
• generated in units of blocks, as described later, it is possible to correct an internal
clock difference between the contents A and B.
[0070]
[Explanation of similarity calculation]
5 FIG. 6 is a view for explaining blocks for which the similarity is calculated.
[0071]
Here, "i" represents a block index of the content A and "j" represents a block
index of the content B. Also, XCi) represents feature amount time series data in the
block of the index "i" of the content A, and YO) represents feature amount time series
10 data in the block of the index "j" of the content B.
[0072]
As illustrated in FIG. 6, regarding n items of XCi) and m items of YO), there
are nxm combinations of similarity calculation targets.
[0073]
15 FIG. 7 is a view for explaining a similarity calculation method.
[0074]
Here, in the matrixes in FIG. 7, the horizontal axis represents the in-block
time interval number indicating the number of in-block time intervals from the head,
and the vertical axis represents the index 'to Also, a white square represents that the
20 time-series data P('t) of feature amounts of the index 't in a time interval of the
corresponding in-block interval number is 0, and a black square represents that the
time-series data P('t) is 1. Further, in the example in FIG. 7, it is assumed that the
number oftime intervals forming a block is four and 't is between 0 and 3.
[0075]
25 As illustrated in FIG. 7, in the case of calculating the similarity between XCi)
and Y(i), fIrst, a logical multiply X(i)nYO) of XCi) and YO) is calculated and a
logical add X(i)uYO) of XCi) and YO) is calculated. For example, as illustrated in
FIG. 7, in the case of calculating the similarity between XCi) and YO) which are each
formed with nine O's and seven I's, fIrst, the logical multiply X(i)nYO) of twelve O's
30 and four I's is calculated and the logical add X(i)uYO) of six O's and ten I's is
calculated.
SP342632XXOO
18/43
• [0076]
Subsequently, by following Equation 2, a similarity A(i,j) of XCi) and Y(j) is
calculated based on Number(X(i)nY(j» indicating the number of 1IS of the logical
multiply X(i)nY(j) and Number(X(i)uY(j» indicating the number of lis of the
5 logical add.
[0077]
A(i,j)=Number(X(i)nY(j»/Number(X(i)uY(j» ... Equation (2)
[0078]
In the example in FIG. 7, smce Number(X(i)nY(j» IS 4 and
10 Number(X(i)uY(j» is 10, the similarity A(i,j) is 0.4.
[0079]
Also, as feature amount time series data, in the case of adopting the total sum
S('t) of periodicity information, as a similarity calculation method, it is possible to
adopt a method of calculating the similarity using a cosine distance.
15 [0080]
[Explanation ofoptimal similarity path search]
FIG. 8 is a view illustrating an example ofa similarity matrix.
[0081]
Also, in the similarity matrix in FIG. 8, the horizontal axis represents an
20 index "j" of a block of the content B and the vertical axis represents an index "i" of a
block of the content A.
[0082]
As illustrated in FIG. 8, in the similarity matrix, a point corresponding to the
index "i" and the index "j" is represented by density corresponding to the similarity
25 A(i,j) such that the point becomes deeper as the similarity A(i,j) becomes larger.
[0083]
Using dynamic programmmg, the optimal path searching section 124
searches a path in which an integrated value of the path similarities on the similarity
matrix is maximum, as an optimal similarity path.
30 [0084]
For example, in a case where a time difference between the contents A and B
SP342632XXOO
19/43
• occurs due to only a delay of one of them, since a difference between "i" and "j" to
maximize the similarity A(ij) is constant at any time, a straight path from the upper
left to the lower right on the similarity matrix is searched as an optical similarity
path. Meanwhile, in a case where an internal clock difference occurs between the
5 contents A and B, an optical similarity path is not straight.
[0085]
Here, although the optimal path searching section 124 performs a search on
all possible paths on the similarity matrix using dynamic programming, in a case
where it is found in advance that an internal clock difference does not occur between
10 the contents A and B, the search may be performed on only all possible straight paths
on the similarity matrix.
[0086]
The optimal path searching section 124 generates a difference i-j of indices
corresponding to the similarities on the optical similarity path, as synchronization
15 information.
[0087]
[Example of synchronization information]
FIG. 9 is a view illustrating an example of synchronization information.
[0088]
20 As illustrated in FIG. 9A, in a case where an internal clock difference
between the contents A and B does not occur and both the contents A and B are not
edited, that is, in a case where a time difference between the contents A and B occurs
due to only a delay of one of them, a constant value is generated as synchronization
information. In the example in FIG. 9, a difference between the indices "i" and "j"
25 corresponding to the similarity on an optimal similarity path is "80" at any time and
"80" is generated as synchronization information at any time.
[0089]
Meanwhile, as illustrated in FIG. 9B, in a case where an internal clock
difference between the contents A and B does not occur and the content B is edited
30 on the way, respective constant values are generated as synchronization information
before an edit point and synchronization information after the edit point. In the
'SP342632XXOO
20/43
• example in FIG. 9, there is an edit point in the block with the index "j" of 44.
Therefore, although synchronization information indicating a difference between the
indices "i" and "j" corresponding to similarities on the optimal similarity path is 40 at
any time in a case where the index "j" is between 41 and 44, the synchronization
5 information is 80 at any time in a case where the index "j" is between 85 and 88.
[0090]
Also, as illustrated in FIG. 9C, in a case where an internal clock difference
between the contents A and B occurs and the contents A and B are not edited, a
difference between the indices "i" and "j" corresponding to similarities on the
10 optimal similarity path varies so as to correct the internal clock difference. In the
example in FIG. 9, although a difference between the indices "i" and "j" is 79 in a
case where the index "i" is 1 and 2, since a synchronization difference due to internal
clock difference integration becomes large in a case where the index "i" is 3, the
difference between the indices "i" and "j" is incremented by 1 and becomes 80. That
15 is, the content B in the block with the index "j" of 82 is cut.
[0091]
Also, although the difference between the indices "i" and "j" is still 80 in a
case where the index "i" is 4, since a synchronization difference due to internal clock
difference integration becomes large in a case where the index "i" is 5, the difference
20 between the indices "i" and "j" is incremented by 1 and becomes 81. That is, the
content B in the block with the index "j" of 85 is cut.
[0092]
Further, although the difference between the indices "i" and "j" is still 81 in a
case where the index "i" is 6, since a synchronization difference due to internal clock
25 difference integration becomes large in a case where the index "i" is 7, the difference
between the indices "i" and "j" is incremented by 1 and becomes 82. That is, the
content B in the block with the index "j" of 88 is cut. Subsequently, in a case where
the index "i" is 8, the difference between the indices "i" and "j" is still 82.
[0093]
30 Therefore, "79" is generated as synchronization information in blocks with
the index "i" of 1 and 2, and "80" is generated as synchronization information in
8P342632XXOO
21/43
• blocks with the index "i" of 3 and 4. Also, "81" is generated as synchronization
information in blocks with the index "i" of 5 and 6, and "82" is generated as
synchronization information in blocks with the index "i" of7 and 8.
[0094]
5 [Processing in signal processing apparatus]
FIG. 10 is a flowchart for explaining synchronization information generation
processing in the signal processing apparatus 12 in FIG. 1. This synchronization
information generation processing starts when a content is input from the imaging
apparatus 11.
10 [0095]
Also, the processing in steps 811 to 817 in FIG. 10 is performed every
content.
[0096]
In step 811 in FIG. 10, the frequency band dividing section 101 divides the
15 acoustic data of contents supplied from the imaging apparatus 11 in FIG. 1, into time
intervals of around several tens of milliseconds to a hundred milliseconds, using a
window function.
[0097]
In step 812, the frequency band dividing section 101 divides the acoustic data
20 of the time intervals into four frequency bands using a plurality of band-pass filters.
The frequency band dividing section 101 supplies the acoustic data of the frequency
bands to the periodicity detecting sections 102-1 to 102-4, respectively.
[0098]
In step 813, the periodicity detecting section 102 extracts periodicity
25 information of time intervals by calculating an autocorrelation function x(b,'t) of the
acoustic data per time interval in a predetermined frequency band, which is supplied
from the frequency band dividing section 101, and sup-plies the periodicity
information to the periodicity intensity detecting section 103. Here, the processing in
step 813 is performed every periodicity detecting section 102.
30 [0099]
In step 814, the periodicity intensity detecting section 103 calculates the
SP342632XXOO
22/43
• periodicity intensity per time interval, based on the periodicity information of time
intervals supplied from the periodicity detecting section 102. Subsequently, the
periodicity intensity detecting section 103 binarizes the periodicity intensity per time
interval depending on whether it is over a threshold, and generates and supplies
5 periodicity intensity information per time interval to the periodicity information
integrating section 104. Here, the processing in step S14 is performed every
periodicity intensity detecting section 103.
[0100]
In step S15, using above Equation 1, the periodicity information integrating
10 section 104 performs periodicity integration processing based on the periodicity
information of time intervals supplied from the periodicity detecting section 102 and
the periodicity intensity information per time interval supplied from the periodicity
intensity detecting section 103. The periodicity information integrating section 104
supplies a total sum S('t) of the periodicity information of time intervals, which is
15 acquired as a result of the periodicity integration processing, to the peak detecting
section 105.
[0101]
In step S16, the peak detecting section 105 performs peak detection for the
total sum S('t) of periodicity information supplied in each time interval from the
20 periodicity information integrating section 104 and generates peak information p('t).
[0102]
In step S17, the peak detecting section 105 supplies the peak information P('t)
of time intervals as feature amount time series data of time intervals to the feature
amount storing section 22 in FIG. 1.
25 [0103]
In step S18, the block integrating section 121 of the synchronization
information generating section 23 reads the feature amount time series data of time
intervals of the content A, which is stored in the feature amount storing section 22-1
in FIG. 1 in step S17. Also, the block integrating section 122 reads the feature
30 amount time series data of time intervals of the content B, which is stored in the
feature amount storing section 22-2 in FIG. 1 in step S17.
SP342632XXOO
23/43
• [0104]
In step S19, using consecutive (for example, 64) time intervals as one block,
the block integrating section 121 integrates the feature amount time series data of
time intervals of the content A in units of blocks and supplies the results to the
5 similarity calculating section 123. Also, using consecutive (for example, 64) time
intervals as one block, the block integrating section 122 integrates the feature amount
time series data of time intervals of the content B in units of blocks and supplies the
results to the similarity calculating section 123.
[0105]
10 In step S20, the similarity calculating section 123 calculates the similarities
between the time-series data of block-units feature amounts supplied from the block
integrating section 121 and the time-series data of block-units feature amounts
supplied from the block integrating section 122, and generates a similarity matrix
representing the similarities between these blocks. The similarity calculating section
15 123 supplies the similarity matrix to the optimal path searching section 124.
[0106]
In step S21, the optimal path searching section 124 searches an optimal
similarity path from the similarity matrix supplied from the similarity calculating
section 123 and generates synchronization information. Subsequently, the optimal
20 path searching section 124 outputs the synchronization information.
[0107]
As described above, in the signal processing system 10, since the signal
processing apparatus 11 generates the synchronization information based on the
periodicity information, it is possible to generate the synchronization information in
25 a robust manner.
[0108]
Here, for example, a block subsequent to the signal processing system 10
continually compares the playback positions of the contents A and B, and, using the
synchronization information, checks whether a synchronization difference occurs.
30 Also, in a case where a synchronization difference occurs, by changing the playback
position of one of the contents A and B using the synchronization information, the
SP342632XXOO
24/43
• block subsequent to the signal processing system 10 corrects the synchronization
difference between the contents A and B. At this time, if the content is simply
thinned, since a discontinuous point occurs at the thinning point and a noise occurs, it
is desirable to use a method ofwarping only time while maintaining a steady pitch.
5 [0109]
Thus, by correcting the synchronization difference between the contents A
and B, the block subsequent to the signal processing system 10 can perform a
synchronization edit of the contents A and B and output acoustic data of one of the
contents A and B instead of acoustic data of the other.
10 [0110]

[Explanation of outline of second embodiment of signal processing system]
FIG. 11 is a view for explaining an outline of the second embodiment of the
signal processing system to which the present disclosure is applied.
15 [0111]
In components illustrated in FIG. 11, the same reference numerals are
assigned to the same components as in FIG. 1. An overlapping explanation will be
adequately omitted.
[0112]
20 A configuration of a signal processing system 140 in FIG. 11 differs from the
configuration of FIG. 1 in that an imaging apparatus 141 is newly set and a signal
processing apparatus 142 is set instead of the signal processing apparatus 12. The
signal processing system 140 generates synchronization information of contents
acquired by three imaging apparatuses.
25 [0113]
To be more specific, similar to the imaging apparatus 11, the imaging
apparatus 141 of the signal processing system 140 includes a video camera or the
like. The imaging apparatus 141 acquires a dynamic image and sound of the
identical event, which are different from those of the imaging apparatuses 11-1 and
30 11-2. Similar to the imaging apparatus 11, the imaging apparatus 141 transmits a
content C of the resulting dynamic image and sound to the signal processing
SP342632XXOO
25/43
• apparatus 142.
[0114]
The signal processing apparatus 142 includes the feature amount calculating
section 21-1, the feature amount calculating section 21-2, a feature amount
5 calculating section 151, the feature amount storing section 22-1, the feature amount
storing section 22-2, a feature amount storing section 152 and a synchronization
information generating section 153.
[0115]
The feature amount calculating section 151 is configured in the same way as
10 the feature amount calculating section 21 in FIG. 2. The feature amount calculating
section 151 calculates feature amount time series data based on the acoustic data of
the content C supplied from the imaging apparatus 141 and supplies the time-series
data to the feature amount storing section 152. The feature amount storing section
152 includes a memory and a hard disk, and so on, and stores the feature amount
15 time series data of the content C, which is supplied from the feature amount
calculating section 151.
[0116]
The synchronization information generating section 153 reads the feature
amount time series data of the content A from the feature amount storing section 22-
20 1, reads the feature amount time series data of the content B from the feature amount
storing section 22-2 and reads the feature amount time series data of the content C
from the feature amount storing section 152. Similar to the synchronization
information generating section 23, the synchronization information generating
section 153 generates synchronization information used at the time of synchronizing
25 the contents A and B, based on the feature amount time series data of the content A
and the feature amount time series data ofthe content B.
[0117]
Also, similar to the synchronization information generating section 23, the
synchronization information generating section 153 generates synchronization
30 information used at the time of synchronizing the contents A and C, based on the
feature amount time series data of the content A and the feature amount time series
•
SP342632XXOO
26/43
data of the content C. The synchronization information generating section 153
outputs the synchronization information of the contents A and B and the
synchronization information ofthe contents A and C.
[0118]
5 Also, the synchronization information generating section 153 may generate
synchronization information of other combinations than the combination of the
contents A and B and the combination of the contents A and C. Also, the
synchronization information generating section 153 generates synchronization
information of the contents B and C and compares the synchronization information
10 of the contents B and C with the synchronization of the contents A and B and the
synchronization information of the contents A and C, thereby increasing the accuracy
of synchronization information.
[0119]

15 [Configuration example of signal processing system]
FIG. 12 is a view for explaining an outline of the third embodiment of the
signal processing system to which the present disclosure is applied.
[0120]
Here, in the components illustrated in FIG. 12, the same reference numerals
20 are assigned to the same components as in FIG. 1. An overlapping explanation will
be adequately omitted.
[0121]
A signal processing system 370 in FIG. 12 includes the imaging apparatuses
11-1 and 11-2, a signal processing apparatus 371 and a server 372.
25 [0122]
The signal processing apparatus 371 includes the feature amount extracting
sections 21-1 and 21-2 in FIG. 1. The signal processing apparatus 371 calculates
feature amount time series data based on the acoustic data of the content A supplied
from the imaging apparatus 11-1, and calculates feature amount time series data
30 based on the acoustic data of the content A supplied from the imaging apparatus 112.
The signal processing apparatus 371 transmits the feature amount time series data
SP342632XXOO
27/43
• ofthe contents A and B to the server 372.
[0123]
The server 372 includes the feature amount storing sections 22-1 and 22-2
and the synchronization information generating section 23 in FIG. 1. The server 372
5 receives and stores the feature amount time series data of the content A transmitted
from the signal processing apparatus 371, and receives and stores the feature amount
time series data of the content B. The server 372 reads the feature amount time
series data of the content A and the feature amount time series data of the content B,
generates synchronization information based on the feature amount time series data
10 ofthe content A and the feature amount time series data of the content B, and outputs
the synchronization information.
[0124]
Also, assignment of the functions of the signal processing apparatus 371 and
the server 372 are not limited to the above assignment. For example, the signal
15 processing apparatus 371 may store the time-series data of the contents A and Band
read and transmit them to the server 372.
[0125]
Also, in the ftrst to third embodiments, although the frequency band dividing
section 101 divides acoustic data intp four frequency bands, the division number is
20 not limited to four and the acoustic data may not be subjected to band division.
[0126]
[Conftguration example of feature amount calculating section]
FIG. 13 illustrates a conftguration example of the feature amount calculating
section 21 in a case where acoustic data is not subjected to band division.
25 [0127]
The feature amount calculating section 21 in FIG. 13 includes a frequency
band dividing section 391, a periodicity detecting section 392 and a peak detecting
section 393.
[0128]
30 The frequency band dividing section 391 divides the acoustic data of the
contents supplied from the imaging apparatus 11 in FIG. 1, into time intervals of
SP342632XXOO
28/43
• around several tens of milliseconds to a hundred milliseconds, using a window
function, and supplies the acoustic data of the time intervals to the periodicity
detecting section 392.
[0129]
5 The periodicity detecting section 392 extracts periodicity information of time
intervals by calculating an autocorrelation function x(b,'t) ofthe acoustic data oftime
intervals supplied from the frequency band dividing section 391. The periodicity
detecting section 392 supplies the extracted periodicity information of time intervals
to the peak detecting section 393.
10 [0130]
The peak detecting section 393 performs peak detection on the periodicity
information supplied in each time interval from the periodicity detecting section 392
and generates peak information p('t).
[0131]
15 Here, in a case where periodicity information in the peak position 'tp is less
than a predetermined threshold, the peak detecting section 393 may set peak
information P('tp) in the peak position 'tp to O. By this means, it is possible to reduce
a noise of the peak information P('tp).
[0132]
20 The peak detecting section 393 outputs the peak information P('t) of time
intervals as feature amount time series data of time intervals.
[0133]
Here, as a feature amount, it is possible to adopt information such as an
acoustic data level and a rising position of the acoustic data level in addition to the
25 peak information P('t).
[0134]
For example, in the case of using acoustic data level information as a feature
amount, the acoustic data is divided into time intervals of several tens of
milliseconds and an RMS (Root Mean Square) value or average value ofthe acoustic
30 data per time interval is calculated as feature amount time series data. Subsequently,
a mutual correlation function between the feature amount time series data is
SP342632XXOO
29/43
• calculated as a similarity.
[0135]
Here, in this case, level information of acoustic data divided into a plurality
of frequency bands by band-pass filters may be used as a feature amount. In this
5 case, the mutual correlation function is calculated every frequency band and a total
sum of the mutual correlation functions of the frequency bands is calculated as a
similarity.
[0136]
Also, in a case where information of a rising position of an acoustic data level
lOis used as a feature amount, for example, the feature amount is information in which
a value in the rising position of acoustic data, that is, a value in a position with a
differential value equal to or greater than a threshold is 1 and values in other
positions are O.
[0137]
15 Also, as a feature amount, it is possible to use Zero Crossing Rate,
SpectrumFlux, Spectrum Centroid or Roll-Off alone or in combination, among
feature amounts disclosed in George Tzanetakis and Perry Cook, Musical genre
classification of audio signals, IEEE Transactions on Speech and Audio Processing,
10(5): 293-302, July 2002. In the case of using these feature amounts in
20 combination, the mutual correlation function of each feature amount is calculated as
a similarity.
[0138]
Further, as a feature amount, instead of the above information itself, it is
possible to use information in which, among peak positions of the information, that
25 is, among positions in which a differential value changes from a positive value to a
negative value, a value in a position with a level equal to or greater than a threshold
is 1 and values in other positions are O.
[0139]
Also, the apparatuses to acquire contents in the first to third embodiments
30 may be an apparatus to acquire at least acoustic data, and a recording apparatus such
as an IC (Integrated Circuit) recorder is possible.
SP342632XXOO
30/43
• [0140]

[Configuration example ofcomputer]
The above series of processing by the signal processing apparatus or the
5 server can be performed by hardware or software. In the case of performing the
series of processing by software, programs forming the software are installed in a
computer. Here, the computer includes a computer mounted in dedicated hardware
or a general-purpose personal computer that can execute various functions by
installing various programs.
10 [0141]
FIG. 14 is a block diagram illustrating a configuration example of hardware
of a computer that executes the above series of processing by programs.
[0142]
In the computer, a CPU (Central Processing Unit) 501, a ROM (Read Only
15 Memory) 502 and a RAM (Random Access Memory) 503 are connected to each
other via a bus 504.
[0143]
The bus 504 is further connected to an input/output interface 505. The
input/output interface 505 is connected to an inputting section 506, an outputting
20 section 507, a storing section 508, a communicating section 509 and a drive 510.
[0144]
The inputting section 506 includes a keyboard, a mouse and a microphone,
and so on. The outputting section 507 includes a display and a speaker, and so on.
The storing section 508 includes a hard disk and a nonvolatile memory, and so on.
25 The communicating section 509 includes a network interface and so on. The drive
510 drives removable medium 511 such as a magnetic disk, an optical disk, a
magnetic-optical disk and a semiconductor memory.
[0145]
In the computer configured as above, for example, the CPU 501 loads the
30 programs stored in the storing section 508 onto the RAM 503 via the input/output
interface 505 and the bus 504 and executes the programs, thereby performing the
SP342632XXOO
31/43
• above series of processing.
[0146]
The programs executed by the computer (i.e. CPU 501) can be recorded in
the removable medium 511 such as a package medium and provided. Also, the
5 programs can be provided via a wired or wireless transmission medium such as a
local area network, the Internet and digital satellite broadcasting.
[0147]
In the computer, by attaching the removable medium 511 to the drive 510, it
is possible to install the programs in the storing section 508 via the input/output
10 interface 505. Also, it is possible to receive the programs in the communicating
section 509 via the wired or wireless transmission medium and install them in the
storing section 508. In addition, it is possible to install the programs in advance in
the ROM 502 or the storing section 508.
[0148]
15 Also, the programs executed by the computer may be programs to perform
processing in chronological order along the order explained in the present
specification, or may be programs performed in parallel or at requested timings such
as the time when a calling is performed.
[0149]
20 Also, in the present specification, a system denotes an aggregation of multiple
components (such as an apparatus and a module (i.e. part)), and it does not matter
whether all components are in the identical chassis. Therefore, a plurality of
apparatuses stored in difference chassis and connected via a network, and one
apparatus in which a plurality of modules are stored in one chassis, are equivalent to
25 the system.
[0150]
It should be understood by those skilled in the art that various modifications,
combinations, sub-combinations and alterations may occur depending on design
requirements and other factors insofar as they are within the scope of the appended
30 claims or the equivalents thereof.
[0151]
SP342632XXOO
32/43
• For example, in the signal processmg system 10 according to the first
embodiment and the signal processing system 370 according to the third
embodiment, the number of imaging apparatuses may be three or more. Also, in the
signal processing system 10 according to the first embodiment and the signal
5 processing system 140 according to the second embodiment, feature amount time
series data may not be stored in the signal processing apparatus but may be stored in,
for example, a server outside the signal processing apparatus.
[0152]
Additionally, the present technology may also be configured as below.
10 [0153]
(1) A signal processing apparatus including:
a first periodicity detecting section detecting periodicity information of an
acoustic signal included in a first content, as first periodicity information;
a second periodicity detecting section detecting the periodicity information of
15 an acoustic signal included in a second content, as second periodicity information;
a similarity calculating section calculating a similarity between the first
periodicity information detected by the first periodicity detecting section and the
second periodicity information detected by the second periodicity detecting section;
and
20 a synchronization information generating section generating synchronization
information used at a time of synchronizing the first content and the second content,
based on the similarity calculated by the similarity calculating section.
(2) The signal processing apparatus according to (1), further including:
a first band dividing section performing a band division on an acoustic signal
25 included in the first content;
a second band dividing section performing a band division on an acoustic
signal included in the second content;
a fust periodicity information integrating section integrating the first
periodicity information of all bands; and
30 a second periodicity information integrating section integrating the second
periodicity information of all bands,
SP342632XXOO
33/43
• wherein the first periodicity detecting section detects the periodicity
information of the acoustic signal subjected to the band division by the first band
dividing section, in each band, as the first periodicity information;
wherein the second periodicity detecting section detects the periodicity
5 information of the acoustic signal subjected to the band division by the second band
dividing section, in each band, as the second periodicity information;
wherein the first periodicity information integrating section integrates the first
periodicity information of the all bands detected by the first periodicity detecting
section; and
10 wherein the second periodicity information integrating section integrates the
second periodicity information of the all bands detected by the second periodicity
detecting section.
(3) The signal processing apparatus according to (2), further including:
a first periodicity intensity detecting section detecting an intensity of the first
15 periodicity information detected by the first periodicity detecting section; and
a second periodicity intensity detecting section detecting an intensity of the
second periodicity information detected by the second periodicity detecting section,
wherein the first periodicity information integrating section integrates the first
periodicity information of the all bands based on the intensity of the first periodicity
20 information detected by the first periodicity intensity detecting section; and
wherein the second periodicity information integrating section integrates the
second periodicity information of the .all bands based on the intensity of the second
periodicity information detected by the second periodicity intensity detecting section.
(4) The signal processing apparatus according to anyone of (1) to (3),
25 wherein the similarity calculating section calculates a similarity per time
interval between the first periodicity information and the second periodicity
information; and
wherein the synchronization information generating section generates the
synchronization information per time interval, based on the similarity per time
30 interval.
(5) The signal processing apparatus according to (4),
SP342632XXOO
34/43
• wherein the similarity calculating section generates a similarity matrix
indicating the similarity per time interval between the fIrst periodicity information
and the second periodicity information; and
wherein the synchronization information generating section generates
5 information indicating an optical path on the similarity matrix, as the synchronization
information.
(6) The signal processing apparatus according to anyone of(1) to (5),
wherein the fIrst periodicity information is an autocorrelation function of the
acoustic signal included in the fIrst content; and
10 wherein the second periodicity information is an autocorrelation function of
the acoustic signal included in the second content.
(7) A signal processing method in a signal processing apparatus, including:
a fIrst periodicity detecting step of detecting periodicity information of an
acoustic signal included in a fIrst content, as fIrst periodicity information;
15 a second periodicity detecting step of detecting the periodicity information of
an acoustic signal included in a second content, as second periodicity information;
a similarity calculating step of calculating a similarity between the fust
periodicity information detected by processing in the fIrst periodicity detecting step
and the second periodicity information detected by processing in the second
20 periodicity detecting step; and
a synchronization information generating step of generating synchronization
information used at a time of synchronizing the fust content and the second content,
based on the similarity calculated by processing in the similarity calculating step.
(8) A program that causes a computer to function as:
25 a fIrst periodicity detecting section detecting periodicity information of an
acoustic signal included in a fIrst content, as fIrst periodicity information;
a second periodicity detecting section detecting the periodicity information of
an acoustic signal included in a second content, as second periodicity information;
a similarity calculating section calculating a similarity between the fust
30 periodicity information detected by the fIrst periodicity detecting section and the
second periodicity information detected by the second periodicity detecting section;
SP342632XXOO
35/43
• and
a synchronization information generating section generating synchronization
information used at a time of synchronizing the fIrst content and the second content,
based on the similarity calculated by the similarity calculating section.
5 (9) An electronic device including:
a fIrst periodicity detecting section detecting periodicity information of an
acoustic signal included in a fIrst content, as fIrst periodicity information;
a second periodicity detecting section detecting the periodicity information of
an acoustic signal included in a second content, as second periodicity information;
lOa similarity calculating section calculating a similarity between the fIrst
periodicity information detected by the fIrst periodicity detecting section and the
second periodicity information detected by the second periodicity detecting section;
and
a synchronization information generating section generating synchronization
15 information used at a time of synchronizing the fIrst content and the second content,
based on the similarity calculated by the similarity calculating section.
(10) A signal processing apparatus including:
a receiving section receiving fIrst periodicity information corresponding to
periodicity information of an acoustic signal included in a fIrst content, and second
20 periodicity information corresponding to the periodicity information of an acoustic
signal included in a second content;
a similarity calculating section calculating a similarity between the fIrst
periodicity information and the second periodicity information received by the
receiving section; and
25 a synchronization information generating section generating synchronization
information used at a time of synchronizing the fIrst content and the second content,
based on the similarity calculated by the similarity calculating section.
(11) A signal processing method in a signal processing apparatus, including:
a receiving step of receiving fIrst periodicity information corresponding to
30 periodicity information of an acoustic signal included in a fIrst content, and second
periodicity information corresponding to the periodicity information of an acoustic
SP342632XXOO
36/43
• signal included in a second content;
a similarity calculating step of calculating a similarity between the first
periodicity infonnation and the second periodicity infonnation received by
processing in the receiving step; and
5 a synchronization infonnation generating step of generating synchronization
infonnation used at a time of synchronizing the first content and the second content,
based on the similarity calculated by processing in the similarity calculating step.
(12) Aprogram that causes a computer to function as:
a receiving section receiving first periodicity infonnation corresponding to
10 periodicity infonnation of an acoustic signal included in a first content, and second
periodicity infonnation corresponding to the periodicity infonnation of an acoustic
signal included in a second content;
a similarity calculating section calculating a similarity between the first
periodicity infonnation and the second periodicity infonnation received by the
15 receiving section; and
a synchronization infonnation generating section generating synchronization
infonnation used at a time of synchronizing the first content and the second content,
based on the similarity calculated by the similarity calculating section.
(13) An electronic device including:
20 a receiving section receiving first periodicity infonnation corresponding to
periodicity infonnation of an acoustic signal included in a first content, and second
periodicity infomiation corresponding to the periodicity infonnation of an acoustic
signal included in a second content;
a similarity calculating section calculating a similarity between the first
25 periodicity infonnation and the second periodicity infonnation received by the
receiving section; and
a synchronization infonnation generating section generating synchronization
infonnation used at a time of synchronizing the first content and the second content,
based on the similarity calculated by the similarity calculating section.
30 (14) A signal processing system including:
a first periodicity detecting section detecting periodicity infonnation of an
SP342632XXOO
37/43
• acoustic signal included in a first content, as first periodicity information;
a second periodicity detecting section detecting the periodicity information of
an acoustic signal included in a second content, as second periodicity information;
a similarity calculating section calculating a similarity between the first
5 periodicity information detected by the first periodicity detecting section and the
second periodicity information detected by the second periodicity detecting section;
and
a synchronization information generating section generating synchronization
information used at a time of synchronizing the first content and the second content,
10 based on the similarity calculated by the similarity calculating section.
(15) A signal processing method in a signal processing system, including:
a first periodicity detecting step of detecting periodicity information of an
acoustic signal included in a first content, as first periodicity information;
a second periodicity detecting step of detecting the periodicity information of
15 an acoustic signal included in a second content, as second periodicity information;
a similarity calculating step of calculating a similarity between the first
periodicity information detected by processing in the first periodicity detecting step
and the second periodicity information detected by processing in the second
periodicity detecting step; and
20 a synchronization information generating step of generating synchronization
information used at a time of synchronizing the first content and the second content,
based on the similarity calculated by processing in the similarity calculating step.
(16) A signal processing apparatus including:
a first band dividing section performing a band division on an acoustic signal
25 included in a first content;
a first periodicity detecting section detecting periodicity information of the
acoustic signal subjected to the band division by the first band dividing section, as
first periodicit)': information;
a first periodicity intensity detecting section detecting an intensity of the first
30 periodicity information detected by the first periodicity detecting section;
a first periodicity information integrating section integrating the first
SP342632XXOO
38/43
• periodicity information of all bands, based on the intensity of the first periodicity
information detected by the first periodicity intensity detecting section;
a second band dividing section performing a band division on an acoustic
signal included in a second content;
5 a second periodicity detecting section detecting the periodicity information of
the acoustic signal subjected to the band division by the second band dividing
section, as second periodicity information;
a second periodicity intensity detecting section detecting an intensity of the
second periodicity information detected by the second periodicity detecting section;
lOasecond periodicity information integrating section integrating the second
periodicity information of all bands, based on the intensity of the second periodicity
information detected by the second periodicity intensity detecting section;
a similarity calculating section calculating a similarity between the first
periodicity information integrated by the first periodicity information integrating
15 section and the second periodicity information integrated by the second periodicity
information integrating section; and
a synchronization information generating section generating synchronization
information used at a time of synchronizing the first content and the second content,
based on the similarity calculated by the similarity calculating section.
20 [0000]
The preserit disclosure contains subject matter related to that disclosed in
Japanese Priority Patent Application JP 2012-039765 filed in the Japan Patent Office
on February 27, 2012, the entire content of which is hereby incorporated by
reference.
25

SP342632XXOO
39/43
• What is claimed is:
1. A signal processing apparatus comprising:
a first periodicity detecting section detecting periodicity information of an
acoustic signal included in a first content, as first periodicity information;
5 a second periodicity detecting section detecting the periodicity information of
an acoustic signal included in a second content, as second periodicity information;
a similarity calculating section calculating a similarity between the first
periodicity information detected by the first periodicity detecting section and the
second periodicity information detected by the second periodicity detecting section;
10 and
a synchronization information generating section generating synchronization
information used at a time of synchronizing the first content and the second content,
based on the similarity calculated by the similarity calculating section.
15 2. The signal processing apparatus according to claim 1, further comprising:
a first band dividing section performing a band division on an acoustic signal
included in the first content;
a second band dividing section performing a band division on an acoustic
signal included in the second content;
20 a first periodicity information integrating section integrating the first
periodicity information of all bands; and . .. .
a second periodicity information integrating section integrating the second
periodicity information of all bands,
wherein the first periodicity detecting section detects the periodicity
25 information of the acoustic signal subjected to the band division by the first band
dividing section, in each band, as the first periodicity information;
wherein the second periodicity detecting section detects the periodicity
information of the acoustic signal subjected to the band division by the second band
dividing section, in each band, as the second periodicity information;
30 wherein the first periodicity information integrating section integrates the first
periodicity information of the all bands detected by the first periodicity detecting
•
SP342632XXOO
40/43
section; and
wherein the second periodicity information integrating section integrates the
second periodicity information of the all bands detected by the second periodicity
detecting section.
5
3. The signal processing apparatus according to claim 2, further comprising:
a first periodicity intensity detecting section detecting an intensity of the first
periodicity information detected by the first periodicity detecting section; and
a second periodicity intensity detecting section detecting an intensity of the
10 second periodicity information detected by the second periodicity detecting section,
wherein the first periodicity information integrating section integrates the first
periodicity information of the all bands based on the intensity of the first periodicity
information detected by the first periodicity intensity detecting section; and
wherein the second periodicity information integrating section integrates the
15 second periodicity information of the all bands based on the intensity of the second
periodicity information detected by the second periodicity intensity detecting section.
4. The signal processing apparatus according to claim 1,
wherein the similarity calculating section calculates a similarity per time
20 interval between the first periodicity information and the second periodicity
information; and
wherein the synchronization information generating section generates the
synchronization information per time interval, based on the similarity per time
interval.
25
5. The signal processing apparatus according to claim 4,
wherein the similarity calculating section generates a similarity matrix
indicating the similarity per time interval between the first periodicity information
and the second periodicity information; and
30 wherein the synchronization information generating section generates
information indicating an optical path on the similarity matrix, as the synchronization
SP342632XXOO
41/43
• information.
6. The signal processing apparatus according to claim 1,
wherein the first periodicity information is an autocorrelation function of the
5 acoustic signal included in the first content; and
wherein the second periodicity information is an autocorrelation function of
the acoustic signal included in the second content.
7. A signal processing method in a signal processing apparatus, comprising:
lOafirst periodicity detecting step of detecting periodicity information of an
acoustic signal included in a first content, as first periodicity information;
a second periodicity detecting step of detecting the periodicity information of
an acoustic signal included in a second content, as second periodicity information;
a similarity calculating step of calculating a similarity between the first
15 periodicity information detected by processing in the first periodicity detecting step
and the second periodicity information detected by processing in the second
periodicity detecting step; and
a synchronization information generating step of generating synchronization
information used at a time of synchronizing the first content and the second content,
20 based on the similarity calculated by processing in the similarity calculating step.
8. Aprogram that causes a computer to function as:
a first periodicity detecting section detecting periodicity information of an
acoustic signal included in a first content, as first periodicity information;
25 a second periodicity detecting section detecting the periodicity information of
an acoustic signal included in a second content, as second periodicity information;
a similarity calculating section calculating a similarity between the first
periodicity information detected by the first periodicity detecting section and the
second periodicity information detected by the second periodicity detecting section;
30 and
a synchronization information generating section generating synchronization
•
SP342632XXOO
42/43
infonnation used at a time of synchronizing the fIrst content and the second content,
based on the similarity calculated by the similarity calculating section.