SP263152WOOO - Description Title of Invention MOVING PICTURE PROCESSING DEVICE, MOVING PICTURE PROCESSING 5 METHOD AND PROGRAM . '" Technical Field [0001] The present disclosure relates to a moving picture processing device, a 10 moving picture processing method, and a program. Background Art [0002] A video (moving' picture) generally includes many cuts. In image 15 expressions, the cut composition may be devised such as repeating a series of mutually different cuts (also called a cutback) according to the creator's intention. Incidentally, for those who enjoy video or those who use video as a material of other video, information about how video objects are grasped, in other words, how video is expressed in temporal context of video may be important. 20 Summary ofInvention Technical Problem [0003] Thus, identifYing cut pairs repeated as a series of mutually different cuts 25 from a moving picture and generating a cut composition image in which the representative image of each cut is arranged according to the order of cut transitions while the boundary of cut pairs being specified is proposed. [0004] However, if the cutback becomes more frequent and the number of cut pairs 30 increases, the cut composition image may become larger than a display area in which the cut composition image is displayed. If an attempt is made to display the cut SP263152WOOO - composition image at a time to maintain at-a-glance visibility, representative images will be reduced at a higher rate and displayed, leading to lower visibility of the cut composition image. If an attempt is made to partially display the cut composition image to maintain visibility ofthe cut composition im~ge, the cut composition image 5 will be divided and displayed, leading to low~r at-a-glance visibility. 'Thus, in any case, the user will not be able to easily grasp the cut composition of a moving picture '" through the cut composition image. [0005] Therefore, the present disclosure provides a moving picture processing 10 device capable of generating a cut composition image allowing one to easily grasp the cut composition of a moving picture, a moving picture processing method, and a program. Solution to Problem [0006] 15 According to an embodiment of the present disclosure, there is provided a moving picture ·processing device, including a cut transition detection unit that detects, from a moving picture containing a plurality of cuts, transitions between the cuts, a cut pair identification unit that categorizes the plurality of cuts into a plurality of cut groups having mutually different feature amounts, and identifies a plurality of 20 cut pairs including two or more sequential cuts belonging to the mutually differe,nt cut groups and being repeated in the moving picture, a cut pair generation unit that generates a predetermined number of the cut pairs, which are less than the plurality of cut pairs in number, from the plurality of cut pairs by combining at least a portion of the plurality of cuts in a manner that two or more cuts constituting each cut pair 25 belong to mutually different cut groups and a context of cut transitions in the moving picture is maintained, and a cut composition image generation unit that generates a cut composition image including the generated cut pairs. [0007] The cut pair generation unit may categorize the plurality of cut pairs into the 30 predetermined number of pair groups, and then, for each of the pair groups, may generate one cut pair from cut pairs contained in each of the pair groups by J '. > 2, .' .~-". ) " SP263152WOOO combining at least a portion of the cuts contained in each of the pair groups in a manner that two or more cuts constituting each cut pair belong to mutually different cut groups and a context of cut transitions in the moving picture is maintained. [0008] 5 The cut pair generation unit may categorize the plurality ofcut 'pairs into the predetermined number ofpair groups based on feature amounts of cuts. ... [0009] The q,ut pair generation unit may categorize the plurality of cut pairs into the predetermined number ofpair groups based on feature amounts of cut pairs. 10 [0010] The cut pair generation unit may generate one cut pair by combining the plurality of cuts based on feature amounts of cuts. [0011] The cut pair generation unit may generate one cut pair by combining the 15 plurality of cuts based on feature amounts of cut pairs. [0012] The cut pair generation unit may sort out cuts based on feature amounts of cuts for each cut group, and may generate one cut pair by combining·the plurality of sorted cuts. 20 [0013] The cut pair generation unit may sort out cuts based on feature amounts of cuts for a first cut group, and may generate one cut pair by combining a plurality of cuts belonging to a same cut pair as the sorted cuts. [0014] 25 The cut' pair may be generated based on an inner product of similarity matrices indicating a similarity between cut pairs. .[0015] • ---:: - 30 The cut pair with a maximum total of the inner products of the similarity matrices may be generated as a representative cut pair representing the plurality of cut pairs. [0016] J '> '.'....'.•.¥ " '. .) "'~,', . . ~';" SP263152WOOO The cut pair whose similarity to the representative cut pair is low may be generated together with the representative cut pair. [0017] The cut pair may be generated based on a scalar value of a similarity matrix 5 indicating a similarity between cut pairs. [0018] '" The cut pair with a maximum scalar value of the similarity matrix may be generated as a~representative cut pair representing the plurality of cut pairs. [0019] 10 The cut pair whose similarity to the representative cut pair is low may be generated together with the representative cut pair. [0020] The predetermined number may be set in accordance with display conditions ofthe cut composition image. 15 [0021] According to an embodiment of the present disclosure, there is provided a moving picture processing method including detecting, from a moving picture containing a plurality of cuts, transitions between the cuts, categorizing the plurality of cuts into a plurality of cut groups having mutually different feature arilOunts, and 20 identifies a plurality of cut pairs including two or more sequential cuts belonging to the mutually different cut groups and being repeated in the moving picture, generating a predetermined number of the cut pairs, which are less than the plurality of cut pairs in number, from the plurality of cut pairs by combining at least a portion of the plurality of cuts in a manner that the two or more cuts constituting each cut 25 pair belong to mutually different cut groups and a context of cut transitions in the moving picture is maintained, and generating a cut composition image including the generated cut pairs. [0022] According to another aspect of the present disclosure, a program to cause a 30 computer to execute the moving picture processing method is provided. The - J SP263152WOOO .. program may be provided by using a computer readable recording medium or via a communication method. Advantageous Effects of Invention 5 [0023] According to the present disclosure described above, a moving picture " processing device capable of generating a cut composition image allowing one to easily grasp tl,le cut composition of a moving picture, a moving picture processing method, and a program can be provided. 10 Brief Description of Drawings [0024] [Fig. 1] Fig. 1 is a flow diagram showing a procedure for a movmg picture processing method according to an embodiment of the present disclosure. 15 [Fig. 2] Fig. 2 is a block diagram showing the configuration of a moving picture processing device. [Fig. 3] Fig. 3 is a flow diagram showing an overall operation procedure for the moving picture processing device. [Fig. 4] Fig. 4 is a diagram exemplifying a cut composition. 20 [Fig. 5] Fig. 5 is a flow diagram showing a procedure for identifying a cut pair. [Fig. 6] Fig. 6 is a diagram exemplifying cut pair identification results. [Fig. 7] Fig. 7 is a diagram exemplifying a cut composition array. [Fig. 8] Fig. 8 is a flow diagram showing a display optimization procedure of the cut composition. 25 [Fig. 9A] Fig. 9A is a diagram (113) showing the display optimization procedure of the cut composition. [Fig. 9B] Fig. 9B is a diagram (2/3) showing the display optimization procedure of the cut composition. [Fig. 9C] Fig. 9C is a diagram (3/3) showing the display optimization procedure of 30 the cut composition. [Fig. 10] Fig. lOis a flow diagram showing the procedure for generating cut pairs. J 5 10 15 20 25 SP263152WOOO [Fig. 11] Fig. 11 is a diagram showing the procedure for generating two cut pairs. [Fig. 12] Fig. 12 is a diagram exemplifying conditions for generating cut pairs. [Fig. 13] Fig. 13 is a diagram exemplifying the procedure for generating cut pairs based on the number offrames ofthe cut pair in a first generation procedure. [Fig. 14] Fig. 14 is a diagram exemplifying the procedure for generating cut pairs based on the number of frames ofthe cut in the first generation procedure. " [Fig. 15] Fig. 15 is a diagram exemplifying the procedure for generating cut pairs based on volume fluctuations between cuts in the first generation procedure. ~ [Fig. 16] Fig. 16 is a diagram exemplifying the procedure for generating cut pairs based on the number of frames ofthe cut pair in a second generation procedure. [Fig. 17] Fig. 17 is a diagram exemplifying the procedure for generating cut pairs based on the number of frames ofthe cut in the second generation procedure. [Fig. 18] Fig. 18 is a diagram exemplifying the procedure for generating cut pairs based on volume fluctuations between cuts in the second generation procedure. [Fig. 19] Fig. 19 is a diagram exemplifying the procedure for generating cut pairs based on an image brightness histogram between cuts in the second generation procedure. [Fig. 20] Fig. 20 is a diagram exemplifying calculation results of similarities of feature amounts between cuts. [Fig. 21] Fig. 21 is a diagram (1/2) exemplifying the procedure for generating cut pairs based on an inner product of similarity matrices. [Fig. 22] Fig. 22 is a diagram (2/2) exemplifying the procedure for generating cut pairs based on the inner product of similarity matrices. [Fig. 23] Fig. 23 is a diagram exemplifying the procedure for generating cut pairs based on a scalar value of the similarity matrix. [Fig. 24] Fig. 24 is a diagram exemplifying a cut composition image. [Fig. 25] Fig. 25 is a diagram showing corrections of the cut composition image based on a contrast ratio. .. --- - 30 Description of Embodiments [0025] J SP263152WOOO Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the appended drawings. Note that, in this specification and the drawings, elements that have substantially the same function and structure are denoted with the same reference signs, and repeated explanation is 5 omitted. [0026] '" [1. Overview ofMoving Picture Processing Method] First, an overview of a moving picture processing method according to an ~ embodiment of the present disclosure will be described with reference to Fig. 1. 10 Fig. 1 shows a procedure for a moving picture processing method according to an embodiment of the present disclosure. [0027] In the moving picture processing method according to an embodiment of the present disclosure, as shown in Fig. 1, transitions between cuts are first detected from 15 a ~oving picture MP containing a plurality of cuts (step Sl). Next, the plurality of cuts is categorized into a plurality of cut groups having different feature amounts S (generic term for feature amounts of cut groups) to identify a plurality of cut pairs comprised of two or more sequential cuts belonging to different cut groups and repeated in the moving picture MP (step S3). 20 [0028] Then, at least a portion of the plurality of cuts is combined to generate a predetermined number N of cut pairs from the plurality of cut pairs so that the cut pairs comprised of two or more sequential cuts belong to mutually different cut groups and the content of cut transitions in the moving picture MP is maintained 25 (step S5). The number N of cut pairs is preset in accordance with display conditions (such as the display range and display size) ofa cut composition image CI (generic term for cut composition images). Further, the cut composition image CI comprised of generated cut pairs is generated (step S7). [0029] 30 Accordingly, the cut composition image CI capable of maintaining at-aglance visibility of the cut composition and visibility of the cut composition image .f ,n5 SP263152WOOO CI can be generated by generating the predetennined number N of cut pairs by combining at least a portion of a plurality of cuts so that predetennined conditions are satisfied and generating the cut composition image CI comprised of generated cut pairs. Then, the user can easily grasp the cut composition ofthe moving picture MP through the cut composition image CI generated as described above. [0030] '" - [2. Moving Picture Processing Device 1] Next., a moving picture processing device 1 according to an embodiment of the present disclosure will be described with reference to Fig. 2. Fig. 2 shows a 10 main function configuration of the moving picture processing device 1. As shown in Fig. 2, the moving picture processing device 1 includes a data acquisition unit 11, a cut transition detection unit 13, a cut pair identification unit 15, a display optimization unit 17, a cut pair generation unit 19, a meta infonnation generation unit 21, a cut composition image generation unit 23, a cut composition image output unit 15 25, a cut composition infonnation output unit 27, and a data storage unit 29. [0031] The data acquisition unit 11 acquires moving picture data MP containing a plurality of cuts to supply the moving picture data MP to the cut transition detection unit 13, the cut pair identification unit 15, the cut pair generation unit 19, the meta 20 infonnation generation unit 21, and the cut composition image generation unit 23. The moving picture data MP is generally data in frame fonnat and may be image data only or may be combined with audio data. The moving picture data MP may be acquired from the data storage unit 29 or an external device (not shown). [0032] - 25 'The cut transition detection unit 13 detects cut transitions in the moving picture MP based on the moving picture data MP and supplies detection results to the cut pair identification unit 15 and the cut composition infonnation output unit 27. The cur transition means a change of cuts in the moving picture MP. The cut transition is detected based on similarities of feature amounts detennined for images 30 and/or audios in succeeding frames. As a feature atnount of an image and/or audio, a color histogram, facial image detection, correlation between images and/or a sound J SP263152WOOO - volume, tone/rhythm and the like can be used. Feature amounts determined for detecting cut transitions may be stored in the data storage unit 29 for use in other processing. [0033] 5 Though details will be described later, the cut pair identificati~n unit 15 categorizes the plurality of cuts into cut groups based on feature amounts S' of each cut. Identification results of cut pairs are-" supplied to the display optimization unit 17, the cut pair generation unit 19, and the cut composition information output unit 27 together with detection results of cut transitions. As the feature amount S' of a 10 cut, a color histogram of an image contained in the cut, facial image detection, correlation betw~en images and/or a sound volume, tone/rhythm and the like or a combination of these can be used. The feature amounts S' determined for identifying cut pairs may be stored in the data storage unit 29 for use in other processing. 15 [0034] The cut'group means a combination of cuts having the feature amounts S' similar to each other. The cut pair means a combination of a series of mutually different cuts repeated in a cut combination and is comprised of two or more temporally sequential cuts. The cut pair identification unit 15 attaches attribute 20 information (a group ID, pair ill or the like described later) representing the cut group or cut pair to each cut based on identification results of cut pairs. The attribute information may be supplied to, in addition to the display optimization unit 17 and the cut pair generation unit 19, the cut composition information output unit 27, the data storage unit 29 and external devices. 25 [0035] , Though details will be described later, the display optimization unit 17 optimizes the display of the cut composition image CI in accordance with display conditions (such as the display range and display size) of the cut composition image CI. More specifically, the number N of cut pairs displayed as the cut composition 30 image CI is optimized to an optimal display number Nopt to maintain visibility of the cut composition image CI along with at-a-glance visibility of the cut composition. J ", SP263152WOOO ---::- -- Optimization results are supplied to the cut pair generation unit 19. [0036] Though details will be described later, the cut pair generation unit 19 generates cut pairs displayed as the cut composition image CI in accordance with cut 5 pair identification results and display optimization results. More specifically, cut pairs are generated so that the optimal cut pair number Nopt is satisfied based on the '" feature amounts S' ofcuts according to generation conditions for cut pairs. Cut pair generation res.,ults are supplied to the cut composition image generation unit 23, but may also be supplied to the cut composition information output unit 27, the data 10 storage unit 29 and external devices. [0037] The meta information generation unit 21 generates meta information MI (generic term for meta information) showing features of audio and images contained in each cut. The meta information generation unit 21 extracts audio or image 15 features from the moving picture data MP based on the moving picture data MP and detection results'of cut transitions. Then, the meta information MI showing audio or image features of each cut is generated and supplied to the cut composition image generation unit 23. [0038] 20 The cut composition image generation unit 23 generates the cut composition image CI based on the moving picture data MP and cut pair generation results. The cut composition image CI is an image in which representative images I of cuts contained in a generated cut pair are arranged in the order of cut transitions while the boundary of cut pairs being specified. In the generation of the cut composition 25 image CI, the representative image I is extracted from images contained in cuts of a generated cut pair according to predetermined criteria. The cut composition image CI may contain the meta information MI supplied by the meta information generation unit 21. The representative image I is an image that represents each cut and is extracted, for example, as an image corresponding to the center frame of the 30 cut. The cut composition image CI is supplied to the cut composition image output unit 25. J ", ..J SP263152WOOO [0039] The cut composition image output unit 25 outputs the cut composition image CI supplied by the cut composition image generation unit 23 so that the user can grasp the cut composition of the moving picture MP. The cut composition 5 image CI may be output to a display device, printing device, storage 'device, or external device (none of these devices shown) connected to the moving picture '" processing device 1. [0040] The cut composition information output unit 27 outputs cut transition 10 detection results, cut categorization results, or cut pair identification results as cut composition information so that the user can use the information to grasp the cut composition. The cut composition information may be output to a display device, printing device, storage device, or external device (none of these devices shown) connected to the moving picture processing device 1. 15 [0041] The cut composition information can be used, for example, as data to realize a moving picture search in consideration of the cut composition. For example, some cut may be set as a reference cut to search for a cut pairedwith the reference cut or some cut pair may be set as a reference cut pair to search for a cut pair 20 composed in the same manner as the reference cut pair. Also, the moving picture MP containing many cut pairs or the moving picture MP containing many cutbacks can be searched for: [0042] - 25 30 The data storage unit 29 stores the moving picture data MP and data attached to the moving picture data MP. The data storage unit 29 has cut composition information stored by being associated with the moving picture data MP. Incidentally, the cut composition image CI may be stored in the data storage unit 29. In Fig. 2, the marking of connection of the data storage unit 29 with other components is partially omitted. [0043] In the above function configuration, the data acquisition unit 11, the cut J ~ , SP263152WOOO transition detection unit 13, the cut pair identification unit 15, the display optimization unit 17, the cut pair generation unit 19, the meta information generation unit 21, the cut composition image generation unit 23, the cut composition image output unit 25, and the cut composition information output unit 27 are configured as 5 a processing unit such as a CPU nsp (digital signal processor) or the like. ' The data storage unit 29 is configured by an internal storage device such as a flash memory or '" an external storage device such as a hard dusk drive or blu-ray disk dive. The CPU , realizes the m~)Ving picture processing method by expanding a program read from a ROM or the like on a RAM and executing the program. The above function 10 configuration may at least partially be configured as hardware such as a dedicated logic circuit. [0044] [3. Procedure for Generating Cut Composition Array Mo] Next, the procedure for generating a cut composition array Me will be 15 described with reference to Figs. 3 to 7. Fig. 3 shows an overall operation procedure for the"moving picture processing device 1. [0045] As shown in Fig. 3, the data acquisition unit 11 first acquires the moving picture data MP (step S11) and supplies the moving picture data MP to the cut 20 transition detection unit 13. The cut transition detection unit 13 detects cut transitions in the moving picture data MP based on the moving picture data !vIP (step S13) and supplies' detection results to the cut pair identification unit 15. The cut transition is detected based on similarities of feature amounts of images and/or audios in succeeding frames. A serial number showing the order cut transitions is 25 attached to each cut as the cut ID. [0046] • .-- - 30 Fig. 4 exemplifies the cut composition determined from cut transition detection results. The cut composition is shown by using representative images 11 to 115 of cuts 1 to 15 to facilitate the understanding. As shown in Fig. 4, the cuts 1, 3, 6, 8, 11, 13 are similar to each other, the cuts 2, 4, 7, 9, 12, 14 are similar to each other, and the cuts 5, 10, 15 are similar to each other. (2- SP263152WOOO - [0047] Next, the cut pair identification unit 15 performs cut pair identification processing to identify a cut pair by categorizing each cut into cut groups. Fig. 5 shows a procedure for cut pair identification processing. In the cut pair 5 identification processing, as shown in Fig. 5, initialization processing is first performed (step S31). In the initialization processing, a group number m and a pair '" ill are initialized (m=2, pair ID=I). A group ID=1 and a group ill=2 are attached to the cuts 1, 2 r~spectively and the pair ID=1 is attached to the cuts 1, 2. [0048] 10 The group number m shows the number of cut groups (the cut groups 1, 2 are identified during initialization processing) identified from the moving picture data MP. The group ill and the pair ID are attached to each cut to indicate the group ID and the pair ID to which each cut belongs. [0049] 15 Next, the feature amount S' of the cut 1 is calculated and stored in the data storage unit 29 br the like as a feature amount SI of a cut group 1 (step S33). Similarly, the feature amount S' of the cut 2 is calculated and stored in the data storage unit 29 or the like as a feature amount S2 of a cut group 2 (step S35). The feature amount S of a cut group (generic term for the feature amount of acut group) 20 is calculated as a color histogram, facial image detection, correlation between images and/or a sound volume, tone/rhythm and the like or a combination ofthese. [0050] Next, whether the subsequent cut to be processed is present is checked (step S37). If the subsequent cut is present ("Yes" in step S37), the feature amount S' of 25 the subsequent cut is calculated (step S39) and the similarity between the feature amount S' ofthe subsequent cut and feature amounts S1 to Sm'of cut groups 1 to m is determined (step S41). When similarities are determined, similarities between the feature amounts S may preferentially be determined to a cut group having a larger group ill than that of a cut immediately before. This is because when belonging to 30 the same cut pair, the group ill of a cut group to which a subsequent cut belongs becomes larger than that of a cut group to which a cut immediately before belongs. J . ' ·oJ 5 10 15 20 25 SP263152WOOO [0051] If the similarity between the feature amount S' of the subsequent cut and one of the feature amounts SI to Sm of the cut groups 1 to m is determined to be equal to a predetermined threshold or more ("Yes" in step step S41), the group ID of the cut group x (1 cut ID of the sub-cut), the subcut 6 may be sorted out, instead of the sub-cut 2, so that the order of cut transitions 15 becomes nonnal between the main cut and the sub-cut. [0095] - 20 25 30 Instead of sorting out the main cut and the sub-cut separately based on the respective numbers of frames, the main cut may be selected based on the number of frames to subsequently select the sub-cut belonging to the same cut pair as the selected main cut. In this case, if, for example, the main cuts 5, 9, 11 are sorted out, the sub-cuts 6, 10, 12 are automatically selected. -"[0096] Also instead of the number of frames of the main cut, the cut pairs' 1 to 7 may be categorized based on the number of frames of the sub-cut or the number of frames of the cut pair. Also instead of two main cuts whose number of frames is the largest or the second largest, the positions of two main cuts with the smallest two numbers of frames may be used as delimiters. Also instead of sorting out the main cut and the sub-cut with the largest numbers of frames for each pair group, the main cut and the sub-cut belonging to the cut pair with the largest number of frames may be selected. [0097] SP263152WOOO Fig. 15 shows an example in which cut pairs are generated based on volume fluctuations between cuts. First, like the example shown in Fig. 13, the cut pairs 1 to 7 are categorized into the pair groups 1 to 3 based on the numbers ofthe cut pairs. [0098] 5 Next, the main cut with the maximum volume fluctuations is sorted from main cuts (cuts 1, 3, 5, 7, 9, 11, 13) belonging to the cut group 1 for each of the cut '" groups 1 to 3. The volume fluctuations are calculated as a ratio of volume of each main cut to t~e average volume of main cuts contained in each pair group. In the above example, the main cut 3 (volume fluctuation: -6.7), the main cut 7 (volume 10 fluctuation: 5.0), and the main cut 11 (volume fluctuation: 5.0) with the maximum absolute values of volume fluctuations to the average volumes 18.3, 15.0, 20.0 of main cuts contained in the pair groups 1, 2, 3 respectively are sorted out (see the item of Group 1 volume fluctuations). When two or more absolute maximum values are present in the same pair group, the main cut having the smallest cut ID is sorted out 15 . for convenience sake. [0099] • 20 25 30 Similarly, the sub-cut with the maximum volume fluctuations is sorted from sub-,cuts (cuts, 2, 4, 6, 8, 10, 12, 14) belonging to the cut group 2 for each of the cut groups 1 to 3. The volume fluctuations are cil1culated as a ratio of volume of each sub-cut to the average volume of sub-cuts contained in each pair group. In the above example, the sub-cut 6 (volume fluctuation: -6.7), the sub-cut 8 (volume fluctuation:"-5.0), and the sub-cut 12 (volume fluctuation: 2.5) with the maximum absolute values of volume fluctuations to the average volumes 18.3, 15.0, 12.5 of sub-cuts contained in the pair groups 1, 2, 3 respectively are sorted out (see the item of Group 2 volume fluctuations). When two or more absolute maximum values are present in the same pair group, the sub-cut having the smallest cut ill is sorted out for convenience sake. Thus, a cut pair comprised of the cuts 3, 6, a cut pair comprised of the cuts 7, 8, and a cut pair comprised of the cuts 11, 12 are generated to generate the cut composition image CI. [0100] Instead of categorizing cut pairs into pair groups based on the numbers of _..- - J I 2. j r .J SP263152WOOO - frames of the cut pairs, like the example shown in Fig. 14, two cuts with the largest two volume fluctuations may be selected from all cuts to categorize the cut pairs 1 to 7 into pair groups by using the positions of the selected cuts as delimiters. The volume fluctuations are calculated as a ratio of volume of each cut to the average 5 volume ofall cuts contained in the cut pairs 1 to 7. [0101] ... The cut with the largest volume fluctuations may be sorted from all cuts contained in ~ach pair group without distinguishing the main cut and the sub-cut for each of the pair groups 1 to 3. For example, the cut 3 (volume fluctuation -6.7 from 10 the average volume 18.3) with the largest volume fluctuations may be sorted from the cuts 1 to 6 for the pair group 1 to sort out the cut 4 belonging to the same cut pair 2 as the selected cut 3. [0102] Also instead of sorting out the main cut and the sub-cut with the largest 15 volume fluctuations separately for each pair group, the main cut and the sub-cut belonging to the· cut pair with the largest volume fluctuations may be sorted out. The volume fluctuations are calculated as a ratio of volume of each cut pair to the average volume of all cut pairs contained in each pair group. [0103] 20 Next, a case when cut pairs are generated from the cut pairs 1 to 7 so that the optimal display number Nopt=3 is satisfied according to the second generation procedure will be described. Processing conditions and the cut configuration shown below are only examples to describe the generation processing ofcut pairs. [0104] 25 Fig. 16 shows an example in which cut pairs are generated based on the numbers of frames of cut pairs. Three cut pairs with the largest three numbers of frames are selected from the cut pairs 1 to 7. Then, cuts contained in the selected cut pairs are sorted out. In the above example, the cuts 5 to 8, 13, 14 corresponding to the cut pairs 3, 4, 7 with the numbers of frames 60, 60, 60 respectively are sorted 30 out (see the item of Number of pair frames). Thus, three cut pairs comprised of the cut pairs 3, 4, 7 are generated to generate the cut composition image CI. J 2{ ..J SP263152WOOO - [0105] Instead of three cut pairs with the three largest numbers of frames, three cut pairs with the average number of frames may be selected or one cut pair with the average number of frames and two cut pairs with the two largest numbers of frames 5 .may be selected. [0106] '.. Fig. 17 shows an example in which cut pairs are generated based on the numbers of :frames of cuts. First, three cuts with the three largest numbers of frames are selected from all cuts contained in the cut pairs 1 to 7. Next, cuts 10 belonging to the same cut pairs as the selected cuts are selected. In the above example, after the cuts 2,5,9 with the numbers offrames 25,20,25 being sorted out, the corresponding cuts 1, 6, 10 are sorted out respectively. Thus, three cut pairs comprised ofthe cut pairs 1,3,5 are generated to generate the cut composition image CI. 15 [0107] Three cilts with the three largest numbers of frames may be sorted out from one of main cuts and sub-cuts, instead of all cuts. Also, one cut with the average number of frames may be selected from one of main cuts and sub-cuts and two cuts with the two largest numbers of frames may be selected from the other. 20 [0108] Fig. 18 shows an example in which cut pairs are generated based on volume fluctuations between cuts.·First, three cuts with the three largest volume fluctuations are selected from all cuts contained in the cut pairs 1 to 7. The volume fluctuations are calculated as a ratio of volume of each cut to the average volume of 25 cuts contained in the cut pairs 1 to 7. Next, cuts belonging to the same cut pairs as the selected cuts are selected. In the above example, after the cuts 3, 6, 11 with the volume fluctuations -8.2 for all being sorted out, the corresponding cuts 4, 5, 12 are sorted out (see the item Group 1, 2 volume fluctuations). Thus, three cut pairs comprised ofthe cut pairs 2, 3, 6 are generated to generate the cut composition image 30 CI. [0109] J SP263152WOOO • Three cuts with the three largest volume fluctuations may be sorted out from one of main cuts and sub-cuts, instead of all cuts. The volume fluctuations are calculated as a ratio of volume of each cut to the average volume of main cuts or subcuts contained in the cut pairs 1 to 7. 5 [0110] Fig. 19 shows an example in which cut pairs are generated based on screen '" brightness fluctuations between cuts. First, histograms that represent normalized screen brightI!,ess are calculated based on image processing for representative images of the cuts 1 to 14. Next, an average histogram of seven cuts is calculated for each 10 cut group. A histogram represents the frequency in each section when brightness of pixels contained in a representative image is sectioned at predetermined brightness intervals. Fig. 19 shows normalized histograms of representative images II to II4 and average histograms for cut groups, along with the representative images II to II4 of the cuts 1 to 14. 15 [0111] Next, three cuts with the three largest fluctuations with respect to the average histogram are sorted out. Fluctuations of a histogram are calculated as differences between the normalized histogram of each cut and the average histogram of the cut group to which each cut belongs. Then, cuts belonging to the same cut 20 pairs as the selected cuts are selected. In the above example, fluctuations ofthe cuts 1, 11, 14 are the three largest and three cut pairs comprised of the cut pairs 1,6, 7 are generated to generate the cut composition'image CI. [0112] Next, a case when cut pairs are generated based on similarities of feature 25 amounts between cuts will be described. A case when cut pairs are generated from the cuts 1 to 14 constituting the cut pairs 1 to 7 so that the optimal display number Nopt=3 is satisfied. [0113] Fig. 20 shows results of calculating similarities of feature amounts among 30 the cuts 1 to 14. In the calculation results shown in Fig. 20, the cuts 1 to 14 are categorized into the cut group 1 (cuts 1,3,5, 7, 9, 11, 13) and the cut group 2 (cuts 2, J SP263152WOOO .. 4,6,8, 10, 12, 14) and also into the cut pairs 1 to 7. [0114] In Fig. 20, similarities of feature amounts among the cuts 1 to 14 are shown as values between 0 and 1 relative to feature amounts S1, S2 of the cut groups 1, 2 5 corresponding to feature amounts of the cuts 1, 2. A similarity of feature amounts closer to 1 means that the feature amounts between cuts are more similar. For ',. example, while the cuts 1 and 3 belonging to the same cut group have a high similarity of feature amounts of 0.9, the cuts 1 and 4 belonging to different cut ~ groups have a low similarity of feature amounts of 0.1. 10 [0115] Fig. 21 shows a first similarity matrix Ms1 showing a similarity between the cuts 1,2 and a second similarity matrix Ms2 showing a similarity between the cuts 3, 4. The first and second similarity matrices Msl, Ms2 are matrices extracted from calculation results shown in Fig. 20. Then, the similarity between the cut pair 1 15 (cuts 1,2) and the cut pair 2 (cuts 3, 4) can be calculated by the inner product of the first and second similarity matrices Msl, Ms2. An increasing inner product of the first and second similarity matrices Msl, Ms2 means that the cut pairs are more similar. [0116] 20 As shown in Fig. 21, the first similarity matrix Msl is vectorized as (1.0, 0.2, 0.3, 1.0) and the second similarity matrix Ms2 is vectorized as (0.9, 0.1, 0.2, 0.8). Thus, the inner product of the first and second-similarity matrices Msl, Ms2 is calculated as 1.0xO.9+0.2xO.l+0.3 xO.2+1.0xO.8 .. 1.8. Similarities among the cut pairs 1 to 7 excluding betw.een the cut pairs 1, 2 can be calculated by the same 25 method. [0117] Accordingly, as shown in Fig. 22, similarities among the cut pairs 1 to 7 are calculated. Fig. 22 shows the total of similarities of each of the cut pairs 1 to 7 " along with similarities among the cut pairs 1 to 7. An increasing total of similarities 30 means that the cut pair has an increasing degree of affinity, that is, the probability that the cut pair represents the cut pairs 1 to 7 increases. J .'" SP263152WOOO [0118] When cut pairs are generated based on the inner product of similarity matrices Ms (generic term for similarity matrices), the cut pair 3 with the maximum total (9.1) of similarities is first selected. Secondly, the cut pair 7 with the lowest 5 similarity (1.1) to the cut pair 3 is selected. Thirdly, the cut pair 1 with the lowest similarity (0.9) to the cut pair 7 is selected. Thus, three cut pairs comprised of the '" cut pairs 1,3, 7 are generated to generate the cut composition image CI. [0119] Instead of the criteria of the lowest similarity to the cut pair 7, the cut pair 10 may be selected according to the criteria of the second lowest similarity to the cut pair 3. Also, three cut pairs may also be selected according to the criteria of a cut pair of the maximum total of similarities, a cut pair of the minimum total, and a cut pair closest to the average value. [0120] 15 Accordingly, the representative cut pair representing all cut pairs and other cut pairs dissimil"ar to the representative cut pair can be generated. [0121] • 20 25 30 Fig. 23 shows the first similarity matrix Msl showing similarities between the cuts 1, 2 and the second similarity matrix Ms2 showing similarities between the cuts 3, 4. The first and second similarity matrices Msl, Ms2 are matrices extracted from calculation results shown in Fig. 20. Then, cut pairs can be selected based on a scalar value of a similarity matrix Ms showing'similarities among the cuts 1 to 14. An increasing scalar value of the similarity matrix Ms means that the probability that the cut pair represents the cut pairs 1 to 7 increases. [0122] For example, the scalar value of the first similarity matrix Msl is calculated as 1.0+0.2+0.3+1.0=2.5 and the scalar value of the second similarity matrix Ms2 is calculated as 0.9+0.1 +0.2+0.8=2.0. Thus, of the cut pair 1 (cuts 1 , 2) and the cut pair 2 (cuts 3, 4), the cut pair 1 has a higher probability of being a representative cut pair representing all cut pairs. [0123] J • 1. -<. , ... SP263152WOOO When cut pairs are generated based on the scalar value of the similarity matrix Ms, scalar values of the similarity matrices Ms are calculated among the cut pairs 1 to 7. Next, three cut pairs are selected by replacing the total of similarities with the scalar value and performing processing in the case shown in Fig. 22. Then, 5 cuts contained in the selected cut pairs are sorted out. [0124] Accordingly, the representative cut pair representing all cut pairs and other cut pairs dissi:Q1ilar to the representative cut pair can be generated. [0125] - 10 15 20 25 30 When the cut pair generation processing is completed, as shown in Fig. 3, meta /information MI of each cut is generated by the meta information generation unit 21 (step S21). The meta information generation unit 21 extracts features of images or audios contained in each cut from the moving picture data MP based on the moving picture data MP and detection results of cut transitions. [0126] If, for example, a cut contains a sound (words, sound effects or the like), the sound contained in the cut may be extracted to generate character/image information corresponding to the extracted sound through voice recognition processing. If no sound is contained in a cut, character/image information indicating a silent cut may be generated. Silent cuts may be distinguished between silent cuts containing no words and silent cuts containing neither words nor sound effects. Character/image information indicating the average value/variations of volume of the sound' contained in a cut, ratio of silent intervals and non-silent intervals, and tone, rhythm, or fluctuations ofthe sound may also be generated. [0127] The number of frames contained in a cut or the time needed to reproduce a cut may be calculated to generate character/image information indicating the calculated value. Also, character/image information indicating the average value/variations of brightness of images contained in a cut and content or changes of images may be generated. [0128] J • 1-'" . . ) " SP263152WOOO .. The cut composition image generation unit 23 generates the cut composition image CI based on results of the cut pair generation processing (step S23). The cut composition image generation unit 23 first extracts the representative image I from a series of images belonging to the selected cut according to predetermined criteria 5 based on the moving picture data MP and results of the cut pair generation processing. The representative image I of each cut may also be extracted in '" advance when cut transitions are detected. Next, the cut composition image CI in which the representative images I of cuts are arranged in the order of cut transitions while cut pairs being specified is generated. If the meta information MI of each cut 10 has been generated, the meta information MI is displayed together with the representative image I ofeach cut. [0129] Fig. 24 exemplifies the cut composition image CI generated from results of the cut pair generation processing shown in Fig. 13. In the cut composition image 15 CI shown in Fig. 24, the representative images 15, 16 of the cuts 5, 6 are arranged horizontally, the representative images 17, 18 of the cuts 7, 8 are arranged horizontally below the representative images 15, 16 of the cuts 5, 6, and the representative images 13, 14 of the cuts 3, 4 are arranged horizontally below the representative images 17, 18 of the cuts 7, 8. The cut composition image CI 20 described above facilitates the understanding of the cut composition. However, the composition ofthe cut composition image CI is not limited to the composition shown in Fig:24. [0130] Fig. 25 shows a modification of the cut composition image CI. In the cut 25 composition image CI shown in Fig. 25, the meta information MI of a cut is displayed by being superimposed on the representative image I ofthe cut. The meta information MI of a cut is information indicating features of images or audios contained in the cut. [0131] 30 The meta information MI indicating sound features is, for example, information indicating content of sound (words, sound effects or the like) contained .-- - J '1 2-, ·oJ SP263152WOOO - in each cut, infonnation indicating that no sound is contained in each cut (indicating a silent cut) and the like. The meta infonnation MI indicating image features is, for example, infonnation indicating the number of frames contained in each cut, infonnation indicating the time needed to reproduce each cut and the like. 5 [0132] In a cut composition image CI shown in a state A of Fig. 25, for example, '" meta infonnation MIl, MI3 of the cuts 1, 3, meta infonnation MI5, MI7, MI9 of the cuts 5, 7, 9, "and meta infonnation MIll of the cut 11 are displayed by being superimposed on the representative image II of the cut 1, the representative image 17 10 of the cut 7, and the representative image III of the cut 11 respectively. Accordingly, visibility of the representative image 17 of the cut 11 in which the meta infonnation MI5, MI7, MI9 of three cuts is displayed by being superimposed thereon is reduced. [0133] 15 Thus, the contrast ratio by the meta infonnation MI, that is, the display occupancy of the meta infonnation MI on the representative image I is calculated for the representative images II, 17, III of the cuts 1, 7, 11. In this case, while the contrast ratio of the representative image 17 of the cut 7. is relatively high, the contrast ratio of the representative image III ofthe cut 11 is relatively low. 20 [0134] Thus, in a cut composition image CI' shown in a state B of Fig. 25, the meta infonnation MI9 of the cut 9 on the representative image 17 of the cut 7 is moved onto the representative image III of the cut 11 based on the contrast ratio by the meta infonnation MI. Accordingly, when compared with the state A, the contrast 25 ratio becomes lower in the representative image 17 of the cut 7 so that visibility of the representative image 17 can be maintained. [0135] According to the moving picture processing method according in the present embodiment, as described above, the cut composition image CI capable of 30 maintaining at-a-glance visibility of the cut composition and visibility of the cut composition image CI can be generated by generating a predetennined number of cut J ~]' SP263152WOOO pairs by combining at least a portion of a plurality of cuts so that predetermined conditions are satisfied and generating the cut composition image CI comprised of generated cut pairs. [0136] 5 Although the preferred embodiment of the present disclosure is described in detail so far with reference to the appended dTawings, the present disclosure is not '" limited to such an example. It is clear that one of ordinary skill in the technical field to which the present disclosure pertains may conceive of various. variations or ~ modifications without departing from the technical idea recited in claims, and it is 10 understood that they naturally pertain to the technical scope ofthe present disclosure. [0137] In the above embodiment, for example, the composition image CI is described as an image in which cut pairs of the optimal display number Nopt=3 are arranged in three rows. However, the composition image CI may be generated with 15 a different optimal display number Nopt in accordance with display conditions thereof or even if the optimal display number Nopt is the same, the composition image CI may be generated as an image in which the representative images I are arranged in a different number of rows/columns. 20 Reference Signs List [0138] - 25 30 1 11 13 15 17 19 21 23 25 27 .. Moving picture processing device Data acquisition unit Cut transition detection unit . Cut pair identification unit Display optimization unit Cut pair generation unit Meta information generation unit Cut composition image generation unit Cut composition image output unit Cut composition information output unit J 29 Data storage unit MP Moving picture (data) I Representative image Nopt Optimal display number 5 CI Cut composition image Ad Display area ... Ro Display occupancy ~ • -~ - SP263152WOOO J .~ SP263152WOOO - \ . CLAIMS Claim 1 A moving picture processing device, comprising: a cut transition detection unit that detects, from a moving picture containing 5 a plurality of cuts, transitions between the cuts; a cut pair identification unit that categorizes the plurality of cuts into a '.. plurality of cut groups having mutually different feature amounts, and identifies a plurality of cl{t pairs including two or more sequential cuts belonging to the mutually different cut groups and being repeated in the moving picture; 10 a cut pair generation unit that generates a predetennined number of the cut pairs, which are less than the plurality of cut pairs in number, from the plurality of cut pairs by combining at least a portion of the plurality of cuts in a manner that two or more cuts constituting each cut pair belong to mutually different cut groups and a context ofcut transitions in the moving picture is maintained; and 15 a cut composition image generation unit that generates a cut composition image including the generated cut pairs. Claim 2 The moving picture processing device according to claim 1, wherein the cut 20 pair generation unit categorizes the plurality of cut pairs into the predetennined number of pair groups, and then, for each of the pair groups, generates one cut pair from cut pairs contained in each of the pair groups by combining at least a portion of the cuts contained in each of the pair groups in a manner that two or more cuts constituting each cut pair belong to mutually different cut groups and a context of cut 25 transitions in the moving picture is maintained. Claim 3 The moving picture processing device according to claim 2, wherein the cut pair generation unit categorizes the plurality of cut pairs into the predetennined 30 number of pair groups based on feature amounts of cuts. J SP263152WOOO • Claim 4 The moving picture processing device according to claim 2, wherein the cut pair generation unit categorizes the plurality of cut pairs into the predetermined number ofpair groups based on feature amounts of cut pairs. 5 Claim 5 '" The moving picture processing device according to claim 1, wherein the cut pair generatiop unit generates one cut pair by combining the plurality of cuts based on feature amounts of cuts. 10 Claim 6 The moving picture processing device according to claim 1, wherein the cut pair generation unit generates one cut pair by combining the plurality of cuts based on feature amounts of cut pairs. 15 Claim 7 The moving picture processing device according to claim 1, wherein the cut pair generation unit sorts out cuts based on feature amounts of cuts for each cut group, and generates one cut pair by combining the plurality of sorted cuts. 20 Claim 8 The moving picture processing device according to' claim 1, wherein the cut pair generation unit sorts out cuts based on feature amounts of cuts for a first cut group, and generates one cut pair by combining a plurality of cuts belonging to a 25 same cut pair as the sorted cuts. Claim 9 The moving picture processing device according to claim 1, wherein the cut pair is generated based on an inner product of similarity matrices indicating a 30 similarity between cut pairs. J ", 5 -. SP263152WOOO Claim 10 The moving picture processing device according to claim 9, wherein the cut paIr with a maximum total of the inner products of the similarity matrices is generated as a representative cut pair representing the plurality of cut pairs. - 10 15 20 25 30 . Claim 11 ... The moving picture processing device according to claim 10, wherein the cut pair whos~ similarity to the representative cut pair is low is generated together with the representative cut pair. Claim 12 The moving picture processing device according to claim 1, wherein the cut pair is generated based on a scalar value of a similarity matrix indicating a similarity between cut pairs. Claim 13 The moving picture processing device according to claim 12, wherein the cut pair with a maximum scalar value of the similarity matrix is generated as a representative cut pair representing the plurality of cut pairs. Claim 14 The moving picture processing device accbrding to claim 13, wherein the cut pair whose similarity to the representative cut pair is low is generated together with the representative cut pair. Claim 15 The moving picture processing device according to claim 1, wherein the predetermined number is set in accordance with display conditions of the cut composition image. Claim 16 J ·oJ r _ J.. SP263152WOOO .. A moving picture processing method, comprising: detecting, from a moving picture containing a plurality of cuts, transitions between the cuts; categorizing the plurality of cuts into a plurality of cut groups having 5 mutually different feature amounts, and identifies a plurality of cut pairs including tWo or more sequential cuts belonging to th~ mutually different cut groups and being repeated in the moving picture; generaf-ing a predetermined number of the cut pairs, which are less than the plurality of cut pairs in number, from the plurality of cut pairs by combining at least 10 a ,portion of the plurality of cuts in a manner that the two or more cuts constituting each cut pair belong to mutually different cut groups and a context of cut transitions ,in the moving picture is maintained; and generating a cut composition image including the generated cut pairs. 15 Claim 17 A program causing a computer to execute a moving picture processing method, comprising: detecting, from a moving picture containing a plurality of cuts, transitions between the cuts; 20 categorizing the plurality of cuts into a plurality of cut groups having mutually different feature amounts, and identifies a plurality of cut pairs including two or more sequential cuts belonging to the mutually different cut groups and being repeated in the moving picture; generating a predetermined numBer of the cut pairs, which are less than the 25 plurality Qf cut pairs in number, from the plurality of cut pairs by combining at least a portion of the plurality of cuts in a manner that the two or more cuts constituting each cut pair belong to mutually different cut groups and a context of cut transitions in the moving picture is maintained; and generating a cut composition image including the generated cut pairs.