TNFORMA'llON I'ROCESSTNG DT^VTCE, TNFORMA'IION I'ROCESSING 5 METHOD, AND PROGRAM Technical Field [0001] Tlic present disclosure relates to an infonnation processing device, an 10 infonnalion processing method, and a pro-am. Tn particular, the present disclosure relates tn an iiitbrmation processing device, an information processing method^ and a program, which pBrfoim an augmented reality (AR) display. Background Art 15 iOOOl] For example, an image, in ivJiieh a virtual image other tlian a captured image ts superimposed on a captured image of a camera, is called an au^nented reality (AR) image and has recently bccii used in various fields. In particular, in recent years, porlahle terminals, sudi as a smartphone 20 having a camera funclien and a display as well as a commuuLcalLon funetion, have heen widespread, anti a|>plicatious that apply au AR image to snch a smarlphone are widely used. [C003] As nne example of use of UieAR image, there is the following example. 25 For example, a poster of a person is cajifiired using a camera function of a portable terminal such as a smarlphune ami is displayed on a display section of llie smartphone. A data processing section of the smaitphone identifies a eaplured poster or a marker set to tlie poster and acquires image data, which coirespouds to a person 3fl moved to the poster, from a storage section or an exteiual server. Furtheimore, the acquired image is superimposedly (lisplayed on the SP34fi7n4WO00 2/57 captured image. Such proccsning rriakes it possible to display and observe an image such that a person jiops out from the poster. Incidentally, as sn example of related art disclosing the gencmtion and display processing of an AR image, there is Patent Literature 1 (JP20I2-5RR3SA). 5 [0004] Fuflhctinori;, by enlarging a pi^rson image popping out troiu the poster with Ibc passage of lime, il is possible lo display an Ai^ image sucli that die person approaches a iisor side holding Ihi; smailpbone. [00051 10 Por example, when a camera captures a poster in an oblique direelion, a person printed on the poster is displayed on a captnred image as an imaj^e viewefl in an oblique direction, in order to generate a natural AR im^e such ^at a person viewed in an oblique direction pops out from a poster, an unage to be superimposed also needs to be an image of a person viewed in an oblique direction, just like the Id posICT. [00061 - ■I TTowevcr, if the image of (he person viewed in the obliquely direelion is generated as an image approaching a user side widi die passage of time, it heeoukes an unnatural im^e because the |>erson viewed in die oblique direction ap|]roaehes lo 'AO the user side, while keeping an oiientation as it is. Cilalion Lis( PalcntT.itcraturc [O0O7I 2h Patent Literature 1: JP20I2-5SB3KA Summaiy of Invention 'leclmical Problem [OOOKJ 30 The present diselobure has been madi; in view of, for example, the above problems, and provides an infomialion processing device, an information processing SP.^4fi7n4WO00 3/57 inclhod, and a program. Aecording lo the present disclosure, m a eonfiguralion that auperunposedly dijiptays a virtual image on a captured image capluied by a camera ofa portable temiinal held by a user and displayed on a display acdion, for example, a more natural AR image can be displayed by performing control of changing a G position or an angle of a virtual imafie, suchas a person, according to time, Solution lo Problem I0D09J Accordinft (o a first aspect of the present disclosure^ there is provided an infonnalion process iitg device including: 10 an acquisition section conQgiired to accpire a captured imagi; capEiircd by an imaging section; and a data processing secUon configured to supeEimposey changing an input image on the captured image in a display seeiion, and 30 wherein the program causes the data processing section lo display, on the display sceljon, the virtual im<^e generated by changing one of a relative position SP34G704WOnO 8/57 and a relative angle of tlie imaging section and the input image, which arc virtually set, in a time series. [0026] According lo a stvcnih aspeel of Ihe present disclosure^ there is provided a 5 progTJim for causing an information processing device lo execute inibrm^ion processing, wlierein the information proix^sing device includes an imaging section configured to perform image capturing, and a data processing fiCcUon conlignred lo generate moving image content of a virtual image, based on a captured image of Ihe lU imaging section^ and wherein (he program causes the data processing section to calculate paramelers, including a relative position parameler Rpos to be applied for determining the display position of (he virliial image and a iclative angle parameter Rrot Co be applied for determining Ihe display angle of the virtual image^ with respect lb to Irames constituting the moving image content of the virtual image upon processing of geiLemling the moving imagecontentof the virtual image, and slorc Ihe calculated parametei's in a storage section BR metadata corresponding to image Jramcs. [0027] Nole Ihal Ihe program according lo (he present disclosure is a program (hat 20 can be provided in a slorage mcdiumor comnmnication medium that is provided in a computer-readable form for an information proecssing device or a computer system that is capable of execnting various types of program eixle^ for c^iamplc Providing this soit of program in a computer-readahle form malces it possible Eo implemcnl Ihe processuig according to Ihe program in flie information prncessing device or the 25 eomputcr syslem. [0028] Tlie objeel, feahircs, and advanlages of the present disclosure will be made clear later by a more ^lelailwl cxplanalion Ihal is based on (he embodiments of die present disclosure and Ihe appended drawings. Furtheiuiore, the system iit this 3(1 speeiiication is not limited to being a eonHgurdCion (hal logically aggregates a plurality of devices, all of wliieh are contained wilhin Ihe same housing. SP34G70dWO0a 9/57 AdvanCaficous TiflecLs of Enveiition [0029] According to the conligiiraliort of one einbodiinent of the present disclosure, b a viitual image, of which n display position or a display angle is changed, can be supeiimposedly displayed on a captured earners image displayed on a display scelion with the passage of lime. Spceifically, t^ie conflguratian includes an inii^mg seclioa, a display scclioti configured to display a captured image of the imaging seclion, and a data processing 10 section configured to superimposedly display a vu^lual image on [he captured image displayed on tlie display scclion. The data processing section super!mjiosedly displays the moving image of the virtual image^ of which die display position and the display angle are changed in a time series, on the captured image. The dalu processing section acquires the parameters, that is, the relative posiliun. paranieler 15 Rpoa lo be applied for determiinng the display posilion oflhe virhial image and (he relative angle parameter Rrot to be applied for dctcnnuiing (he display angle of the vulual image, as the metadata coirespondmg to the frame of the virtual image content, applies the acquimd pardmelcrs, and generates and superimposedly displays tlie virtual image, of whicti Qjc display position or t^ic display angle is changed in a 20 time series. According (o such eonfigurations, the virtual image, of which Ihe display position or Ihe display angle ha.^; heen changed widi lite passage of tune, can be superimposedly displayed on the captured camera image. 25 BriefDcscriplion ufUfavvings [00301 [FIQ IJ I'lQ 1 is a diagram describing an overview of processing of the present disclosure. [FIQ 2J FIQ 2 is a diagram illustrating a flowchart describing a sequence of content ,10 generalionprocessiiig of the present disclo.'iure. [FIQ 3] HG 3 is a diagrairj illnstrating a flowchart describing a sequence of content SP!t46704WO00 10/57 display processing of the present disclosiiTC. [FIG 4] HG 4 is a diagram describing a display angle conlrol of a viilual image lo be superimpnsedly displayed. [FIQ 5J l'[Q 5 is a diagiTim describing a display angle control of a virtual image lo 5 be snperimposedly displayed. [FIG 6] FIQ 6 is a diagram desciibing a display angle connul of a virtual image to be suptjdmposedly displayed. [FIG 7] FIG 7 is a diagram desciibing a conliguialion example of an information processing device. ID [FTG SI FTG 8 is a diagram describing a eonfigUTalion of Ijmilalion upon generation ofvirtual image content. [l-IG 9J FIG 9 is a diagram describing a display example of description infonoalion and icons for lealizmg a conliguration of limitation upon generation ofvirtual image conlenl. 15 [FIG 10] FIG 10 is a diagram describing a spcciHc example of adaptive masking processing. i ■I [Frc IT] FTG I! is a diagram illnstratiTig a Iluwchajl deseiiblng a se(|uence of adaptive masking processing. [FIQ 12] FIG 12 is a digram describing a configuration example of an iiiformalion 20 processing device that performs adaptive masking processing. [FIQ 13] FIG 13 is a diagram describing a configuration example of an information processing device Ihal perfonns objeel clipping processing. DescripMoii of Fmbodimenls 2n [on:ii] Hereinafter, an information processing device, an information processing method, and a program according to die present disc[o.sure will be described in detail wilb reference to the appended drawings. Note that description will be provided in the following order 30 1. Overview of I*rocessing of l*resenl Disclosure 2. First Embodiment of Present Disclosure SP3dG704WO00 11/57 2-\. Processingof Generating Moving [mage Content of Virtual Image 2-2, Processing of .Superimposedly Displaying Moving Image Content of Virtual image on Captured linage 3. ConHgurafionExumpleoflnfoiuialion Processing Device 5 4. Modifications 4-1, Embodimoil in Which RcslriclEon un Posilion of Siibjccl is Sc( upon Gcnerdlion of Vitliial Lnagi; Contuol 4-2. Embodiment Hi at Performs Image Edge Processing upon Virtual Tnicigc Display ProeesRing 10 4-3. Emijodiment that Performs Processing on PEuralify of Virtual Images 4-4. Processing Example in Which Part of Processing is Performed in Network-connected Server 5. Sumniiuy of Conliguralion of l'ic:^cnl Disclosure [0032] IJj fl- Overview ofProccssing ofPrtscnl Disclosure] Firstj the overview of processing of the present disclosure will he described 1 with reference to FIG. 1. I'[Q I illustrates a display processing example of tlie following two AR images. 20 (A) Vulual Image Display Processing wiEhoul Angle Coulrol (B) Virtual Image Displiiy Processing wilh Angle Conlrol [0033] Both are diagrams in which a user captures a poster H, on which a person is printed, by using a jiorlahle terminal such as a smartphone with a camera hinclion 2^ and a c^tured camera im^e 15 is displayed on a dispby section of the porfable terminal with the passage of time (tO to t4). In both of (A) and (B), a virtual im<^, not a captured image, is superimposedly displayed on a captured image. [0034] JU l^c virtual image Ls^ for example^ Iwo-duncnsional image dalu of (he game reaE person as Ihc person prinlcd on lljc posler 11 and is^ for example^ image dala SP34G704WO00 12/57 slofcd in a storage section of apoitable terminal such as a stnarfphonc or image d^a provided from a server tliroi^h a netwoti. [00351 Tlie user captures the poshJ 11 ijn an oblique direction, [n the virtual 5 image display processing v/ithout angle conlro! illustrated in FTQ 1(A), in order to display an image sui;h Ihal (he person pops ont from the obliquely captured poster, a virtual image 21 to be superimposedly displayed is also an image of a person viewed in an obliqucdireetion^ just like the poster. [003 6J 10 After tliaE, moving image display processing, lo vvhieh the image viewed in Ihc oblique direction is applied, is performed so as lo displ^ an image such tliat the person approaches theuser side with. Ihe passage of time (tO to 1*1). However, in spite of approaching tlie camera, the virtual image 21 of time (t!i) or {t4) illustrated in FIG ICA) is still an image in which the person is viewed IJj from the oblique direction. Tlnis^ tlie virtual image 21 is recognized as un unnatural image as approaching the user. Tlierefore, it is apparent that Ihc virtual image 21 will he recognised as an image pasted separately from the eapUircd image. T\\h is caused by direct application of an image eapliiral Jn the same capturing direction as an initial image of lime (tOj. 20 f0037] In conhasi, vhtiial image display processing with an^e control of FIG: 1^) is a display processing example of a virtual image to which die processing of Ihe present disclosure, which is to he described helow, is applied. In Ihc cxamiile illustrated in l-IQ 1(B), as opposed lo FJG KA), a display 25 auf^c of a virtual image 31 is changed frT>m an ohhque dircelion to a front direction with the passage of time (11 —it4}. Such an angle control of (be virEiial image makes it possible to provide an image in which a person popping out Jrom a poster a|)proaclies a user more naturally. In the example itiushated in FJG 1{R), the jirocessing of changing the ,H0 direction or the like of the virtual image is performed on the image captured in die same capturing direction as Che initial image of time (tO) by performing the angle SP34G704WO00 13/57 control according to» for example, a dialanct of iJic virtual image. Due tn flics processing, a more natural virlual image tan be displayed. Hereinafter, Ibtpioecasingof the present disclosure, which realizes such an image display, will be de^ribed in delail. S [0038] In order to perform the im^e display as illustrated in FJQ 1(B), for example, a {riilio of) relative jiosition and angle of a camera is sel as metadata to Ibe rcspeelive frames constituting moving image content of the virtual image. The metadata is set as data corresponding to distance information of an 10 object to be displayed as the virtual image, for example, an object such a person, Incidentally, foi example, tbe distance information can use a depth map in which a subject distance from the camera is set by a pixel unit, 'f he depth map can be generated in advance by processing usiitg data captured by a compound eye camera oi by using a device that acquires distance information (depth) sepaialely 15 from (he camera. Also, even when there is no depth map, if an object subjected to distance ■r meawrement is a person, a distance can be deteimined liom a size of a face by using a face detection. Also, a user or an fipcralor may triannally sctflie distance information. 20 L0t)39j Tncidenlally, although described in the following embodiment in delail, various sellings can be performed as the mode of the angle conhcl processing of the person popping out thim the poster, for example, as illustrated in FIG 1(B). For example, after flie person displayed as the virlual image ap|iroaches the 25 camera or moves far away from the camera with Ibe passage of time, the contml can be perliirmed such fliat the angle between the virlual image and flie camera is set to be constant, suchasalroni {lireclion aU fltctinte. [0040] Incidentally, tlie device thai performs &\e image capture and display is not :m limited to the smarlphone and can be realized by various iufojmation processing devices, for example, a PC or a glass-type AR glass. SP3'l(i704WO00 14/57 10041J Ry applying (he pfoccssing of Ihe present disclosure, a more natural virtual image can be displayed in a configuration that superimposedly displays a virtual image sucli as a pei'son on a liase image snch as A captured CEimera imuge displayed 5 on Qie infommlioii processing device s^ich as the srtiarlplione or the AR glass. SpetiJjeally^ by changing a display position or a display angle willi the passage of lime, a more nalural virtual image can he displayed, that is, an image can be displayc^l such that a virtual image exists on a base image siieh as a captured image that is a snperimposition Uugei. IQ [0042] Incidentally, when the virtual image is, for CKumplc, a real person image, tlie permeability of pixel portions otiier than tlie person is maximized and image data widi o^ channel information of pixel imil, called peinieabiEily information or mask infonnatiun such as setting the permeability of a person region to 0, can be used as 15 the vnlual image to be su|]erimposeil. By superimposing content with such ct chaniLel Entormalion on tlic captured image, irnly the person region included in the virtual image is sujierimposefl on the captured image of the client and Hie captured image is directly displayed on the image region other Ihan tht person. [0043] 20 [2. First Embodiment of Present Disclosnrcl AUrstctnbodimcnlof the present disclosure will be described. The following two processings are required tor reali7ing the processing of superimpo.scdly displaying a virtual image on a captured camera image as described above with reference to FTG IQ^)- 25 (a) Processing of generating moving image conlcnl of virtual image (b) Processing of supcrjrnposcdly displaying moving image content of virtual image, which is genei'ated in the processing (a), on captured image Hertinafler, such processing will be sequentially described. [0044] 30 [2-1.1'rucessing of (ieneiating Moving Image Content of Virtual Image] FirsI, the sequence of the processing of generating the moving image SP346704WO0Q 15/57 conleul of Ihe virliial image for displaying the virtual image 31 iiliislralod in FIG 1(B) will bedeseribeU wilh reference to atlow illustrated in FiG. 2. Ineidcntilly, the processing of generating the moving image eonCent may be performed in, for example, the infomiaiion proeessing device sneli as the smarfphone, 5 or may be performed in other infbnnation processing device such as a PC, [00451 Incidentally, the processing illustraleil in iKc flow is performed under the control of a data processing seelion, Ibat is, a data processing section liaving a Ci'U or the like wilh a program cxecnEion tUnction, according to a pmgram slorcd In a 10 storage section of Ihe information processing device. Hereinafter, details of the processing of Qn: icspeelive steps will be scqiicnliatly described, [00461 (StepSiO!) Ii> First, die dala processing section of the information processing device allows the u^cr lo inpiil inodc setting information throngh au inpul scc(ion and detemiines howlo direct a "pop-oiit effect". A:; an example of a selling mode, Ihcrc are (he following two types. (1) Simple Mode 20 A simple mode is a mode dial delermincs an angle and a position formed with a virtual image subjecled lo cEimern snpcrimposiCion display simply according to a distance between a virtual image and a can^era (infijrmatioit processing device). (2) ConslanI Modt A constant mode is a mode that performs processing of setting an angle as fomied wilh acatnerii to be constant, for example, a front direclion, aflcr popping out a virtual image trom a poster or ti\c like diat is a captured object of a base image beiag a captured image, [0047J Incidentally, such modes may be set (o an cilher-or mode, Imt may also be 30 configured to be set to a mode between Ihe simple mode and the con,'itant mode. Also, it may be configured such that a speed alwliich tlie aitgle formed with SPa4G704WO00 16/57 die camera is dxangod is input as a parameter throiifih the input section by the user, and the dala proi;cssiQi^ section poifonns cotilfol according to the input parametcr. [0048] (Step Sf 02) 5 Next, in step .SI02, processing of acquiring a camera image by applying an imaging section (section) of die information processing device is pcrlonncd. Tbis is the processing of acquiring the cultured image made up of, fori^xample, a person to be displayed as a virtual image. For example, a pcisOQ in cupliircd in a sliidio with greenback equipmenl. 10 [0049] (Step SI 03) NcTct, in .step ,SI0.1, a distance to an object of the person being a subject to be displayed as avirtual image is calculated. Incidentally, as a niediod of acquiring subject distance iufoimation, various 15 methods can be applied. Specifically, for example, Ibt following inclhods can be applied. (a) A dcpib map is j-cncralctl by iicqiiitrng a distance with a combination of a ncar-inftarcd camera as in a stereo version using a compound eye camera or Kinect. An average depth in a region of a cli|iped object, which has been im^e-processed 2D vising a generated depdi maji, is.^tas a distance. (b) When a subject object to be displayed as a vir(ua] image is a person, a distance is estimated Itoni a size by using a face detection or a human body deteelion. (c) A distance is estimated fi'om a size of a region area of a clipped object that lias been imagc-prueessed. 25 [0050] (Step SI 04) Subscquentty, in .'vtep 8104, the information processing device sets a relative position and angle of an object, for example, a person, which is to be displayed as a virtual ijnage, as metadata (tribute infoimation) corresponding to the respective 30 frames con.^tituting moving image content of the virtual Image. [005 n SF3'*67n4WO00 17/57 HcrcinafEer, a dctaiLed algorithm of die metadata setting processing will be desciibi^, Incidentally, the metadata setting piocessing is diHeicnl according lo (he above-described mod&s. G Tliat is, die processing is different in the '^simplt mudc" and liic "conslanl mode". Hereinafter, meladala selling processing algorithms corresponding to Hie respective modes will be described. [U052] 10 (Regarding Metadata Setting Algoridim in Simple Mode) As described above, a simple mode is a mode thai delenuincs an. angle and a position of a virtual image subjected to camera supcrimpusilLOii display simply accordmg to a distance belween a virtual image ami a camera (information processing device). 15 [0053] ' First, (he following variables a^ defined. Dislancc al operation start point: Dstart Distance at processing time: Dproc Minimum distance at moving image sequence: Dmin 20 Tor example, the distance Dstart at operaliou slurl puint cotresponds lo the distance from the camera lo the poster in the example of HG l(R), 'Ilie distance Dproc at processing lime eorrcsponds to the distance from the camera to Ihe virtual image at lhclimc(tO to t4) in the example of FIG. 1(0). Tlic ininLinuni distance Dmin at moving im^e sequence coii'esponds to the 26 dislaiicc from the camera, to the virtual image at the time (t4) in the example of FlCi 1(B). [005-1] When Dslait >- Dproc >= Dmin >=' 0, a relative position pammeter Rpos and a relative angle parameter Krot are iin defined as follows. Rpos = (Dstart - Dproe)/(Dslarl - Dmin) RP34e704WOl)0 18/57 Rrot = (Dstart - Dproc)/(Dsrait - Dmin) [0055] As CiQi he seen Irom the above tbrnmla, relative posilion parameter: Rpos = 0 to 1 a relative angle parameter: Rrot = 0 to I These values can be aequiied. When (ho dislanee Uproe al processing time is equal to the distance Ds(ar| at operation startpoint, that is, for example, aj (he lime (ill) ofthe setting of FIQ l(R), Rpos ^ 0 10 Rrot = 0 [0056] Also, when the dislance Dproc at processing time is equal to tlic iiimjiniim dj&latiee Dinm al moving image sequence, that is» for example, at the time (t4) of (he selling ofFIGi(B), 35 Rpos = I 4 Rrot- I [0057] Ihal is, hodi tlie relative position parameter Rpos and the rclalive angle parameter Krot are parameters tlmt continuously change tlie value of 0 to 1 Irom Ihe 20 operation aUai point 1o the mirumum distaitce. Virtual image display processing, lo which these pammeters are applied, wilt be rtescnbcd in (he la((efparl. For examj^le, relative position parameter: Rpos = 0, 'A5 relative angle paramelei" Rrot-0 In such parameter setting, as in thcCime (10) of FIG 1(B), Ihc processing of supcrimposedly displaying Ihe same virtual image as the poster in performed. Thai is, (he position and the angle of the virtual object are set to be equal (o theposler of the captured image. 30 [0058] Also, SP3 46 704 WOOD 19/57 relative position paramelei: Rpos — 1» relative angle parameter: Kiol = 1 In sudi parameter selling, as in the time (t4) of FIG 1(B), the proeessingof locating the viilual iimigc lo a predetermined position close^l lo Ihc eaiRcra nitd G disjilayingthe virluylimuge iim front direction is perfomied. Virlual image display processing, to which these parameters are ^plied, will be described in Ihe latter part, [0059] Incidentally, the parameter definilionfoimiilii, lliat is, 10 relative position parameler: Rpos - (Dstart - D|iroc)/(Dstart - Dmm) relative angle puraineier: Riot = (Dstart - Dpro(i)/(Dstarl - Dmm) iTiese parami^ler definition fi:>rmulas are an example and various modifications ean be made. [0060J 15 For orample, hy introducing a dislanee Ihrcahold value Dlim at which the angle formed with the camera is equal lo 0, parameters may be set according lo the foUowing algorism. Relative posilion parameter: Rpos = (Dstart - Dproc)/(Dslarl - Dmin) If Dproc=TlimTlien Rrol -1.0 Rise Rmt ^ (Dstart - Dproc)/(Dslail - Dlim) Rndif 10066J [ncidentalty, Ihe idiitive position parameter Rpos may be set as in ihs simple mode. In thi; above algorithm, tlie virtual image can bi; displayed sue!) that "the 10 angle fonned wiUi the camera after popping out oneejs always zero". [0067] Tn die constant mode, the relative position parameter Rpos and the relative angle parameter Rrot are sequenltally set as metadata to the frames constituting ttte moving image content oflheviriual image in a time series. 15 For j^xample, when the moving image content is l^ODO liames made of tramesOlo 1,000, " ■r relative position parameter: Rpos = 0 to 1» relative angle [^nuueter: Riot = 0 to 1, These jjarameters are sequentially set to the frair*es constituting the moving 20 image eontenf of die virtual image in a lime series. |006R] (StepS105) Subsetjuently, ill step SI 05, die information processing device perfonns the processing of^oicrating moving image content of the virtual ima^c. a5 Specifieally, die processing of generating Ihe image in wliich the object to be supcrimpusedly displayed, for example, only theperson, iselipped is performed. For example, when the virtual image is a real person image, the permeability of pixet portions other than Ihe person is maximized and image data with a ehaniiE^E itiformation of pixel unit, called peinn;ability information or mask 30 information such as selling the permeability of a jierson region to 0, is generated. By superirtipcjsing content with such a channel inforaialioii on the captured image, SP346704WO00 22/57 only the person region included in the virtual image is superimposed on ihe captured image of the client and the captured image is directly displayed on the image re^on other than the person. [0069] G (Step SI06) SubsH]ut;itlly, Ihc information processing device records flie metadata generated instep SllM and Iheimage data generated in step SI05 in as^ciation with each olhen For example, llie processing of recording in media such as a hard disk or a flash memory is performed, 10 [0070] (Step SI07) Finally, instep SI 07, the finish detennimition is perfornied. When a next Irame is present in the moving image sequence being processed^ the pioeessiug T^IUCHS IO step S102 and the next frame is processes]. 15 When all Ihe proeesaing has been finished, the processing is ended. [0071] '^ [2-2. Processing of Sujierimposedly Displaying Moving Image Content of Virtual Image on Captured Image Next, die sequence of the processing of displaying Ihe moving image 20 content made up of the virtual image foi superimposition display, which is generated accoiding to the flow illustfated in FIG 2, will he desern>ed with reference to a flowcliait illustrated in FIG. 3. Incidentally, the proecssiiift of .■'iiperimpo.'Kdly displaying the moving image content of tiic virtual image on the capmred image is peifomied by» for example, die 25 information jirocessing device including flie imaging section (ciuncra) sueh as the sraEirtplione and the display section. Encidentally, in addition, in any configuration including die imaging section (camera) and Ihe displuy seelion, for ejfamplc, llie processing can be performed in various devices sueh as !i FC or an AR glass. [0072] 30 Incidi^Ially, the pioeessing ilhistrated in tlii^ flmv is performed nndcr the control of a data processing seelion, that i^ !i data processing section having a CFll SP346704WO00 TMSl or die like wiOi a program execudoii function, according to a program slorcd in a storage seelion of the information processing device, Hercinafler, dtlails of die processing of die respective steps will be sequentially described. 5 [0073] (StepS20l) First, the dafa proeessmg seetion of the information processing device performs image captures an image thron^i llie ima^^g section (camera) of die infonnation processing device and acquires a captured irniige. 10 For example, the captured image is an image of time (10) as illustratod in FTG 1(B). [00741 (Step S202) Sulu^qucnEly, in ^tcp S202, the infonnation processing device determines 15 whether information acquisition for calculating posilion iind angle infomialion of die camera, which is necessary at a later stage, is succcssflll- An example of the information for calculating Ihc position and angle information of Ihe camera is a marker included in the captured image. Tlic marker is a two-lication information can be variously sel. [00761 SP3dG70dWO00 24/57 For example, jusi like Ihu t^oognition of a Ihree-dimen.^ional position of the camera by extracting and Iratking y foyUirc point from an image cultured liy the camera of the infoitnalion procc^ing device, jiosilion identitication processing to which y simiiSUinixius lociilization nnd mapping (SLAM) technology is applied may 5 be perfonned. Incidentally, the proee,'^''ing to which the SLAM is applied will be dustribcxl in, for example, JP 20II-43419A, filed by the same appliCEint as Ihc ypplicanl of [he prc^nt application. [0077] Also, in addition, as the inforaiylion for eabiilatmg the position and angle ]0 inftirmation of the cameiu, infoimaEion of a sensor iiicluded in die information processing deviceiayy he applied. As sueh, in^lcp S!£02, it is determined ^edter the acquisition of the camera posilion and angie information, which is performed by Ihe infonnylion processing device, is siicecssfii]. IS When the information is not acquired, il in delcrmitiixl (hat subsequent AR image generation processing is impossible, ynd Ihc processing is ended, When the infoimaiion is yc^uircd, Ihc processing proceeds to step S20.1-1007 8 J (Slep S203) 20 SubsC{|Ucnlly, in step S203, tlie information processing device cylculales y cutrcnl position and orientation of the infomiylion piui;essing device (camery) by applying the iiiformation acquired in step S202. [00791 (Step S204) 25 Subsequeiilly in slep S204, Ihc infortnalion processing device performs the processing of decoding Ihe virluyl imyge conlent lo be su|)erimposedly displayed, [ooao] (SEcp S205) Subscqiienlly, in slep S2{t5, the information processing device performs Ihe ao processing of generating the virtual image to be displayed on Ihe display seeliun of die infonnalion processing device by using die camera posilion ynd angle SP34(;70iWO00 25/57 JtifoTmalion Jictpiircd in step S203 and the paiamclcrs set as the metadata corresponding lu each image frame of the virtual iniagu content. [0081] Tneidentally, the paiamefers se( as (he metadata nf the respective Ji'ames of 5 the virtual image content are tlie following parameter.i described above wilh reference to the iiow of FIQ 2. Relative position parameler: Rpos, Relative angle parameter: Rrot [O0R2J 10 Speeilltiilly, for example, wlien the paramc(ers set to the virtcal image frame are an image set as tbllows: Kpos = 0 Rrot^O ftie virtual im;^e is superimposed at the position and angle pasted in the IG marker of the captured image. Forcxample^ the ■virtual image is superimposedly displayed on the person of the poster 11 of time (tO) of FIG IR at the same posilion and angle. [uoa3] On l^ie other hand, for example^ when the parameters set to the virtual image ^0 frame are an image set as follows: Rpos = 1 Rrot-2 the virtual image is superimposed at a position closesl (o the information processing deviee side and in tlie front direction. as For example, the virtual image correspunding to the virtual image 31 illustrated in lime (13) ofTTG IR is sujieiimposedly displayed. [0084J Incidentally, as described above wilh reference to the flow of hlCi 2, the vnlnal image content is the moving image eonteiit ai^rt the following parameters are 30 scj)arately set to the respective frami:s eonstitnting the moving image conlent. Relative position parameler: Kpos = 0 to I, 26/57 Rclalivc iinglc paramela: Itioi - 0 to 1, [00R5] Tlie information processing device acquires Ihc Iwo or more piirarueters correKjionding to the frame for each image frame of the virtual image eonleDl^ and A calculates a dijiplay position and a display angle of (Ite virtual irnage of eaeh im^ge frame. Fuithennore, image conversion processing, to which ^ese parameters are applied, is performed on die virtual image of each image frame to gciieralc a eonveited virtual image with a display position and a display angle corresponding (o each image liarac. The converted virtual image generated by the conversion 10 processing is snpcrimposcdly dtspluyed. In the image conversion processing, for example, tlic foliowing proi;ussing is included with respect to each image Irame of flie virtual image content. (a) Rotation processing 15 (b) Scale change processing (zoom in, zoom out) (e) Parallel IrimslaEion processing [0087] In order to periorm tlie above processings (a) io (e) on each image Irame of the virtual image content, tlic information processing device calculates processing 30 parameters with respectto the respective processings (a) to (e) corresponding lo Ihe respective image Jiames. [008a] The procussii^ parameters are calculated in the following order. Firsts Ihc following Iwo model view matrixes are acquired. 3G (A) Initial state model view mulrix (Mmaikcr) Mode! view matrix of the virtual image of IUB same position and angle as die virtual image heing the initial state of the virtual image content^ for example^ the person of die poster i I of the time (tO) illustrated in FTG l(R): Mniarker (B) Final state model view matrix (Mdisplay) 30 Model view matrix of the virtual image that is the virtual image being the final state of the virtual image content, for example, the virtual image that is closest sps-isTo-iwoon 27/57 lo t!ic information processing device (tJimora) of the time (t3) illustrated inUCi 1(B) and is directed in the fiont direcliou, and Ihe virtual image normalized in a display: Mdisplay These are ueqiihcd from the stnr^e section or generated based on each 5 image. [0089J Incidenlally, Ihe niodel view matrix is a maliix that indiciites a position and an orienluliun of a model (virtual image coirespunding lo Ihe person in the example illushaled in FTG I) and is a transfoini malrix lliat transforms three-dimensional 10 coordinates of a reference coordmaie syslcm into a camera coordiimte system. Tncidentally, these model view matrixes m^ be calculated upon eonleni generation with reference to FTG 2 and stored in die storage section of Ihc information proecssing dcvicCj and may be calculated liom each image upon the processing of step S205 of the flow illustrated in FIQ 3. 15 [O090] Subsequently, die following paiamclers for transforming the matrix Mroaritcr into the matrix Mdisplay are caleiilatcd by using Ihe two model view matrixes Mmarker and Mdisplay. Rotation axis :Vrot 20 Rotation angle: Orot Seale: Vscalc Uianblution component: Vtranslate Thcst parameters are calculated. [0091] as Furtliermore, by using Ihe metadata set corresponding to each image frame, Ihatis, the parameters Relative position pyiumclcr: Rpos = 0 to I, Relative angle paTainctcr: Rrot = Oto I, the following offset infurniation is calcnialed as parameleis that convcil the ^a virtual image of each image frame, (I) Rotation angle offset: Riol x Orot spa4(i7odwooo 28/57 (2) Scale oHset: Rpos >; Vscale (1) Translation contjionent offset: Kpos s Vliimslale [0092] For exampb, tlie (1) rolalion. iUiglc oUscf Sjot x 0ro( is set to a vahic 5 illusdalcd ui a graph of FIG 4 according to the relative angle parameter Rrot = 0 to 1 set to each image frame of the virtual image content. FIG 4 illustrates tlie relative angle parameter Itiol = 0 to I stl lo cuth iiuagc frame of the virtual image coutent, on a hoiizoulal axis» and Ihc roEalion angle oJTsct Rrot X Orot calculated corresponding to each image, oil a veilital axia. 10 The relalive angle parameter Rrot = 0 to I of the horizontal axis is a paiamcler Ihal is set to Ibc respective image frames constituting the moving image, Ihal is, Ihc respective frames of the zeroth frame f(tO) to the last frame lT(n). [0093] At the zeroth frame i(tO), relative angle parameter Rrot = 0» and 15 rolation angle ofi^etRrot x Orot- 0. 'Iliat ia, in (his ease, Ibi; rolalioTi processing is not performed on the virtual image of the zerolh Irame i(lO) of Ihe virUial irnage content, and the image fnr supeiimposHion display is generated, [0094] 20 Also, at the nth fhime f(tn), relative angle parameter Rrot = l,and rolalion angle offset Rrot x Grot = Orot TTiat is, in this case, the image for superiniposilion display is generaled by perlorming the rotation processing on Ibc virLnal ima^e of Ibe nib frame f(ln) of Ibe virtual image content according to the rotation angle Orot for Irausfoiming Qie mitial 25 state model view Jnalrix (MjnaHiO-) in!o Ihc final state model view matrix (Mdisplay). [0095] For the respective frames from the zerotb frame fi;tO) to Ilie nth frame i{ta)y as ill Ibe grajili illustrated in FIG 4, Uie rolalion angle of[aet Rrol x Grot is sequentially changed from 0 to Orot 30 according to the relative angle parameter Riol = 0 to 1 siul as Ibe meladala corresponding to the respective images. sra^sTO-iwoao 29/57 'iJial is, Iho model of the respeetive virtual image iVames, for example^ tlia person of the viilUiil imu^e 31 illustrated in FIG KRX is displ^ed while being sequentially and gradually rotated, [00961 5 The example of the rotation angle offset Rrotx Grot is illustrated in FIG. 4. However, Ihc same proecssing is perihrmed on other scale offset Rpos x Vscale and other tratislatiori comjionent oflset Rpos x Vtronslate. According lo the following parameters, which are metadata lo whieh llie sealc change processing and the translation processing arc set to caeh linage, tliat is, 10 Relative position paranielcr: Rpos, Relalive angle parameter: Rrot the offset infoimulion such ua the seale ofH^et Rpos x Vscale and the translation compuuenl offset Rpos x Vtranslate in determined. [0097] Ifi The infoimation praeessing deviee calculates sueh offset information, that is, (1) Rotation angle offeet: Rratx Grot <2) Scale oflset: Rpos x Vscale (3) Translation component offset: Rpos y- Vtranslate by applying the relative position parameter Rpos = 0 to 1 and the lelalive ao angle parameter Rrot = 0 to I set as the metadata coirespondii^ lo each image. The information pmcessing device perfoims the image conversion of each image by applying the calculated offset information and generates (he converted virtual image to he superimposed ly displji^cd. liicidentaLly, for OEumple, in Che example of FIG. I{R), the moving image 2Ji content of Ihc virtual image to be eonverted can use the moving image content in which the person of Ihe poster is capt^ired in an obhque direction, like tlie initial image of lime ((0). fiy performing diflerent image conversions on the image capturee displayed on tlie display seelLon 167. A dab iiequisiMon secCion 164 acquires the moving image coTtlcrit stored in 15 (he moving image content database 141, acquires Ihe metadata stored in the metadata storage section 142, and ouipuls (Itc moving image content and the metadata to die supcrim|)osed virtual image genecation seelion 165. Ineidenluliy, when (he content is encoded and recorded, Ihi; dula ac(|Uisilion section 164 decodes Ihe conlcnl and outpAits the decoded content to llie superimposed an virtual image gcneralion section 165, [0113] The supcrimpo.'^ed virtual image generation section 165 injmis the moving image content of the vhtual imj^e and Ihe me(ada(a corresponding to the respective frames tlirou^ die data acquisidon seelion 164, 2S Furthermore, (he superimposed virtual image generation section 165 acquirtij the camera pusilion/orientation information from tiie recognition detenu ination section 163. 10114J 'fhe superimposed virtual image generation section 165 performs Ihc abovc-30 described processing of step ,S205 of the flow ofFlCi 3. SP34G704WO0D 34/57 Titat is, by using tliB meladala set lo each image framc^ Iliat is, the parameters Relative position paiamolcr: Rpos = 0 to I, Relaltve aQglc parameter: Rrol = 0 to I, a like following oJTsel infonnation is calculated as jiarameters that convert the virtual image of each imafic frame, (i) Rotation angle oDsct: Krol < Grot (2) ticale oflset: Rpos K Vaeale (3} Translalioti component oflset: Rpos x Vtranstale 10 By using-such offset infoimalion, Ihe image frame conversion proecssiug is perlbrmed to geneiale a eonveilcd virtual image to be displayed, 10115] A moving image superimpnsition action Ififi performs tlie processing of steps S206 and S207 of the flow illustrated in FlCt 3, that is, outputs an AR image by 15 superimposing the virtual image on the captured camera image thai is displayed on the display section 167. [01161 Incidenlally. the eonQguralion illuslralcd in 1?|G 7 is a diagram iElustrating die main coiiligumlion of die itdurmidLOii processing device, Resides the ao coLiHguralion illu^liaEed in Ihe thawing, the information processing device include, for exainpic, the control section having the CPU or the like thai conhols (he processing described with reference to FIGS. 2 and 3^ or the storage section thai slores Che program to he periomied by the control section. [0117] 25 In this way, the virtual image is supeilmpusijijly displayed on [he captured image that is displayedon tlie display seeticn of the client. [01! SJ [4. Modifications] Hereinahcr, modifiealions in which the configuration and the processing of 30 the infoimalioii processing device of the embodiment described above are partially modified will be described. SP34fi704WO00 35/57 T\K following three modili[;alions will be sequentially describid. (1) Embodiment in Which Reslriclion on Position of Kubjeci is Stl upon Generation of Virtual Iniagu CoEitont (2) EnibodimenI (bal Performs Image Edge I'roeessing upon Virtual Image 5 Display Processing (3)Embodiment that Performs Processing on Plurality of Virtual Images (4) I'roeessing Re supeumposedly displayed as the virtual image, can previously restrict the subject position wilh resptet to the camera and generate the virlual image content in which Ihe subject (object) to be superimposedly 25 displayed is eajituredwithin an allowable pusilion, [0121] ^jiecitlcally, regarding Ihe ciipSiiring angle of flie subject wiOi lespeel lo the camera, the following restrictions are pix>vided arid the virtual image conlenl for snpcrimposition 30 display is geneialed: a preset predetermmed vertical angle range, SP346704WO00 36/57 ji pr^el piedetennined horizontal angle, and a presel predetemiined distance range for llic subjuci disl-mce from the camera. [01221 a TTG K(a) is a diagram illustrating a setting example of a borjzonud angle restriction of acamerawitli respect lo asubjed. F[Q S(lj) is a dia^^m illualrdling a setting example of a verlical anglu reslriction of a camera with respect lo a subject aad a dislance restriction betwcc" a subjm;! and a camera. 10 [0123] In Ciiptiifing Ihe viitual image object in such a manner, Ihc expulsion of a more natural AR is realized due lo (he restriction on the capturing angle and the distance. Incidentally, data of infonualion on the vertical and horizontal angle 15 resh^itlious and the distance restriction is linked (o caeh. image. AhernativelK the dala of Ihe information may be linked to each frame of the moving image. [0124] For example, when considering the content .™ch as popping oul of the person from Ihe pusler, if the camera is shaken in a horizonlal direction, it looks 20 natural, Imt it looks umiiitiital aa if the person pops out in a direction Ilmt eaimot be if shaken in a veitical direction. Ceiierjily, when considering a I'eco^ition object, called a postei- stuck on a vertical wall, it is difficult to assume the great vertical shaking, floweverj in the sense of fail-safe, a veitical limit angle is designaleci by Ihe delinilion such as "viewing aiifite" in each moving image content. When the 2b poster is reeugnized in a state of exceeding the limit angle, the induction is perfottned to coiieclly Jit into the angle, and the superimpositiun of AR moving image content is sEarEcd from that. [0125] Also, the selling of Ihc allowable angle and distance ranges is flependent on 30 the content. For example, it is unnatural if a person pops out in a vertical dircetion, hut it is iiamral in the case of a bird. When a bird is capmred as avirtual object (o sraisTOdwooo 37/57 be superunposcilly displayed, the vertical resEiiction is relaxed. A lablc of Ihe allowahle angles or Ihe allowable distances according to types of eoiilcnt is stored in flie storage section of Iho inforinatinn processing device, it is desirable that, if necessary, a content producer selects a type of conleni to be 5 taplured, checks an allowable angle and Uislancc, and captures an image. 10126] Incidenlally, depending on Ihc allowahle angle infoimation or the allowable distance infonnalion lead from the table according to the eonlenC, the information pmcessing device displays information of whether Ibe current captured image is 10 within Ibe allowable range on the display section Ibat ilisplay^'i the captured imj^e, and displays an instruction icon or s dcsciiption for prompting an image capture wilhin the allowaljle range. A display example of the mslmefion icon and the desciiplion is illuslraled m FrG9, Ifi 10127] ilGS, 9(al)to (a3) illustrate display examples of captured images and icons when a laaximuni allowahle distance (Xinax) between a eaincra and a subject is set to 5 m. (al) to(a!i) illustrate the following states- 20 (al) A subject dislance (Xaetual) of a captured image is 10 m, and an instruction icon and instiiiclion information for bringing a eanieia close (o a subject are displayed. {a2) A biibjeet distance (Xactual) of a captured image is 7 m, and an instruction icon and instruction information for bringing a camera close to a subject 25 are displaycd. (a3) A subject distance (Xactual) of a capluTcd image is 5 m and is matched with a maximum allowable distance pCmax). Tims, it is in a state in whicli the (lisplayof an instiuction icon and inslrucliioii information has disappeared. [0128] 30 FIGS. 9(bl)io (b3) ilhislratcdi.iplay examples of captured images ami icons when a maximum allowable veitical angle (<&max) between a camera and a subject is SP34e704WO00 38/57 set to 15 dcfirccs. (bl) lo (b3) illustrate the following stales. (bl) A v^licid angle (1»aclual) of a camera with respect to a subject of a captured image is ^5 degrees, and an instruction icon and iitstmction information for fj directing the vertical angle of the camera in a more vertical direction. {b2) A vertical angle (Oactnal) of a camera with respect to a subject of a captured image is 25 degiees, and an mstiiiction icon and instruction information for directing Ihe vcilLcal angle of the camera in a moie vertical direclion. {a3) A vertical angle (actua]) of a camera with respect to s snhjeet of a ]0 captured image is 15 degrees and is matelied with a maximum allowable angle (0max). Thus, it is in a slale in which the display of an iiistruclion icon and instruction information has disappeared. 10I29J Hierefore, when capmring virtual image content, the instruction information 15 is displayed so thai Ihe usei can caplure content in Ihe allowable range. [0130] [4-2. Embodiment lliat Perfoims Image Edge Processing upon Virtual Image Display Processing] Next, the embodiment that performs the image edge processing upon Ihe 30 virtual image display processing will he described. The present embodiment is an embodiment that perfotms edge processing (mask processing) such as edge hluiTing of an edge portion of the viitual image, whieh is a boundary region of Ihe captured objecl, so as to perform superimposition display of Ihe virUial ijiiagc with no discomfort with respect lo Ihe captured image a& \ipon processing of displaying the virtual ifriage according lo Ihe iloweharl ofFiQ 3 dcseribcd above. The present configuration realizes the way to show an image with minimized discomfort even when a p^t of an object Eo be superimposed is deviated fiom an angle of view when capmringAR moving image content 30 10131] Upon superimposition display of a viitual image, for example, wheit a part HP34e704WOnn 39/57 tif an object to l>e superimposed is devinlod from an angle of view and the objtel lu be superimposed is alt fitted in an auglcof view of a display of s client (enuinai^ Ihu diseoinforl is reduced by proetssing such as edge btuning of a viilual image objeel. For example, RG 10(A) is a display example of a viiUial object when the 5 mask processing of iheprtHcnl embodiment is not applied. in an edge region 312 of a lower end of a virUial image (person) 3! 1 displayed on a eaplurcd image, a subject image is cul in a siraiglit line and thus looks imnalmal. |0132] 10 On the other hand, FIQ 10(B) is a display example of a virlual object when Ihe mask processing of the presenl embodiment is applied. Tlie mask applicalion processing such as bluiring is ptafonned on the edge region 322 of the lower end of the virtual image (person) 321 displayed on ihe capmred image, and unnaliiralncss is eliminated. Ift 10133] a FIG 11 illustrates a flowdiarl describing a processing sequence of adaplivc masking processing of the present embodiment. Tncidentally, Ihe processing illustrated in this flow is performed in the "processing of superimposing Ihe virtual image" of step S206 in the flow described with reference to FIQ 3. 20 \m3A] Ulie processing of the respective steps of (he ilow of FIG 11 will be described. (Slep S301) Firsts in step S30I, the infoimalion processing device performs virlual 115 image content frame end determinalion. Tn the processing frame of Uie virlual image content to be drawn^ il is determined whether Ihe virtual image object is fitted in the angle of view of [he content. Specifically, il is checked whether an alpha valut of 1 is continued in successive n pixels or more in caeli of vertical and horizontal ouler lines of the 30 viitual image eonleni lo be superimposed, n is an arbitrary Ihresliold value. It is determined wlielhtr objuet ends corresponding to the verlital and hori?A>ntal lines are SPa-lG 70-1 WOOD 40/57 CUE off. Incidenlally^ Ihis processing may be performed in advance in tlie generation side and realized by lecciving mtladatar [0135] (Step S302) 5 SiibscqitCiitly, in step S302, captui-ed camera image end non-^oUisiun dcictmiuation processing is performed. Tn step SZ30 L, when the object end of (he processing frame of the virdia! image is cut oft' it is detennmed whether lo make llic ciil-ofF portion be visible at a position at which the conlenlis actually superimposed on the camera image. in There is no problon if the cut-off portion in the original moving image source is couipklcly ciiE olf even when superimposed, hut discomfoil is great if the end portion appears when su|ierimposed. 'Ilrnt is, as illustrated in FIG 10(A), an unnatural straight-line edge apiiears. Ill such a case, the processing proceeds to step S303. 15 [0136] (StepS303) * In step ^303, image processing is performed on the edge of die vittual image eonlcnl. hi step S302, whcu JC is determined that a cut-off poiliou is captured in a 20 ciiptuied camera image, blurring processing is adapEively peifuinied on only the end of that direction. Tliat is, as illustrated in hlQ 10(B), blutring proeessing is performed on tlie edge region. [0137] as (Step S304) Finally, the iinal result, that is, the virtual [mage, in whicEi bluiring is peifomied on the edge of tlie virtual image existing in a huundaiy region of the virtual imaec and the captuhed image, is output as illustrated in FIQ 10(B). [0138] JO Tncidentally, in the case of applying Ihe processing, in a state io which the virtiiEil image end gets in and out hum the camera hame, a Irame that is not made of SP34G704WOOn 41/57 a Irame on which edge image processing is performed is alternaCely generated in H flappmg manner. 'Jlierefore, in the actual operalinn^ it is suitable thai a low-paas nUm is inserted and smoothing is perfoimed to some degree in a time series, [0139] 5 FIG 12 illustrates a configuration example of the iiifoEmation proce.spjing device that performs the adaptive masking proet:ssing according lo the flow illustrated in FIG 11, Tlie information processing device iJInstrated in FIG 12 diflbis Jrom the information piTicessing device described with reference to FIG 7 in that an adaptive 10 mask processing section 171 is added to the content display ccintrnl section 160, lie adaptive mask processing section 171 performs the processing deseribud with icferencu toFiCi II. Hie rentaiiiing eonliguration is the same as the eonliguiation described with reference loFIG 7. 15 [0140] [4-3. Embodimentthat Performs Processing on Plnrality of Virtual Images] Next, an embodiment that performs prneesflin]* on a plurality of virtual imagi^ will be described. 'I'he present embodiment is an embodiment in a case where there is a 20 pluruliEy of objects to be supmimpusediy displayed as a virtual image when moving image content of a virtual image is generated aceorduig to (lie Howehari of FIQ 2 described above. [OMl] En a case where a plurality of objects when a virtual image content is 25 captured, il may be assumed that a distance is different fnr each object. In such a case, in the present embodiment, clipping processing of the object is perfoTmed Avhen Ihc conti^t is gemmated, and each is treated as separate content and synthesized upon supeiimpositJon, so Lhat the plurality of contents is treated with no faiLui'e. [0142] 30 TncidentaMy, tlicpluralily of objects, that is, Ihe plurality of virtual objects to be supeiimpusedly tli^layed, la overlapped when viewed from the capturing camera, SP3d670'iWO00 42/57 it is treated as the sunte virlual object for convenience, Tlowever, when assuming that the piurahty of objecls, of which (he distance is greatly different, is superimposed, i1 is desirable to capture an image widiout overlapping. [0143] ft However, in a case where the distance is matched to some exleuf when Ihe pluralily of objecfs is sQ|ierimposed, tor example, in a ease where Iwo persons are set such that Iwo pcr.sons appear 1rom different directions, hug eadi other, and rcUim (o Oic poster, a total of three objects are treated us follows: Ihe first appearing person is an object A, the second appearing person is an object B, and the overlapping of the 10 first person and die second person is an object C. [0L44J ¥10. 13 illnstrafes a configuration example of the infonnaliun processing device Ihal performs (Ifc processing of the present embodinienh The information processing device illustrated m HG 13 differs Irom Ihe IG information pi-ocessing device described wilh reference to FTG 7 in that an object clipping section 18i is add^ lo Ihe conlenl j^eneration section 120, ^ As described above, when a pluralily of virtual objects exists in an image to he captured upon gcneralioji of Ihe virtual image content, the object clipping seclion IS! perfonns clipping processing on each virtual object SO Incidcnlally, \wUh respect to die clipped viitual objecls, aiX*a thai, st^par^lc mcladnita, tliat is, the aliove-described relative posilion parameter Rpos and relative angle parameter Rrot ace individually calculated and are stored in the storage section as Ehc sejiarate metadata. [01451 2fi Also, when liie virtual imiige is supcrimposedly displayed on the content djsjilay control section 160, a superimposed image generation section 165 performs the proi;essing of generating separate superimposed virtual images in which Ihe respetlive paraiiielers arc applied to tJie respective objecls, and superimposedly displays die generale^l victual images, 30 The remaining configuration is the same as the configuration described with reference lo FTG 7. Si'3^6704WO00 43/57 rOHfil [-1-4. Processing Exjiraplc in Wbicli Part of Processing is Perfoimed lu Network-coiunicliid Scivtr] The iibovc-dcscfibed embodiment has been described ^ the cmbodhnciit in 5 whjcli the whole processing is performed in flieinfomiationprocessiuj; device. However, it may he configured such Ihul a part of such processing is perfomtcd in a server tJiat is networfc-coniiected to (he infonnalion processing device. [0147] For example, it may be configured such that the information processing 10 device such as the smarlphoue puforms only the processing of capturing by the camera and ths seivei provides Ihc virtual image to he supeiimposedly displiiyj^. lliestirvcrpcr/brms the |irocessing described above wilh icrerencc loFJCt 2, and provides the information processing device (client) with Ihe virtual moving conlenl, of which the display position and the display angle have been controlled, 15 acoora.';ed on a captured image of die imaging liection, wherein the data processing section calculates parameieis, including a . G relative position parameter Rpos tobe applied fbrdeEenniniug die display position nf the virtual image and a lelalive angle parauielei l^rol (o be applii^ for delennining the display angle of die virlual image, with lespecl to Jrames cunsliluling the movu^ ini^B conleni of Ihe virEual image upon processing of gcnciuling Ibe moving image confenl of Ibc virliial imugc, and alotcs Ibc calcalHled patiimcLcrs in a storage ^c(ion 10 as mcladala corresponding lo image frames. (13) Tite infiirmation processing device according to (12), wherein the data processing section sets values of die relative-position parameter Rpos and die relative angle parameter Riot in a range of 0 lo 1 according 15 lo asubjecl dislancc of Ibc virlual image in each moving uuugc frame of Ibc moving image content. [0155] (M) An information jsroceasing device iiicluding: 21) an imaging section configured to performing image capturing; a display secliun conliguied lo display a caplurcd image of Ihe imaguig section; and a data processing section conCgiircd lo siipcrimposedly display a virtual image on (he caplnrcd image displays! on Ihe display scclion, 2& \v}icrciii file ilala |iroeessing section acquires, from a server, a moving jmagc of die virlual image, of which at least one of a display position and a display angle is changed in a time .^ries, and superimposedly displays the acquired viitual image nn the capmred image. [0!56] 30 h'urthei; a method of processing performed in die above apparatus and system and a piogram lo execute the pi'ocessiug are included in die configuration of Sl^34e704wonn 48/57 the present disclosme. [01571 Furihennoie, (be processing seqnence tliat is explained in Iht spccificadan can be implemunlcd by bard^vare, by software and by a ciuiiigiiralion that combines fi hardware and sofiwarc. Til a case where the processing is implcmciilcd by sottwarCj it is pussibic to in^stall in meinoiy within a compnCcr {Hat is inci^riKirated into dediciilcd hardware a program in whid] llie pro^jessing sequence is encoded and to exceuie the program. It is also possible lo install a program in a general-purpose ooinpiitcr that is capable of perfoiming various (ypes of processing and lo execute (he 10 progi-am. For example, (ht program can be installed in advance in a sloragc medium. In addition (o being installed in a computer from (he s(ordge medium, the progrjun can also be received through a network, such as a local area network (T,AN) or the Inlcmct, and ean be installed in a storage medium such as a hard disk or die like that is built into tlic computer. 15 [0158] Note that tlie various (ypes of processing that are destribed in this specification may not only be perfonned in a tempoi-a! sequence as has been described, buf may also bo performed in parallel or individually, in aecorilance with the processing eapaeily of (he device that performs the processing or as iiccricd- 20 FuTthennore, Ihe syslcm in this specitlcation is no( limilcd Eo being a coniiguration tba( logically aggregates a plurality of devices^ all of which are contained within the same housing. Industrial Applicability [01591 25 As described above, according to the conJiguiBtiou of one embodiment of the i^resent disclosure, (he virliial image, of which the display position or (he display angle is clianged wiih llic passage of time, can be superimposcdly displayed on the captured camera image displayed on the display secliun. Lipecilieally, Ihe confignration includes an unagtug seelion, a display section no coTiii};iired to display a captured image of Ihe imaging section, and a d^a processing section eouEgured to snperiin|Nisedly display a virtual image on the capmred image SP346704WO00 ^19/57 displayed on die displ^ section. 11ie dafa processing section supeiiniposedly displays the moving image of tiie virtual image, of which die display position and die displi^ angle ore changed in a (inm series, on Qie captured ima^. Thu data processing secrion acquires Ihe paranicleis, Qiiil is, Ihc relalive position paramclcr jj llpos to be applied for delemiining die display position of Qic virtual image and Ihc relative angle parameter Rrot to be applied for determining the display angle of the virtual image, as the mctadala corresponding to the frame of the virtual image conleni, applies the aequired parameters, and generates and superimposedty displays the virtLial image, of which the display position or die display angle is changed in a 10 timeseries. According to sucli coiiligurations, the virtual image, of which Qie display posidon or the display angle is chunged with the passage of time, can be superimposcdiy displayed on the captured eamcra image, Ifi Referenee Si^s List 20 2ft 3U roififJi i> II If poster 15 captured camera image 21 viituaE image 31 virtual image 120 content generation seetion 121 imagJi^g section (camera) 122 distance estimation seetion |2:i input section 124 relative position and angle calculation section 125 image proce.ssiiig section 140 storage section 141 metadata 142 moving image content database 160 cuntenl djsplay control section 161 ima^ng section (ciimcra) RP34e7a4WO00 50/57 162 image lecognition secEJnn 163 recognition determination section 164 data acquisition section 165 sii|icriniposcd viilual image generation sccQon 166 moving image superim position Ejection 167 di.splay section 171 adaptive mask processing section ISI object clipping scclion WE CLAIM:- An mfomiBfion proccssmg device coinpii^mg: an ae^iuEsition secCion eonliguTed to acqiint: a captured image cset, in a (iiiie series. Claim 2 The information pfoccsaing device according to claim 1, IS wherein tlie data processing section generates the virtual image, of winch at least i>ne of die display position and the display angle \R changed in a time series, by l^rforming image conversion pmeessing to which metadata set in association whh each frame of a moving image content ofthcvirliial image is applied. 2U Claim .'i The infonnytion processing device according to elami 2, wherein the data processing section acquires parameters, inclnding a relative posilion parameter Rpos to be applied for determining the display position of flie virtual image and a relative angle parameter Urot to Ije applied for determining the 25 display' angle of flte virtual image, as metadata set in association with each frame of fltc moving image content of the virfiml image, appliti the acquired parameters, generates the vitlual iinage> of which at least one of the di.splay position and Ihe display BSi^lc is changed in a time series, and siipcriinposediy display?: (he virtual image on Che captured image. 30 Claim ^ SP34fi7n4WO00 52/57 T)ic information pmcessing device according to claim 3, whi^^^in the data pioeessing section calculates a transfonu paramclur thai transforms a model view matrix Mmarker coirespondmg lo an initial image of Uie virtual image into a model view Diahix. Mdisplay corresponding to a last image of the G virtual image, multiplies the relative poailion parameter Rpos or (he relative angle parameter Rrot, which are metadata set corresponding to each virtual image trame» with respect to the caleuLali^d h^iD^form paracneter, calculates ofl^et infoimation to he applied to conversion proeessing of each virtual image frame, performs conversion processing of Ehe virltial image frame to which tlie calculated offset information is 10 apphcd, and generates the moving image of the virtual image of which at Icasl oflc of Ihe display po.iilion and the display angle is changed in a time swics. Claim 5 Tlie infbrmalionproctHsingdeviec according to claim 4, Ijj wherein each of the relative position jiarameter Rpos and the relative angle parameter Krut is a vahic that is sequentially changed in a range of U to 1 in each " moving image frame from the initial image lo the last image of Ihe virtual image. Claim 6 20 Tlie information processingdeviee according to cbim J, wherein the dala processing section performs hlurring process on the virtual image of a boundary portion between Ihe virtual image and the cultured image upon theprocessnigofsupcrimposcdly displaying the victual image on the captured image. 25 Claim 7 The information jiroc&ising device according to claim 1» wherein the data processing section pcrformH processing of generating moving image content of tlie vhtual image, calculates parameters, including a relative jsosition paameter Rpos lo be apphed for determining Ihe display position of 30 the virtual image and a relative angle pardmelcr Rrot to be applied tor determining the display angle of the virtual image, with respect to frames constituting the moving SP346704WO0n 53/57 imiigc coiilcnt of flic virtual image upon processing of generating the moving imago eonlonl of Ihevirfua] imago, and sloros Iho oaloiflalod parairiolers in a storage aoolioa a.s metadata correspnndmg to image frame.^, R Claim 8 The information processing device according lo claim 7, wlierein die data processing section sets values of tlie relative position parameter Rpos and llie relative angle paranieler Rrol in a range of 0 lo 1 according lo usubjecl distanee of Ihe virtual image in caeh moviirg imago Jrame of Ihe moving 10 iiniigo eonlcdl. Claim *) l1ie informatLon processing device according to claim 7, wlierein flie data processing section sets values, wliicli are sequentially 15 ehar^od in a range of 0 lo 1 wilk respuel lo each, movii^ image Jiamo irom an initial image lo a last image of Iho viriual image, as values of the relative poailion parEuneter Rpos and tiio rolalive angle paramolor Krol. Claim 10 an Tlie information processing device according to claim 9, wherein the data processuig section sets values of the relative position parameter Rpos and the relative angle parameter l^ot with respect to each moving image frame eonstiluling the virtual image in difTeront aspt:e1s according lo a mode sclthig. 25 Claim 11 The intermation processing device according fo claim 7, wherein the data processing section outputs restriction information indicating an angle and a distance of a preset allowable range to a display section 30 upon processing of generating moving image content of the virtual image, and generates moving image content of the virtual image including a virtual object within SP34fi704WO0U 5-1/57 the angle and thsi distance of t^ie iillowable mnge, Claim 12 An informiilion processing dcvico comprising: C an imaging seclion configured lo perloiming image capUiring; and a data processing seclion configured lo generate moving image content of a viriual image, based on a [;apluied image of Ihe imaging section, wherein the data processing section calculates parameters, including a relative position parameter Kpos lube applied for delL^miining the displiiy position of 10 tlie virtual imugi; and a relative angle parameter Itrot to be applied for delermining Ihe display angle of Ihe virtual image, \vi(h respect to frames conslitutiug tlie moving image <;ontent of the virtnai image upon processing of gencraling (he moving image eonlen( of the virtual image, and stores the calculated parameters in a storage section as metadata corresponding lo image frames, 15 Claim n "Die infoniiation processing device according to claim 12, wherein Ihe data processing section sets vah'es of the relative posiliou parameter Rpos and lire relative angle parameter Rrot ill a range of 0 to I aeeording 20 lo a subject distance of the virtual image in eacli moving image frame of tlie moving image content. Claim 14 An information processing device comprising: 2& an imaging section configured to performing image capmrin^ a display section configured to display a c^pEured im^e of the imaging section; and a data jirocessing section eonHgurHl to superimposediy display a virtual image on the captured image displayed on Ihe display section, 30 wherein the dala processing section ae<|uires, from a sccver, a moving image of Ihe virtual image, of which at least one of a display position cind a display angle is SPJ4G704WOnn 55/57 changed in a time scries^ afid supi^impu^dly displays the acquired virtual image on the captured image. Claim 15 5 An information processing method^ whieh is perfonned by an information proecasing deviee^ wherein the infoniiation processing device eomprises an aequisilion section eonfigiifed lo acquire a captured image captured by an imaging scelion, and a data processing section eonfiguTcd lo supeiimposedly display a virtual image generated 10 by changing an input iiiLagu on a display section, and wherein the data processing section displays^ on the display section, the virtual image generated by changing one of a relaEivc position and a relative angle of Ibc imaging section and the input image, which are vitUially sel, in a time series. I& Claim 16 An information proccssii^" meUiod, which is |ierforined by an informalion proces-sing device, wherein the information processing device comprises an imaging .'vcction configured to perform image capturing, ajid a data processing section configured to 2Q geueiate moving image content of a virtual hfiage, based on a captured im^e of the imagitig seelion, and wherein the data processing section calculates parameters, including a relative ijosiljon paramelei Kpos to be apphed for determining the display position of the virtual image and a relalive angle parameter Rrot to lie applied for deteimuiing as the display angle of the virtual image, wilh respect to fl'ames constituting tlie moving image content of the virtual Image upon processing of generating the moving image content of the viitual image, and stores the ealelllaled parameters in a storage .action as metadata corresponding fo image frames. 30 Claim 17 A program for causing an infi>rinatioii processing device to execute fiP34G70lWOOO 56/57 informaliun processing, ivhcreui the informalion processing device comprises: an acquisition section coEitigured to acquire a caplurixl image captured by Jin imaging section; anil Vt a data proce:^sing section configured t'} snperimposedty Jisplay a virtiiyl image generali^ by clianging an input image on flie capturetl image in a display section, and wherein the pruj^am causes the data pruccasing section to tlisplay, on the tliidpiay section, the virluaj image generated by ciianging one of H relative position 10 add a lelative angle of the imaging section wnd the lEipul image, which are virtnafty ' set, in a time stetieSr Claim IS A program tor causing an iLttojination pruccssiifg (Jevicc to execute IG iiifoimalion processing, wherein tlie infoTinatioii processing device comprises an imaging sccUon - conligured to perform ijoagc cajjturiug, and a data processing section conligured to ■ generate moving image content of a virtual image, based on a captured image of the inia^p, section, and 20 wherein Ibc progiam causes llie data processing section to calculate parameters, inclnding a relative pojjition parameter Rpos to be applied for determining the display posiEion of di« viitual image and a relative angle jsarameler Krotto be applied for determining Ihc display angle oflhevirtual image, with respect to frames eonstituting the moving image content of the virtual image upon :iG processing of generating the moving, image content of the virfuai image, and store the calculated pjirameteis in a sloraj^c section as metadata corresponding to image frames.