Sign In to Follow Application
View All Documents & Correspondence

An Apparatus For Determining A Spatial Output Multi Channel Audio Signal

Abstract: An apparatus (100) for determining a spatial output multi channel audio signal based on an input audio signal and an input parameter. The apparatus (100) comprises a decomposer (110) for decomposing the input audio signal based on the input parameter to obtain a first decomposed signal and a second decomposed signal different from each other. Furthermore, the apparatus (100) comprises a renderer (110) for rendering the first decomposed signal to obtain a first rendered signal having a first semantic property and for rendering the second decomposed signal to obtain a second rendered signal having a second semantic property being different from the first semantic property. The apparatus (100) comprises a processor (130) for processing the first rendered signal and the second rendered signal to obtain the spatial output multi-channel audio signal.

Get Free WhatsApp Updates!
Notices, Deadlines & Correspondence

Patent Information

Application #
Filing Date
07 February 2011
Publication Number
16/2011
Publication Type
INA
Invention Field
COMMUNICATION
Status
Email
Parent Application
Patent Number
Legal Status
Grant Date
2020-04-20
Renewal Date

Applicants

FRAUNHOFER-GESELLSCHAFT ZUR FOERDERUNG DER ANGEWANDTEN FORSCHUNG E.V.
HANSASTR. 27C, 80686 MUNICH, GERMANY

Inventors

1. SASCHA DISCH
TURNSTR. 7, 90763 FUERTH, GERMANY
2. VILLE PULKKI
YLAEPORTTI 4, A 7, 02210 ESPOO/FINLAND
3. MIKKO-VILLE LAITINEN
ALBERGANESPLANADI 2 A 26, 02600 ESPOO, FINLAND
4. CUMHUR ERKUT
LAUTTASAARENTIE 46 A 2, 00200 HELSINKI, FINLAND

Specification

The present invention is in the. field of audio processing, especially processing of spatial audic properties, Audic processing and/or coding has advanced in many ways. More and more dfmand is fenjsataj fey spatial audio applications, jn many applie.sti^ns. audis signal pjroeessing is utilised to deepfTfelati or reridef signal-'?* Such applications may, for example Gasry QUt msn^to^stsreo up-* mix, mono/stereo to multi'channel up-?mix, artificial reverberation, stereo widening or user interactive mixing/rendering. for certai classes of signals as e.g. noise-like signals as for instace applausee-like signals, conventional methods and systems suffer from either unsatisfactory perceptual quality or, if an object-orientated approach is used, high computational xomjplexity due to the number of auditory events to be modeled or processed, other examples of audio material, which is problematic, are generally ambience material like, for example, the noise that is emitted by a flock of birds, a sea shore, galloping horses, a division of marching soldiers, etc. Conventional concepts use, for example, parametric stereo or MPEG^surround eoding (M?|G * Wgyinq Picture,? Expert Group), Fig. 6 shows a typical application of a decorrelator in a mono'to^ste?©© up^mixer. Fig, 6 §how§ a mono input signal provided t© a deaprrelator 610, which provides a decorrelated input signal at its output. The original input signal is provided to an up"mix metric 620 together with the decorrelated signal, pependent on up-mi^ control parameters 630, a stereo output signal is rendered, The signal decorrelator 610 generates a dasorrelat^d signal D fed to the matrixing stage 620 along with the dry mono signal M. Inside the: mixing matrix 620, the sterso channels L (L *> l.eft ^terso channel) and R (R ^ Right stereo channel) are f©spied according %q a mixing matrix 8. The coefficients in the matrix 8 s$n be tix®4( signal dependent or controlled by a usff. Alternatively, the matrix can be, tscjnt railed by side information, transmitted ai$ng wi|h the down'-mix, containing a p§rarn«|fi«3 <499$y4p%4$i# pr§>£§i853, This is typically eJQhS in £*£«,$$£?; i§ ^pati^l m;dia ceding as, fQj? example, in fafajtgtrif §t{ft£jgQi gf, J, grt|ej:!a.|r/t, §,' van d§ ?#?« A; ISphlraujsh, g;, pgftyije^i, ^H:ic|h^§u|litiy Fafa.me|ris. Spatial JUjdio £qd.ing a£ hoy |itra|e$?' in K%§ l%$* Ganventisri, i^rlipi, Pr-epfinJj $078, M§y ?QQ4 and in M^|§ ^urj?^und, Qft J. Mt??e» $, JfJ^Siin^i #> Breeba^rt, ft: ai., »SJPE)Q Sucreund " the I^Q/J^ffS gtandjrd fe,r Efficient and §,5>ro^atifele ^uiti^ehannel &«Sli© C$dinfff in fr^Ggedingg of the 122nd Agjg Qonvention, Via.nni&i fcus^ia^ Mf¥ ^0Q7. $ typieai ftrueiurf of a pa^amjijEig gte&^s© stegpeter ii shown in fig. 7, Jn this $xajnpleu fchj dejgr;SSl.3t4©n process i^ pfrformjsd in 3 transform dp^in* whifh is indie.a/t«?c} by the analysis interbank 719? which trannferjns an inpi^t mono signal t© the, transform domain its, for sample, th§ frequency domain in terms of a. numJaer of frequency bands, In the frequency domain, the dscorrelator 720 generates the according decorrelated signal, which is to be up-mined in the up-mix matrix 730. The up-mix matrix 7 30 considers up- mix parameters, which are provided by the parameter modification box 740, which is provided with spatial input parameters and coupled to a parameter control stage 7 50. In the example shown in Fig. 7, the spatial parameters can be modified by a user or additional tools as, for example, postprocessing for binaural rendering/presentation, jFl this case, the up-mix parameter? can be merged with the pafametejrs from the binaural filters fee form the input parameter? fpr the upnriix matrix 730, fhe. measuring of the parameters may be carried py| lay |h<§ parameter mof.ilflection bipqk; 740, The pul.put ©f the vig-.mjL^ matrix 730 is then provided to a. synthesis filteflaf&nK 16Q« wWcH determine,!?! this stereo ©ytput algnal, Aj deseriiaed ab$ve, th§ ©yfcpt £/$ 9l ths mixing m%$i% H $%n fe# Qgftiputfts f£©*n t*h$ »f»89 iflpul ^signal M iN £H$ d^qfrfla^ad §ifrja,l $, fpc ff$jMty@3-£ ^fge^irig to in |^@ mining fptsi#? t?M ^Bjt^fc ¦©£ d^§g£f$iated siQund fstd, to |ht puiput agn &* £sQ&t*piif4 §ri %hf ^#§!i$ ©I kf-anfwniist^d. p^fa^ftgff© a.$, |©r twpiplfi ?@$ ll§§ * Jnterfihjnoel SQrjr^latipn) and/pr miatd q? ug$jp»djf$.n$g| settings, Another conventional appreifch is established Jay th§ temporal permutation method, A dgdiemted prppagai on deeosselatlon of applause-lilse signals §an b§ found, for example, in Gerard Hotho, Steven van d« ^ar, Jeroen Breeb^aart, ^Multichannel Coding Qi Applause Signals," in EUHA3IE Journal on Advances in Signal Processing, Vol. l, Art. 10, 2008. Here, a monophpnic audio signal is segmented into overlapping time segments, which are temporally permuted pseudo randomly within a "super""feloc)c to form the decorrelated output channels. The permutations are mutually independent for a number n output channels. Another approach is the alternating channel swap of original and delayed copy in order to obtain a decorrelated signal, cf. German patent application 102007018032.4-55. In some conventional conceptual object ^orientat-pd systems, e.g, in Wagner, Andreas; Walth^f/ Andsfag/ Melehg»i-)r, Frank; Sferauft, Michael,* ^6$ners|tipn of Highly immers.iy® AtyRQapbtffg for Wav« Fi&ld Synthesis f^pr^duetiph" at li6a' International B$fj Contention, lejelin, $004 < it if described haw to or©ate> an inp^rgivB iisine. out of m#ny pjajeqts §§ for ess$mpi«§ sin^l® slifSU fey &ppU§$fc£©n $f © mv?. fifld synthesis, ¥§t §noth$r §pps§ii§h is %h$ s^^ii^d, »di ration 31 audi© QQiinq" (|ir-&q * $&?fa$49$i^ h$ty$ Sp^i^f), ^ish ig I mtfchodi ?©£ 8p»^i*J, f@un^ f$pifta$j|ti§n* fi£$4«afei* fm dlffir^nt ?Qunc| $9£!¥g$viet£e$ $p|§?n§f stf. lulMtei, ViUfij '¦Spatiii iSp.und $fp£0di}§t4§n WiSH RiEgotionjl $udio Ceding" in J, ftudio ©ng, poa?, Vol, $$< No* 6, 3QQ^, in the anaiyfig past? fchf ^Iffujsnjsii $0 dijffptjgn pf jarr-i^i $£¦ $©und are ejtimatfd in a singly i^gtipn deg^sM^ o*r fci$<9 and frequency. Ib the. gynth§§i§ park* miorpphon© a.|.|nsip are first divided into npn^diffujgf and diffuse pa.rtg and are then reproduced using difff^int strategies, Conventional approaches have a number of disadvantages. For example, guided or unguided up^mix of audio signals haying content such as applauds may require a strong decorrelation. Consequently, on the one hand, strong decorrelation is needed to restore the ambience sensation of being, for example, in a concert hail. On the other hand, suitable decorrelation filters as, for example, all-? pass filters, degrade a reproduction of quality of transient events, like a single handclap by introducing temporal smearing effects such as pre- and post-echoes and filter ringing. Moreover, spatial panning of single clap events has to be done on a rather fine time grid, while ambience decorrelation should be quasi-stationary over time. State of the art systems according to J. Breebaart, S. van de Par, A. Kohlrausch, E. Schuijers, "High-Quality Parametric Spatial Audio Coding at Low Bitrates" in AES 116th Convention, Berlin, Preprint 6072, May 2.004 and J. Herre, K. Kjorling, J. Breebaart, et. al., "MPEG Surround - the ISO/MPEG Standard for Efficient and Compatible Multi- Channel Audio Coding" in Proceedings of the 122nd AES Convention, Vienna, Austria, May 2007 compromise temporal resolution vs. ambience stability and transient quality degradation vs. ambience decorrelation. A system utilizing the temporal permutation method, for example, will exhibit perceivable degradation of the output sound due to a certain repetitive quality in the output audio signal. This is because of the fact that one and the same segment of the input signal appears unaltered in every output channel, though at a different point in time. Furthermore, to avoid increased applause density, some original channels have to be dropped in the up-tnix and, thus, some important auditory event might be missed in the resulting up-mix. In object-orientated systems, typically such sound events are spatialized as a large group of point-like sources, which leads to a computationally complex implementation. It is the object of the present invention to provide an improved concept for spatial audio processing. This object is achieved by an apparatus according to claim 1 and a method according to claim 16. It is a finding of the present invention that an audio signal can be decomposed in several components to which a spatial rendering, for example, in terms of a decorrelation or in terms of an amplitude-panning approach, can be adapted. In other words, the present invention is based on the finding that, for example, in a scenario with multiple audio sources, foreground and background sources; can be distinguished and rendered or decorrelated differently. Generally different spatial depths and/or extents of audio objects can be distinguished. One of the key points of the present inventior. is the decomposition of signals, like the sound originating from an applauding audience, a flock of birds, a sea shore, galloping horses, a division of marching soldiers, etc. into a foreground and a background part, whe::eby the foreground part contains single auditory events originated from, for example, nearby sources and the background part holds the ambience of the perceptually-fused far-off events. Prior to final mixing, these two signal parts are processed separately, for example, in order to synthesize the correlation, render a scene, etc. Embodiments are not bound to distinguish only foreground and background parts of the signal, they may distinguish multiple different audio parts, which all may be rendered or decorrelated differently. In general, audio signals may be decomposed into n different semantic parts by embodiments, which are processed separately. The decomposition/separate processing of different semantic components may be accomplished in the time and/or in the frequency domain by embodiments. Embodiments may provide the advantage of superior perceptual quality of the rendered sound at moderate computational cost. Embodiments therewith provide a novel decorrelation/rendering method that offers high perceptual quality at moderate costs, especially for applause-like critical audio material or other similar ambience material like, for example, the noise that is emitted by a flock of birds, a sea shore, galloping horses, a division of marching soldiers, etc. Embodiments of the present invention will be detailed with the help of the accompanying Figs., in which Fig. la shows an embodiment of an apparatus for determining a spatial audio multi-channel audio signal; Fig. lb shows a block diagram of another embodiment; Fig. 2 shows an embodiment illustrating a multiplicity of decomposed signals; Fig. 3 illustrates an embodiment with a foreground and a background semantic decomposition; Fig. 4 illustrates an example of a transient separation method for obtaining a background signal component; Fig. 5 illustrates a synthesis of sound sources having spatially a large extent; Fig. 6 illustrates one state of the art application of a decorrelator in time domain in a mono-to-stereo up-mixer; and Fig. 7 shows another state of the art application of a decorrelator in frequency domain in a monc-to- ster^o up-mixer scenario. Fig, 1 shows a.n embodiment of an S]pjP§ratus 100 fp? determining a spatial output muiti-ehanneX audio signal based on an input audio signal, In spina embodiments the. apparatus can be. adapted, for further baling the spatial output my^ti-shanne^ audio fign^i on an input parameter, The input parameter may be generated; issflly p.r grovidesd with the input sudio signal, for example., as? side information. In the embodiment depicted in Fig, X, the apparatus 1QQ comprises a decomposer 1X0 for decomposing the input audio signal to obtain a first decomposed signal having a first semantic property and a second decomposed signal having a second semantic property being different from the first semantic? property. The apparatus 100 further comprises a tenderer 120 for rendering the first decomposed signal using a first rendering characteristic to obtain a first rendered signal having the first semantic property and for rendering the second decomposed signal using a second rendering characteristic to obtain a second rendered signal having the. second semantic property, A semantic property may correspond to a spatial property, as close or far, focused or wide* and/or $ dynamic; property as e.g. whether a signal is tonal, stationary or transient and/or a dominance property as e.g. whether the signal is foreground or background, a, measure thereof respectively, Moreover, in the. embodiment, the apparatus 100 comprises a processor 130 for processing the first rendered signal and the second rendered signal to obtain the spatial output multi-channel audio signal. In other words, the decomposer X10 is adapted for decomposing the input audio signal, in some embodiment § based on the input parameter. The decomposition of the Input audio signal is adapted to semantic , e,g, spatial, properties of different parts e£ the input audio signal, Moreover, rendering carried out by the tenderer 120 according to the first and second rendering characteristics i can also be adapted to the spatial properties, which allows, for example in a scenario where the first decomposed signal corresponds to a background audio signal and the second decomposed signal corresponds to a foreground audio signal, different r^ndaring or decorrelators may be applied, the. other way around respectively. In the following the term "foreground'' i§ understood to refer to an audio object being dpmiricint in a,n audio environment, such that a potential listener would notice a foreground-audio object. A foreground audio object or source may be distinguished or differentiated from a background audio object or source? A background audio object or source, rn^y not be. noticeable by a potential listener in an audio environment as teeing less dominant than a foreground audio object or source, jn embodiments; foreground audio objects or sources may be, but ire not limited %ot a point^lik^ §udio fou^qe, whist background audio objects or sour-ces may $Q££99PQ&0 to spatijiHy widfr audio objects or spurqfs, In other words f in embodiments the. first rendering • characteristic can be. based m Q£ mjtghed to the first semantic property and the geccmd pandering cha.ract§ri§tie can be based on o£ matched to the. second sjmintip property, in one embodiment the first semantic property and the first rendering characteristic correspond fee $ foreground audio source or object and the. £§ndere,r i|Q pan be adapted to apply amplitude panning to the. £|r-$t deefm^ofed signal, The renderer 120 may th§n be further- idajted for providing as the fir-st rendered sigMi two amplitude panned versions of the first decomposed signal, In this embodimentf tha second semantic property and the second rendering characteristic correspond %Q a background audio source, qr object, a. plurality thereof respectively, and the tenderer 120 can be adapted to apply a. decoJ-relation to the second d«com.pos©d signal and provide as second rendered signal the second decomposed signal and the decorrelated version thereof. in embodiments, the renderer 12Q can be further adapted for rendering the first decomposed signal such that the first rendering characteristic does not have a delay introducing characteristic, In other words, there may be no decorrelation of the first decomposed signal. In another embodiment, the first rendering characteristic may have a delay introducing characteristic having a first delay amount and the second rendering characteristic may have a second delay amount, the second delay amount being greater than the first delay amount. In other words in this embodiment, both the first decomposed signal and the second decomposed signal may be decorrelated, however, the level of decorrelation may scale with amount of delay introduced to the respective decorrelated versions of the decomposed signals. The decorrelation may therefore be stronger for the second decomposed signal than for the first decomposed signal. In embodiments, the first decomposed signal and the second decomposed signal may overlap and/or may be time synchronous. In other words, signal processing may be carried out block-wise, where one block of input audio signal samples may be sub-divided by the decomposer 110 in a number of blocks of decomposed signals. In embodiments, the number of decomposed signals may at least partly overlap in the time domain, i.e. they may represent overlapping time domain samples. In other words, the decomposed signals may correspond to parts of the input audio signal, which overlap, i.e. which represent at least partly simultaneous audio signals. In embodiments the first and second decomposed signals may represent filtered or transformed versions of an original input signal. For example, they may represent signal parts being extracted from a composed spatial signal corresponding for example to a close sound source or a more distant sound source. In other embodiments they may correspond to transient and stationary signal components, etc. In embodiments, the renderer 3.20 ^ay be sub-divided in 3 first renderer and a second renderer, where t u? first renderer can be adapted for rendering the first decomposed signal and the second renderer can be adapted for rendering the second decomposed signal. In embodiments, the renderer IZQ may be implemented in software, for example;, as a program stored in a, memory to be run on a processor or a digital signal processor which, in turn, is adapted for rendering the decomposed signals sequentially. The renderer 120 can be adapted for deeorrelating the first decomposed signal to obtain a £irs;t deeorreiatsd signal and/or for decstrreiating the second decomposed signal to obtain a second deqorseiated. s^gjial. In other wards, the? renderer 120 may be adaptfd for deserrelating both decomposed signals, however, using 4%£f®sm% decprr#.lation or rendering characteristics, in tmbodiments, the renderec 12Q m#y be adapted for applying amplitude panning to either. one. of tH$ first o? second dsee-mposed signals instead or in addition to dscorrelation^ The renderer 1£0 may be adsptsd for rendering the JHrst and second rendered signals ea,eh having a,s many components as channels in the spatial output multi*sha.nnel au.dio signal and the processor 130 may be adapted, for combining the components of the first and fepond rendered signals to obtain the spatial output muiti»«e.hanhei §udio signal, in other embodiments the rendered 120 can be, adapted for rendering the first and second r-snder-edi signals ^a<:h having less components than the spatial output multi-channel audio signal and wherein the processor 130 can be adapted for up*- mixing the components $£ the first and second rendered signals to obtain the spatial output multi^ehannel audio signal. Fig, lb shows another embodiment of an apparatus 100 { comprising similar components as were introduced with the help of Fig, la. However, Fig. lb shows an embodiment having more details. Fig. lb shows a decomposer liQ receiving the input audio signal and optionally the input parameter. As can be seen from Fig. lb, the decomposer, is adapted for providing a first decomposed signal and a second decomposed signal to a renderer 120, which is indicated by the dashed lines. |n the embodiment shown in Fig. lb, it is assumed that the first decomposed signal corresponds to a point^like aiadio source as. th§ first semantic property and that thf renderer 3,20 is adapted for applying amplitudes-panning as the first rendering characteristic to the first decomposed signal. In embodiments the first and second decomposed signals ar§ exchangeablet irt- in other tmbodimepts amplitudy-rpanning may be applied to the second decomposed signal. In the emtoodimtnt depicted in fig, 1%, |h§ rgmejersr 120 shews? in the signal pith of the fi^pt dfeompo^tP signal, tp salable am|>llfi§E§ 11% and \%%^ whigh are djd*iPted for iimpiilying %m sspi??. p$ thjs fir.fiti g9$Q$pqs*d. signal differently, The dtfJffSfBfc f^plili??ftiQCi faptjQjcg u$£$ may, in evmteod.im^nt$, b§ d^tfrminfcd. frgm, %$Q i^puS ^ftraii'etfr, in othts fmbadimenrtp? £H?y &§y N djt>prmin8d frost the input aujaio signal* it may fei p!3p;r; 130, whgEe only the first dfoompo^sd iignal #nd. a panning fasten may b© provided by the renderer 130, h§ oan be. seen in Fig, lij, the R£8S?9«g9? 130 oan b«i adapted for prQ.c rendered background dec©jnpps$d part to the up^mi* 330, cprresppnding to the prpegespr- 1301 Th§ foreground decomposed signal part is? provided tp an ^mpiitud^ pinning D2 stage 34Q, which cprreaponds tp the renderer 120, Locally-generated lpw-*pass nplse 350 is also provided to the amplitude panning stage 340, which can then provide the fpregrpund-deqomppsed signal in an amplitudes-panned configuration to the up-^min 330, The amplitude panning D2 stage 340 may determine its output by providing a scaling factor k for an amplitude selection between two of a stereo set of audio channels, The sealing factor k may be based on the lowpass noise. As can be seen from Fig. 3, there is only one ar:rcv> between the amplitude panning 340 and the up~mix 330. This one arrow may as well represent amplitude-panned signals, i.e. in case of stereo up-mix, already the left and the right channel. As can be seen from Fig. 3, the up-mix 330 corresponding to the processor 130 is then adapted to process or combine the background and foreground decomposed signals to derive the stereo output. Other embodiments may use native processing in order to derive background and foreground decomposed signals or input parameters for decomposition. The decomposer 110 may be adapted for determining the first decomposed signal and/or the second decomposed signal based on a transient separation method. In other words, the decomposer 110 can be adapted for determining the first or second decomposed signal based on a separation method and the ether decomposed signal based on the difference between the first determined decomposed signal and the input audio signal. In other embodiments, the first or second decomposed signal may be determined based on the transient separation method and the other decomposed signal may be based on the difference between the first or second decomposed signal and the input audio signal. The decomposer 110 and/or the renderer 120 and/or the processor 130 may comprise a DirAC monosynth stage and/or a DirAC synthesis stage and/or a DirAC merging stage. In embodiments the decomposer 110 can be adapted for decomposing the input audio signal, the renderer 120 can be adapted for rendering the first and/or second decomposed signals, and/or the processor 130 can be adapted for processing the first and/or second rendered signals in terms of different frequency bands. Embodiments may use the following approximation for applause-like signals. While the foreground components can be obtained by transient detection or separation methods, cf. Pulkki, Ville; "Spatial Sound Reproduction with Directional Audio Coding" in J. Audio Eng. Soc, Vol. 55, No. 6, 2007, the background component may be given by the residual signal. Fig. 4 depicts an example where a suitable method to obtain a background component x' (n) of, for example, an applause-like signal x(n) to implement the semantic decomposition 310 in Fig. 3, i.e. an embodiment of the decomposer 120. Fig. 4 shows a time-discrete input signal x(n), which is input to a DFT 410 (DFT = Discrete Fourier Transform) . The output of the DFT block 410 is provided to a block for smoothing the spectrum 420 and to a spectral whitening block 430 for spectral whitening on the basis of the output of the DFT 410 and the output of the smooth spectrum stage 430. The output of the spectral whitening stage 43p i§ %hm provided to a spegtrai pea,k*-pic;k;ing stage 440, whi^h separates the spectrum and provide twp outputs, i.e. a noise and transient residual signal and a, tonal signal? The? noise and transient residual signal is provided to an lpc filter 450 (LPq * Linear. Prediction gotfinj) of which tjis residual noise signal ia provided to the mining g-tage 4S.Q together with the tonal signal a? output of the speetraJ. peak*-picking stage 440. The output of the mixincj atag® 46Q is then provided to a spectral Shaping itage 4 70, which shapes the spectrum on the kajis of the gmiptheei spectrum provided b^y the smoothed $pe@trum st^gf 410, $he output of the spectral shaping stage 470 is, then provided to the synthesis filter 4f 0, i,e, an inverse discrete Fpuritr transform in order to obtain x/ (n) representing the background component. The foreground gpmp^nent san then b§ derived as the differenoe between the input signal and the. output signal, i.e. $s x(n)^x<(n5- Embodiments of the present invention may be operated, in a virtual reality applications aj, lor example, 3D gaming. In such applications., th© synthesis of sound, isource.3 with a large spatial extent may fee complicated and c.omplgx- when £ased on conventional concepts, iueh soureef might, for example^ tee a seashore, a bird fiosH? galloping horsey, the division of marching soldiers, or an applauding audience, Typically, such sound events are spatialiged as a large group of point-like sources, ¦ which leads to Gomputationally^complex implementations, cf. Wagner, Andreas; Walther, Andreas; Melchoir, Frank; Straufi, Michael; "Generation of Highly Immersive Atmospheres for Wave Field Synthesis Reproduction" at 116tri International EA8 Convention, Berlin, 2004. Embodiments may carry out a method, which performs the, synthesis of tb£ extent of sound s^urcts plausibly but, at the same time, having a lower stryeturil gnd computational complexity, Embodiments may ba bas^d ©n §ir&e (PirAC * pirsctionsl Audio Coding), sf, Fulkki, villi; ''Spatial Pound Reproduction wilH Directiona.,1. $u4iQ Coding^ in J, Au^ie |n<3f. See,, Vgl, §§» Np. §, 2001 ? In othtr w^cta, in emteodimfnfes, the decomposer 13LQ and/®r the refndar§r 120 and^cir UN processor 130 m§y I?© adapted for pr^c^sing pirAS signals, In otht* worlds, tbn d8cs^mpos#r lip may corapsige. $irAC monosynth stages, the. i^ndfrer i?@ may PQmprise a PirAC synthesis, st$ge and/or th§ processor may comprise a Bi^AC merging stags, Smfes$§iments may be to*0$g on PirAQ proejgsinf, if or sample e using, only two synth^s^s gtrue?tuj?fs? fpj gasamplt, ont for foreground found SQuree.8 and ©,nt lo* baelsgrQund sound spiiPP^s, fhe, f@r$gir©unsl sound, mjy be applied, to a singly DirAS stream with controlled 4i£9f$Aor^| data, resulting in the perception of nearby poin^'"!^© sources. TN bsieskground, sound may also be reproduced by using a s.ingls direct stream with di£ferently-?qontr@lled directional data, which leads to the perception of spatially«apread sound objects, The two DirAC streams may then b® merged and decoded for arbitrary loudspeaker set-up or for headphones, for example. Fig, 5 illustrates a synthesis of sound sources having a spatially^large extent, Fig. 5 shows an upper monosynth block 610, which create? a mono-PirAC stream leading to a perception of a nearby point-like sound source, such as the nearest clappers of an audience. The lower monosynth block 620 is used to create a. monQ?-C!ir&C stream leading to the perception of np«»tialiyspr«^ sound, which ig, for example,, suitable, to gtner^t^ b^gkfr^ynd spund as the, plopping sound farojn the a:y$ie.n.gs., The. outputs of the. two Pir&Q monofiyhlh teloe.ks 610 %%$ $%Q are $h@n. rnerfeisi in the Dir^? merge stap 130. Fig, § fthews that only two PipM; $yn:tha§ig bloekjs flQ %ni S2Q a?s u§§d in this, ^rflbodirrifnt, Qm of them i§ used to $r§a.t§ the $m®$ $va^t§? vhj^h arg in the. fgjje.gr^und, such as c|§§£sit ®S nearby airds or piogejt $r nearby persons in a.n applauding a^ijgnc?? a^d th§ o|h©,r jjene^stej a fo^ckgrsnpd 9©un^? thi spntinusus fed figgk s§unjl, atg, The. foreground. fQURd. i§ 69n¥£jr$$£ into § m$nQ."Qirfiq str^m. With Qir^gw^qnjaynth feliPgk f|8 in a W|V £h*t the ajslmutj-i djta i§ kept constant wi|h ffsgu^ney,, however, chsnQi^ ggtEigggily or Go^|g^|i«»d fey §n ^tajfigj. pifQgpss in t\$m, Th§ d^ffyaeness p§i?aH\e.fc§r \|i is |$$ to p, i»§. re^ra anting a point^liHf sourfi. The. au4iQ input; to £h# telpck 618 i.a ap§um§d to, be temporarily njn^ovjrla^ping §oyndj, such as distinct bird calls or ha^d slips * which g©nsra.t$ the pf.ree.pt ion of nearby sound source^, such as bircls ar Clapping persons. The spatial extent of the foreground sound events is controlled by adjusting the 6 and 9rangej?Qregre>uncu which means that individual sound events will be perceived in 0i9range_forepoun0 directions, however, a single event may be perceived p©int-4ik§. In other words, point-like sound sources are generated where the possible positions of the point are limited to the. range "i"range_f oreground < The background block 620 takes as input audio stream, a signal, which contains all other sound events not present in the foreground audio stream, which is intended to include lots of temporarily overlapping sound events, for example hundreds of birds 03? a great number of |ar~away clappers, The attached asimyth values are. then get random both in time and frequency, within $i-v#n constraint'. azimuth values ©±fMngS_t.scKg?sunr determining an input parameter #$ a santrol parameter from, the input audio signal, The apparatus (IQQ) of om of th© Glaims 1 %p 5, wh^rgirs the renderer (120) i$ adapted, fop r^ndfring th$ first deepmpogsed signal and the gsseond daeompoasd signal bage.d on different tim§ grids. Thf apparatus (1QQ) of one. of th§ e^aimp 1 to 8, ^herein the deepjijppser (110) ia adapted foje d§t§r-minin&! i;hj firgt decomposed signal and/or the second decomposed Signal based cm a transient separation method, The apparatus (IQQ) of clai.ni 7, wherein, the d©qompQsar (11Q) ia adapted for determining erne of the first desompor^d pignals or the seqond decomposed signal by a transient separation method and the other one bsa§d an the di|f^Jf^nce between t,h§ ©ne and the input audio signal, The apparatus (IQQ) of one oj the claims l to 8, whertin the decomposer (110) is adapted for d§compoplng the input audio signal, the renderer (120) is adapted for rendering the first and/or second decomposed signals, and/or the processor (130) t is adapted for processing the first and/or second rendered signals in terms of different frequency bands. 10. The apparatus of claim 1, in which the processor is configured to process the first rendered signal, the second rendered signal, and the background signal part to obtain the. spatial output multi-channel audio signal 11. A method for determining a spatial output multi^chc.nnel audio signal based on an input audio signal and «*n input parameter comprising the steps of: pemantieal^y d,©cpmppsing thi input au^io signal to pbtain @ first dspomppsfsd signal Having a f^rst ssma^ie property, the, first deosmppsed signal fcfing a. fpr$f£§und. signal part, and a seppnji deepmppsed signtl having a gespemd, §Sm#nt;i£ property being different from the first j?©m$nii|.P jpppjrty, thp second decomposed signal being a kap^ippund signal m?%i rendering the fprsgrpunj signal p«|it ujifMjf pipWtw^ panning to pbtain a fif^t r§nd@r$$ ?lgnj4 hiding the, first semantic property, py prpc,es§in| the fpssfsfun^ signal gsst in sn amplitudt panning ?t

Documents

Orders

Section Controller Decision Date

Application Documents

# Name Date
1 599-KOLNP-2011-RELEVANT DOCUMENTS [08-09-2023(online)].pdf 2023-09-08
1 599-kolnp-2011-specification.pdf 2011-10-06
2 599-kolnp-2011-pct request form.pdf 2011-10-06
2 599-KOLNP-2011-RELEVANT DOCUMENTS [08-09-2022(online)].pdf 2022-09-08
3 599-kolnp-2011-pct priority document notification.pdf 2011-10-06
3 599-KOLNP-2011-IntimationOfGrant20-04-2020.pdf 2020-04-20
4 599-KOLNP-2011-PatentCertificate20-04-2020.pdf 2020-04-20
4 599-KOLNP-2011-PA.pdf 2011-10-06
5 599-KOLNP-2011-Written submissions and relevant documents [25-02-2020(online)].pdf 2020-02-25
5 599-kolnp-2011-international search report.pdf 2011-10-06
6 599-kolnp-2011-international publication.pdf 2011-10-06
6 599-KOLNP-2011-Correspondence to notify the Controller (Mandatory) [09-01-2020(online)].pdf 2020-01-09
7 599-KOLNP-2011-HearingNoticeLetter-(DateOfHearing-10-02-2020).pdf 2020-01-07
7 599-kolnp-2011-form-5.pdf 2011-10-06
8 599-KOLNP-2011-Information under section 8(2) (MANDATORY) [14-09-2019(online)].pdf 2019-09-14
8 599-kolnp-2011-form-3.pdf 2011-10-06
9 599-kolnp-2011-form-2.pdf 2011-10-06
9 599-KOLNP-2011-Information under section 8(2) (MANDATORY) [10-04-2019(online)].pdf 2019-04-10
10 599-kolnp-2011-form-1.pdf 2011-10-06
10 599-KOLNP-2011-Information under section 8(2) (MANDATORY) [17-08-2018(online)].pdf 2018-08-17
11 599-KOLNP-2011-FORM 3-1.1.pdf 2011-10-06
11 599-KOLNP-2011-Information under section 8(2) (MANDATORY) [27-02-2018(online)].pdf 2018-02-27
12 599-KOLNP-2011-ABSTRACT [23-11-2017(online)].pdf 2017-11-23
12 599-KOLNP-2011-FORM 18.pdf 2011-10-06
13 599-KOLNP-2011-CLAIMS [23-11-2017(online)].pdf 2017-11-23
13 599-kolnp-2011-drawings.pdf 2011-10-06
14 599-kolnp-2011-description (complete).pdf 2011-10-06
14 599-KOLNP-2011-DRAWING [23-11-2017(online)].pdf 2017-11-23
15 599-kolnp-2011-correspondence.pdf 2011-10-06
15 599-KOLNP-2011-FER_SER_REPLY [23-11-2017(online)].pdf 2017-11-23
16 599-KOLNP-2011-CORRESPONDENCE 1.2.pdf 2011-10-06
16 599-KOLNP-2011-PETITION UNDER RULE 137 [23-11-2017(online)].pdf 2017-11-23
17 599-KOLNP-2011-Information under section 8(2) (MANDATORY) [15-09-2017(online)].pdf 2017-09-15
17 599-KOLNP-2011-CORRESPONDENCE 1.1.pdf 2011-10-06
18 599-kolnp-2011-claims.pdf 2011-10-06
18 599-KOLNP-2011-Information under section 8(2) (MANDATORY) [24-08-2017(online)].pdf 2017-08-24
19 599-KOLNP-2011-ASSIGNMENT.pdf 2011-10-06
19 599-KOLNP-2011-FER.pdf 2017-05-24
20 599-kolnp-2011-abstract.pdf 2011-10-06
20 Other Patent Document [18-02-2017(online)].pdf 2017-02-18
21 Other Patent Document [12-08-2016(online)].pdf 2016-08-12
22 599-kolnp-2011-abstract.pdf 2011-10-06
22 Other Patent Document [18-02-2017(online)].pdf 2017-02-18
23 599-KOLNP-2011-ASSIGNMENT.pdf 2011-10-06
23 599-KOLNP-2011-FER.pdf 2017-05-24
24 599-KOLNP-2011-Information under section 8(2) (MANDATORY) [24-08-2017(online)].pdf 2017-08-24
24 599-kolnp-2011-claims.pdf 2011-10-06
25 599-KOLNP-2011-Information under section 8(2) (MANDATORY) [15-09-2017(online)].pdf 2017-09-15
25 599-KOLNP-2011-CORRESPONDENCE 1.1.pdf 2011-10-06
26 599-KOLNP-2011-CORRESPONDENCE 1.2.pdf 2011-10-06
26 599-KOLNP-2011-PETITION UNDER RULE 137 [23-11-2017(online)].pdf 2017-11-23
27 599-kolnp-2011-correspondence.pdf 2011-10-06
27 599-KOLNP-2011-FER_SER_REPLY [23-11-2017(online)].pdf 2017-11-23
28 599-kolnp-2011-description (complete).pdf 2011-10-06
28 599-KOLNP-2011-DRAWING [23-11-2017(online)].pdf 2017-11-23
29 599-KOLNP-2011-CLAIMS [23-11-2017(online)].pdf 2017-11-23
29 599-kolnp-2011-drawings.pdf 2011-10-06
30 599-KOLNP-2011-ABSTRACT [23-11-2017(online)].pdf 2017-11-23
30 599-KOLNP-2011-FORM 18.pdf 2011-10-06
31 599-KOLNP-2011-FORM 3-1.1.pdf 2011-10-06
31 599-KOLNP-2011-Information under section 8(2) (MANDATORY) [27-02-2018(online)].pdf 2018-02-27
32 599-kolnp-2011-form-1.pdf 2011-10-06
32 599-KOLNP-2011-Information under section 8(2) (MANDATORY) [17-08-2018(online)].pdf 2018-08-17
33 599-kolnp-2011-form-2.pdf 2011-10-06
33 599-KOLNP-2011-Information under section 8(2) (MANDATORY) [10-04-2019(online)].pdf 2019-04-10
34 599-kolnp-2011-form-3.pdf 2011-10-06
34 599-KOLNP-2011-Information under section 8(2) (MANDATORY) [14-09-2019(online)].pdf 2019-09-14
35 599-kolnp-2011-form-5.pdf 2011-10-06
35 599-KOLNP-2011-HearingNoticeLetter-(DateOfHearing-10-02-2020).pdf 2020-01-07
36 599-kolnp-2011-international publication.pdf 2011-10-06
36 599-KOLNP-2011-Correspondence to notify the Controller (Mandatory) [09-01-2020(online)].pdf 2020-01-09
37 599-KOLNP-2011-Written submissions and relevant documents [25-02-2020(online)].pdf 2020-02-25
37 599-kolnp-2011-international search report.pdf 2011-10-06
38 599-KOLNP-2011-PatentCertificate20-04-2020.pdf 2020-04-20
38 599-KOLNP-2011-PA.pdf 2011-10-06
39 599-kolnp-2011-pct priority document notification.pdf 2011-10-06
39 599-KOLNP-2011-IntimationOfGrant20-04-2020.pdf 2020-04-20
40 599-KOLNP-2011-RELEVANT DOCUMENTS [08-09-2022(online)].pdf 2022-09-08
40 599-kolnp-2011-pct request form.pdf 2011-10-06
41 599-kolnp-2011-specification.pdf 2011-10-06
41 599-KOLNP-2011-RELEVANT DOCUMENTS [08-09-2023(online)].pdf 2023-09-08

Search Strategy

1 ForD1_17-03-2017.pdf

ERegister / Renewals

3rd: 07 Jul 2020

From 11/08/2011 - To 11/08/2012

4th: 07 Jul 2020

From 11/08/2012 - To 11/08/2013

5th: 07 Jul 2020

From 11/08/2013 - To 11/08/2014

6th: 07 Jul 2020

From 11/08/2014 - To 11/08/2015

7th: 07 Jul 2020

From 11/08/2015 - To 11/08/2016

8th: 07 Jul 2020

From 11/08/2016 - To 11/08/2017

9th: 07 Jul 2020

From 11/08/2017 - To 11/08/2018

10th: 07 Jul 2020

From 11/08/2018 - To 11/08/2019

11th: 07 Jul 2020

From 11/08/2019 - To 11/08/2020

12th: 07 Jul 2020

From 11/08/2020 - To 11/08/2021

13th: 30 Jul 2021

From 11/08/2021 - To 11/08/2022

14th: 01 Aug 2022

From 11/08/2022 - To 11/08/2023

15th: 28 Jul 2023

From 11/08/2023 - To 11/08/2024

16th: 26 Jul 2024

From 11/08/2024 - To 11/08/2025

17th: 11 Aug 2025

From 11/08/2025 - To 11/08/2026