Sign In to Follow Application
View All Documents & Correspondence

Apparatus, Method And Computer Program For Upmixing A Downmix Audio Signal

Abstract: An apparatus for upmixing a downmix audio signal describing one or more downmix audio channels into an upmixed audio signal describing a plurality of upmixed audio channels comprises an upmixer configured to apply temporally variable upmixing parameters to upmix the downmix audio signal in order to obtain the upmixed audio signal. The apparatus also comprises a parameter interpolator, wherein the parameter interpolator is configured to obtain one or more temporally interpolated upmix parameters to be used by the upmixer on the basis of a first complex-valued upmix parameter and a subsequent second complex-valued upmix parameter. The parameter interpolator is configured to separately interpolate between a magnitude value of the first complex-valued upmix parameter and a magnitude value of the second complex-valued upmix parameter, and between a phase value of the first complex-valued upmix parameter and a phase value of the second complex-valued upmix parameter, to obtain the one or more temporally interpolated upmix parameters. A respective method can be implemented, for example, as a computer program.

Get Free WhatsApp Updates!
Notices, Deadlines & Correspondence

Patent Information

Application #
Filing Date
22 July 2011
Publication Number
44/2012
Publication Type
INA
Invention Field
ELECTRONICS
Status
Email
Parent Application
Patent Number
Legal Status
Grant Date
2022-03-31
Renewal Date

Applicants

FRAUNHOFER-GESELLSCHAFT ZUR FOERDERUNG DER ANGEWANDTEN FORSCHUNG E.V.
HANSASTRASSE 27C, 80686 MUENCHEN GERMANY

Inventors

1. MATTHIAS NEUSINGER
BERGSTRASSE 10, 91186 ROHR GERMANY
2. JULIEN ROBILLIARD
INNERER KLEINREUTHER WEG 25 A, 90408 NUERNBERG GERMANY
3. JOHANNES HILPERT
HERRNHUETTESTRASSE 46, 90411 NUERNBERG GERMANY

Specification

APPARATUS, METHOD AND COMPUTER PROGRAM FOR UPMIXING A DOWNMIX AUDIO SIGNAL Background of the Invention Embodiments according to the invention are related to an apparatus, a method, and a computer program for upmixing a downmix audio signal. Some embodiments according to the invention are related to a magnitude-preserving upmix parameter interpolation for parametric multi-channel audio coding. In the following, the context of the invention will be described. Recent development in the area of parametric audio coding delivers techniques for jointly coding a multi-channel audio (e.g. 5.1) signal into one (or more) downmix channels plus a side information stream. These techniques are known as Binaural Cue Coding, Parametric Stereo, and MPEG Surround etc. A number of publications describe the so-called "Binaural Cue Coding" parametric multi-channel coding approach, see for example references [1][2][3][4] [5] . "Parametric Stereo" is a related technique for the parametric coding of a two-channel stereo signal based on a transmitted mono signal plus parameter side information [6][7]. "MPEG Surround" is an ISO standard for parametric multi- channel coding [8]. The abovementioned techniques are based on transmitting the relevant perceptual cues for a human's spatial hearing in a compact form to the receiver together with the associated mono or stereo downmix-signal. Typical cues can be inter-channel level differences (ILD), inter-channel correlation or coherence (ICC), as well as inter-channel time differences (ITD) and inter-channel phase differences (IPD). These parameters are in some cases transmitted in a frequency and time resolution adapted to the human's auditory resolution. The update interval in time is determined by the encoder, depending on the signal characteristics. This means that not for every sample of the downmix-signal, parameters are transmitted. In other words, in some cases a transmission rate (or transmission frequency, or update rate) of parameters describing the abovementioned cues may be smaller than a transmission rate (or transmission frequency, or update rate) of audio samples (or groups of audio samples). Since the decoder may in some cases have to apply the parameters continuously over time in a gapless manner, e.g. to each sample (or audio sample), intermediate parameters may need to be derived at decoder side, typically by interpolation between past and current parameter sets. Some conventional interpolation approaches, however, result in poor audio quality. In the following, a generic binaural cue coding scheme will be described taking reference to Fig. 7. Fig. 7 shows a block schematic diagram of a binaural cue coding transmission system 800, which comprises a binaural cue coding encoder 810 and a binaural cue coding decoder 820. The binaural cue coding encoder 810 may for example receive a plurality of audio signals 812a, 812b, and 812c. Further, the binaural cue coding encoder 810 is configured to downmix the audio input signals 812a-812c using a downmixer 814 to obtain a downmix signal 816, which may for example be a sum signal, and which may be designated with "AS" or "X". Further, the binaural cue coding encoder 810 is configured to analyze the audio input signals 812a-812c using an analyzer 818 to obtain the side information signal 819 ("SI") . The sum signal 816 and the side information signal 819 are transmitted from the binaural cue coding encoder 810 to the binaural cue coding decoder 820. The binaural cue coding decoder 820 may be configured to synthesize a multi-channel audio output signal comprising, for example, audio channels yl, y2, ... , yN on the basis of the sum signal 816 and inter-channel cues 824. For this purpose, the binaural cue coding decoder 820 may comprise a binaural cue coding synthesizer 822 which receives the sum signal 816 and the inter-channel cues 824, and provides the audio signals yl, y2,..., yN. The binaural cue coding decoder 820 further comprises a side information processor 826 which is configured to receive the side information 819 and, optionally, a user input 827. The side information processor 826 is configured to provide the inter-channel cues 824 on the basis of the side information 819 and the optional user input 827. To summarize, the audio input signals are analyzed and downmixed. The sum signal plus the side information is transmitted to the decoder. The inter-channel cues are generated from the side information and local user input. The binaural cue coding synthesis generates the multi-channel audio output signal. For details, reference is made to the articles "Binaural Cue Coding Part II: Schemes and applications," by C. Faller and F. Baumgarte (published in: IEEE Transactions on Speech and Audio Processing, vol. 11, no. 6, Nov. 2003). However, it has been found that many conventional binaural cue coding decoders provide multi-channel output audio signals with degraded quality if the side information is received at a lower update frequency than the downmix signal. In view of this problem, there is a need for an improved concept of upmixing a downmix audio signal into an upmixed audio signal, which reduces a degradation of the hearing impression if the update frequency of the side information is smaller than the update frequency of the downmix audio signal. Summary of the Invention An embodiment according to the invention creates an apparatus for upmixing a downmix audio signal describing one or more downmix audio channels into an upmixed audio channel describing a plurality of upmixed audio channels. The apparatus comprises an upmixer configured to apply temporally variable upmixing parameters to upmix the downmix audio signal in order to obtain the upmixed audio signal. The apparatus further comprises a parameter interpolator, wherein the parameter interpolator is configured to obtain one or more temporally interpolated upmix parameters to be used by the upmixer on the basis of a first complex-valued upmix parameter and a subsequent second complex-valued upmix parameter. The parameter interpolator is configured to separately interpolate between a magnitude value of the first complex-valued upmix parameter and a magnitude value of the second complex-valued upmix parameter, and between a phase value of the first complex-valued upmix parameter and a phase value of the second complex-valued upmix parameter, to obtain the one or more temporally interpolated upmix parameters. Embodiments according to the invention are based on the finding that a separate temporal interpolation of the magnitude value of an upmix parameter and of the phase value of the upmix parameter brings along a good hearing impression of the upmixed audio signal because a variation of the magnitude of the interpolated upmix parameter is kept very small. It has been found that an unnecessarily large variation of the amplitude of the upmix parameter may result in an audible and disturbing modulation of the upmixed audio signal. In contrast, by separately interpolating the amplitude of the complex-valued upmix parameters from the phase value thereof, the amplitude variation caused by the interpolation is kept small (or even minimized) , even in the presence of a large phase difference between the complex value of the first (or initial) upmix parameter and the complex value of the second (or subsequent) upmix parameter. Accordingly, an audible and disturbing modulation of the upmixed output audio signal is reduced when compared to some other types of interpolation (or even completely eliminated). Thus, a good hearing impression of the upmixed output audio signal can be obtained, even if the side information is transferred from a binaural cue coding encoder to a binaural cue coding decoder less frequently than samples of the downmix audio signal. In an embodiment according to the invention, the parameter interpolator is configured to monotonically time interpolate between a magnitude value of the first complex-valued upmix parameter and the magnitude value of the second (subsequent) complex-valued upmix parameter to obtain magnitude values of the one or more temporally interpolated upmix parameters. Furthermore, the parameter interpolator may preferably be configured to linearly time-interpolate between a phase value of the first complex-valued upmix parameter and the phase value of the second complex-valued upmix parameter, to obtain phase values of the one or more temporally interpolated upmix parameters. Further, the parameter interpolator may be configured to combine the one or more magnitude values of the interpolated upmix parameters with corresponding phase values of the interpolated upmix parameters in order to obtain the one or more complex-valued interpolated upmix parameters. In an embodiment according to the invention, the parameter interpolator is configured to linearly time-interpolate between the magnitude value of the first complex-valued upmix parameter and the magnitude value of the second, subsequent complex-valued upmix parameter, to obtain magnitude values of the one or more temporally interpolated upmix parameters. By performing a monotonic or even linear time interpolation between magnitude values of the subsequent complex-valued upmix parameters, a disturbing amplitude modulation of the upmixed audio signal (which would be caused by other interpolation schemes) can be avoided. Regarding this issue, it has been found that the human auditory system is particularly sensitive to amplitude modulation of audio signals. It has also been found that the auditory impression (or hearing impression) is significantly degraded by such a parasitic amplitude modulation. Accordingly, obtaining a smooth and non-modulated variation of the upmix parameters, which results in a smooth and non-modulated temporal evolution of the audio signal amplitude, is an important contribution to the improvement of the hearing impression of an upmix signal in the presence of an interpolation of the upmix parameters. In an embodiment of the invention, the upmixer is configured to perform a linear scaled superposition of complex-valued subband parameters of a plurality of upmixer audio input signals in dependence on the complex-valued interpolated upmixing parameters to obtain the upmixed audio signal. In this case, the upmixer may be configured to process sequences of subband parameters representing subsequent audio samples of the upmixer audio input signals. The parameter interpolator may be configured to receive subsequent complex-valued upmix parameters, which are temporally spaced by more than the duration of one of the subband audio samples, and to update the interpolated upmixing parameters more frequently (e.g. once per subband audio sample). Thus, the upmixer may be configured to receive updated samples of the upmixer audio input signals at an upmixer update rate, and the parameter interpolator may be configured to update the interpolated upmix parameters at the upmixer update rate. In this way, the update rate of the upmix parameters may be adapted to be the update rate of the upmixer audio input signals. Accordingly, particularly smooth transitions between two subsequent sets of upmix-parameters received by the apparatus (e.g. at an update rate smaller than the upmixer update rate) may be obtained. In a preferred embodiment of the invention, the upmixer may be configured to perform a matrix-vector multiplication using a matrix comprising the interpolated upmix parameters and a vector comprising one or more subband parameters of the upmixer audio input signals, to obtain as a result a vector comprising complex-valued subband samples of the upmixed audio signals. By using a matrix-vector multiplication, a particularly efficient circuit implementation can be obtained. The matrix-vector multiplication defines, in an efficient-to- implement form, the upmix-parameter-dependent linear superposition of the audio input signals. A matrix-vector- multiplication can be efficiently implemented in a signal processor (or in other appropriate hardware or software units) if the entries of the matrix are represented split-up into a real part and an imaginary part. Handling of complex values split-up into a real part and an imaginary part can be performed with relatively little effort, as the real- part/imaginar-part splitting is well-suited both for a multiplication of complex numbers and, particularly, for an addition of the results of the multiplication. Thus, while other number representations bring along severe difficulties either with respect to a multiplication or with respect to an addition (which operations are both needed in a matrix-vector- multiplication) , the usage of a real-part/imaginary-part number representation provides for an efficient solution. In an embodiment of the invention, the apparatus is configured to receive spatial cues describing the upmix parameters. In this case, the parameter interpolator may be configured to determine the magnitude values of the upmix parameters in dependence on inter-channel level difference parameters, or in dependence on inter-channel correlation (or coherence) parameters, or in dependence on inter-channel level difference parameters and inter-channel correlation (or coherence) parameters. Further, the parameter interpolator may be configured to determine the phase values of the upmix parameters in dependence on inter-channel phase difference parameters. Accordingly, it can be seen that in some cases it is possible, in a very efficient manner, to obtain the magnitude values and the phase values of the upmix parameters separately. Thus, the input information required for the separate interpolation can be efficiently obtained even without any additional magnitude-value/phase values separation unit if the abovementioned parameters (ILD, ICC, IPD, and/or ITD) or comparable parameters are used as input quantities to the parameter interpolator. In an embodiment of the invention, the parameter interpolator is configured to determine a direction of the interpolation between the phase values of subsequent complex-valued upmix parameters such that an angle range passed in the interpolation between a phase value of the first complex- valued upmix parameter and a phase value of the (subsequent) second complex-valued upmix parameter is smaller than, or equal to, 180°. In other words, in some embodiments it is ensured that a phase variation caused by the interpolation is kept sufficiently small (or even minimized). Even though the human auditory perception is not particularly sensitive to phase changes, it may be advantageous to limit the phase variation. For example, fast phase variation of the upmix parameters might result in difficult-to-predict distortions such as frequency shifts or frequency modulation. Such distortions can be limited or eliminated by carefully deciding how to interpolate the phase values of the upmix parameters. Another embodiment according to the invention creates a method for upmixing a downmix audio signal. Yet another embodiment according to the invention creates a computer program for upmixing a downmix audio signal. Brief Description of the Figures Embodiments according to the invention will subsequently be described taking reference to the enclosed figures, in which: Fig. 1 shows a block schematic diagram of an apparatus for upmixing a downmix audio signal, according to an embodiment of the invention; Fig. 2a and 2b show a block schematic diagram of an apparatus for upmixing a downmix audio signal, according to another embodiment of the invention; Fig. 3 shows a schematic representation of a timing relationship between samples of the downmix audio signal and a decoder input side information; Fig. 4 shows a schematic representation of a timing relationship between the decoder input side information and temporally interpolated upmix parameters based thereon; Fig. 5 shows a graphical representation of an interpolation path; Fig. 6 shows a flow chart of a method for upmixing a downmix audio signal, according to an embodiment of the invention; and Fig. 7 shows a block schematic diagram representing a generic binaural cue coding scheme. Detailed Description of the Embodiments Embodiment according to Fig. 1 Fig. 1 shows a block schematic diagram of an apparatus 100 for upmixing a downmix audio signal, according to an embodiment of the invention. The apparatus 100 is configured to receive a downmix audio signal 110 describing one or more downmix audio channels, and to provide an upmixed audio signal 120 describing a plurality of upmixed audio channels. The apparatus 100 comprises an upmixer 130 configured to apply temporally variable upmixing parameters to upmix the downmix audio signal 110 in order to obtain the upmixed audio signal 120. The apparatus 100 also comprises a parameter interpolator 140 configured to receive a sequence of complex-valued upmix parameters, for example a first complex-valued upmix parameter 142 and a subsequent second complex-valued upmix parameter 144. The parameter interpolator 140 is configured to obtain one or more temporally interpolated upmix parameters 150 to be used by the upmixer 130 on the basis of the first (or initial) complex-valued upmix parameter 142 and the second, subsequent complex-valued upmix parameter 144. The parameter interpolator 140 is configured to separately interpolate between a magnitude value of the first complex-valued upmix parameter 142 and a magnitude value of the second complex-valued upmix parameter 144 (which magnitude value interpolation is represented at reference numeral 160), and between a phase value of the first complex-valued upmix parameter 142 and a phase value of the second complex-valued upmix parameter 14 4 (which phase value interpolation is represented at reference numeral 162). The parameter interpolator 140 is configured to obtain the one or more temporally interpolated upmix parameters 150 on the basis of the interpolated magnitude values (also designated as amplitude values, or gain values)(which is represented with reference numeral 160) and on the basis of the interpolated phase values (also designated as angle values)(which is shown at reference numeral 164). In the following, some details regarding the functionality of the apparatus 100 will be described. The downmix audio signal 110 may be input into the upmixer 130, for example in the form of a sequence of sets of complex values representing the downmix audio signal in the time-frequency domain (describing overlapping or non-overlapping frequency bands or frequency subbands at an update rate determined by the encoder not shown here). The upmixer 130 is configured to linearly combine multiple channels of the downmix audio signal 110 in dependence on the temporally interpolated upmix parameters 150, or to linearly combine a channel of the downmix audio signal 110 with an auxiliary signal (e.g. de-correlated signal) (wherein the auxiliary signal may be derived from the same audio channel of the downmix audio signal 110, from one or more other audio channels of the downmix audio signal 110 or from a combination of audio channels of the downmix audio signal 110) . Thus, the temporally interpolated upmix parameters 150 may be used by the upmixer 130 to decide upon the amplitude scaling and a phase rotation (or time delay) used in the generation of the upmixed audio signal 120 (or a channel thereof) on the basis of the downmix audio signal 110. The parameter interpolator 140 is typically configured to provide temporally interpolated upmix parameters 150 at an update rate which is higher than the update rate of the side information described by the upmix parameters 142, 144. For this purpose, subsequent complex-valued upmix parameter are obtained (e.g. received or computed) by the parameter interpolator 140. A magnitude value and a phase value of the complex-valued upmix parameters 142, 144 are separately (or even independently) processed using a magnitude value interpolation 160 and a phase value interpolation 162. Thus, temporally interpolated magnitude values of the upmix parameters and temporally interpolated phase values of the upmix parameters are available separately, and may either be fed separately to the upmixer 140, or may be fed to the upmixer 130 in a combined form (combined - after separate interpolation - into a complex-valued number). The separate interpolation brings along the advantage that an amplitude of the temporally interpolated upmix parameter typically comprises a smooth and monotonic temporal evolution between subsequent instances in time at which the updated side information is received by the apparatus 100. Audible and disturbing artifacts, such as an amplitude modulation of one or more subbands, which are caused by other types of interpolation, are avoided. Accordingly, the quality of the updated audio signals 120 is superior to the quality of an upmix signal which would be obtained using conventional types of upmix parameter interpolation. Embodiment according to Fig. 2 Further details regarding the structure and operation of an apparatus for upmixing an audio signal will be described taking reference to Figs. 2a and 2b. Figs. 2a and 2b show a detailed block schematic diagram of an apparatus 200 for upmixing a downmix audio signal, according to another embodiment of the invention. The apparatus 200 can be considered as a decoder for generating a multi-channel (e.g. 5.1) audio signal on the basis of a downmix audio signal and a side information SI. The apparatus 200 implements the functionalities which have been described with respect to the apparatus 100. The apparatus 200 may, for example, serve to decode a multi-channel audio signal encoded according to a so- called "binaural cue coding", a so-called "parametric stereo", or a so-called "MPEG Surround". Naturally, the apparatus 200 may similarly be used to upmix multi-channel audio signals encoded according to other systems using spatial cues. For simplicity, the apparatus 200 is described which performs an upmix of a single channel downmix audio signal into a two- channel signal. However, the concept described here can be easily extended to cases in which the downmix audio signal comprises more than one channel, and also to cases in which the upmixed audio signal comprises more than two channels. Input signals and input timing The apparatus 200 is configured to receive the downmix audio signal 210 and the side information 212. Further, the apparatus 200 is configured to provide an upmixed audio signal 214 comprising, for example, multiple channels. The downmix audio signal 210 may, for example, be a sum signal generated by an encoder (e.g. by the BCC encoder 810 shown in Fig. 7) . The downmix audio signal 210 may, for instance, be represented in a time-frequency domain, for example in the form of a complex-valued frequency decomposition. For instance, audio contents of a plurality of frequency subbands (which may be overlapping or non-overlapping) of the audio signal may be represented by corresponding complex values. For a given frequency band, the downmix audio signal may be represented by a sequence of complex values describing the audio content in the frequency subband under consideration for subsequent (overlapping or non-overlapping) time intervals. The subsequent complex values for subsequent time intervals may be obtained, for example, using a filterbank (e.g. QMF Filterbank) , a Fast Fourier Transform, or the like, in the apparatus 100 (which may be part of a multi-channel audio signal decoder), or in an additional device coupled to the apparatus 100. However, the representation of the downmix audio signal described here is typically not identical to the representation of the downmix signal used for a transmission of the downmix audio signal from a multi-channel audio signal encoder to a multi-channel audio signal decoder, or to the apparatus 100. Accordingly, the downmix audio signal 210 may be represented by a stream of sets or vectors of complex values. In the following, it will be assumed that subsequent time intervals of the downmix audio signal 210 are designated with an integer-valued index k. It will also be assumed that the apparatus 200 receives one set or vector of complex values per interval k and per channel of the downmix audio signal 210. Thus, one sample (set or vector of complex values) is received for every audio sample update interval described by time index k. To facilitate the understanding, Fig. 3 shows a graphical representation of a timing relationship between samples of the downmix audio signal 210 ("x") and the corresponding decoder side information 212 ("SI"). Audio samples ("AS") of the downmixed audio signal 210 received by the apparatus 200 over time are shown at reference numeral 310. As can be seen from the graphical representation 310, a single audio sample AS is associated with each audio sample update interval k, as described above. The apparatus 200 further receives a side information 212 describing the upmix parameters. For instance, the side information 212 may describe one or more of the following upmix parameters: inter-channel level difference (ILD), inter- channel correlation (or coherence) (ICC) , inter-channel time difference (ITD), and inter-channel phase difference (IPD). Typically, the side information 212 comprises the ILD parameters and at least one out of the parameters ICC, ITD, IPD. However, in order to save bandwidth, the side information 212 is typically only transmitted towards, or received by, the apparatus 200 once per multiple of the audio sample update intervals k of the downmix audio signal 210 (or the transmission of a single set of side information may be temporally spread over a plurality of audio sample update intervals k) . Thus, there is typically only one set of side information parameters for a plurality of audio sample update intervals k. This timing relationship is shown in Fig. 3. For example, side information is transmitted to (or received by) the apparatus 200 at the audio sample update intervals k=4, k=8, and k=16, as can be seen at reference numeral 320. In contrast no side information 212 is transmitted to (or received by) the apparatus 200 between said audio sample update intervals. As can be seen from Fig. 3, the update intervals of the side information 212 may vary over time, as the encoder may for example decide to provide a side information update only when required (e.g. when the decoder recognizes that the side information is changed by more than a predetermined value) . For example, the side information received by the apparatus 200 for the audio sample update interval k=4 may be associated with the audio sample update intervals k=3,4,5. Similarly, the side information received by the apparatus 200 for the audio sample update interval k=8 may be associated with the audio sample update intervals k=6,7,8,9,10, and so on. However, a different association is naturally possible, and the update intervals for the side information may naturally also be larger or smaller than shown in Fig. 3. Output signals and output timing However, the apparatus 200 serves to provide upmixed audio signals in a complex-valued frequency composition. For example, the apparatus 200 may be configured to provide the upmixed audio signals 214 such that the upmixed audio signals comprise the same audio sample update interval or audio signal update rate as the downmix audio signal 210. In other words, for each sample (or audio sample update interval k) of the downmix audio signal 210, a sample of the upmixed audio signal 214 is generated. Upmix In the following, it will be described in detail how an update of the upmix parameters, which are used for upmixing the downmix audio signal, can be obtained for each audio sample update interval k, even though the decoder input side information is updated only in larger update intervals (as shown in Fig. 3). In the following the processing for a single subband will be described, but the concept can naturally be extended to multiple subbands. The apparatus 200 comprises, as a key component, an upmixer which is configured to operate as a complex-valued linear combiner. The upmixer 230 is configured to receive a sample x(k) of the downmix audio signal 210 (e.g. representing a certain frequency band) associated with the audio sample update interval k. The signal x(k) is sometimes also designated as "dry signal". Also, the upmixer is configured to receive samples representing a decorrelated version of the downmix audio signal. Further, the apparatus 200 comprises a decorrelator (e.g. a delayer or reverberator) 240, which is configured to receive samples x(k) of the downmix audio signal and to provide, on the basis thereof, samples q(k) of a de-correlated version of the downmix audio signal (represented by x(k)). The de- correlated version (samples q(k)) of the downmix audio signal (samples x(k)) may be designated as "wet signal". The upmixer 230 comprises, for example, a matrix-vector multiplier 232 which is configured to perform a complex-valued linear combination of the "dry signal" (x(k)) and the "wet signal" (q(k)) to obtain a first upmixed channel signal (represented by samples yi(k)) and a second upmixed channel signal (represented by samples y2

Documents

Orders

Section Controller Decision Date

Application Documents

# Name Date
1 3128-KOLNP-2011-(01-09-2011)-PA.pdf 2011-09-01
1 3128-KOLNP-2011-RELEVANT DOCUMENTS [04-09-2023(online)].pdf 2023-09-04
2 3128-KOLNP-2011-(01-09-2011)-CORRESPONDENCE.pdf 2011-09-01
2 3128-KOLNP-2011-IntimationOfGrant31-03-2022.pdf 2022-03-31
3 3128-KOLNP-2011-PatentCertificate31-03-2022.pdf 2022-03-31
3 3128-KOLNP-2011-(21-09-2011)-CORRESPONDENCE.pdf 2011-09-21
4 3128-KOLNP-2011-FORM 3 [07-12-2021(online)].pdf 2021-12-07
4 3128-KOLNP-2011-(21-09-2011)-ASSIGNMENT.pdf 2011-09-21
5 abstract-3128-kolnp-2011.jpg 2011-10-07
5 3128-KOLNP-2011-FORM 3 [07-06-2021(online)].pdf 2021-06-07
6 3128-kolnp-2011-specification.pdf 2011-10-07
6 3128-KOLNP-2011-Information under section 8(2) [08-12-2020(online)].pdf 2020-12-08
7 3128-kolnp-2011-pct request form.pdf 2011-10-07
7 3128-KOLNP-2011-Information under section 8(2) [03-07-2020(online)].pdf 2020-07-03
8 3128-kolnp-2011-pct priority document notification.pdf 2011-10-07
8 3128-KOLNP-2011-FORM 3 [04-06-2020(online)].pdf 2020-06-04
9 3128-kolnp-2011-international search report.pdf 2011-10-07
9 3128-KOLNP-2011-Written submissions and relevant documents [18-05-2020(online)].pdf 2020-05-18
10 3128-kolnp-2011-international publication.pdf 2011-10-07
10 3128-KOLNP-2011-Written submissions and relevant documents [15-05-2020(online)].pdf 2020-05-15
11 3128-kolnp-2011-international preliminary examination report.pdf 2011-10-07
11 3128-KOLNP-2011-Written submissions and relevant documents [23-03-2020(online)].pdf 2020-03-23
12 3128-KOLNP-2011-Correspondence to notify the Controller [02-03-2020(online)].pdf 2020-03-02
12 3128-kolnp-2011-form-5.pdf 2011-10-07
13 3128-kolnp-2011-form-3.pdf 2011-10-07
13 3128-KOLNP-2011-HearingNoticeLetter-(DateOfHearing-13-03-2020).pdf 2020-02-28
14 3128-kolnp-2011-form-2.pdf 2011-10-07
14 3128-KOLNP-2011-Information under section 8(2) (MANDATORY) [26-12-2019(online)].pdf 2019-12-26
15 3128-KOLNP-2011-FORM-18.pdf 2011-10-07
15 3128-KOLNP-2011-Information under section 8(2) (MANDATORY) [07-06-2019(online)].pdf 2019-06-07
16 3128-kolnp-2011-form-1.pdf 2011-10-07
16 3128-KOLNP-2011-Information under section 8(2) (MANDATORY) [10-01-2019(online)].pdf 2019-01-10
17 3128-KOLNP-2011-Information under section 8(2) (MANDATORY) [20-08-2018(online)].pdf 2018-08-20
17 3128-kolnp-2011-drawings.pdf 2011-10-07
18 3128-KOLNP-2011-ABSTRACT [05-03-2018(online)].pdf 2018-03-05
18 3128-kolnp-2011-description (complete).pdf 2011-10-07
19 3128-KOLNP-2011-CLAIMS [05-03-2018(online)].pdf 2018-03-05
19 3128-kolnp-2011-correspondence.pdf 2011-10-07
20 3128-kolnp-2011-claims.pdf 2011-10-07
20 3128-KOLNP-2011-DRAWING [05-03-2018(online)].pdf 2018-03-05
21 3128-kolnp-2011-abstract.pdf 2011-10-07
21 3128-KOLNP-2011-FER_SER_REPLY [05-03-2018(online)].pdf 2018-03-05
22 3128-KOLNP-2011-(20-01-2012)-FORM 3.pdf 2012-01-20
22 3128-KOLNP-2011-PETITION UNDER RULE 137 [05-03-2018(online)].pdf 2018-03-05
23 3128-KOLNP-2011-(20-01-2012)-CORRESPONDENCE.pdf 2012-01-20
23 3128-KOLNP-2011-PETITION UNDER RULE 137 [05-03-2018(online)]_13.pdf 2018-03-05
24 Other Patent Document [27-07-2016(online)].pdf 2016-07-27
24 3128-KOLNP-2011-Proof of Right (MANDATORY) [24-02-2018(online)].pdf 2018-02-24
25 3128-KOLNP-2011-Information under section 8(2) (MANDATORY) [09-01-2018(online)].pdf 2018-01-09
25 Other Patent Document [19-01-2017(online)].pdf 2017-01-19
26 3128-KOLNP-2011-FER.pdf 2017-09-06
26 3128-KOLNP-2011-Information under section 8(2) (MANDATORY) [20-07-2017(online)].pdf 2017-07-20
27 3128-KOLNP-2011-FER.pdf 2017-09-06
27 3128-KOLNP-2011-Information under section 8(2) (MANDATORY) [20-07-2017(online)].pdf 2017-07-20
28 3128-KOLNP-2011-Information under section 8(2) (MANDATORY) [09-01-2018(online)].pdf 2018-01-09
28 Other Patent Document [19-01-2017(online)].pdf 2017-01-19
29 3128-KOLNP-2011-Proof of Right (MANDATORY) [24-02-2018(online)].pdf 2018-02-24
29 Other Patent Document [27-07-2016(online)].pdf 2016-07-27
30 3128-KOLNP-2011-(20-01-2012)-CORRESPONDENCE.pdf 2012-01-20
30 3128-KOLNP-2011-PETITION UNDER RULE 137 [05-03-2018(online)]_13.pdf 2018-03-05
31 3128-KOLNP-2011-(20-01-2012)-FORM 3.pdf 2012-01-20
31 3128-KOLNP-2011-PETITION UNDER RULE 137 [05-03-2018(online)].pdf 2018-03-05
32 3128-kolnp-2011-abstract.pdf 2011-10-07
32 3128-KOLNP-2011-FER_SER_REPLY [05-03-2018(online)].pdf 2018-03-05
33 3128-kolnp-2011-claims.pdf 2011-10-07
33 3128-KOLNP-2011-DRAWING [05-03-2018(online)].pdf 2018-03-05
34 3128-KOLNP-2011-CLAIMS [05-03-2018(online)].pdf 2018-03-05
34 3128-kolnp-2011-correspondence.pdf 2011-10-07
35 3128-KOLNP-2011-ABSTRACT [05-03-2018(online)].pdf 2018-03-05
35 3128-kolnp-2011-description (complete).pdf 2011-10-07
36 3128-KOLNP-2011-Information under section 8(2) (MANDATORY) [20-08-2018(online)].pdf 2018-08-20
36 3128-kolnp-2011-drawings.pdf 2011-10-07
37 3128-kolnp-2011-form-1.pdf 2011-10-07
37 3128-KOLNP-2011-Information under section 8(2) (MANDATORY) [10-01-2019(online)].pdf 2019-01-10
38 3128-KOLNP-2011-FORM-18.pdf 2011-10-07
38 3128-KOLNP-2011-Information under section 8(2) (MANDATORY) [07-06-2019(online)].pdf 2019-06-07
39 3128-kolnp-2011-form-2.pdf 2011-10-07
39 3128-KOLNP-2011-Information under section 8(2) (MANDATORY) [26-12-2019(online)].pdf 2019-12-26
40 3128-kolnp-2011-form-3.pdf 2011-10-07
40 3128-KOLNP-2011-HearingNoticeLetter-(DateOfHearing-13-03-2020).pdf 2020-02-28
41 3128-KOLNP-2011-Correspondence to notify the Controller [02-03-2020(online)].pdf 2020-03-02
41 3128-kolnp-2011-form-5.pdf 2011-10-07
42 3128-kolnp-2011-international preliminary examination report.pdf 2011-10-07
42 3128-KOLNP-2011-Written submissions and relevant documents [23-03-2020(online)].pdf 2020-03-23
43 3128-kolnp-2011-international publication.pdf 2011-10-07
43 3128-KOLNP-2011-Written submissions and relevant documents [15-05-2020(online)].pdf 2020-05-15
44 3128-kolnp-2011-international search report.pdf 2011-10-07
44 3128-KOLNP-2011-Written submissions and relevant documents [18-05-2020(online)].pdf 2020-05-18
45 3128-KOLNP-2011-FORM 3 [04-06-2020(online)].pdf 2020-06-04
45 3128-kolnp-2011-pct priority document notification.pdf 2011-10-07
46 3128-kolnp-2011-pct request form.pdf 2011-10-07
46 3128-KOLNP-2011-Information under section 8(2) [03-07-2020(online)].pdf 2020-07-03
47 3128-kolnp-2011-specification.pdf 2011-10-07
47 3128-KOLNP-2011-Information under section 8(2) [08-12-2020(online)].pdf 2020-12-08
48 abstract-3128-kolnp-2011.jpg 2011-10-07
48 3128-KOLNP-2011-FORM 3 [07-06-2021(online)].pdf 2021-06-07
49 3128-KOLNP-2011-FORM 3 [07-12-2021(online)].pdf 2021-12-07
49 3128-KOLNP-2011-(21-09-2011)-ASSIGNMENT.pdf 2011-09-21
50 3128-KOLNP-2011-PatentCertificate31-03-2022.pdf 2022-03-31
50 3128-KOLNP-2011-(21-09-2011)-CORRESPONDENCE.pdf 2011-09-21
51 3128-KOLNP-2011-(01-09-2011)-CORRESPONDENCE.pdf 2011-09-01
51 3128-KOLNP-2011-IntimationOfGrant31-03-2022.pdf 2022-03-31
52 3128-KOLNP-2011-(01-09-2011)-PA.pdf 2011-09-01
52 3128-KOLNP-2011-RELEVANT DOCUMENTS [04-09-2023(online)].pdf 2023-09-04

Search Strategy

1 PatSeer1_12-04-2017.pdf
1 PatSeer3_12-04-2017.pdf
2 PatSeer2_12-04-2017.pdf
3 PatSeer1_12-04-2017.pdf
3 PatSeer3_12-04-2017.pdf

ERegister / Renewals

3rd: 08 Jun 2022

From 12/01/2012 - To 12/01/2013

4th: 08 Jun 2022

From 12/01/2013 - To 12/01/2014

5th: 08 Jun 2022

From 12/01/2014 - To 12/01/2015

6th: 08 Jun 2022

From 12/01/2015 - To 12/01/2016

7th: 08 Jun 2022

From 12/01/2016 - To 12/01/2017

8th: 08 Jun 2022

From 12/01/2017 - To 12/01/2018

9th: 08 Jun 2022

From 12/01/2018 - To 12/01/2019

10th: 08 Jun 2022

From 12/01/2019 - To 12/01/2020

11th: 08 Jun 2022

From 12/01/2020 - To 12/01/2021

12th: 08 Jun 2022

From 12/01/2021 - To 12/01/2022

13th: 08 Jun 2022

From 12/01/2022 - To 12/01/2023

14th: 09 Jan 2023

From 12/01/2023 - To 12/01/2024

15th: 29 Dec 2023

From 12/01/2024 - To 12/01/2025

16th: 28 Dec 2024

From 12/01/2025 - To 12/01/2026