Abstract: Disclosed herein is a method and a video generator for generating video response to user queries. The video generator receives a visual image of a character of interest from the user and generates a frontal face of the visual image. Further, facial expressions of the character of interest are mapped with an audio/video sequence of one or more textual responses for generating a human like video response to the user queries. In an embodiment, the video generator detects gender of the character of interest, and modulates and matches voice of the video response based on the gender of the character of interest. The instant method can synthesize a video with the face of a character of interest to the user, thereby providing a wholesome communication experience to the user. FIG. 4
Claims:WE CLAIM:
1. A method for generating video response (112) for user queries (107), the method comprising:
receiving, by a video generator (101), a visual image (105) of a character of interest from the user (103);
generating, by the video generator (101), a frontal face (209) of the character of interest;
generating, by the video generator (101), an audio sequence (110) and a video sequence (111) for one or more predetermined textual response (109) generated in response to the user queries (107);
mapping, by the video generator (101), the video sequence (111) to one or more facial expressions of the character of interest; and
generating, by the video generator (101), the video response (112) by combining the video sequence (111) and the audio sequence (110).
2. The method as claimed in claim 1, wherein the one or more facial expressions of the character of interest comprises lip movement and eye movement of the character of interest, wherein the lip movement matches pronunciation of the one or more predetermined textual response (109).
3. The method as claimed in claim 1, wherein mapping the video sequence (111) is based on training an interacting framework of Convolutional Neural Network (CNN) image encoder, a convolutional Long Short-Term Memory (LSTM) video encoder, a Gated Recurrent Unit (GRU) encoder and a Conditional Pixel CNN (CPCNN) decoder using training data.
4. The method as claimed in claim 1 further comprises determining gender of the character of interest based on the visual image (105).
5. The method as claimed in claim 1 further comprises modulating vocal rhythm of the audio sequence (110) based on gender of the character of interest.
6. The method as claimed in claim 1, wherein combining the audio sequence (110) and the video sequence (111) further comprises synchronizing the audio sequence (110) with the one or more facial expressions of the character of interest.
7. A video generator (101) for generating video response (112) for user queries (107), the video generator (101) comprising:
a processor (203); and
a memory (205) communicatively coupled to the processor (203), wherein the memory (205) stores processor-executable instructions, which, on execution, causes the processor (203) to:
receive a visual image (105) of a character of interest from the user (103);
generate a frontal face (209) of the character of interest;
generate an audio sequence (110) and a video sequence (111) for one or more predetermined textual response (109) generated in response to the user queries (107);
map the video sequence (111) to one or more facial expressions of the character of interest; and
generate the video response (112) by combining the video sequence (111) and the audio sequence (110).
8. The video generator (101) as claimed in claim 7, wherein the one or more facial expressions of the character of interest comprises lip movement and eye movement of the character of interest, wherein the lip movement matches pronunciation of the one or more predetermined textual response (109).
9. The video generator (101) as claimed in 7, wherein to map the video sequence (111), the processor (203) is configured to train an interacting framework of Convolutional Neural Network (CNN) image encoder, a convolutional Long Short-Term Memory (LSTM) video encoder, a Gated Recurrent Unit (GRU) encoder and a Conditional Pixel CNN (CPCNN) decoder using training data.
10. The video generator (101) as claimed in 7, wherein the processor (203) is further configured to determine gender of the character of interest based on the visual image (105).
11. The video generator (101) as claimed in claim 7, wherein the processor (203) is further configured to modulate vocal rhythm of the audio sequence (110) based on gender of the character of interest.
12. The video generator (101) as claimed in claim 7, wherein to combine the audio sequence (110) and the video sequence (111), the processor (203) is further configured to synchronize the audio sequence (110) with the one or more facial expressions of the character of interest.
Dated this 3rd day of April, 2017
SWETHA S.N
OF K & S PARTNERS
AGENT FOR THE APPLICANT
, Description:TECHNICAL FIELD
The present subject matter is related, in general to audio-video response system, and more particularly, but not exclusively to a system and method for generation of human like video response for user queries.
| # | Name | Date |
|---|---|---|
| 1 | Power of Attorney [03-04-2017(online)].pdf | 2017-04-03 |
| 2 | Form 5 [03-04-2017(online)].pdf | 2017-04-03 |
| 3 | Form 3 [03-04-2017(online)].pdf | 2017-04-03 |
| 4 | Form 18 [03-04-2017(online)].pdf_224.pdf | 2017-04-03 |
| 5 | Form 18 [03-04-2017(online)].pdf | 2017-04-03 |
| 6 | Form 1 [03-04-2017(online)].pdf | 2017-04-03 |
| 7 | Drawing [03-04-2017(online)].pdf | 2017-04-03 |
| 8 | Description(Complete) [03-04-2017(online)].pdf_223.pdf | 2017-04-03 |
| 9 | Description(Complete) [03-04-2017(online)].pdf | 2017-04-03 |
| 10 | 201741012047-Proof of Right (MANDATORY) [09-12-2017(online)].pdf | 2017-12-09 |
| 11 | Correspondence by Agent_Form 1_13-12-2017.pdf | 2017-12-13 |
| 12 | 201741012047-FER.pdf | 2020-08-03 |
| 13 | 201741012047-PETITION UNDER RULE 137 [31-01-2021(online)].pdf | 2021-01-31 |
| 14 | 201741012047-OTHERS [31-01-2021(online)].pdf | 2021-01-31 |
| 15 | 201741012047-FORM 3 [31-01-2021(online)].pdf | 2021-01-31 |
| 16 | 201741012047-FER_SER_REPLY [31-01-2021(online)].pdf | 2021-01-31 |
| 17 | 201741012047-DRAWING [31-01-2021(online)].pdf | 2021-01-31 |
| 18 | 201741012047-CLAIMS [31-01-2021(online)].pdf | 2021-01-31 |
| 19 | 201741012047-PatentCertificate25-10-2022.pdf | 2022-10-25 |
| 20 | 201741012047-IntimationOfGrant25-10-2022.pdf | 2022-10-25 |
| 21 | 201741012047-PROOF OF ALTERATION [12-01-2023(online)].pdf | 2023-01-12 |
| 1 | 2020-07-3116-17-29E_31-07-2020.pdf |