DESC:Cross-Reference to Related Applications
[001] The present application claims priority to Indian Provisional Patent Application No., which are hereby incorporated by reference herein.
Background
[002] Video conferencing is a meeting with two or more participants who are participating in different locations. Typically, a computer connection, audio, and video are used to connect. At its simplest, video conferencing provides the transmission of static images and text between two locations. At its most sophisticated, it provides transmission of full-motion video images and high-quality audio between multiple locations. There are several different types of video conferencing systems including Telepresence Video Conferencing (multiple screens or monitors are used to make everyone feel like they are joining the meeting in-person), Desktop Video Conferencing (the conferencing hardware and software is built-in to a computer or laptop), and Room-Based Video Conferencing (video conferencing technology is built-in to the room itself).
[003] Video conferencing technology is used for team Meetings, collaborative work, webinars, product demos, one-on-one training and support, job Interviews, and for other purposes. An advantage of video conferencing is to record calls and get them transcribed. Anyone can join, regardless of their location with no need to relocate. The teleconferencing can include additional meeting data and insights and enable a digital workforce.
Summary
[004] Video recordings in a video conference can be done using video conferencing apps, where video recording functionality is provided, or using third-party online screen recorders to record video conferences. These recorded video clips or segments may be stored locally in computers or in clouds and archived when necessary. Existing video recording functionality is limited in the flexibility and control provided to the participant. Moreover, it is demanding of memory and storage, which can be a significant problem for some client devices such as handheld wireless devices. So an enhanced conference recording functionality, as described herein, is desired to address these issues.
Brief Description of the Drawings
[005] FIG. 1 depicts a diagram of a live-custom recording teleconferencing system.
[006] FIG. 2 depicts a server-side flow diagram for maintaining sliding history window.
[007] FIG. 3 depicts a server-side flow diagram for live-custom recording.
[008] FIG. 4 depicts a client-side flow diagram for maintaining sliding history window.
[009] FIG. 5 depicts a client-side flow diagram for live-custom recording.
[0010] FIG. 6 depicts a diagram of engines of a live-custom recording teleconferencing system.
Detailed Description
[0011] FIG. 1 depicts a diagram 100 of a live-custom recording teleconferencing system. The diagram 100 is intended to represent a teleconferencing environment in which live-custom recording is available. The diagram includes a network 102, a live-custom recording teleconferencing system 104, a host device 106, and a participant device 108-1 to a participant device 108-n (individually, the participant device 108; collectively, the participant devices 108).
[0012] The network 102 is intended to represent a network of computer systems. A “computer system,” as used herein, may include or be implemented as a specific purpose computer system for carrying out the functionalities described in this paper. In general, a computer system will include a processor, memory, non-volatile storage, and an interface. A typical computer system will usually include at least a processor, memory, and a device (e.g., a bus) coupling the memory to the processor. The processor can be, for example, a general-purpose central processing unit (CPU), such as a microprocessor, or a special-purpose processor, such as a microcontroller. As used in this paper, a means for computing, initiating, or stopping (in a computer system context) includes a processor of some kind, and is intended to include a hardware processor executing instructions, if applicable.
[0013] Memory of a computer system includes, by way of example but not limitation, random access memory (RAM), such as dynamic RAM (DRAM) and static RAM (SRAM). The memory can be local, remote, or distributed. Non-volatile storage is often a magnetic floppy or hard disk, a magnetic-optical disk, an optical disk, a read-only memory (ROM), such as a CD-ROM, erasable programmable read-only memory (EPROM), or electrically erasable programmable read only memory (EEPROM), a magnetic or optical card, or another form of storage for large amounts of data. During execution of software, some of this data is often written, by a direct memory access process, into memory by way of a bus coupled to non-volatile storage. Non-volatile storage can be local, remote, or distributed, but is optional because systems can be created with all applicable data available in memory.
[0014] Software in a computer system is typically stored in non-volatile storage. Indeed, for large programs, it may not even be possible to store the entire program in memory. For software to run, if necessary, it is moved to a computer system location appropriate for processing, and for illustrative purposes in this paper, that location is referred to as memory. Even when software is moved to memory for execution, a processor will typically make use of hardware registers to store values associated with the software, and a local cache that, ideally, serves to speed up execution. As used herein, a software program is assumed to be stored at an applicable known or convenient location (from non-volatile storage to hardware registers) when the software program is referred to as “implemented in a computer system location.” A processor is considered “configured to execute a program” when at least one value associated with the program is stored in a register readable by the processor.
[0015] In one example of operation, a computer system can be controlled by operating system software, which is a software program that includes a file management system, such as a disk operating system. One example of operating system software with associated file management system software is the family of operating systems known as Windows from Microsoft Corporation of Redmond, Wash., and their associated file management systems. Another example of operating system software with its associated file management system software is the Linux operating system and its associated file management system. The file management system is typically stored in the non-volatile storage and causes the processor to execute the various acts required by the operating system to input and output data and to store data in the memory, including storing files on the non-volatile storage.
[0016] The bus of a computer system can couple a processor to an interface. Interfaces facilitate the coupling of devices and computer systems. Interfaces can be for input and/or output (I/O) devices, modems, or networks. I/O devices can include, by way of example but not limitation, a keyboard, a mouse or other pointing device, disk drives, printers, a scanner, and other I/O devices, including a display device. Display devices can include, by way of example but not limitation, a cathode ray tube (CRT), liquid crystal display (LCD), or some other applicable known or convenient display device. Modems can include, by way of example but not limitation, an analog modem, an integrated services digital network (ISDN) modem, a cable modem, and other modems. Network interfaces can include, by way of example but not limitation, a token ring interface, a satellite transmission interface (e.g., “direct PC”), or other network interface for coupling a first computer system to a second computer system. An interface can be considered part of a device or computer system. As used in this paper, a means for sending, requesting, providing, or receiving includes an interface of some kind (potentially including a user interface).
[0017] Computer systems can be compatible with or implemented as part of or through a cloud-based computing system. As used in this paper, a cloud-based computing system is a system that provides virtualized computing resources, software and/or information to client devices. The computing resources, software and/or information can be virtualized by maintaining centralized services and resources that the edge devices can access over a communication interface, such as a network. “Cloud” may be a marketing term and for the purposes of this paper can include any of the networks described herein. The cloud-based computing system can involve a subscription for services or use a utility pricing model. Users can access the protocols of the cloud-based computing system through a web browser or other container application located on their client device.
[0018] A computer system can be implemented as an engine, as part of an engine, or through multiple engines. As used in this paper, an engine includes at least two components: 1) a dedicated or shared processor or a portion thereof; 2) hardware, firmware, and/or software modules executed by the processor. A portion of one or more processors can include some portion of hardware less than all of the hardware comprising any given one or more processors, such as a subset of registers, the portion of the processor dedicated to one or more threads of a multi-threaded processor, a time slice during which the processor is wholly or partially dedicated to carrying out part of the engine’s functionality, or the like. As such, a first engine and a second engine can have one or more dedicated processors, or a first engine and a second engine can share one or more processors with one another or other engines. Depending upon implementation-specific or other considerations, an engine can be centralized, or its functionality distributed. An engine can include hardware, firmware, or software embodied in a computer system for execution by the processor. The processor transforms data into new data using implemented data structures and methods, such as is described with reference to the figures in this paper.
[0019] The engines described in this paper, or the engines through which the systems and devices described in this paper can be implemented, can be cloud-based engines. As used in this paper, a cloud-based engine is an engine that can run applications and/or functionalities using a cloud-based computing system. All or portions of the applications and/or functionalities can be distributed across multiple computing devices and need not be restricted to only one computing device. In some embodiments, the cloud-based engines can execute functionalities and/or modules that end users access through a web browser or container application without having the functionalities and/or modules installed locally on the end-users’ computing devices.
[0020] As used in this paper, datastores are intended to include repositories having any applicable organization of data, including tables, comma-separated values (CSV) files, traditional databases (e.g., SQL), or other applicable known or convenient organizational formats. Datastores can be implemented, for example, as software embodied on a general- or specific-purpose machine, in firmware, in hardware, in a combination thereof, or in an applicable known or convenient device or system. Datastore-associated components, such as database interfaces, can be considered “part of” a datastore, part of some other system component, or a combination thereof, though the physical location and other characteristics of datastore-associated components is not critical for an understanding of the techniques described in this paper.
[0021] Datastores can include data structures. As used in this paper, a data structure is associated with a way of storing and organizing data in a computer so that it can be used efficiently within a given context. Data structures are generally based on the ability of a computer to fetch and store data at any place in its memory, specified by an address, a bit string that can be itself stored in memory and manipulated by the program. Thus, some data structures are based on computing the addresses of data items with arithmetic operations; while other data structures are based on storing addresses of data items within the structure itself. Many data structures use both principles, sometimes combined in non-trivial ways. The implementation of a data structure usually entails writing a set of procedures that create and manipulate instances of that structure. The datastores, described in this paper, can be cloud-based datastores. A cloud based datastore is a datastore that is compatible with cloud-based computing systems and engines.
[0022] The network can be an applicable communications network, such as the Internet or an infrastructure network. The term “Internet” as used in this paper refers to a network of networks that use certain protocols, such as the TCP/IP protocol, and possibly other protocols, such as the hypertext transfer protocol (HTTP) for hypertext markup language (HTML) documents that make up the World Wide Web (“the web”). More generally, a network can include, for example, a wide area network (WAN), metropolitan area network (MAN), campus area network (CAN), or local area network (LAN), but the network could at least theoretically be of an applicable size or characterized in some other fashion (e.g., personal area network (PAN) or home area network (HAN), to name a couple of alternatives). Networks can include enterprise private networks and virtual private networks (collectively, private networks). As the name suggests, private networks are under the control of a single entity. Private networks can include a head office and optional regional offices (collectively, offices). Many offices enable remote users to connect to the private network offices via some other network, such as the Internet.
[0023] A LAN is a private network that connects devices within a limited area like a residence, an office, a building or a campus. A wired LAN uses Ethernet cables to connect computers together directly or more commonly, through a hub, switch, or router. Wireless networks use radio waves to connect devices such as laptops to the Internet and other hand held devices.
[0024] The live-custom recording teleconferencing system 104 (which can also be referred to as a live-custom recording teleconferencing engine) is intended to represent a teleconferencing system that enables live-custom recording in a teleconference. The live-custom recording teleconferencing system 104 can be implemented in a cloud. Instead or in addition, components of the live-custom recording teleconferencing system 104 can be implemented on the host device 106, the participant devices 108, or other devices (not shown). In an implementation in which the host device 106 and/or the participant devices 108 are client devices, the live-custom recording teleconferencing system includes a teleconferencing server that comprises one or more entities, such as servers, that provide various functionalities for teleconferencing such as transcoding, mixing, switching etc. A conference management engine in the teleconferencing sever system provides functionalities including but not limited to resource allocation, management of participants (e.g. joining and exiting), conference session management and media stream management, and processing for the server-side teleconferencing operation and functionality. The media streams handled by the conference management engine may be, e.g., voice and video streams using RTP and RTP Control Protocol (RTCP). The live-custom recording teleconferencing system 104 may support MCU and/or SFU functionalities dynamically as needed. Several configurations are possible depending on functionalities desired for a teleconferencing environment.
[0025] Video conferencing protocols may be broadly classified into H.323 and SIP. Designed by ITU (International Telecommunication Union) and widely used in IP based videoconferencing, VoIP, and Internet telephony, H.323 provides standards for equipment, computers, and services for multimedia communication across packet based networks and specifies transmission protocols for real-time video, audio and data details. Designed by IETF (Internet Engineering Task Force), SIP (Session Initiation Protocol) is a signaling protocol used to create, modify, and terminate a multimedia session over the Internet Protocol. SIP takes the help of SDP (Session Description Protocol) which describes a session and RTP (Real Time Transport Protocol) used for delivering voice and video over IP networks.
[0026] Network Address Translation (NAT) is a process in which one or more local IP address is translated into one or more Global IP address and vice versa in order to provide Internet access to the local hosts. A firewall is a network security device that monitors incoming and outgoing network traffic and permits or blocks data packets based on a set of security rules. A proxy server is a computer that acts as a gateway between a client and a larger-scale network such as the internet. Proxy servers provide increased performance and security.
[0027] In mesh topology (P2P), the audio/video streams are passed from one user to another user of the same level. The data is passed directly from one user to another user without any intermediate support like the server. Peer-to-Peer topology is easy to use and implement. When the number of participants increases, the quality of the video is degraded and leads to frozen video and cut-offs in sound.
[0028] MCU (Multipoint Control Unit) acts as a gateway in a multipoint videoconferencing system. Each user sends the audio/video streams to the MCU. The MCU does the encoding and decoding. It then combines all the streams into one stream and distributes it to the users. There is one outgoing stream from the user to MCU and one incoming stream from the MCU to the user. MCU supports different users with different devices. It supports protocols such as S.I.P and H.323. It normalizes the audio/video streams from the user. It also filters noise and reduces echo in audio streams. The centralized encoding and decoding at the MCU helps in reducing the bandwidth and computational load on the user side.
[0029] In SFU video conferencing architecture, the audio/video streams from the user are passed to the SFU (Selective Forwarding Unit). The SFU collects the stream and selectively forwards it to the users. There is one outgoing stream from the user to SFU and one or more incoming streams from the SFU to the user.
[0030] SVC (Scalable Video Coding) is a video compression technique that requires less bandwidth for video transmission. An SVC video stream is constructed with a minimum base layer and additional (multiple) enhanced layers. This layering technique requires less bandwidth and has the capacity to recover quickly from error. SVC can support a broad range of devices and networks.
[0031] In Simulcast video conference architecture, different independent versions of the same stream with different resolutions are sent simultaneously from a user. The collected stream is encoded and transmitted with different resolutions and frame-rates ranging from minimal quality to highest necessary quality as required for different users.
[0032] The host device 106 and the participant devices 108 are intended to represent devices used by the host and participants to host or participate in a teleconferencing session. Such devices can include a phone, laptop, desktop, table, or the like. Hardware typically includes a microphone and speaker, headphones, or earphones for voice input/output and a camera or webcam and display device for video input/output. A modem can be used to convert data from one form to another so that it can be transmitted from one computer to another and a codec can be used to compress and decompress data used in voice over IP (VoIP), video conferencing, and streaming media. Video conferencing software enables a host to initiate and conduct live conferences and remote meetings by transmitting audio, video, and text. It often includes features like file sharing, electronic whiteboards, and file transfers. Acoustic Echo Cancellation (AEC) software removes echoes generated by the enclosed loudspeaker from the microphone signal. With web Real Time Communication (webRTC), meetings can be scheduled directly from a browser. Advantages of browser-based video conferencing includes no need to download, store, or update apps; it is easy to launch; it is compatible with most devices, OS, and browsers; and security is handled by the browser.
[0033] In a specific implementation, the host device 106 and the participant devices 108 communicate via a teleconferencing server system e.g. a video conferencing system. The host device 106 and the participant devices 108 can be referred to as “client devices” in an implementation in which they act as clients for a teleconferencing sever system. In such an implementation, a client conference management engine in a client device provides functionalities including but not limited to resource allocation, conference session management and media stream management & processing for client-side teleconferencing operation and functionality. The media streams handled by the client conference management engine may be, e.g., voice and video streams using RTP and RTCP. In some scenarios there may be peer to peer communication and direct conferencing between client devices.
[0034] In an example of operation, using a server-client description for illustrative purposes, via the network 102 in coordination with the live-custom recording teleconferencing system 104, a speaker using the host device 106 and an audience using participant devices 108 participate in a teleconference. An audience member may miss what was communicated by the speaker due to inattentiveness, inadequate or low bandwidth and network connectivity issues that result in choppy audio, a video feed that keeps freezing up, screen sharing failure, and extended, unexplained delays, or for other reasons. Live-custom recorder functionality is useful to capture segments or clips of the teleconference meeting, especially when an audience member feels they have missed something. More generally it provides flexibility and control to an audience member to capture notes or highlights of the teleconference in the form of live-custom recording segments or clips customized to the audience member’s interests. It may be noted that in some teleconferencing sessions, one speaker can be replaced with another speaker (including members of the audience); if applicable, tools for switching speakers are assumed.
[0035] In a specific implementation, an audience member clicks a live-custom recorder button during the teleconference to capture a segment of the teleconference, which causes a recording to begin as of some time before the button was clicked (e.g., a few minutes prior to pressing the live-custom recorder button). Later, such as when the audience member feels that the recording is sufficient, the audience member can stop the recording. Capturing only a few such live-custom recording segments or clips rather than recording the entire meeting will save conserve computer resources, most obviously storage space. In a specific implementation, live-custom recording functionality is also designed to provide options to record in a range of formats, including those that have lower memory and storage requirements e.g. lower bit rate video, audio only, and speech transcription only (e.g. speech to text captioning) formats. Another useful aspect provided in the design of the live-custom recording functionality is that an audience member can have immediate access to live-custom recording segments or clips. These live-custom recording segments or clips can be played in a separate window that can be private or shared with other participants (e.g. the speaker, a subset of the audience members, or all participants). For private review, the participant may also play the live-custom recording segments using any suitable media player that can play the live-custom recording content.
[0036] In a specific implementation, the live-custom recording functionality requires storing a specified maximum window of history of the teleconference continuously, e.g., last 10 min of the teleconference history. The specified maximum window of history can be configured or modified by a teleconference meeting organizer, a server-side administrator in live-custom recording session, or by a client-side participant in live-custom recording session.
[0037] In a specific implementation, when a participant triggers live-custom recording, the participant can, if allowed, specify or select a past duration to record via a user interface in any of the following ways: (a) entering a time duration e.g. 7 minutes (b) choosing from suggested time duration options e.g. 2, 4, 6, 8, or 10 minutes. A default past duration for live-custom recording may be applied in the absence of a default specification or selection by the participant. The value of this default may be configured or modified similar to the specified maximum window of history. Essentially the participant can set the past duration for live-custom recording which is always bounded or limited by the specified maximum window of history. If the specified maximum window of history is updated during the conference, the options of past duration for the live-custom recording presented on the user interface to the participant may need to be updated to stay within the new bound or limit specified by the update. Each of these specifications can occur in real time with immediate implementation of the most recent specification though, depending upon implementation-, configuration-, or preference-specific factors, the window may need time to fill if the newer specification is for a larger window than was previously specified.
[0038] In another scenario when a participant triggers a live-custom recording, the participant may be allowed to choose a future duration to record from the time of live-custom recording request and append to the live-custom recording segment or clip. Essentially the duration of the live-custom recording request contains a past duration and a future duration (as measured) from the time of the live-custom recording request. The design and configuration are analogous to that of the past duration to include in a live-custom recording segment. One point of difference is that the participant may chose the future duration to record dynamically by ending (in real-time) the live-custom recording from the user-interface e.g. via a button for ending a live-custom recording in progress. When a participant triggers live-custom recording, the participant may also be allowed to specify or select a future duration to record (in addition to the past duration) via a user interface in any of the following ways: (a) entering a time duration e.g. 5 minutes (b) choosing from suggested time duration options e.g. 1, 3, 5 minutes. A default future duration for live-custom recording may be applied in the absence of a specification or selection by the participant. The value of this default may be configured or modified at anytime. Essentially the participant can set the future duration for live-custom recording which is always bounded or limited by the specified maximum future window. The specified maximum future window may be configured or modified at anytime by the teleconference meeting organizer or administrator in a server-side live-custom recording functionality and by a participant in a client-side live-custom recording functionality. If the specified maximum future window is updated at anytime during the conference, the options of future duration for the live-custom recording presented on the user interface to the participant may need to be updated to stay within the new bound or limit specified by the update.
[0039] In a specific implementation, a participant can specify via the user-interface, for each live-custom recording request, a type of media format for live-custom recordings such as a video format with a bit rate, an audio only format with bit rate or speech transcription only format (e.g. speech to text captioning). The type of media format may be set in advance and used for all subsequent live-custom recordings until it is updated or changed by the participant.
[0040] In a specific implementation, when a live-custom recording is triggered or requested, a live-custom recording initiation message goes to server along with participant identifier, desired live-custom recording format, selected or specified live-custom recording duration, and a live-custom recording sequence number for the live-custom recording request. The live-custom recording sequence number reflects the chronological order of the live-custom recording requests and segments. It may be used to reorder the live-custom recording segments when they are out of order due to different processing times for the live-custom recording requests. The processing times may be impacted by the length of segments, any transcoding requirements, or delays due to computing resources for the processing being distributed. The participant identifier, live-custom recording sequence number, and a time stamp corresponding to the time of live-custom recording request together form a unique identifier (and also serve as meta data) for a live-custom recording segment.
[0041] In a specific implementation, a participant can specify via the user-interface, for each live-custom recording request, a tag/name for the live-custom recording segment or clip to help with indexing and search ability e.g. using appropriate keywords. The tag or name may be specified along with the live-custom recording request if the participant wishes. A tag or name for a live-custom recording request from a participant may or may not be specified after the live-custom recording request is sent for processing so as to not delay the processing. The tag or name for a live-custom recording is preferably obtained from the participant (via the user interface) as soon as possible (at the time of the live-custom recording request or after the live-custom recording request is sent for processing) so as not to delay processing; it may be desirable for a server-side engine to provide the tag or name if one is not provided expeditiously. The tag or name is associated with the unique identifier for the live-custom recording segment and may or may not be included as part of the name for the live-custom recorder segment. If the tag or name is to be included as part of the name at the time of providing access to a completed live-custom recording, it should be obtained before the completion of the processing for live-custom recording.
[0042] FIG. 2 depicts a server-side flow diagram 200 for maintaining sliding history window. The diagram includes nodes 202, 208, and 210 of a conference management engine and nodes 204, 206, and 212 of a live-custom recorder engine. In a specific implementation, the conference management engine and live-custom recorder engine are implemented as part of the live-custom recording teleconferencing system 104 of FIG. 1. In order to support live-custom recording functionality the live-custom recorder engine stores and maintains a specified maximum window of history. This is a sliding window and data older than or outside this window is discarded. The window can be specified by the teleconference organizer or administrator as a maximum history window for live-custom recording. The live-custom recorder engine stores and maintains data from media streams corresponding to participants of a teleconference for a specified sliding window dynamically. Live-custom recordings may be stored, for example in RTP & RTCP format or transcoded to a convenient/suitable format based on the capabilities of devices used by the participants. For illustrative purposes, storage of live-custom recording segments/clips using RTP & RTCP format as described here, with the understanding storage for live-custom recording segments can be in any desired media type and format. The media content in RTP & RTCP format can be transcoded to another media type and format e.g. MPEG video, if desired.
[0043] The diagram 200 is intended to illustrate shows message flow between a conference management engine and a live-custom recorder engine. The conference management engine notifies the live-custom recorder engine to start a sliding window history (represented by the arrow from the node 202 to the node 204). In one scenario this notification message may be received when the teleconference begins e.g. when a teleconference is started by the organizer. In another scenario this notification message may be received when a participant joins the teleconference. In yet another scenario this notification message may be received when a participant opts for live-custom recording either by activating a corresponding command on the user interface or by responding to a recording preference/poll message (e.g. from the conference management engine).
[0044] When maintenance of the sliding window history begins, the live-custom recorder engine requests packets from the conference management engine (represented by the arrow from the node 206 to the node 208). Start time is computed by subtracting past duration (d) from current time (c). For example, for c = 12:23 PM and d = 5 minutes, start time would be 12:18 PM. Depending upon configuration-, implementation-, or preference-specific factors, the start time can have a different granularity than minutes (e.g., to the second).
[0045] The conference management engine sends packets to the live-custom recorder engine (represented by the arrow from the node 210 to the node 212). The live-custom recorder engine then maintains a sliding history window for a duration specified by the teleconference meeting organizer or administrator. Maintenance of the sliding history window may be stopped under various scenarios (e.g. resource constraints or conservation, participant input choice or conference exiting) on receiving by the live-custom recorder engine (e.g. from the conference management engine ) a notification message for stopping the sliding window history maintenance.
[0046] FIG. 3 depicts a server-side flow diagram 300 for live-custom recording. The diagram 300 includes nodes 302, 306, 312, and 314 of a client device and nodes 304, 308, 310, and 316 of a live-custom recorder engine. The diagram 300 is intended to illustrate a message flow between a client device and a live-custom recorder engine. When a live-custom recording is initiated/triggered (from the user interface of a participant) with a specified duration, a live-custom recording request message is sent from the client device to a teleconferencing server that includes the live-custom recorder engine.
[0047] In a specific implementation, when a participant triggers a start recording, a message containing participant identifier, timestamp, a live-custom recording sequence number, media type preference (e.g., multimedia, audio only, or speech transcription), and format preference is sent from the client device to the live-custom recorder engine (represented by the arrow from the node 302 to the node 304). The message may include a past duration or the past duration can be determined on the server side. The live-custom recorder engine computes the starting time for live-custom recording based on the past duration for live-custom recording (e.g., time of live-custom recording request minus a specified past duration for live-custom recording) and starts recording in association with the participant identifier.
[0048] The live-custom recorder engine stores relevant stream packets from the live-custom recording start time by copying from the available live-custom recorder history and applies any transcoding if necessary (e.g. for a media format, bit rate etc.). When the participant requests stop recording (represented by the arrow from the node 306 to the node 308), the live-custom recorder engine stores the recorded media clip with participant identity and sequence number (which can also include timestamp and tag, if specified). The live-custom recorder engine provides a link of the recorded clip to the client device (represented by the arrow from the node 310).
[0049] The participant assigns a tag for the recorded clip (represented by the arrow from the node 312). Alternatively, the tag could be included in the message from the node 302 or assigned at the server side. In this sense, the flow from the node 312 is optional. If applicable, the participant requests the recorded clip with the tag (represented by the arrow from the node 314). Instead or in addition, the participant can request the recorded clip using some other identification of the clip. The live-custom recorder engine then sends the recorded clip with the tag (represented by the arrow from the node 316).
[0050] Advantageously, the live-custom recorder engine makes the live-custom recording segment or clip identified with a name (e.g. including participant identifier, time- stamp, a sequence number, a tag name if specified) immediately available or accessible to the participant. This access may be provided via a user interface on the client device or on some other device. In some embodiments one or more messages between the client device and live-custom recorder engine are communicated via some other engine e.g. a conference management engine.
[0051] Sharing permissions of a live-custom recording on the server-side can be specified by the participant for each live-custom recording clip. By default a live-custom recording is private in the absence of any permission setting by the participant. However participant can modify permission setting for a live-custom recording by adding and removing permissions for other participants.
[0052] In one scenario the server may only store the active speaker’s media streams. The selection of the active speaker’s media streams may be based on voice activity detection. This would reduce the storage size requirement which is especially useful in client devices with limited memory/storage.
[0053] FIG. 4 depicts a client-side flow diagram for maintaining sliding history window. The diagram includes nodes 402, 408, and 410 of a conference management engine and nodes 404, 406, and 412 of a live-custom recorder engine.
[0054] As teleconferencing starts, the live-custom recorder engine requests media packets from client conference management engine. Sliding history window is maintained by the live-custom recorder engine for a duration specified by the teleconference meeting organizer or administrator. When a teleconference starts, the client conference management engine sends notification to start sliding window history to the live-custom recorder engine (represented by the arrow from node 402 to the node 404).
[0055] The live-custom recorder engine requests media packets from the client conference management engine (represented by the arrow from node 406 to the node 408). With client-side recording, recorded data is stored on the client device. All the recordings done by the live-custom recording functionality has a past duration which cannot be beyond or older than a specified maximum window of history. The specified maximum window of history may be configured or modified anytime. It is also a sliding window and data older than or outside this window is discarded. The teleconference organizer or administrator specifies this maximum history window for live-custom recording. The live-custom recorder engine maintains and stores the media streams corresponding to the participants of the teleconference for the specified sliding window dynamically.
[0056] The client conference management engine sends the media packets to the live-custom recorder engine (represented by the arrow from the node 410 to the node 412) as requested.
[0057] FIG. 5 depicts a client-side flow diagram for live-custom recording. A participant triggers a live-custom recording from a user interface by sending a live-custom recording request message is sent to the client live-custom recorder engine located in the client device. The live-custom recording request message is sent including information such as duration, participant identifier, timestamp, a live-custom recording sequence number, a tag/name if specified, and a preference for media type and format e.g. video, audio only or speech transcription along with relevant format (represented by the arrow from the node 502 to the node 504). The client's live-custom recorder engine initiates recording with the given tag/identifier for the participant. It computes the start time for live-custom recording based on past duration for live-custom recording (which can also be provided in the message or generated by the live-custom recorder engine) and stores the relevant media stream packets from the live-custom recording start time by copying from the available live-custom recorder history. As a default the live-custom recorder engine stores the recorded media clip with participant identity and sequence number. If necessary transcoding is also applied (e.g. for a media format, bit rate etc.).
[0058] When the participant indicates a desire to stop a live-custom recording in the user interface, a request to stop recording is sent to the client engine (represented by the arrow from the node 506 to the node 508). The live-custom recorder engine stops storing information for the live-custom recording segment or clip and makes the live-custom recording segment or clip identified with a name (e.g. including participant identifier, time- stamp, a sequence number, a tag name if specified) immediately made available or accessible to the participant.
[0059] The live-custom recorder engine provides a link (e.g., in the user interface) of the recorded clip to the participant (represented by the arrow from the node 510).
[0060] The participant can assign a tag to the recorded clip (represented by the arrow from the node 512). So later, if the participant wants to access the clip, the participant can request the live-custom recorder engine for the recorded clip using the assigned tag (represented by the arrow from the node 514). The live-custom recorder engine provides the recorded clip with tag in response to the request (represented by the arrow from the node 516). In some embodiments one or more messages between the user interface and live-custom recorder engine may be communicated through an other module or engine e.g. the client conference management engine.
[0061] In client side processing, permission for sharing the live-custom recording can be specified by the participant for each live-custom recording clip. By default a live-custom recording is private in the absence of any permission setting by the participant. However participant can modify permission setting for a live-custom recording by adding and removing permissions for other participants.
[0062] In one scenario the client may only record and store the active speaker’s media streams. In that case the selection of the active speaker’s media streams may be based on voice activity detection. This would reduce the storage size requirement for client devices in user interface with limited memory/storage. In another scenario if the client side is experiencing bandwidth issues (low/no bandwidth) then the system automatically triggers to change the recording process to the server side.
[0063] A live-custom recorder engine providing one or more of the live-custom recording functionalities described herein may be implemented on one or more client devices in the teleconference system. A notification message is received (e.g. from a client conference management engine located in the client device) by the live-custom recorder engine to start a sliding history window and its maintenance. In one scenario this notification message may be received when the teleconference begins e.g. when a teleconference is started by the organizer. In another scenario this notification message may be received when the participant associated with the client device joins the teleconference. In yet another scenario this notification message may be received when the participant associated with the client device opts for live-custom recording either by activating a corresponding command on the user interface or by responding to a recording preference/poll message (e.g. from the conference management engine). The maintenance of the sliding history window may be stopped under various scenarios (e.g. resource constraints or conservation, participant input choice or conference exiting) on receiving by the live-custom recorder engine (e.g. from the client conference management engine) a notification message for stopping the sliding window history maintenance.
[0064] FIG. 6 depicts a diagram 600 of engines of a live-custom recording teleconferencing system. The diagram 600 includes a conference management engine 602, a live-custom recorder engine 604, and a client device or user interface 606. The conference management engine 602 can be implemented server-side, client-side, or on an end user device. The live-custom recorder engine can be implemented server-side, client-side, or on an end user device. In an implementation in which the conference management engine 602 is implemented server-side, the client device or user interface 606 is implemented client-side. In an implementation in which the live-custom recorder engine is implemented server-side, the client device or user interface 606 is implemented client-side.
[0065] In operation, the conference management engine 602 sends a notification to start sliding window history to the live-custom recorder engine 604, as described previously; receives a request for media packets from the live-custom recorder engine 604, as described previously; and sends media packets to the live-custom recorder engine 604 in response thereto, as described previously. Conversely, the live-custom recorder engine 604 receives a notification to start a sliding window history from the conference management engine 602, as described previously; requests media packets from the conference management engine 602, as described previously; and receives media packets from the conference management engine 602, as described previously. This behavior is the same regardless of whether one or both of the conference management engine 602 and the live-custom recorder engine 604 are server-side, client-side, or implemented on an end-user device.
[0066] In operation, the live-custom recorder engine 604 computes a start time by subtracting past duration from current time, as described previously; initiates a live-custom recording of a teleconference from the start time, as described previously; stops the live-custom recording upon receipt of a request to stop recording, as described previously; provides a link of the live-custom recording to a teleconference participant, as described previously; and sends the live-custom recording to the teleconference participant, as described previously. This behavior is the same regardless of whether the live-custom recorder engine is server-side, client-side, or implemented on an end-user device. However, the client device or user interface 606 is properly characterized as a user interface (and not a client device) if the live-custom recorder engine 604 and the client device or user interface 606 are on the same end-user device.
,CLAIMS:WE CLAIM:
1. A system comprising one or more processors and memory storing instructions that, when executed by the one or more processors, cause the system to perform:
computing, at a live-custom recorder engine, a start time by subtracting past duration from current time;
initiating a live-custom recording of a teleconference from the start time;
stopping the live-custom recording upon receipt of a request to stop recording;
providing a link of the live-custom recording to a teleconference participant;
sending the live-custom recording to the teleconference participant.

2. The system of claim 1, comprising instructions that, when executed by the one or more processors, cause the system to perform:
receiving, at the live-custom recorder engine, a notification to start a sliding window history;
requesting media packets from a conference management engine;
receiving media packets from the conference management engine at the live-custom recorder engine.

3. The system of claim 1, wherein the conference management engine includes a client-side conference management engine.

4. The system of claim 1, wherein the conference management engine includes a server-side conference management engine.

5. The system of claim 1, wherein the start time is computed, the initiating a live-custom recording, and the stopping the live-custom recording are performed on a client device.

6. The system of claim 1, wherein the start time is computed, the initiating a live-custom recorded, and the stopping the live-custom recording are performed on a server-side live-custom recorder engine.
7. The system of claim 1, comprising instructions that, when executed by the one or more processors, cause the system to perform tagging the live-action recording.

8. The system of claim 7 wherein the tag is received from a client device prior to the providing a link of the live-custom recording to a teleconference participant.

9. The system of claim 7 wherein the tag is provided by the live-custom recorder engine.

10. A method comprising:
computing, at a live-custom recorder engine, a start time by subtracting past duration from current time;
initiating a live-custom recording of a teleconference from the start time;
stopping the live-custom recording upon receipt of a request to stop recording;
providing a link of the live-custom recording to a teleconference participant;
sending the live-custom recording to the teleconference participant.

11. The method of claim 10, comprising:
receiving, at the live-custom recorder engine, a notification to start a sliding window history;
requesting media packets from a conference management engine;
receiving media packets from the conference management engine at the live-custom recorder engine.

12. The method of claim 10, wherein the conference management engine includes a client-side conference management engine.

13. The method of claim 10, wherein the conference management engine includes a server-side conference management engine.

14. The method of claim 10, wherein the start time is computed, the initiating a live-custom recording, and the stopping the live-custom recording are performed on a client device.
15. The method of claim 10, wherein the start time is computed, the initiating a live-custom recorded, and the stopping the live-custom recording are performed on a server-side live-custom recorder engine.

16. The method of claim 10, comprising instructions that, when executed by the one or more processors, cause the system to perform tagging the live-action recording.

17. The method of claim 16 wherein the tag is received from a client device prior to the providing a link of the live-custom recording to a teleconference participant.

18. The method of claim 16 wherein the tag is provided by the live-custom recorder engine.

19. A system comprising:
a means for computing a start time by subtracting past duration from current time;
a means for initiating a live-custom recording of a teleconference from the start time;
a means for stopping the live-custom recording upon receipt of a request to stop recording;
a means for providing a link of the live-custom recording to a teleconference participant;
a means for sending the live-custom recording to the teleconference participant.

20. The system of claim 19, comprising:
a means for receiving a notification to start a sliding window history;
a means for requesting media packets from a conference management engine;
a means for receiving media packets from the conference management engine.