Tracking Mouse Locally And Microsoft Windows Connectivity Algorithm

Abstract

The present invention relates to an integrated convergence platform for voice, video and data applications including computing, communication and entertainment applications adapted for local mouse implementation facility at the user's access terminal / device involving embedded system having very constrained resources such as small processor and memory. The invention is also directed to windows desktop connectivity for such embedded systems having constrained resources and with or without operating system.

Information

Applicants

Specification

TRACKING MOUSE LOCALLY AND MICROSOFT WINDOWS CONNECTIVITY
ALGORITHM
FIELD OF THE INVENTION
The present invention relates to an integrated convergence platform for voice, video and data applications including computing, communication and entertainment applications adapted for local mouse implementation facility at the user's access terminal / device involving embedded system having very constrained resources such as small processor and memory. Importantly, the local mouse implementation facility is adapted to provide a combination of VNC technology involving the integrated convergence platform and graphic operation technology, which would enable the user to not only implement and emulate the server screen at the user's access terminal by decompressing and display of the screen but also enable carrying our graphic operation. The invention is also directed to windows desktop connectivity for such embedded systems having constrained resources and with or without operating system. Importantly, the invention would favour wide scale and user friendly application and use of integrity convergence platform including a local mouse implemented facility and also favour simple and ready connectivity of windows desktop from embedded systems without dependencies on high load consuming and complex X-servers.
BACKGROUND ART
It is well-known that technology platforms are available for each of the computing-communication-entertainment such as mobile platforms for voice applications, integrated device platforms for voice and data application.
However, mostly such devices or application presently available in the market are all I personal computer based, Video on Demand Set-Top Boxes, Residential Gateways and

Integrated Access Devices. Importantly, such presently available platforms such as the video on demand set top boxes are only capable of providing single function. Such devices which are application specific and personal computer based, are complex to use for mass adaptation. There are a few devices presently available in the market which are capable of handling any two of the above functions but a device that could provide simplicity in operation and use and yet facilitate triple play application selected from computing, communication and entertainment is highly desirable for the users of such system.
In our CO pending PCT Intemational Application No. PCT/IN2005/000380 dated November 25, 2005 there disclosed an integrated simplified convergence platform for triple play applications including computing, communication and entertainment applications comprising
a simple & single platform providing (i)means for computational application ; (ii)means for entertainment applications; and (iii) means for voice and data applications;
plurality of customized user access terminals/devices operatively connected to said common platform for said desired triple play applications.
The above integrated technology platform rests on the server and provides access to the user for said computing or media applications. Importantly, to facilitate the management of the plurality of the user access devices a device management console is available. The said user access terminals /devices are free of any moving components and/or heat generating units.
In our said co-pending application there is also disclosed the integrated access system platform comprising of:

plurality of user access terminal/devices in a server centric Computing environment; a LAN and/or WAN operating connection to server and a device management console.
The server cluster used in the system can be selectively comprised of Content servers and Computing Servers, or the functionality of the cluster could be provided by a single server.
Further, client management software is also provided for managing multiple simplified user terminals / devices in the system. The software is embedded in the hardware to impart desired operational capability to the hardware of the simplified convergence platform of the
invention.
It would be apparent from the above that the convergence platform imparts simplicity and user friendly characteristics to the operation / use of triple play gadgets / components by way of providing for the first time very simple appliance like attributes to the access terminal/devices in the system of the invention. Such appliance like attributes of the access terminals / devices would enable deployment of multiple access terminal / devices in hundred or thousands with very simple to manage network set up. Importantly, the system is adapted such that there is no additional burden or complexity of managing typical personal computer for individual work station.
While in our above discussed said co-pending PTC International application there is disclosed a simple and single technology platform to host variety of access devices selected from computing, communication and entertainment applications to facilitate multiple functions from common platforms and the server has been provided to operatively connect to such user's access terminal PCs to emulate the server screen at the user's access terminal, in the said system the mouse ftinctionalities are essentially carried out in the server i.e. in the

thick server and thin client architecture using VNC technology. In such system the mouse events are sent to server and the changes happen at the server end and then it is reflected at the client's user terminal. The mouse response is thus a part of the screen update to the client. Because of this, there has been a delay involved in moving the mouse and seeing the update on the screen. This delay depends on the network traffic and also on the server response.
It would be apparent from the above that the system involving the mouse update from the server that there is necessarily delay involved between the movement of the mouse by the user and the actual update of the screen. Thus the mouse movement is not synchronized with respect to the hand and the eyes of the user. Also when there is a heavy flash or screen update, the traffic in the network is heavy thus making the mouse response further slow. The mouse movement generated by the user in such system is thus almost ignored. It is thus very painful situation for the user as he loses control over the screen temporary.
It is therefore experienced that the above problem of mouse implementation in turn affected the application and integrities of the simplified convergence platform with multiple access terminal involving PC with constrained resources. The problem as discussed above is basically because the system of the convergence platform is a server based computing product. Importantly, it is found that though the server supported the local mouse, the packet format did not support the user terminal requirements and resulted in the above loss of control over the screen by the user due to delay in the movement of the mouse and its actual update on the screen based on server input.
Moreover, apart from the above problems of mouse implementation it is found that windows desktop connectivity from embedded systems including in the simplified convergence

platform in our said co-pending application, the windows desktop connectivity from such embedded system having very constrained resources such as small processor and small memory but involving dependencies on X-server required the running of the system with heavy application like X-server. Such required dependence of the system on the X-server in environments especially with embedded systems with constrained resources is found to be taxing and not desirable. There has thus been the need in the art to eliminate such dependencies on X-11 and X-server especially for achieving windows desktop connectivities from embedded system having limited and constrained resources.
In particular, in such presently available windows desktop connectivity from an embedded platform , the application have a communication protocol that interacts with windows server and uses X-server on client*s side for providing input and output. Such existing applications work well for large processor architecture like normal X-86 based platform, since such platforms have enough resources in terms of processor's speed and memory to support a heavy resource consuming application like X-server. X-servers are thus required by existing applications for input and output which is found to be lot taxing for embedded system with limited resources to support X-server which require lot of memory and processor's cycles.
OBJECTS OF THE INVENTION
It is thus the basic object of the present invention to provide for simplified convergence platform adapted to facilitate triple play application / use including computing, communication and entertainment which would enable operative simple and user friendly access terminal having facilities for implementing the local mouse technology and thus avoid any afore discussed problems and complexities due to any possible loss of control over the screen in the server based mouse applications/response.

Another object of the present invention is directed to providing the local mouse implementation as a combination of VNC technology and graphic operation technology, which would not only favour the user terminal / PC in embedded systems to emulate the server screen by de-compressing and displaying of the screen but also favour graphic operation.
Yet another object of the present invention is directed to local mouse implementation as a combination of VNC technology and the graphic operation technology.
A further object of the present invention is directed to a system where the mouse updates from the server and delay involved between the movement of the mouse by the user and the actual update of the screen could be avoided and the mouse movement synchronized with respect to the hand and the eyes of the user.
Yet further object of the invention is directed to achieving the mouse movement synchronized with the users movement of the mouse even in case of heavy flash or screen update and in environments where the traffic in the network is heavy and therefore take care of the problems of the user losing control over the screen due to delay action of mouse movement.
Yet further object of the present invention is directed to a server based computing product wherein the server is adapted to support local mouse movement and corresponding action.
A further object of the present invention is directed to capturing and drawing mouse movement locally and thereby solving the bottleneck of the mouse response being dependent of the network traffic and the server.

A further object of the present invention is directed to simple and cost-effective windows desktop connectivity from an embedded system having very constraint resources such as small processor and memory, which would specifically remove the dependency of X-server and be very useful in embedded environment with constant resources where it is taxing to run heavy application like X-server.
Yet another object of the present invention is directed to a windows desktop connectivity from an embedded system having very constraint resources, small processor and small memory, which would enable connecting embedded devices to support windows desktop connectivity thus saving cost on additional resources such as processing capabilities, memory and the like.
Yet further object of the present invention is directed to windows desktop connectivity from an embedded system, which can be used as an architectural model for thin client having minimal processing capabilities.
SUMMARY OF THE INVENTION
Thus according to the basic aspect of the present invention there is provided an integrated convergence platform with local mouse implementation facility adapted for voice, video, and data applications including computing, communication and entertainment applications comprising:
plurality of user access terminal/devices having respective network processor with constrained resources with or without additional network processors of varied capabilities in a server centric Computing environment;

a LAN and/or WAN operating connection to server means comprising a network server and a media server;
a local mouse implementation facility comprising said media processor adapted to store locally the initial mouse shape received form the server; said network processor adapted to capture the coordinates whenever the user moves the mouse and to forward the coordinates to the said media processor and the server; said media processor adapted to draw the mouse on the screen with respect to the x and y coordinates ;said server adapted to recognize any change in shape of the mouse and forward the pattern and mask to the to the network processor; the network processor further adapted to receive the same as a separate mouse packet and forward the data to the media processor; and said media processor adapted to recognize and handle the mouse packet and draw the packet on the screen as a graphics operation.
In the above integrated convergence platform with local mouse implementation facility the network server is adapted to provide other screen updates as a regular VNC screen update and is decoded and displayed by the client/network processor. Also, the network server is adapted to support such as 2 byte mask information for communicating with said access terminal/devices having respective network processor with constrained resources. Also, the network server is adapted to strip the unused portion of the mouse area and forward only the modified area for mouse drawing operations to said access terminal/devices having respective network processor with constrained resources. Moreover, the network server is modified only to the extent of supporting the access terminal/devices having respective network processor with constrained resources.

According to further aspect in the integrated convergence platform with local mouse implementation facility the client/network processor is adapted to forward mouse packet to the server and also use the data within itself for drawing the mouse. Importantly also, the client/network processor is adapted to identify the mouse packet and the screen updates as two different packet types and handles it differently such that when the packet is corresponding to the mouse packet it does graphic operation and draws the mouse locally , otherwise , the screen update packet is drawn as in the original VNC protocol.
Advantageously, in the integrated convergence platform with local mouse implementation facility it is possible to achieve server scalability for said access terminal/devices having respective network processor with constrained resources and also all other clients connected to the same server through additional network processors of varied capabilities.
The integrated convergence platform with local mouse implementation facility comprises said platform having embedded customized software for providing computing, entertainment and voice capabilities. Preferably, a device management console is provided for managing said plurality of user access terminals/devices.
The said user access terminals/devices are free of any moving components and said device operates menu based graphical user interface (GUI).The said device connects to an operating system server also providing computing applications as one of its featiires. The device comprises embedded software specifically developed to control and provide the said features and are equipped with specially developed management software for changing configuration parameters and/or updating the device configuration. Such devices are application class computer access terminals with multimedia capabilities.

The said server means can comprising Content servers and Computing Servers.
In accordance with an aspect the integrated convergence platform with local mouse
implementation facility comprises any or all of the following depending on the users'
application requirement -
General Purpose Processor having random access memory means;
media processor having random access memory means;
flash storage means for storing software codes; and
system bus for operative connection of various components with the central processing units.
The system can also be adapted for embedded software and / or hardware as an interface to smart card reader and /or biometric sensor means to secure access to the computing /content as provided by the platform.
In accordance with another aspect the integrated convergence platform with local mouse
implementation facility comprising the following hardware blocks -
Ethernet MAC connectivity;
Universal Asynchronous Transmitter and receiver means;
Serial Peripheral Interface;
SPORT means;
Peripheral Component Interconnect Bus means;
USB Controller;
Audio Coder-Decoder means;
Video Digital to analog converter and / or VGA Controller;
Speaker/MIC means;

Interface to Monitor and / or TV;
A power source,
One Voice over Internet Protocol (VoIP) Port
An optional PCI expansion slot, modem, Ethernet switch, and
Infra-Red Receiver
In accordance with yet another aspect of the invention there is provided a computer readable information recording medium which records a program /algorithm for providing an integrated convergence platform for voice, video, and data applications including computing. communication and entertairmient applications with local mouse implementation facility as discussed above.
According to yet further aspect of the present invention there is disclosed a remote windows desktop connectivity from embedded systems with constrained resources comprising:
a remote desktop protocol operatively connecting to windows terminal server at one end and to a wrapper means at the other end ;
said wrapper means disposed there between said remote desktop protocol and a custom display hardware device having embedded graphical operations.
In the above remote windows desktop connectivity for input events, the remote desktc^p protocol is adapted to directly communicate with said hardware and comprises a key map wrapper for identifying the key strokes and converting the same to necessary format.

In particular, the remote windows desktop connectivity comprises a remote desktop protocol network layer adapted for interacting with Windows Terminal Server, a remote desktop protocol decoder means adapted to decode the data that passes to the wrapper means, said wrapper means provided as an interface between said display device and said decoder means.
The remote desktop protocol display device comprises a graphical Operations Core operatively connected to a Display Driver and a Device Cache said Display Driver in turn operatively connected to an output means whereby the core interacts with the Display Drivci to pass the output to the appropriate output device. The graphical operations core comprises all graphical operations including preferably memory blitting, text drawing.
The embedded system for remote connectivity can have no operating system or adapted to run on embedded Linux operating system.
According to an aspect the remote windows desktop connectivity can comprise plurality of said remote desktop protocols and remote user access desktop device comprising embedded customized software for providing computing, entertainment and voice capabilities.
Preferably, a device management console for managing said plurality of user access terminals/devices is provided. Advantageously, the said user access terminals /devices arc free of any moving components and the device operates menu based graphical user interface (GUl),The said device connects to said operating system server also providing computing applications as one of its features. Importantly, device comprises embedded software specifically developed to control and provide the said features. In particular, such devices are equipped with specially developed management software for changing configuration

parameters and/or updating the device configuration. Preferably, the devices are application class computer access terminals with multimedia capabilities.
The said windows terminal server means can comprising Content servers and Computing Servers.
The remote windows desktop connectivity can be included in systems comprising any or all
of the following depending on the users' application requirement -
General Purpose Processor having random access memory means;
media processor having random access memory means;
flash storage means for storing software codes; and
system bus for operative connection of various components with the central processing units.
The remote windows desktop connectivity including remote access protocol adapted for embedded software and / or hardware as an interface to smart card reader and /or biometric sensor means to secure access to the computing /content as provided by the platform.
The said remote windows desktop connectivity include remote access protocol with the
following hardware blocks -
Ethernet MAC cormectivity;
Universal Asynchronous Transmitter and receiver means;
Serial Peripheral Interface;
SPORT means;
Peripheral Component Interconnect Bus means;
USB Controller;
Audio Coder-Decoder means;

Video Digital to analog converter and / or VGA Controller;
Speaker/MIC means;
Interface to Monitor and / or TV;
A power source,
One Voice Over Internet Protocol (VoIP) Port
An optional PCI expansion slot, modem, Ethernet switch, and
Infra-Red Receiver
In accordance with another aspect there is provided a computer readable information recording medium which records a program /algorithm for providing remote windows connectivity from an embedded platform using the said remote windows desktop connectivity as detailed above.
It is thus possible by way of the above system of local mouse implementation to impart better and effective user-friendly characteristics to thin client computing system. In particular, the above system of the invention involving the local mouse implementation facility would provide for a combination of VNC technology and graphic operation technology in thin client computing environment. Thus the disadvantages and limitation in effecting transfer of mouse update from the server and delay involved between the movement of the mouse by the user and the actual update on the screen can be avoided. Also, the system architecture of the invention would enable achieving mouse movement generated by the user almost instantaneously without requiring the user to lose control over the screen temporarily. The system would take care of constraints of thin client computing environments such as the integrated convergence platform for voice, video and data application and bring in the much required compatibility between the server and the local access /terminal and overcome the

bottleneck of implementing the local mouse technology due to unsupported packet format of existing VNC technology.
Moreover, the constraints of access terminal / PC with limited resources are found to have problems to process data due to memory limitation. This has been taken care of at the server by stripping the unused portion of the mouse area and sending only the modified area for the mouse drawing operation. Also the server side changes are adapted such that it does not affect other regular X-86 client's connected to it and only the user terminal / PC with limited resources have the effect of such change in the server configuration.
In accordance with the further aspect of the invention the windows desktop connectivity from an embedded system having constraint resources such as small processor and small memory have been achieved in an maimer whereby it is possible to avoid the dependencies on X-server and make for the much desired effective use of embedded environment with constraint resources where it is taxing to run heavy applications like X-server, Using such system ii would now be possible for the embedded devices to be able to support windows desktop connectivity, thus saving cost on additional resources such as processing capabilities, memory etc.
Advantageously, the above remote desktop protocol eliminates any known dependency on X-11 and X-server. The wrapper between the present RDP and a custom display hardware has been provided for such purpose in the system. The graphical operations are directly embedded on disk play hardware. For input events too the present RDP communicates directly to hardware and has a key map wrapper for identifying the keystrokes and converting the same to the necessary format.

Thus by way of the above system the embedded devices would be able to support windows desktop comiectivity thus saving cost on additional resources.
The details of the invention, its object and advantages are explained hereunder in greater detail in relation non-limiting exemplary illustration as per the following accompanying figures
BRIEF DESCRIPTION OF THE ACCOMPANYING FIGURES
Figure 1: Illustrates a multimedia capable server based computing environment provided by the convergence platform as in our co-pending PCT International application;
Figure 2: Illustrates a system block diagram describing the various high level system blocks as used in the system of our said co-pending application;
Figure 3: is a block diagram illustrating the mouse movement in a typical existing VNC protocol as per the existing system;
Figure 4: is a block diagram illustrating the mouse movement involving the VNC technology and graphic operation technology in accordance with the present invention for use with the embedded systems in particular the systems as per figures 1 and 2 above;
Figure 5: is an illustration of the architecture diagram of remote desktop application with X-server of the existing art;
Figure 6: is an illustration of architecture diagram of remote desktop application without X-server and involving the wrapper in accordance with the present invention; and

Figure 7: is an illustration of the display device used in the system involving the remote desktop application with wrapper and without X-server in accordance with the invention.
Reference is first invited to accompanying figure 1 which illustrates the provision of thin client computing environment in accordance with the system providing for the simple and effective operative connection to plurality of access terminal / devices and the triple p!ay application area comprising content server [2], computing server [3] etc.
As clearly illustrated in Figure 1 the simplified convergence platform provides for the much desired server centric computing environment by operative communication with plurality of access terminal / devices (AT) [6] and with the required servers for the triplet play applications including the content server (CS) [2] and computing server (CS) [3], As further illustrated in same figure, the management of the plurality of the access terminal (AT) [6| is achieved by means of the device management console (DMC) [5], Importantly, such device management console (DMC) [5] is simple and user friendly since the same is required to only manage very simplified access terminal (AT) [6] which do not use any moving parts / components or have any heat generating resources. As also illustrated in said figure, llic operative connection of the convergence platform to the operative servers and the management console is attend through LAN / WAN environment.
Reference is now invited to accompanying figure 2 which illustrates the system block diagram describing the various high level system blocks. As apparent from said figure, the system basically comprises of General Purpose Processors (GPP) [7] to communicate over the network, handle tasks and manage the peripheral interfaces and a media processor (MP) [8] which is also a central processing unit adapted to handle media specific part of ihc platform.

The memory block (RAM) [9] indicates random access memory modules used by one or both the processors [8]. It is required for temporary calculation for running programs,
A flash means (FM) [10] which is basically a persistence storage area for software code that is also used along with optional provision of a smart card reader (SMR) [11] for high security environments.
The various high level system blocks used in the convergence platform and its purposes are discussed hereunder in greater detail.
The General Purpose Processor (GPP) [7] is a general purpose CPU (Central Processing Unit). Its main function is to communicate over the network, handle tasks and manage the many peripheral interfaces available to it. It may or may not be present in all configurations of the hardware platform described, In which case the Media Processor [8] takes over the general purpose processing.
The Media Processor (MP) [8] is the CPU that handles the media specific part of the platform and interfaces to output devices like TV, monitor [34], speaker [33], etc. For example, the access devices [6] in the present system, when used in a server centric environment, have full multimedia capability unlike thin clients, allowing the use of the device as a media access client or as Video conferencing equipment. Where the General Purpose Processor [7] is unavailable in certain configurations, it takes over its role as well.
The System Bus (SB) [13] is a set of wires that connects together various components with the CPU.

A Smart Cards Reader means (SCR) [11] is preferably provided to read authentication cards such as Credit Cards, ATM Cards, Identity Cards, Access Cards, etc. In high security environments, where typically a user needs to be identified with much more than a user name and password, this can be used to check if a user is actually producing a valid card. The Smart Card Reader (SCR) [11] is basically a hardware device used in the system that is used to read information from Smart Cards [11].
The Simplified Convergence Platform design offers the following hardware blocks ~
The Ethemet MAC (EMAC) [14] or the EMAC [14] which is the Ethernet controller that is responsible for the hardware connectivity part of the Networking Subsystem.
Universal Asynchronous Transmitter and Receiver (UART) [16] that is used for serial communication.
The Serial Peripheral Interface (SPI) [18] is a serial bus used to communicate with other SPl [18] capable devices. Since the media processor [8] has this capability, it is used to communicate between the General Purpose [7] and Media processors [8].
Means identified as (SPORT) [19] comprising a Serial port. In the system, it is used for inter-processor communications.
The Peripheral components Interconnect Bus (PCI) are an industry standard bus that can be used to connect various commercially available expansion slots that add functionality to the system.

The component USB COntr [23] is the USB controller that helps communicate between the General Purpose Processor [7] and various USB devices. In the system, the USB controller [23] controls devices such as the Mouse [28] , the Keyboard [27] , the Fingerprint scanner [29] used for user authentication and Web Camera [26] used for Video Conferencing.
The (PCI) expansion slot [24] is an optional block that can be present to accommodate commercially available hardv^are expansion slots like graphics cards or sound cards.
The Audio CODEC [31] is an Audio Coder-Decoder that helps with digital audio encoding and decoding. Internally, the system, like all computers deals with digital signals. But any devices like speakers [33] and VGA monitors [34] need analog signals. The Video DAC [32] or Digital to Analog converter is one such device that converts the digital signals delivered by the Media processor [8] into analog signals that the VGA monitor can display.
The system further includes speakers/mic (SP) [33] which is a normal set of speakers and microphone used commonly in computers. Also monitors (MN) [34] comprise a regular PC monitor.
The system is powered by a power supply (PS) [35] which is the unit that supplies DC power to the whole system.
Reference is now invited to Figure 3 which illustrates the mouse movement and the cause of delayed response from the server in the existing VNC protocol as per accompanying Figures 1 and 2. As shown in the figure the mouse movement is sent to the server (SV) and mouse update is also a part of the screen update. Thus since the mouse update is from the server (SV) in such existing system there is delay involved between the movement of the mouse by

the user and the actual update of the screen. As a result, mouse movement is not synchronized with respect to the hand and eyes of the user (CL1/CL2)). This is more in case of heavy flash or screen updates where the traffic in the network is heavy which lead to further slowing of the mouse response in such conventional system.
Reference is now invited to Figure 4 which illustrates the manner of achieving an integrated convergence platform with local mouse implementation facility in accordance with the present invention. As desired, the block at the client shows that the client (CLl) is sending the mouse packet to the server (SV) as well as using the data within itself tor drawing the mouse. Only when there is a change in mouse shape, the server is adapted to send the new shape to the client as a separate packet.
The client (CLl) identifies the mouse packet and the screen update packet as to different packet type and handles it differently. If the packet is corresponding to the mouse packet, it does graphic operation and draws the mouse locally. Otherwise the screen update is drawn as in the original VNC protocol. The methodology of operation of thin client computing systems as per Figures 1 and 2 with possible local mouse implementation in accordance with the present invention is detailed further hereunder:
The initial mouse shape is received from the server and stored locally in the media processt^i. Whenever the user moves the mouse, the networks processor captures the coordinates. The processors send the coordinates to both the media processor and the server.
The media processor draws the mouse on the screen with respect to the X and Y coordinates send by the network processor. If there is a mouse shape change, a server recognizes the change and sends the pattern and the mask to the client. The client (network processor)

receives this as a separate mouse packet and gives the data to a media processor. The media processor recognizes and handles the mouse packet and draws the packet on the screen as it it is graphic operation.
The other screen update comes from the server as a regular VNC screen updates and is decoded and displayed by the client. Since the mouse movement is recognized and handled on priority basis, the mouse movements are faster thus the technology solves the problem in delay in mouse response due to network traffic. This achieves a PC like experience for the user.
At the server side, the X-11 cursor option sends the mask as 1 bit information. Whereas packet type use in the access terminal PC supports only 2 byte information. Hence thai is modified in the server to support 2 byte mask information. Also, the access device / PC of the system which has constraints to process only 1,000 words of data for the mouse packet due to memory limitation. This is taken care at the server by stripping the unused portion of the mouse area and sending only the modified area for mouse drawing operation. Also the servei side changes are done such that it does not affect the regular X-86 client connected to it but only the clients/ user terminal PC with constraint resources have the effect of this change.
Reference is now invited to accompanying figure 5 which shows a conventional Remole Desktop Protocol (RDP) which communicates with X-server (XSV) with the help of X-11 (Xll) library. X-11 serves as a wrapper between X-clients and X-server, isolating the underlying graphical implementation for application calls, X-server initiates input / ouipui devices and waits for request from X-clients. The X-clients like remote desktop connects to the X-server (XSV) and starts sending graphical operations such as text, bitmaps, blitting ele.

X-server is also responsible for receiving inputs from input devices like mouse, keyboard etc. on behalf of the X-clients,
The remote desktop protocol avoiding the dependency on X-11 (XI1) and X-server (XSV) in accordance with the present invention is illustrated in accompanying figure 6. As apparent from said Figure 6 in the present system a wrapper (WR) between the remote desktop protocol and custom display hardware (CDH) has been provided. This enables the graphical operations directly embedded on the display hardware. For input events too, the remote desktop protocol involving the wrapper means of the invention directly communicates with hardware and has a key map wrapper for identifying the key strokes and converting the same to necessary format.
As clearly apparent from the Figure 6, in case of remote desktop protocol involving wrapper (WR) of the invention, the remote desktop protocol network layer (RDPNL) is responsible for interacting with windows terminal server (WTSV) . The remote desktop protocol decoder is adapted to decode the data that passes to wrapper, and interface between the architecture as clearly illustrated further in figure 6.
Reference is now invited to accompanying figure 7 which illustrate by way of block diagram the display device architecture used in the system of the invention, As shown in the figure 7 the graphical operation code (GOC) contains all the graphical operations including memory blitting, text dravdng to name a few. This core (GOC) interacts with display driver (DD) to pass the output to the appropriate output device (OD),
It is thus possible by way of the above connectivity to remove dependency on X-server and thereby advancing the utility of the same in the embedded environment with constraint

resources where it is taxing to run heavy application like X-server. By way of the invention embedded devices would be able to support window desktop connectivity thus saving cost all additional resources and processing capabilities, memories etc. Importantly, the system can be advantageously used as architecture for thin clients with minimal processing capabilities.

WE CLAIM:
1. An integrated convergence platform with local mouse implementation facility adapted for
voice, video, and data applications including computing, communication and entertainment
applications comprising:
plurality of user access terminal/devices having respective network processor with constrained resources with or without additional network processors of varied capabilities in a server centric Computing environment;
a LAN and/or WAN operating connection to server means comprising a network server and a media server;
a local mouse implementation facility comprising said media processor adapted to store locally the initial mouse shape received form the server; said network processor adapted to capture the coordinates whenever the user moves the mouse and to forward the coordinates to the said media processor and the server; said media processor adapted to draw the mouse on the screen with respect to the x and y coordinates ;said server adapted to recognize any change in shape of the mouse and forward the pattern and mask to the to the network processor; the network processor further adapted to receive the same as a separate mouse packet and forward the data to the media processor; and said media processor adapted to recognize and handle the mouse packet and draw the packet on the screen as a graphics operation.
2. An integrated convergence platform with local mouse implementation facility as claimed
in claim 1 wherein the network server is adapted to provide other screen updates as a regular
VNC screen update and is decoded and displayed by the client/network processor.

3. An integrated convergence platform with local mouse implementation facility as claimed in anyone of claims 1 or 2 wherein the network server is adapted to support 2 byte mask information for communicating with said access terminal/devices having respective network processor with constrained resources.
4. An integrated convergence platform with local mouse implementation facility as claimed in anyone of claims 1 to 3 wherein the network server is adapted to strip the unused portion of the mouse area and forward only the modified area for mouse drawing operations to said access terminal/devices having respective network processor with constrained resources.
5. An integrated convergence platform with local mouse implementation facility as claimed in anyone of claims 1 to 4 wherein the network server is modified only to the extent of supporting the access terminal/devices having respective network processor with constrained resources.
6. An integrated convergence platform with local mouse implementation facility as claimed in anyone of claims 1 to 5 wherein the client/network processor is adapted to forward mouse packet to the server and also use the data within itself for drawing the mouse.
7. An integrated convergence platform with local mouse implementation facility as claimed in anyone of claims 1 to 6 wherein the client/network processor is adapted to identify the mouse packet and the screen updates as two different packet types and handles it differently such that when the packet is corresponding to the mouse packet it does graphic operation and draws the mouse locally, otherwise, the screen update packet is drawn as in the original VNC protocol.

8. An integrated convergence platform with local mouse implementation facility as claimed in anyone of claims 1 to 7 adapted for required server scalability for said access terminal/devices having respective network processor with constrained resources and also all other clients connected to the same server through additional network processors of varied capabilities.
9. An integrated convergence platform with local mouse implementation facility as claimed in anyone of claims 1 to 8 comprising said platform having embedded customized software for providing computing, entertainment and voice capabilities.
10. An integrated convergence platform with local mouse implementation facility as claimed
in anyone of claims 1 to 9 comprising a device management console for managing said
plurality of user access terminals/devices.
11 An integrated convergence platform with local mouse implementation facility as claimed in anyone of claims 1 to 10 wherein said user access terminals /devices are free of any moving components.
12. An integrated convergence platform with local mouse implementation facility as claimed in anyone of claims 1 to 11 wherein said device operates menu based graphical user inleiiaee (GUI).
13. An integrated convergence platform with local mouse implementation facility as claimed in anyone of claims 1 to 12 wherein the said device connects to an operating system scv\cv also providing computing applications as one of its features.

14. An integrated convergence platform with local mouse implementation facility as claimed in anyone of claims 1 to 13 wherein said device comprises embedded software specifically developed to control and provide the said features.
15. An integrated convergence platform with local mouse implementation facility as claimed in anyone of claims 1 to 14 wherein said devices are equipped with specially developed management software for changing configuration parameters and/or updating the device configuration.
16. An integrated convergence platform with local mouse implementation facility as claimed in anyone of claims 1 to 15 wherein said devices are application class computer access terminals with multimedia capabilities.
17. An integrated convergence platform with local mouse implementation facility as claimed in anyone of claims 1 to 16 wherein said server means comprising Content servers and Computing Servers.
18. An integrated convergence platform with local mouse implementation facility as claimed in anyone of claims 1 to 17 comprising any or all of the following depending on the users' application requirement-
General Purpose Processor having random access memory means;
media processor having random access memory means;
flash storage means for storing software codes; and
system bus for operative connection of various components with the central processing units

19. An integrated convergence platform with local mouse implementation facility as claimed in anyone of claims 1 to 18 adapted for embedded software and / or hardware as an interface to smart card reader and /or biometric sensor means to secure access to the compound /content as provided by the platform.
20. An integrated convergence platform with local mouse implementation facility as claimed in anyone of claims 1 to 19 comprising anyone or more of the following hardware blocks Ethernet MAC connectivity;
Universal Asynchronous Transmitter and receiver means;
Serial Peripheral Interface;
SPORT means;
Peripheral Component Interconnect Bus means;
USB Controller;
Audio Coder-Decoder means;
Video Digital to analog converter and / or VGA Controller;
Speaker/MIC means;
Interface to Monitor and / or TV;
A power source,
One Voice Over Internet Protocol (VoIP) Port
An optional PCI expansion slot, modem, Ethernet switch, and
Infra-Red Receiver
21. A computer readable information recording medium which records a program /algoriihin
for providing an integrated convergence platform for voice, video, and data applications

including computing, communication and entertainment applications with local mouse implementation facility as claimed in claim 1.
22. A remote windows desktop connectivity from embedded systems with constrained
resources comprising:
a remote desktop protocol operatively connecting to windows terminal server at one end and to a wrapper means at the other end ;
said wrapper means disposed there between said remote desktop protocol and a custom display hardware device having embedded graphical operations.
23. A remote windows desktop connectivity as claimed in claim 22 wherein for input events
the remote desktop protocol is adapted to directly communicate with said hardware and
comprises a key map wrapper for identifying the key strokes and converting the same to
necessary format,
24. A remote windows desktop connectivity as claimed in anyone of claims 22 to 23 comprising a remote desktop protocol network layer adapted for interacting with Windows Terminal Server, a remote desktop protocol decoder means adapted to decode the data that passes to the wrapper means, said wrapper means provided as an interface between said display device and said decoder means.
25. A remote windows desktop connectivity as claimed in anyone of claims 22 to 24 wherein said remote desktop protocol display device comprises a graphical Operations (Ore operatively connected to a Display Driver and a Device Cache said Display Driver in turn operatively connected to an output means whereby the core interacts with the Display Drivei to pass the output to the appropriate output device.

26. A remote windows desktop connectivity as claimed in anyone of claims 22 to 25 wherein said graphical operations core comprises all graphical operations including preferably memory blitting, text drawing.
27. A remote windows desktop connectivity as claimed in anyone of claims 22 to 26 wherein the embedded system is selectively having no operating system or adapted to run on embedded Linux operating system.

28. A remote windows desktop connectivity as claimed in anyone of claims 22 to 11 comprising plurality of said remote desktop protocols and remote user access desktop device comprising embedded customized software for providing computing, entertainment and voice capabilities.
29. A remote windows desktop connectivity as claimed in anyone of claims 22 to 28 comprising a device management console for managing said plurality of user access terminals/devices.
30. A remote windows desktop connectivity as claimed in anyone of claims 22 to 29 where! n said user access terminals /devices are free of any moving components.
31. A remote windows desktop cormectivity as claimed in anyone of claims 22 to 30 whereni said device operates menu based graphical user interface (GUI).
32. A remote windows desktop connectivity as claimed in anyone of claims 22 to 31 wherein the said device connects to said operating system server also providing computing applications as one of its features.

33. A remote windows desktop connectivity as claimed in anyone of claims 22 to 32 wherein said device comprises embedded software specifically developed to control and provide the said features.
34. A remote windows desktop connectivity as claimed in anyone of claims 22 to 33 wherein said devices are equipped with specially developed management software for changing configuration parameters and/or updating the device configuration.
35. A remote windows desktop connectivity as claimed in anyone of claims 22 to 34 wherein said devices are application class computer access terminals with multimedia capabilities.
36. A remote windows desktop connectivity as claimed in anyone of claims 22 to 35 wherein said windows terminal server means comprising Content servers and Computing Servers.
37. A remote windows desktop connectivity as claimed in anyone of claims 22 to 36
comprising any or all of the following depending on the users' application requirement -
General Purpose Processor having random access memory means;
media processor having random access memory means;
flash storage means for storing software codes; and
system bus for operative connection of various components with the central processing units.
38. A remote windows desktop cormectivity as claimed in anyone of claims 22 to 37
including remote access protocol adapted for embedded software and / or hardware as an
interface to smart card reader and /or biometric sensor means to secure access to the
computing /content as provided by the platform.

39. A remote windows desktop connectivity as claimed in anyone of claims 22 to 38
comprising remote access protocol with the following hardware blocks -
Ethernet MAC connectivity;
Universal Asynchronous Transmitter and receiver means;
Serial Peripheral Interface;
SPORT means;
Peripheral Component Interconnect Bus means;
USB Controller;
Audio Coder-Decoder means;
Video Digital to analog converter and / or VGA Controller;
Speaker/MIC means;
Interface to Monitor and / or TV;
A power source.
One Voice Over Internet Protocol (VoIP) Port
An optional PCI expansion slot, modem, Ethernet switch, and
Infra-Red Receiver
39. A computer readable information recording medium which records a program /algorithm for providing remote windows cormectivity from an embedded platform using the said remote windows desktop connectivity as claimed in claim 22.

40. An integrated convergence platform with local mouse implementation facility adapted for voice, video, and data applications including computing, communication and entertainment applications and a remote windows desktop connectivity from embedded systems with constrained resources substantially as herein described and illustrated with reference to the accompanying figures.

Documents

Orders