Claims:We claim:
1. A dual processor multi FPGA device (100) integrated with multiple FPGAs to facilitate realization of layered time critical control applications through multiple sub devices in a multi-tasking environment, the dual processor multi FPGA device (100) comprising of:
a central processor sub device (CPD) (102) having a processor (108) to execute control functions with appropriate priorities in the multi-tasking environment under a hard real time operating system (RTOS);
a communication control device (CCD) (101) connected with the CPD (102) via a high speed serial bus (PCIe) using PCIe switch to offload from the processor (108) of the CPD (102) the task of highly deterministic multi-channel communication; and
an input output (I/O) processor sub device (IPD) (103) connected with CPD (102) via a PCIe interface 116, the IPD (103) has a processor (109) with a double precision floating point unit (FPU) for data processing and a high speed FPGA for data acquisition, the IPD (103) connected with the CPD (102) for exchanging deterministic data among the sub devices.
2. The dual processor multi FPGA device (100) as claimed in claim 1, wherein dual processor multi FPGA device (100) performs layered time critical control tasks, high speed data acquisition and highly deterministic communication tasks simultaneously with an integrated and tightly coupled architecture of three types of sub devices.

3. The dual processor multi FPGA device (100) as claimed in claim 1, wherein the CPD (102) performs time critical control tasks and the RTOS ensures a tight scheduling environment with well defined time slices for the time critical control tasks irrespective of system load.

4. The dual processor multi FPGA device (100) as claimed in claim 1, wherein the CPD (102) creates a virtual local data image of each of the ports of CCD (101) and the IPD (103) for making faster data availability in synchronization with the control tasks of the CPD (102).

5. The dual processor multi FPGA device (100) as claimed in claim 1, wherein the CCD (101) communicates with the CPD (102) over the hard PCIe end-point built into the FPGA with the protocol stack implemented using the register/latches of the FPGA.

6. The dual processor multi FPGA device (100) as claimed in claim 1, wherein each CCD (101) has eight fibre optic ports connected to the FPGA in a parallel manner which ensures data from each of the fibre optic ports are available at the CPD (102) at the same instance for processing.

7. The dual processor multi FPGA device (100) as claimed in claim 1, wherein the IPD (103) provides high speed analog and digital process data to and from the CPD (102); wherein the IPD (103) executes tier two control tasks locally in coordination with the CPD (102).

8. The dual processor multi FPGA device (100) as claimed in claim 1, wherein the FPGA of the IPD (103) and the FPGA of the CCD 101 communicates with the CPD over the inbuilt hard PCIe end-point using Raw Ethernet protocol with data frames customized for low overhead and more data throughput.

9. The dual processor multi FPGA device (100) as claimed in claim 1, wherein the overall control function is realized through a layered mechanism in an optimal decentralized multi task scheduling environment with appropriate hard processing core based signal mechanism.

10. The dual processor multi FPGA device (100) as claimed in claim 1, wherein the CPD (102), the CCD (101), and the IPD (103) ensure high speed data exchange with each other through high speed serial interconnect bus (PCIe bus) and a PCIe switch.

11. The dual processor multi FPGA device (100) as claimed in claim 1, wherein the CPD (102) has a shared memory with a the virtual local data image for each of the CCD (101) ports and the IPD (103) for making faster data availability in synchronization with the control task, in the shared memory Rx data buffers are updated by the respective CCDs CCD (101) and the IPD (103), a background daemon task on the CPD (102) refreshes the entire received shared memory data buffer for a particular port once arrival of fresh data is detected by the FPGAs of the CCD (101) and the IPD (103), the application programs on the CPD (102) considers the data available on the shared memory to be the most recently acquired one. , Description:A DUAL PROCESSOR MULTI-FPGA BASED ELECTRONIC DEVICE INTEGRATED WITH MULTIPLE HIGH SPEED DETERMINISTIC COMMUNICATION INTERFACES FOR TIME CRITICAL CONTROL APPLICATIONS
FIELD OF INVENTION:
[001] The present subject matter described herein, relates to harnessing of multiple processing subsystems built using Field Programmable Gate Arrays (FPGAs) and processors of different architectures working under the control of multiple real time operating systems (RTOSs) to realize high speed control applications incorporating process interlocks and delivering time deterministic communication at all subsystem levels.
BACKGROUND AND PRIOR ART:
[002] There are commercially available systems to address time critical control/data processing applications. Some of them are multi-module multi-processor systems or their combinations thereof while others use a simpler approach of multiple modules having limited or no processing capabilities coupled to a single processor module or a single processor-FPGA module that does all the necessary processing. Some of the relevant solutions are as mentioned below:
[003] The patent US6757761B1 refers to a quad-processor arrangement having cache-less communication (data) paths between each pair of processors used for signal and image processing. Each processor is provided with a local memory operated under the control of the local processor as well as remote ones. The inter memory communication is handled by FPGAs with the communication paths following a parallel bus architecture. This arrangement, representing a tiled architecture, where the subsystems are similar in terms of design and functioning, is suitable for compute and data intensive offline applications, like image and signal processing which require parallel processing. This arrangement is not suitable to realize the high speed data acquisition, distributed data communication and to implement the multiple time deterministic control algorithms, with some requiring independent execution and others executed in a mutually dependent manner with inter processor data transfer.
[004] The patent US7475175B2 depicts an arrangement comprising of a plurality of logically independent processors, a system bus, and a cache control and bus bridge device in communication with the plurality of processors. The processors, cache control and bus bridge device are operated in a manner so that the data to each processor is processed parallely and independently. The solution is more in keeping with increasing the throughput of legacy systems by the brute force method of increasing the computing power available without changing the system architecture or the form factors. It is more suitable for general purpose computing. Moreover, the computing functions are segregated at the processor level only and there exists no modules with dedicated functions. The absence of FPGAs in this arrangement makes it unsuitable for high speed data acquisition required in control applications.
[005] The patent US9089067B2 depicts an automation device that consists of a central processor board with decentralized I/O devices having field bus connectivity to the processor. The arrangement is more like a normal DCS with all the processing done centrally. This arrangement is not suitable for high speed control applications requiring availability of high speed acquisition of field data, multiprocessor architecture for in-situ distributed execution.
[006] The patent application US2012/0128364A1 depicts a fibre optic based distributed I/O system for wind turbine control. It consists of a centralized controller connected with plurality of I/O boards through fibre optic cables. The I/O boards themselves consist of a processor to relay field data and own health information to the processor. Due to fixed number of fibre optic ports some of the boards communicate to the controller in a half-duplex mode. The controller communicates to the on-board fibre optic transceivers using UART and SPI through digital expanders. This arrangement is not suitable for high speed data acquisition and control as the all the functions are carried out centrally by the controller. Moreover, due to the half-duplex nature of data communication between the controller and remote I/Os it is not time deterministic.
[007] The patent WO2011/150929A1 depicts a wind power computer system comprising of two separate operating systems running their own set of application programs on a single hardware platform. A hypervisor program allows the OSs and their associated programs to share the hardware platform as virtually separate systems. This arrangement necessitates different programs and OSs to use the same hardware and is not suitable in the case where different set of functions requires different hardware platforms tailor-made for that set of functions.
[008] The patent US8219992B2 depicts a system using a plurality of OS and a plurality of processors. The communication between the processors is carried out by a management program resident in each of the OSs using a communication protocol like serial, TCP/IP or the like. The arrangement is more suitable for offline multiplexed data processing applications requiring parallel processing on multiple processors (or processor cores) than with time critical control applications that needs high speed real time data processing and execution of control functions simultaneously.
[009] Therefore there is requirement of a system to simultaneously realize multiple time critical control tasks and highly time deterministic distributed communication functions in true real time for a suitable process.
OBJECTS OF THE INVENTION:
[0010] The principal objective of the present invention is to provide a device to realize time critical control and highly deterministic communication functions in true real time for a suitable process.
[0011] Another object of the present subject matter is to provide a device which should have tightly coupled configurable architecture with dedicated sub devices for exclusive functions viz. data acquisition, communication and control to support layered control schemes.
[0012] Another object of the present subject matter is to provide a device topology that should support seamless data availability from local sub devices as well as remote devices at precise time instants for effective control of the overall process.
[0013] Another object of the present subject matter is to provide a device which should support implementation of high speed control tasks utilizing the individual functions of the sub devices with overall coordination from a main control processor.
[0014] Yet another object of the present invention is to provide a device hardware which should support seamless integration of sub devices using standard interfaces for minimum software overhead.
SUMMARY OF THE INVENTION:
[0015] The subject matter disclosed herein relates to a system to realize time critical control applications, which require time bound data acquisition and processing as well as deterministic data exchange, with a device comprising of dual processors integrated with multiple FPGAs. A distributed scheduling of control and communication tasks under the various processing components in the device spread over three sub devices ensures true execution of time critical layered control tasks with deterministic communication effectively. The processor based sub devices process data under the control of hard RTOSs to guarantee time deterministic behaviour. Each of the sub devices is engaged in executing specific tasks of the control functions which include data pre-processing, distributed data communication, process control, etc. The individual sub devices communicate in a deterministic manner so that the data from any sub device or a remote device is ensured in determined time for executing time critical control actions effectively. This method of executing time critical control tasks is required in applications where the high speed control actions are to be implemented in real time at different layers of the control architecture in a deterministic manner.
[0016] In order to further understand the characteristics and technical contents of the present subject matter, a description relating thereto will be made with reference to the accompanying drawings. However, the drawings are illustrative only but not used to limit scope of the present subject matter.
BRIEF DESCRIPTION OF THE DRAWINGS
[0017] It is to be noted, however, that the appended drawings illustrate only typical embodiments of the present subject matter and are therefore not to be considered for limiting of its scope, for the invention may admit to other equally effective embodiments. The detailed description is described with reference to the accompanying figures. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. The same numbers are used throughout the figures to reference like features and components. Some embodiments of system or methods in accordance with embodiments of the present subject matter are now described, by way of example, and with reference to the accompanying figures, in which:
[0018] Fig. 1 illustrates block diagram of the main device, in accordance with an embodiment of the present subject matter;
[0019] Fig. 2 illustrates arrangement of the sub devices within the main device, in accordance with an embodiment of the present subject matter; and
[0020] Fig. 3 illustrates shared memory structure of the device, in accordance with an embodiment of the present subject matter.
[0021] The figures depict embodiments of the present subject matter for the purposes of illustration only. A person skilled in the art will easily recognize from the following description that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles of the disclosure described herein.

DESCRIPTION OF THE PREFERRED EMBODIMENTS:
[0022] The subject matter disclosed herein relates to a system to realize time critical control applications, which require time bound data acquisition and processing as well as deterministic data exchange, with a device comprising of dual processors integrated with multiple FPGAs. A distributed scheduling of control and communication tasks under the various processing components in the device spread over three sub devices ensures true execution of time critical layered control tasks with deterministic communication effectively. The processor based sub devices process data under the control of hard RTOSs to guarantee time deterministic behaviour. Each of the sub devices is engaged in executing specific tasks of the control functions which include data pre-processing, distributed data communication, process control, etc. The individual sub devices communicate in a deterministic manner so that the data from any sub device or a remote device is ensured in determined time for executing time critical control actions effectively. This method of executing time critical control tasks is required in applications where the high speed control actions are to be implemented in real time at different layers of the control architecture in a deterministic manner.
[0023] It should be noted that the description and figures merely illustrate the principles of the present subject matter. It should be appreciated by those skilled in the art that conception and specific embodiment disclosed may be readily utilized as a basis for modifying or designing other structures for carrying out the same purposes of the present subject matter. It should also be appreciated by those skilled in the art that by devising various arrangements that, although not explicitly described or shown herein, embody the principles of the present subject matter and are included within its spirit and scope. Furthermore, all examples recited herein are principally intended expressly to be for pedagogical purposes to aid the reader in understanding the principles of the present subject matter and the concepts contributed by the inventor(s) to furthering the art, and are to be construed as being without limitation to such specifically recited examples and conditions. The novel features which are believed to be characteristic of the present subject matter, both as to its organization and method of operation, together with further objects and advantages will be better understood from the following description when considered in connection with the accompanying figures.
[0024] These and other advantages of the present subject matter would be described in greater detail with reference to the following figures. It should be noted that the description merely illustrates the principles of the present subject matter. It will thus be appreciated that those skilled in the art will be able to devise various arrangements that, although not explicitly described herein, embody the principles of the present subject matter and are included within its scope.
[0025] Fig. 1 illustrates block diagram of the present system with all sub devices for performing the control, data processing, and communication tasks. The block diagram of the main device 100 as depicted in fig.1 consists of (a) Central Processor sub device (CPD) 102 which has a processor 108 functioning under the control of a hard real time operating system (RTOS), I/O Processor sub device (IPD) 103 which has a processor-FPGA 109-105 combination and an associated RTOS and a Communication Control sub device (CCD) 101 based on a FPGA. The CPD 102 and CCD 101 are connected together over a connector 106 of PCIe form factor. The CPD 102 and IPD 103 are also connected together over a connector 107 of PCIe form factor. As illustrated in figure 2, the CPD 102, the IPD 103 and the CCD 101 are arranged in the manner to ensure sub device inter connectivity using industry standard PCIe (Peripheral Component Interconnect -Express) bus. The present arrangement of the devices, i.e., CPD, CCD, IPD ensures the availability of high speed serial interconnect bus (PCIe bus) to all the three types of sub devices exchanging data at high speed with each other. The present arrangement also ensures that the CPD 102 can host multiple CCDs 101 without sacrificing any speed of data exchange between the CCDs 101 and the CPD 102. The sub device arrangement is designed to facilitate configurable device architecture, where sub devices can also be arranged differently without sacrificing the provision of multiple CCDs 101 in the devices and PCIe interconnect functionality between the three types of sub devices.
[0026] The Central Processor sub device (CPD) 102 is the heart of the system where all the process related control functions are executed with appropriate priorities in a multi-tasking environment. The CPD 102 has a 32-bit processor 108 with 64-bit double precision floating point unit (FPU) for executing the tasks under a hard real time operating system (RTOS) which is QNX Neutrino in the present instance. The various time critical control tasks are executed by the CPD 102 and the RTOS guarantees a tight scheduling environment with well-defined time slices for the tasks irrespective of the system load. The CPD 102 and the CCD 101 are connected over a high speed serial bus (PCIe) 113 using a PCIe switch 112. The switch has multiple ports ensuring connection of multiple CCDs 101 (maximum four numbers as shown in fig. 2) giving provision to have up to thirty two deterministic high speed communication links in a single main device. The CPD 102 possesses two single lane on-chip PCIe (PCIex1) interfaces 115 with one connected to the PCIe switch 112 and the other connected to the IPD 103 to accomplish deterministic data exchange among the sub devices. Apart from the above, the CPD 102 has all the necessary on board peripherals 110 like RAM, Flash memory, EEPROM to store non-volatile data, SD Card for data backup, Gigabit Ethernet ports, serial ports, etc., required for a local processor board.
[0027] The IPD 103 is primarily concerned with providing high speed analog and digital process data to and from the CPD 102 as well as to execute tier two control tasks locally in coordination with the CPD 102. The IPD 103 has a 32-bit processor 109 with 64-bit FPU for data processing and a high speed FPGA 105 for data acquisition. The data pre-processing tasks are implemented under a resident hard RTOS environment (MQX in this instance) for execution within a well-defined time interval every time. The data acquisition is carried out by the high speed FPGA 105. The FPGA 105 also communicates with the CPD 102 over the inbuilt hard PCIe end-point using Raw Ethernet protocol with data frames customized for low overhead and more data throughput. Apart from the above the IPD 103 also has board peripherals 111 like RAM, Flash, EEPROM, Fast Ethernet ports, RS232 ports, etc. found in a typical processor board. The analog and digital field inputs and outputs 118 are connected to the on board FPGA/processor through proper isolation. Moreover, the IPD 103 as shown in fig. 1 has provision for eight fibre optic ports 117 like a CCD 101. This provision has been made to enable need based activation as and when the IPD 103 is used in a separate stand-alone configuration.
[0028] The primary functionality of the CCD 101 is to offload the task of multi-channel communication over point-to-point fibre links from the processor 108 present in the CPD 102. Raw Ethernet with a light weight stack is employed as the communication protocol having minimum overhead with point to point addressing scheme. The CCD 101 handles communication with multiple channels parallely and communicates with the CPD 102 over the hard PCIe end-point built into the FPGA 104 with the protocol stack implemented using the register/latches of the FPGA. Each CCD 101 has eight fibre optic ports connected to the FPGA 104 in a parallel manner. This ensures that data from each of the ports are available at the CPD 102 at the same instance for processing.
[0029] Fig. 3 illustrates the data exchange and mirroring mechanism 300 in which process data from each communication link of CCDs 101 (IPD0 to IPD31 - 301) has a corresponding image in the shared memory structure of the CPD 102, thereby creating virtual local data image for making faster data availability in synchronization with the control tasks. IPD32 data 302 is depicted as being the IPD 103 connected directly over PCIe bus to the CPD 102 in the system. The tunneled memory structures (Rx data buffer 303 – received data buffer) are updated up by the respective CCDs 101 (and the IPD 103) once they receive data on the fibre optic ports (and on the field interfaces). The application programs in the CPD 102 are freed of the burden of the time consuming tasks of acquiring and populating the port data buffers. This task is carried out by a suitable daemon working in the background. The daemon refreshes the entire received data buffer for a particular port the moment arrival of fresh data on that port is detected by the FPGAs of either the CCD(s) 101 or the IPD 103. The outgoing data (TX data buffer 304 – transmitted data buffer) from the CPD 102 is filled on a need basis (explicitly by the application program).
[0030] The present subject matter describes a dual-processor multi FPGA device as illustrated in fig. 1 and fig. 2 to facilitate realization of layered time critical control tasks, high speed data acquisition and highly deterministic communication tasks simultaneously without sacrificing one another’s performance with an integrated and tightly coupled architecture of three types of sub devices. The sub devices are made on the basis of the basic set of functions each sub device has to perform. Thus the overall control function is realized through a layered mechanism in an optimal decentralized multi-task scheduling environment with appropriate hard processing core based signal mechanism. Each layer realizes one of the three functions viz. data acquisition with pre-processing, communication, and control. Each layer is realized in a different sub device enabling clear demarcation in each device’s functionality and thereby eliminating unnecessary and continuous access of a device’s local memory by other devices. The present system employs highly efficient coordinated inter sub-devices distributed data processing and communication architecture suitable for complex control and data processing tasks.
[0031] Further, the sub devices possess the capability to run on different RTOSs with different OS tick sizes. The RTOSs and the tick sizes are chosen depending on the functions the processors on the different sub devices need to perform. The dual processor system employs multi-channel high speed PCIe communication for inter sub-devices communication to achieve simultaneous hierarchical control and decentralized communication functions. In the sub-devices, the CCD 101 possesses capability of eight fibre optic ports connected with the FPGA in a parallel interface configuration and thus data received on each of the eight ports become available to the CPD 102 at the same instant of time.
[0032] Furthermore, as explained in the figure 1 and 2, use of the PCIe switch in CPD 102 with the shared memory concept as in fig. 3 enables the CPD 102 to host multiple CCDs 101 (up to four in number) without sacrificing any speed of data exchange between the CCDs 101 and the CPD 102. The background daemon task on the CPD 102 refreshes the entire received shared memory data buffer for a particular port once arrival of fresh data is detected. Due to this kind of architecture, for all practical purposes, the application programs on the CPD 102 can consider the data available on the shared memory to be the most recently acquired one. The present device architecture provides a scalable, high speed, time deterministic fibre optic communication capability between the electronic device and remote modules having the same communication protocol and physical layer.
[0033] Although embodiments for the present subject matter have been described in language specific to structural features, it is to be understood that the present subject matter is not necessarily limited to the specific features described. Rather, the specific features and methods are disclosed as embodiments for the present subject matter. Numerous modifications and adaptations of the system/component of the present invention will be apparent to those skilled in the art, and thus it is intended by the appended claims to cover all such modifications and adaptations which fall within the scope of the present subject matter.