Field of the Invention
The present invention relates to a memory structure that is optimized for image rotation and mirroring.
Background of the Invention
Image processing is an essential aspect of the functioning of a vast majority of modern day devices, ranging from computing systems to consumer devices such as cellphones and Personal Digital Assistants (PDAs). The Human Interfaces of these devices are becoming increasingly Graphical in nature inorder to provide a more user friendly interface. These Graphical User Interfaces (GUIs) are also increasing in sophistication as greater computing power becomes available in these devices.
Image processing systems are characterized by the requirment of relatively large memory structures that store the images. The image processing activity involves the manipulation of this large amount of imaging data at very high speeds so as to enable real-time visualization of the images and the movement of those images. Typical operations include image translation and rotation. This high-speed manipulation of the image data is performed by Signal Processors, General Purpose Processors or Special purpose Image Processors These image processing engines access the image data in the memory through high-speed buses that connect them together.
Existing memory systems are designed for meeting the needs of normal non-imaging data processing functions. As such, conventional memory system are designed for normal sequential memory access. When such conventional memory systems are used for image processing, the image data (pixels) are packed to improve the memory utilization, for example, if memory datawith is 64-bits then it will contain four 16-bit pixels or two 24-bit pixels and two bytes of the third pixel. The resulting performance is less than optimal as the typical frequently used functions of image translation or rotation very often require non-sequential memory access. Consequently several memory accesses are required for each step of the image processing function resulting in inefficient and slow operation for a given clock speed.
U,S, Patent No.4,716,533 describes a method for improving the performance of a memory system that is used for image rotation by 90 degrees or a multiple of 90 degrees. This invention describes a system and method for movement of image data from an initial position to a subsequent position involving rotation by 90 degrees or a multiple of 90 degrees with or without image translation. This invention however focusses on the determination of the optimum movement of the data regardless of the speed of accessing data from the memory. The invention does not address the issues relating to the speed and efficiency of accessing the data from the memory, hence its effectiveness is limited by the memory access mechanism.
There is a need for a memory architecture that is efficient for image processing applications involving image rotation by 90 degrees or a multiple of 90 degrees.
Objects and Summary of the Invention:
It is an object of the invention to provide a memory architecture that is efficient for image rotation by 90 degrees or a multiple of 90 degrees. It can also be extended to implement rotation by any amount with at par or better performance than the conventional architecture.
It is also an object of the invention to provide a memory architecture that is efficient for image mirroring, flipping and generating planar data (sparating image into separate luma and chroma planes).
To achieve the said objectives, this invention provides a memory architecture in which the memory is arranged to provide multiple data paths of a wide multi-byte data width. The individual bytes of each data path are combined in a configurable manner based on the desired image processing operation to produce a variable data path of the desired multi-byte data width that provides efficient memory access for the desired operation.
Accordingly, the present invention provides a memory architecture for image
processing comprising:
a memory array having multiple multi-byte memory data paths of equal multi-byte data width, and
a multiplexing structure connected to the output of said multiple multi-byte data paths, capable of selectively providing a multi-byte data path of a desired width containing a desired permutation of bytes chosen from one or more of said multiple data paths.
The said multiplexing arrangement comprises a 2-level hierarchy of multiplexers in
which:
the first level of multiplexers combines one or more individual bytes from one or more multi-byte data paths to create multiple data paths of the desired width in the desired permutations based on the desired set of operations, and
the second level of multiplexers selects a desired one of said created multiple multi-byte data paths, based on the desired operation.
The said multiplexing structure is configurable to provide different sets of byte permutations based on changing requirements.
The invention further provides a method for providing an efficient memory architecture
for image processing, comprising the steps of:
structuring the memory array as a multi-byte memory array having multiple data paths of equal multi-byte data width, and providing a multiplexing structure at the output of said memory array that selectively provides a multi-byte data path of a desired width containing a desired permutation of bytes chosen from one or more of said multiple data paths.
The said multiplexing structure is provided by :
combining one or more individual bytes from one or more multi-byte data paths to create multiple data paths of the desired width in the desired permutations based on the desired set of operations, and selecting a desired one of said created multiple multi-byte data paths, based on the desired operation.
The said multiplexing structure can be configured to provide different sets of byte permutations based on changing requirements
Brief Description of the Drawings:
The invention will now be described with the help of the following figures and drawings:
Fig.- 1 shows a block diagram of a prefered embodiment of the invention
Fig- 2 shows a block diagram of the Read and Write Logic blocks, in the preferred
embodiment Fig- 3 shows the format of the address bus, in the preferred embodiment Fig- 4 shows the internal structure of the multiplexing arangement in the preferred
embodiment.
Detailed Description of the Invention:
The instant invention enables efficient utilzation of the system bus bandwidth for image manipulation functions. A preferred embodiment of the invention, shown in Fig- 1, uses six memory cuts each of forty eight data width, along with simple read write control logic and a data multiplexer. This embodiment describes the best bus performance is met for 16-bit and 24-bit per pixel modes. This structure can be extended to support other formats also with slight alteration These signals are used in the data mulitplexer to pack the data so that all the bits can be used. The data read and data write logic blocks are connected with the system bus for receiving data inputs and control signals. In the preferred embodiment the data width used is 48-bit wide, however this is not a necessary restriction and the data width and can be adjusted according to requirements.
The Memory system is organised as six equal memory cuts of 48-bit wide data each. The reason for choosing 48-bit as the data width is to ensure that each single pixel is addressed in each access. The individual chip select and byte select signals are generated for each of the six memory cuts of 48-bit data width, which are in turn connected to the data multiplexer for multiplexing the received data and sent back to the SRAM through its 48-bit data bus. Data multiplexing information has the multiplexing information that is to be used by the glue logic. The system utilizes two level mutiplexing, in which the first level decides which combination of bytes from different cuts make a valid combination depending on the selected mode, rotation of the data and the second level selects one of them should be selected.
Figure 2 describes the system block diagram of the Read and Write according to the preferred embodiment. The system comprises two write channels and one read channel with associated read and write control blocks (Write Channel#0,l and Read channel), bi-directional Write FIFO memories and Read FIFO memory of size 12*48, Request Arbiter, system bus (STBUS T2 in this particular case) I/F, system bus interconnect, configuration registers, SRAM controller, and Glue Logic and SRAM controller. The Write Channels#0 and I receive data and control signals and are coupled to bidirectional Write FIFO memories by read and write control blocks. The write channels are further coupled to the Request Arbiter, which in turn is connected to system bus I/F block. This in turn is connected bi-directionally to a system bus interconnect bus which also connects to the SRAM controller which transmits and receives data from the Glue Logic. The glue logic is connected to SRAMs with 6 sets of these signals. Configuration Registers are connected on the system register bus interface (STBUS Tl in this particular case). The Request Arbiter is coupled to a Read Channel for outputtimg data on receiving a read request. The Read Channel is also bi-directionally coupled to the Read FIFO.
Figure 3 shows the format of the address bus, in the preferred embodiment. The read and write to SRAM are performed in a manner such that only a small gluelogic is required between the SRAM controller and the SRAM.. This also allows the use of the
existing SRAM Controllers. The address bus in system bus is 32-bit wide and all the bits are not used if we use a 256KBytes memory. Hence, upper memory addresses can be used for sending data multiplexing and cut selection information. The 32-bits of address bus are split as shown. Memory location address is the address passed to the memory cuts. The same address is also passed to all the memory cuts. The Cut selected field has one bit for each cut. If the bit is ' 1' then corresponding memory cut is selected (chip select is asserted) and if the bit is '0', memory cut is not selected (chip select deasserted). Bit 13 corresponds to cutO, 14 to cut 1 and so on. Thus, bit 18 corresponds to cut5. More than one cut may be selected at the same time.
Fig.- 4 shows the internal structure of the Data path multiplexers. Data multiplexing information has the multiplexing information that is to be used by the glue logic. Two level multiplexing is used. The first level decides which bytes of different cuts make a valid combination depending on modes (rotation etc.) and the second level decides which one of them is selected. The multiplexing modes are given in the following table:
(Table removed)

a. 010 and Oil are chosen so that BYTESPERPIXEL setting can directly be used (no
logic required to generate select signals). 000 is used when the data read write is
sequential (non-rotation modes) and 010 or 011 is used when rotation is required.
b. Dnm means data byte m of memory cut n. The order is as it would appear in D[47:0].
c. Reserved so that can be used if required for other modes 420 planar.
The addressing scheme for writing data in different modes will be different and is shown in the following table:
(Table removed)

a. Data is 48-bits which means 2 or 3 pixels depending on 3Bytes/pixel 2Bytes/pixel
b. It (cut no = 5) is not necessarily for 1,3 (line/data). It can be for any data depending
on the value of j. It'll start here if j = 1. Same is true for 2,2 (5/k', it's only if j' = 2).
c. n is the data for last pixel of the line
d. j(or j' or j") and k (or k' or k") are the cut no. and address respectively for the last
data of the line
e. m(or m' or m") is the address for the last data of the line. Note that the 6th line will
start at (m+1) address of cut 0. Similarly, 7th at (m'+l) address of cut 1 and so on. m is
also referred to as line_pitch in the following description and tables.
For read operations, the address generation logic needs to take care of the image transformation mode. Following table shows an example of data read pattern for 90 degree rotated image.
(Table removed)

In above table title should be "Memory Access Example for Data Read". Byte enabled must be asserted accordingly.

We claim:
1. A memory architecture for image processing comprising:
a memory array having multiple multi-byte memory data paths of equal multi-byte data width, and
a multiplexing structure connected to the output of said multiple multi-byte data paths, capable of selectively providing a multi-byte data path of a desired width containing a desired permutation of bytes chosen from one or more of said multiple data paths.
2. A memory architecture for image processing as claimed in claim 1 wherein, said
multiplexing arrangement comprises a 2-level hierarchy of multiplexers in
which:
the first level of multiplexers combines one or more individual bytes from one or more multi-byte data paths to create multiple data paths of the desired width in the desired permutations based on the desired set of operations, and
the second level of multiplexers selects a desired one of said created multiple multi-byte data paths, based on the desired operation.
A memory architecture for image processing as claimed in claim 1 wherein said multiplexing structure is configurable to provide different sets of byte permutations based on changing requirements.
A method for providing an efficient memory architecture for image processing, comprising the steps of:
structuring the memory array as a multi-byte memory array having multiple data paths of equal multi-byte data width, and providing a multiplexing structure at the output of said memory array that selectively provides a multi-byte data path of a desired width
containing a desired permutation of bytes chosen from one or more of said multiple data paths.
5. A method for providing an efficient memory architecture for image processing
as claimed in claim 4, wherein said multiplexing structure is provided by :
combining one or more individual bytes from one or more multi-byte data paths to create multiple data paths of the desired width in the desired permutations based on the desired set of operations, and selecting a desired one of said created multiple multi-byte data paths, based on the desired operation.
A method for providing an efficient memory architecture for image processing as claimed in claim 4, wherein said multiplexing structure can be configured to provide different sets of byte permutations based on changing requirements
A memory architecture for image processing substantially as herein described with reference to and as illustrated in the accompanying drawings
A method for providing an efficient memory architecture for image processing substantially as herein described with reference to and as illustrated in the accompanying drawings