FIELD OF INVENTION
The invention generally relates to image processing and particularly to a system and method of multi-axis image processing.
BACKGROUND
Inspection of large and thick specimens under an optical microscope poses a major challenge to the users. Due to limited depth of field, many optical sections along Z direction needs to be taken. Since the specimen size is larger than the field of view (FOV) of the microscope, multiple XY scans with overlap need to be collected to cover the image dimensions.
Well known digital image processing methods like extended depth of field (EDF) is used to analyse the Z stacks to provide an all-in-focus image and depth map. Image stitching is then applied on the images separately to create a large panoramic view of the specimen under analysis.
Existing systems apply EDF and stitching independently and results in an image with artefacts such as missing information between tiles, scale variations across tiles and improper alignment. The volume of image data that needs to be processed is significantly high and poses challenges in terms of memory and execution speed.
Hence, there is a need to devise an image synthesis method in which EDF and stitching works together and address the above-mentioned challenges.
The present invention is directed to overcoming one or more of the problems as set forth above.
SUMMARY OF THE INVENTION
Exemplary embodiments of the invention disclose a system and method for generating a seamless composite image from multi-axis and multi-focus image data of a subject. According to an embodiment, the disclosed system and method captures a plurality of images of the 1 subjectStfBiBerent lo&fibnslihrmulfiplevefiicd'arid.horizbntal planes'. 'The plurality of Images"""'" may be focus stacked in each of the vertical planes to generate a composite image and surface

depth map. The vertical focus stacked images maybe aligned based on a horizontal coordinate transformation model and the aligned images are stitched to render a seamless composite image. In a similar manner, the corresponding depth maps can be stitched together to create a panoramic 3d surface of the specimen.
BRIEF DESCRIPTION OF DRAWINGS
Other objects, features, and advantages of the invention will be apparent from the following description when read with reference to the accompanying drawings. In the drawings, wherein like reference numerals denote corresponding parts throughout the several views:
Figure 1 illustrates an exemplary arrangement for capturing a plurality of images in horizontal (X, Y) and vertical (Z) plane of an object, according to an embodiment of the invention; Figure 2 illustrates a high-level system diagram for generating a seamless composite image from a multi-axis and multi-focus image data of a subject, according to an embodiment of the invention; and
Figure 3 is a flowchart of a method for generating a seamless composite image and depth map from multi-focus image data of a subject, according to an embodiment of the invention.
DETAILED DESCRIPTION OF DRAWINGS
The following description with reference to the accompanying drawings is provided to assist in a comprehensive understanding of exemplary embodiments of the invention. It includes various specific details to assist in that understanding but these are to be regarded as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. In addition, descriptions of well-known functions and constructions are omitted for clarity and conciseness.
According to embodiments of the invention, a system and method for generating a seamless composite image from a multi-axis and multi-focus image data of a subject is disclosed. Figure 1 illustrates an exemplary arrangement for capturing a plurality of images in horizontal (X, Y) ^ahd vertical fZ^plain of an^bje'c't/acbording tb ah'e'mbo'dfm'e'nt dttheiinvehtiorE As illustrated, the disclosed system and method captures a plurality of images of the subject at different

locations in multiple vertical and horizontal planes. According to an embodiment of the invention, the images may be captured by varying the position of the camera in a vertical plane at a specific horizontal (X-Y) location and repeating the process for varied horizontal (X-Y) locations. According to another embodiment, the images may be captured by varying the position of the camera in a horizontal plane (X-Y) at a specific Vertical (Z) location and repeating the process for varied vertical (Z) locations. The captured images may be used to constitute the multi-axis and multi-focus image data.
Figure 2 illustrates a high-level system diagram and Figure 3 illustrates a flowchart of a method for generating a seamless composite image and depth map multi-focus image data of a subject, according to exemplary embodiments of the invention. The process is illustrated using an exemplary thick specimen that is larger than the FOV (field of view) of an exemplary microscope. The exemplary process starts by traversing microscopic camera in vertical (Z direction) and lateral planes (XY Direction) with a fixed focal length. As illustrated in figure 2, the plurality of images may be focus stacked in each of the vertical planes.
According to an embodiment, for vertical focused stacking, each z plane image input may be aligned correctly with respect to each other to account for translation and scale variations. According to exemplary embodiment, one image from the z stack may be used as a reference to align rest of the images. By keeping magnification and angular aperture of an objective lens constant, without any rotation of camera and stage, scale and translation component follows almost a linear relationship with z stage position. Similarly, translation between die other adjacent tiles holds a linear relationship with their X and Y stage positions.
One time calibration procedure may be used to derive coordinate transformations. According to exemplary embodiments, the coordinate transformations may include introducing known transformations in image space; such as but not limited to, checker board, coefficients are derived that may govern transformation as a function of X, Y and Z stage position.
According,to exemplary embodiment, the transformation between Z planes may be calculated by a coordinate transformation model T as: T = F(f,9,z) '^wherein the coordinate transforrhariSn model "T is 'a* functiori'-bf- calibration parameters f, 6 and Z and f denotes focal length of an objective, 0 denotes angular aperture of the objective, and z

equals depth minus stage position of the microscope. The coordinate transformation model may be derived from the microscope hardware and optics parameters.
According to an embodiment, input for the transformation correction module for Z direction movements may be a set of images captured in Z direction. One image from the set of images may be selected as a reference plane. The calibration parameters identified in the calibration module may be used to derive an affine transformation between a reference image and an image that needs to be corrected. The transformation correction module results in a set of translation and scale corrected images captured in z direction. According to an embodiment, the translation and scale corrected images may be further processed to form a set of focused stack images.
According to an embodiment, for vertical focused stack images a focus fusion module may be used. Input to the focus fusion module is the set of translation and scale corrected images captured in Z direction and output is a single all-in-one focus image.
According to an embodiment, the plurality of Z stack images obtained from the transformation correction module may be aligned to a common reference frame and the focus fusion model may be applied to create a single all-in-one focus image or vertical focus stacked images. During the process of focus stacking, any known state of the art techniques of extending depth of focus (EDF) may be applied. According to an embodiment, an energy value may be computed at each pixel and compared against corresponding pixel locations in die Z stack during vertical focus stacking.
According to an embodiment, the vertical focus stacked images may be aligned based on a horizontal coordinate transformation model and the aligned images may be stitched to render a seamless composite image. All EDF images generated from different vertical (X, Y) tile locations need to be corrected for horizontal (XY) translation. The position correction module may include two steps. First step may be prediction of X, Y offsets and the second step may be refinement of X, Y offsets.
(a) Prediction of X, Y offsets
Affine transformation between two adjacent tiles at a specific z position may be computed from : Nc51ibf^tTon tabFeSo proviideTari initial X offseBaridGriitia'l y 'offset'. 1 4 : 5 5

(b) Refinement of X, Y offsets
The initial X and Y translation values may not be accurate because of stage errors and view point variations due to non-tele centric lenses. A smaller section at the overlap region may be extracted and any motion estimation methods may be used to compute the translation between adjacent tiles. The computed values should fall within the threshold of the transformation predictions from the calibration procedure.
The image properties such as illumination may be normalized across multiple XY tiles. A reference value is computed from a selected image and appropriate correction transformation may be applied to all other set of images.
The blending module uses inputs from position correction module and global correction module to combine EDF images outputted from focus fusion module. According to an embodiment, the overlap regions may be blended using conventional blending techniques such as, but not limited to, alpha blending and pyramidal blending. A similar procedure can be used for the individual depth maps and a panoramic 3D surface can be created.
The present invention effectively utilizes of memory and may be executed in conventional CPUs.
In the drawings and specification there has been set forth preferred embodiments of the invention, and although specific terms are employed, these are used in a generic and descriptive sense only and not for purposes of limitation. Changes in the form and the proportion of parts, as well as in the substitution of equivalents, are contemplated as circumstances may suggest or render expedient without departing from the spirit or scope of the invention.
Throughout the various contexts described in this disclosure, the embodiments of the invention further encompass computer apparatus, computing systems and machine-readable media configured to carry out the foregoing systems and methods. In addition to an embodiment consisting of specifically designed integrated circuits or other electronics, the present invention may be conveniently implemented using a conventional general purpose or a specialized digital computer or microprocessor programmed according to the teachings of the present disclosure,

The embodiments of the present invention may be provided as a computer program product that may include a machine-readable medium, having stored thereon instructions which may be used to program a computer (or other electronic devices) to perform a process according to the present invention. The machine-readable medium may include, but is not limited to, floppy diskettes, optical disks, compact disc read-only memories (CD-ROMs), and magneto-optical disks, ROMs, random access memories (RAMs), erasable programmable read-only memories (EPROMs), electrically erasable programmable read-only memories (EEPROMs), magnetic or optical cards, flash memory, or other type of media/machine-readable medium suitable for storing electronic instructions. In addition to an embodiment consisting of specifically designed integrated circuits or other electronics, the present invention may be conveniently implemented using a conventional general purpose or a specialized digital computer or microprocessor programmed according to the teachings of the present disclosure, as will be apparent to those skilled in the computer art.
Appropriate software coding can readily be prepared by skilled programmers based on the teachings of the present disclosure, as will be apparent to those skilled in the software art. The invention may also be implemented by the preparation of application specific integrated circuits or by interconnecting an appropriate network of conventional component circuits, as will be readily apparent to those skilled in the art.

1. A method for generating a seamless composite image from multi-axis and multi-focus
image data of a subject, the method comprising:
capturing a plurality of images of the subject at different locations in multiple vertical and horizontal planes;
focus stacking of the plurality of images captured in each of the vertical planes; .
aligning the vertical focus stacked images based on a horizontal coordinate transformation model; and
stitching the aligned images & depth maps to render a seamless composite image and a 3d panorama
2. The method as claimed in claim 1, wherein the horizontal coordinate transformation model is derived from microscope hardware parameters.
3. The method as claimed in claim 1, further comprising performing illumination and colour correction on the aligned images.
4. The method as claimed in claim 4, further comprising using the calibration parameters to derive affine transformation between a reference image and an image with parallax.