Abstract: ABSTRACT SYSTEM FOR DETERMINING CROP SOWN AREA BASED ON SATELLITE IMAGERY USING DEEP LEARNING MODEL A system and method for automatically determining a crop sown area from one or more satellite images at one or more timestamps using a deep learning model 108 are provided. The system includes a satellite 102, a crop sown area estimation server 106 that includes a deep learning model 108. The server 106 receives satellite images at different timestamps from the satellite 102. The server 106 pre-processes the satellite images. The server 106 extracts spatial features and exclusive temporal features using the deep learning model 108. The server 106 generates clusters of features based on extracted spatial features and extracted exclusive temporal features. The server 106 combines the clusters of features to to generate clusters of features. The server 106 categorize areas where sowing happens and the areas where sowing does not happen using trained deep learning model. The server 106 determines the crop sown area by removing stray pixels. FIG. 1
DESC:SYSTEM FOR DETERMINING CROP SOWN AREA BASED ON SATELLITE IMAGERY USING DEEP LEARNING MODEL
CROSS-REFERENCE TO PRIOR-FILED PATENT APPLICATIONS
[0001] This application claims priority from the Indian provisional application no. 202141001296 filed on January 12, 2021, which is herein incorporated by reference
Technical Field
[0002] The embodiments herein generally relate to deep learning model, more particularly, a system and method for automatically determining a crop sown area from one or more satellite images at one or more timestamps using a deep learning model.
Description of the Related Art
[0003] Satellite Images are an essential source of information. With technological advancements in satellites, global information systems, aviation, digital photography, computer technology, and telecommunications, high-resolution satellite images, and aerial photographs are nowadays available virtually to everyone. However, obtaining satellite imagery for generating earth observation data is extremely challenging, time-consuming, and expensive. The data from various satellites that are available free of cost publicly has its own set of discrepancies. Satellites generate earth observation data in an electromagnetic spectrum. Due to environmental factors, for example, continuous cloudy days during monsoons, etc., there is no data loss or no data possibility in generating earth observation data using multi-spectral satellite or optical satellite images, and sometimes lack of timely information due to satellite visibility issues limits usage of satellite images in crop monitoring applications
[0004] In some existing techniques, earth observation data is analyzed to extract features from a stack of one or more images using sparse autoencoders. The extracted features are applied to flood mapping applications as SAR data which is highly sensitive to water classes. The existing techniques are using only two images for analysing the earth observation data.
[0005] In some other existing techniques, during pre-processing speckle noise is not removed and thereby the existing techniques fail to produce superior quality of image for further analysis. In some existing techniques, a different image is constructed by reducing the intra-class variance gap, increasing inter-class variance gap, and enhancing edges. But the analysis using the different image has failed to achieve superior results.
[0006] Therefore, there arises a need to address the aforementioned technical drawbacks in existing technologies in processing satellite imagery accurately.
SUMMARY
[0007] In view of the foregoing, an embodiment herein provides a system for automatically determining a crop sown area from one or more satellite images at one or more timestamps using a deep learning model. The system includes a satellite that captures one or more satellite images of the earth at one or more timestamps. The satellite images include a set of spectral bands. The system includes a crop sown area estimation server that receives the one or more satellite images at one or more timestamps from the satellite. The satellite images include at least one of a first satellite image that is captured in a first set of spectral bands at a first timestamp, a second satellite image that is captured in a second set of spectral bands at a second timestamp, or a third satellite image that is captured in a third set of spectral bands at a third timestamp. The crop sown area estimation server includes a memory that stores a database, and a processor in communication with the memory, the processor is configured to (i) pre-process the at least one of the first satellite image captured at the first timestamp, the second satellite image captured at the second timestamp, or the third satellite image captured at the third timestamp to generate pre-processed satellite images using an image pre-processing technique, (ii) extract spatial features and temporal features from the pre-processed satellite images using a cross-domain autoencoder, (iii) train the deep learning model by providing historical spatial features, historical temporal features associated with historical clusters of features and historical categorization of areas as training data to generate a trained deep learning model, (iv) generate clusters of features by combining the spatial features, the temporal features using a statistical analysis, the clusters of features are generated by combining the features that are similar to each other into a cluster using a clustering technique, (v) categorize pixels that correspond to areas where sowing happens and the areas where sowing does not happen in the satellite images based on the clusters of features using the trained deep learning model, and (vi) determine the crop sown area from the categorized pixels by removing stray pixels.
[0008] In some embodiments, the processor is configured to pre-process the at least one of the first satellite image captured at the first timestamp, the second satellite image captured at the second timestamp, or the third satellite image captured at the third timestamp by, (i) applying an orbit file to determine an accurate position of the satellite and velocity of the satellite; (ii) removing thermal noise from the satellite images in a cross-polarization channel; (iii) calibrating the satellite images by converting pixel values to radiometrically synthetic aperture radar (SAR) backscatter; (iv) filtering, using speckle filters, speckle-noise from the satellite images; and (v) determining the pre-processed satellite images by performing terrain corrections to compensate geometric distortions in the satellite images.
[0009] In some embodiments, the processor is configured to train the deep learning model by, (i) obtaining the spatial features of the first satellite image and the temporal features of the second satellite image using a first generator; (ii) translating the spatial features of the first satellite image and the temporal features of the second satellite image into a translated second satellite image using the first generator; (iii) obtaining the temporal features of the first satellite image and the spatial features of the second satellite image using a second generator; (iv) translating the temporal features of the first satellite image and the spatial features of the second satellite image into a translated first satellite image using the second generator; (v) providing the translated second image to generate a first loss function when there is a difference in the plurality of spatial features of the first satellite image and the plurality of temporal features of the second satellite image and the translated second image to a first discriminator; (vi) providing the translated first image to generate a second loss function when there is a difference in the temporal features of the first satellite image and the spatial features of the second satellite image and the translated first image to a second discriminator; (vii) backpropagating the first loss function to the first generator and the second loss function to the to the second generator to optimize the translated first image and the translated second image such that the first loss function and the second loss function becomes zero; and (viii) generating an optimized spatial features, and an optimized temporal features associated with the satellite images as the historical spatial features, the historical temporal features as the training data to the trained deep learning model.
[0010] In some embodiments, the spatial features include a common information between the at least one of the first satellite image, the second satellite image, or the third satellite image, the temporal features include specific information of the at least one of the first satellite image, the second satellite image, or the third satellite image.
[0011] In some embodiments, the spatial features and the temporal features are extracted by determining a profile of time series data.
[0012] In some embodiments, the historical categorization of areas includes historical areas where sowing happens, and historical areas where sowing does not happen.
[0013] In one aspect, a processor-implemented method for automatically determining a crop sown area from one or more satellite images at one or more timestamps using a deep learning model is provided. The method includes obtaining the one or more satellite images of the earth at one or more timestamps from a satellite. The satellite images includes at least one of a first satellite image that is captured in a first set of spectral bands at a first timestamp, a second satellite image that is captured in a second set of spectral bands at a second timestamp, or a third satellite image that is captured in a third set of spectral bands at a third timestamp. The method includes pre-processing the at least one of the first satellite image captured at the first timestamp, the second satellite image captured at the second timestamp, or the third satellite image captured at the third timestamp to generate pre-processed satellite images using an image pre-processing technique. The method includes extracting spatial features and temporal features from the pre-processed satellite images using a cross-domain autoencoder. The method includes training the deep learning model by providing historical spatial features, historical temporal features associated with historical clusters of features, and a historical categorization of areas as training data to generate a trained deep learning model. The method includes generating clusters of features by combining the spatial features, the temporal features using a statistical analysis. In some embodiments, the clusters of features are generated by combining the features that are similar to each other into a cluster using a clustering technique. The method includes categorizing pixels corresponds to areas where sowing happens and the areas where sowing does not happen in the satellite images based on the clusters of features using the trained deep learning model. The method includes determining the crop sown area from the categorized pixels by removing stray pixels.
[0014] In some embodiments, the processor is configured to pre-process the at least one of the first satellite image captured at the first timestamp, the second satellite image captured at the second timestamp, or the third satellite image captured at the third timestamp by, (i) applying an orbit file to determine an accurate position of the satellite and velocity of the satellite; (ii) removing thermal noise from the satellite images in a cross-polarization channel; (iii) calibrating the satellite images by converting pixel values to radiometrically synthetic aperture radar (SAR) backscatter; (iv) filtering, using speckle filters, speckle-noise from the satellite images; and (v) determining the pre-processed satellite images by performing terrain corrections to compensate geometric distortions in the satellite images.
[0015] In some embodiments, the processor is configured to train the deep learning model by, (i) obtaining the spatial features of the first satellite image and the temporal features of the second satellite image using a first generator; (ii) translating the spatial features of the first satellite image and the temporal features of the second satellite image into a translated second satellite image using the first generator; (iii) obtaining the temporal features of the first satellite image and the spatial features of the second satellite image using a second generator; (iv) translating the temporal features of the first satellite image and the spatial features of the second satellite image into a translated first satellite image using the second generator; (v) providing the translated second image to generate a first loss function when there is a difference in the plurality of spatial features of the first satellite image and the plurality of temporal features of the second satellite image and the translated second image to a first discriminator; (vi) providing the translated first image to generate a second loss function when there is a difference in the temporal features of the first satellite image and the spatial features of the second satellite image and the translated first image to a second discriminator; (vii) backpropagating the first loss function to the first generator and the second loss function to the to the second generator to optimize the translated first image and the translated second image such that the first loss function and the second loss function becomes zero; and (viii) generating an optimized spatial features, and an optimized temporal features associated with the satellite images as the historical spatial features, the historical temporal features as the training data to the trained deep learning model.
[0016] In some embodiments, the spatial features include a common information between the at least one of the first satellite image, the second satellite image, or the third satellite image, the temporal features include specific information of the at least one of the first satellite image, the second satellite image, or the third satellite image.
[0017] The system and method of processing satellite imagery to estimate crop sown area using a deep learning model are provided. The system provides a classified estimation of crop sown area by removing cloud cover and haze. Also, the system analyzes sowing estimation to certain crop types or regions by the absence of threshold values or correlation functions. The system utilizes a single satellite data source by covering a large area. Hence there are no pre-processing costs involved and the system is computationally light.
[0018] These and other aspects of the embodiments herein will be better appreciated and understood when considered in conjunction with the following description and the accompanying drawings. It should be understood, however, that the following descriptions, while indicating preferred embodiments and numerous specific details thereof, are given by way of illustration and not of limitation. Many changes and modifications may be made within the scope of the embodiments herein without departing from the spirit thereof, and the embodiments herein include all such modifications.
BRIEF DESCRIPTION OF DRAWINGS
[0019] The embodiments herein will be better understood from the following detailed description with reference to the drawings, in which:
[0020] FIG. 1 illustrates a system for automatically determining a crop sown area from satellite images at different timestamps using a deep learning model according to some embodiments herein;
[0021] FIG. 2 illustrates a block diagram of a crop sown area estimation server according to some embodiments herein;
[0022] FIG. 3 illustrates a block diagram of a pre-processing module of FIG. 2 according to some embodiments herein;
[0023] FIG. 4 illustrates a block diagram of a feature extracting module of FIG. 2 according to some embodiments herein;
[0024] FIG. 5 illustrates a block diagram of a deep learning module of FIG. 2 according to FIG. 1 to some embodiments herein;
[0025] FIG. 6 illustrates an exemplary view of an estimation of crop sown area according to some embodiments herein;
[0026] FIGS. 7A- 7B are a flow diagram of a method for automatically determining a crop sown area from one or more satellite images at one or more timestamps using a deep learning model according to some embodiments herein; and
[0027] FIG. 8 is a schematic diagram of a computer architecture in accordance with embodiments herein.
DETAILED DESCRIPTION
[0028] The embodiments herein and the various features and advantageous details thereof are explained more fully with reference to the non-limiting embodiments that are illustrated in the accompanying drawings and detailed in the following description. Descriptions of well-known components and processing techniques are omitted so as to not unnecessarily obscure the embodiments herein. The examples used herein are intended merely to facilitate an understanding of ways in which the embodiments herein may be practiced and to further enable those of skill in the art to practice the embodiments herein. Accordingly, the examples should not be constructed as limiting the scope of the embodiments herein.
[0029] As mentioned, there remains a need for a system and method for automatically determining a crop sown area from satellite images at different timestamps using a deep learning model. Referring now to the drawings, and more particularly to FIGS. 1 through 8, where similar reference characters denote corresponding features consistently throughout the figures, and various embodiments are shown.
[0030] The following terms are referred to in the description, which is briefly described below:
[0031] SAR-Synthetic-aperture radar is a form of radar that is used to create two-dimensional images or three-dimensional reconstructions of objects, such as landscapes.
[0032] Sentinel-1 satellites survey every 12 days around the earth and carry a C-band synthetic-aperture radar (SAR) instrument which provides a collection of optical data in all-weather, day, or night. To achieve frequent revisits and high mission availability, two identical Sentinel-1 satellites (Sentinel-1A and Sentinel-1B) operate together. The satellites are phased 180 degrees from each other on the same orbit. This allows for what would be a 12-day revisit cycle to be completed in 6-7 days.
[0033] FIG. 1 illustrates a system 100 for automatically determining a crop sown area from satellite images at different timestamps using a deep learning model 108 according to some embodiments herein. The system 100 includes a satellite 102, and a crop sown area estimation server 106 that includes a deep learning model 108. The crop sown area estimation server 106 includes a device processor and a non-transitory computer-readable storage medium storing one or more sequences of instructions, which when executed by the device processor causes the processing of satellite imagery for crop sowing area estimation. The satellite 102 captures one or more satellite images of the earth at one or more timestamps. The crop sown area estimation server 106 receives one or more satellite images at one or more timestamps from the satellite 102 through a network 104. The one or more different timestamps may be at time T, T+1, ---T+N. The network 104 may include, but is not limited to, a wireless network, a wired network, a combination of the wired network and the wireless network or Internet, and the like. The satellite 102 may be Sentinel-1.
[0034] The satellite images include at least one of a first satellite image that is captured in a first set of spectral bands at a first timestamp, a second satellite image that is captured in a second set of spectral bands at a second timestamp, or a third satellite image that is captured in a third set of spectral bands at a third timestamp. In some embodiments, the one or more images are included in a time series of Sentinel-1 images in chronological order. In some embodiments, the Sentinel-1 images are stacked for three dates consecutively. The crop sown area estimation server 106 pre-processes the at least one of the first satellite image captured at the first timestamp, the second satellite image captured at the second timestamp, or the third satellite image captured at the third timestamp to generate a pre-processed satellite images using an image pre-processing technique.
[0035] The crop sown area estimation server 106 extracts spatial features and temporal features from the pre-processed satellite images using a cross-domain autoencoder. The first image may be obtained at time T, the second image may be obtained at time T+1, and the third image may be obtained at time T+2.
[0036] The deep learning model 108 is trained by providing historical spatial features, historical temporal features associated with historical clusters of features, and a historical categorization of areas as training data to generate a trained deep learning model. In some embodiments, the historical categorization of areas includes historical areas where sowing happens and historical areas where sowing does not happen. The deep learning model 108 sternly defines and extracts spatial variation information and exclusive variation information. The deep learning model 108 analyzes crop areas by observing certain patterns using spatial variation information and exclusive variation information.
[0037] The crop sown area estimation server 106 generates clusters of features by combining the spatial features, the temporal features using a statistical analysis. The crop sown area estimation server 106 generates the clusters of features by combining the features that are similar to each other into a cluster using a clustering technique. The crop sown area estimation server 106 categorizes pixels corresponds to areas where sowing happens and the areas where sowing does not happen in the satellite images based on the clusters of features using the trained deep learning model. The crop sown area estimation server 106 determines the crop sown area from the categorized pixels by removing stray pixels. The stray pixels are the ones that include distortions due to many causes. The stray pixels may be removed by selecting and deleting manually.
[0038] FIG. 2 illustrates a block diagram of a crop sown area estimation server 106 according to some embodiments herein. The block diagram of the crop sown area estimation server 106 includes a database 202, a pre-processing module 204, a feature extracting module 206, a deep learning model 108, a cluster generating module 208, a categorization of areas module 210 and a crop sown area determining module 212.
[0039] The pre-processing module 204 receives one or more satellite images of the earth at one or more timestamps from the satellite 102 through a network 104. The one or more different timestamps may be at time T, T+1, ---T+N. The satellite 102 may be Sentinel-1. The pre-processing module 204 pre-processes the one or more images at one or more different time stamps. The satellite images include at least one of a first satellite image that is captured in a first set of spectral bands at a first timestamp, a second satellite image that is captured in a second set of spectral bands at a second timestamp, or a third satellite image that is captured in a third set of spectral bands at a third timestamp. In some embodiments, the one or more images are included in a time series of Sentinel-1 images in a chronological order. In some embodiments, the Sentinel-1 images are stacked for three dates consecutively. The pre-processing module 204 pre-processes the at least one of the first satellite image captured at the first timestamp, the second satellite image captured at the second timestamp, or the third satellite image captured at the third timestamp to generate pre-processed satellite images using an image pre-processing technique.
[0040] The feature extracting module 206 extracts spatial features and temporal features from the pre-processed satellite images using a cross-domain autoencoder. The first image may be obtained at time T, the second image may be obtained at time T+1, and the third image may be obtained at time T+2.
[0041] The deep learning model 108 is trained by providing historical spatial features, historical temporal features associated with historical clusters of features, and historical categorization of areas as training data to generate a trained deep learning model. The deep learning model 108 sternly defines and extracts spatial variation information and exclusive variation information. The deep learning model 108 analyzes crop areas by observing certain patterns using the spatial variation information and exclusive variation information.
[0042] The cluster generating module 208 generates clusters of features by combining the spatial features, the temporal features using a statistical analysis . The clusters of features are generated by combining the features that are similar to each other into a cluster using a clustering technique. The categorization of areas module 212 categorizes pixels corresponds to areas where sowing happens and the areas where sowing does not happen in the satellite images based on the clusters of features using the trained deep learning model. The crop sown area determining module 214 determines the crop sown area from the categorized pixels by removing stray pixels
[0043] FIG. 3 illustrates a block diagram of a pre-processing module 204 of FIG. 2 according to some embodiments herein. The pre-processing module 204 receives one or more satellite images at one or more timestamps from the satellite 102. The satellite images include at least one of a first satellite image that is captured in a first set of spectral bands at a first timestamp, a second satellite image that is captured in a second set of spectral bands at a second timestamp, or a third satellite image that is captured in a third set of spectral bands at a third timestamp. The pre-processing module 204 pre-processes the at least one of the first satellite image captured at the first timestamp, the second satellite image captured at the second timestamp, or the third satellite image at the third timestamp to generate a pre-processed satellite images using an image pre-processing technique.
[0044] The block diagram of the pre-processing module 204 includes an apply orbit file module 302, a thermal noise removal module 304, a calibration module 306, a speckle filter module 308, and a terrain correction module 310. The apply orbit file module 302 applies an orbit file to determine an accurate position of the satellite and velocity of the satellite. The thermal noise removal module 304 thermal noise from the satellite images in a cross-polarization channel. The calibration module 306 calibrates the satellite images by converting pixel values to radiometrically synthetic aperture radar (SAR) backscatter. The speckle filter module 308 filters speckle-noise appearing in the one or more images to increase the quality of the one or more images. In some embodiments, the speckle noise is filtered using speckle filters. The terrain correction module 310 performs terrain corrections to compensate geometric distortions in the satellite images.
[0045] FIG. 4 illustrates a block diagram of a feature extracting module 206 of FIG. 2 according to some embodiments herein. The block diagram of the feature extracting module 206 includes a cross-domain autoencoder 402, a shared representation encoder 404A, an exclusive representation encoder 404B. The cross-domain autoencoder 402 extracts one or more spatial features and one or more exclusive temporal features from the at least one of the first satellite image, the second satellite image, or the third satellite image. In some embodiments, the first image may be obtained at time T, the second image may be obtained at time T+1, and the third image may be obtained at time T+2. The cross-domain autoencoder 402 includes the shared representation encoder 404A, and the exclusive representation encoder 404B. The shared representation encoder 404A extracts one or more spatial features from the at least one of the first satellite image, the second satellite image, or the third satellite image. The one or more spatial features include a shared representation that captures common information between the first image, the second image, and the third image. The exclusive representation encoder 404B extracts one or more exclusive temporal features from the first image, the second image, and the third image. The one or more exclusive temporal features include an exclusive representation that captures specific information of each image.
[0046] FIG. 5 illustrates a block diagram of a deep learning model 108 according to some embodiments herein. The block diagram of deep learning model 108 includes a first generator 502A, a second generator 502B, a first discriminator 504A, and a second discriminator 504B. The first generator 502A obtains the spatial features of the first satellite image and the temporal features of the second satellite image. The first generator 502A translates the spatial features of the first satellite image and the temporal features of the second satellite image into a translated second satellite image. The second generator 502B obtains the temporal features of the first satellite image and the spatial features of the second satellite image. The second generator 502B translates the temporal features of the first satellite image and the spatial features of the second satellite image into a translated first satellite image. The translated first image is provided to generate first loss function when there is a difference in the plurality of spatial features of the first satellite image and the plurality of temporal features of the second satellite image and the translated second image to a first discriminator 504A. The translated second image is provided to generate a second loss function when there is a difference in the temporal features of the first satellite image and the spatial features of the second satellite image and the translated first image to a second discriminator 504B. The first loss function is backpropagated to the first generator 502A to optimize the translated first image such that the first loss function becomes zero. The second loss function is backpropagated to the second generator 502B to optimize the translated second image such that the second loss function becomes zero. The deep learning model 108 generates an optimized spatial features, and an optimized temporal features are associated with the satellite images as the historical spatial features, the historical temporal features as the training data to the trained deep learning model.
[0047] FIG. 6 illustrates an exemplary view of an estimation of crop sown area according to some embodiments herein. At 602, the exemplary view includes an exemplary farm that is not captured in the estimation of crop sown area. At 604, the exemplary view includes an exemplary farm that is captured in the estimation of crop sown area. At 606, the exemplary view includes an exemplary farm that includes validation points overlayed on the sowing raster. The validation points are shown in captured polygons at 606A, for example.
[0048] FIGS. 7A- 7B are a flow diagram of a method for automatically determining a crop sown area from one or more satellite images at one or more timestamps using a deep learning model according to some embodiments herein. At step 702, the method includes obtaining the one or more satellite images at one or more timestamps of the earth from a satellite. In some embodiments, the satellite images includes at least one of a first satellite image that is captured in a first set of spectral bands at a first timestamp, a second satellite image that is captured in a second set of spectral bands at a second timestamp, or a third satellite image that is captured in a third set of spectral bands at a third timestamp. At step 704, the method includes pre-processing the at least one of the first satellite image captured at the first timestamp, the second satellite image captured at the second timestamp, or the third satellite image captured at the third timestamp to generate pre-processed satellite images using an image pre-processing technique. At step 706, the method includes extracting spatial features and temporal features from the pre-processed satellite images using a cross-domain autoencoder. At step 708, the method includes training the deep learning model by providing historical spatial features, historical temporal features associated with historical clusters of features and historical categorization of areas as training data to generate a trained deep learning model. At step 710, the method includes generating clusters of features by combining the spatial features, the temporal features using a statistical analysis, the clusters of features are generated by combining the features that are similar to each other into a cluster using a clustering technique. At step 712, the method includes categorizing pixels that correspond to areas where sowing happens and the areas where sowing does not happen in the satellite images based on the clusters of features using the trained deep learning model. At step 714, the method includes determining the crop sown area from the categorized pixels by removing stray pixels.
[0049] A representative hardware environment for practicing the embodiments herein is depicted in FIG. 8, with reference to FIGS. 1 through 7. This schematic drawing illustrates a hardware configuration of a crop sown area estimation server 106/computer system/ computing device in accordance with the embodiments herein. The system includes at least one processing device CPU 10 and at least one graphical processing device GPU 38 that may be interconnected via system bus 14 to various devices such as a random access memory (RAM) 12, read-only memory (ROM) 16, and an input/output (I/O) adapter 18. The I/O adapter 18 can connect to peripheral devices, such as disk units 38 and program storage devices 40 that are readable by the system. The system can read the inventive instructions on the program storage devices 40 and follow these instructions to execute the methodology of the embodiments herein. The system further includes a user interface adapter 22 that connects a keyboard 28, mouse 30, speaker 32, microphone 34, and/or other user interface devices such as a touch screen device (not shown) to the bus 14 to gather user input. Additionally, a communication adapter 20 connects the bus 14 to a data processing network 42, and a display adapter 24 connects the bus 14 to a display device 26, which provides a graphical user interface (GUI) 36 of the output data in accordance with the embodiments herein, or which may be embodied as an output device such as a monitor, printer, or transmitter, for example.
[0050] The foregoing description of the specific embodiments will so fully reveal the general nature of the embodiments herein that others can, by applying current knowledge, readily modify and/or adapt for various applications such specific embodiments without departing from the generic concept, and, therefore, such adaptations and modifications should and are intended to be comprehended within the meaning and range of equivalents of the disclosed embodiments. It is to be understood that the phraseology or terminology employed herein is for the purpose of description and not of limitation. Therefore, while the embodiments herein have been described in terms of preferred embodiments, those skilled in the art will recognize that the embodiments herein can be practiced with modification within the scope of the appended claims. ,CLAIMS:CLAIMS
I/We Claim:
1. A system for automatically determining a crop sown area from a plurality of satellite images at a plurality of timestamps using a deep learning model, the system comprising:
a satellite (102) that captures a plurality of satellite images of the earth at a plurality of timestamps, wherein the plurality of satellite images comprises a set of spectral bands;
a crop sown area estimation server (106) that receives the plurality of satellite images at a plurality of timestamps from the satellite (102), wherein the plurality of satellite images comprises at least one of a first satellite image that is captured in a first set of spectral bands at a first timestamp, a second satellite image that is captured in a second set of spectral bands at a second timestamp, or a third satellite image that is captured in a third set of spectral bands at a third timestamp, wherein the crop sown area estimation server (106) comprises,
a memory that stores a database (202);
a processor in communication with the memory, the processor is configured to:
pre-process, using an image pre-processing technique, the at least one of the first satellite image captured at the first timestamp, the second satellite image captured at the second timestamp, or the third satellite image captured at the third timestamp and generate a plurality of pre-processed satellite images;
characterized in that,
extract, using a cross-domain autoencoder, a plurality of spatial features and a plurality of temporal features from the plurality of pre-processed satellite images;
train the deep learning model (108) by providing a plurality of historical spatial features, a plurality of historical temporal features associated with a plurality of historical clusters of features and a plurality of historical categorization of areas as training data to generate a trained deep learning model;
generate a plurality of clusters of features by combining, using a statistical analysis, the plurality of spatial features, the plurality of temporal features, wherein the plurality of clusters of features are generated by combining the features that are similar to each other into a cluster using a clustering technique;
categorize, using the trained deep learning model, pixels correspond to a plurality of areas where sowing happens (606A) and the plurality of areas where sowing does not happen in the plurality of satellite images based on the plurality of clusters of features; and
determine the crop sown area (604) from the categorized pixels by removing stray pixels.
2. The system as claimed in claim 1, wherein the processor is configured to pre-process the at least one of the first satellite image captured at the first timestamp, the second satellite image captured at the second timestamp, or the third satellite image captured at the third timestamp by:
applying an orbit file to determine an accurate position of the satellite and a velocity of the satellite;
removing thermal noise from the plurality of satellite images in a cross-polarization channel;
calibrating the plurality of satellite images by converting pixel values to radiometrically synthetic aperture radar (SAR) backscatter;
filtering, using speckle filters, speckle-noise from the plurality of satellite images; and
determining the pre-processed plurality of satellite images by performing terrain corrections to compensate geometric distortions in the plurality of satellite images.
3. The system as claimed in claim 1, wherein the processor is configured to train the deep learning model (108) by,
obtaining, using a first generator (502A), the plurality of spatial features of the first satellite image and the plurality of temporal features of the second satellite image;
translating, using the first generator (502A), the plurality of spatial features of the first satellite image and the plurality of temporal features of the second satellite image into a translated second satellite image;
obtaining, using a second generator (502B), the plurality of temporal features of the first satellite image and the plurality of spatial features of the second satellite image;
translating, using the second generator (502B), the plurality of temporal features of the first satellite image and the plurality of spatial features of the second satellite image into a translated first satellite image;
providing, to a first discriminator (504A), the translated second image to generate a first loss function when there is a difference in the plurality of spatial features of the first satellite image and the plurality of temporal features of the second satellite image and the translated second image;
providing, to a second discriminator (504B), the translated first image to generate a second loss function when there is a difference in the plurality of temporal features of the first satellite image and the plurality of spatial features of the second satellite image and the translated first image;
backpropagating the first loss function to the first generator (502A)and the second loss function to the to the second generator (502B) to optimize the translated first image and the translated second image such that the first loss function and the second loss function becomes zero; and
generating an optimized plurality of spatial features, and an optimized plurality of temporal features associated with the plurality of satellite images as the historical spatial features, the historical plurality of temporal features as the training data to the trained deep learning model.
4. The system as claimed in claim 1, wherein the plurality of spatial features comprises a common information between the at least one of the first satellite image, the second satellite image, or the third satellite image, wherein the plurality of temporal features comprises specific information of the at least one of the first satellite image, the second satellite image, or the third satellite image.
5. The system as claimed in claim 1, wherein the plurality of spatial features and the plurality of temporal features are extracted by determining a profile of time series data.
6. The system as claimed in claim 1, wherein the plurality of historical categorization of areas comprises a plurality of historical areas where sowing happens, and a plurality of historical areas where sowing does not happens.
7. A processor-implemented method for automatically determining a crop sown area from a plurality of satellite images at a plurality of timestamps using a deep learning model, the method comprising:
obtaining the plurality of satellite images of the earth at a plurality of timestamps from a satellite, wherein the plurality of satellite images comprises at least one of a first satellite image that is captured in a first set of spectral bands at a first timestamp, a second satellite image that is captured in a second set of spectral bands at a second timestamp, or a third satellite image that is captured in a third set of spectral bands at a third timestamp;
pre-processing, using an image pre-processing technique, the at least one of the first satellite image captured at the first timestamp, the second satellite image captured at the second timestamp, or the third satellite image captured at the third timestamp to generate a plurality of pre-processed satellite images;
characterized in that,
extracting, using a cross-domain autoencoder, a plurality of spatial features and a plurality of temporal features from the plurality of pre-processed satellite images;
training the deep learning model by providing a plurality of historical spatial features, a plurality of historical temporal features associated with a plurality of historical clusters of features and a plurality of historical categorization of areas as training data to generate a trained deep learning model;
generating a plurality of clusters of features by combining, using a statistical analysis, the plurality of spatial features, the plurality of temporal features, wherein the plurality of clusters of features are generated by combining the features that are similar to each other into a cluster using a clustering technique;
categorizing, using the trained deep learning model, pixels corresponds to a plurality of areas where sowing happens (606A) and the plurality of areas where sowing does not happen in the plurality of satellite images based on the plurality of clusters of features;
determining, the crop sown area (604) from the categorized pixels by removing stray pixels.
8. The processor-implemented method as claimed in claim 7, wherein the method comprises pre-processing the first satellite image captured at the first timestamp and the second satellite image captured at the second timestamp by,
applying an orbit file to determine an accurate position of the satellite and velocity of the satellite;
removing thermal noise from the plurality of satellite images in a cross-polarization channel;
calibrating the plurality of satellite images by converting pixel values to radiometrically synthetic aperture radar (SAR) backscatter;
filtering, using speckle filters, speckle-noise from the plurality of satellite images; and
determining the pre-processed plurality of satellite images by performing terrain corrections to compensate geometric distortions in the plurality of satellite images.
9. The processor-implemented method as claimed in claim 7, wherein the deep learning model (108) is trained by,
obtaining, using a first generator (502A), the plurality of spatial features of the first satellite image and the plurality of temporal features of the second satellite image;
translating, using the first generator (502A), the plurality of spatial features of the first satellite image and the plurality of temporal features of the second satellite image into a translated second satellite image;
obtaining, using a second generator (502B), the plurality of temporal features of the first satellite image and the plurality of spatial features of the second satellite image;
translating, using the second generator (502B), the plurality of temporal features of the first satellite image and the plurality of spatial features of the second satellite image into a translated first satellite image;
providing, to a first discriminator (504A), the translated second image to generate a first loss function when there is a difference in the plurality of spatial features of the first satellite image and the plurality of temporal features of the second satellite image and the translated second image;
providing, to a second discriminator (504B), the translated first image to generate a second loss function when there is a difference in the plurality of temporal features of the first satellite image and the plurality of spatial features of the second satellite image and the translated first image;
backpropagating the first loss function to the first generator (502A)and the second loss function to the to the second generator (502B) to optimize the translated first image and the translated second image such that the first loss function and the second loss function becomes zero;
generating an optimized plurality of spatial features, and an optimized plurality of temporal features associated with the plurality of satellite images as the historical spatial features, the historical plurality of temporal features as the training data to the trained deep learning model.
10. The processor-implemented method as claimed in claim 7, wherein the plurality of spatial features comprises a common information between the first satellite image and the second satellite image, wherein the plurality of temporal features comprises specific information of the first satellite image and the second satellite image.
Dated this 10th day of January, 2022
Signature of Patent Agent:
Arjun Karthik Bala
(IN/PA1021)
| # | Name | Date |
|---|---|---|
| 1 | 202141001296-STATEMENT OF UNDERTAKING (FORM 3) [12-01-2021(online)].pdf | 2021-01-12 |
| 2 | 202141001296-PROVISIONAL SPECIFICATION [12-01-2021(online)].pdf | 2021-01-12 |
| 3 | 202141001296-PROOF OF RIGHT [12-01-2021(online)].pdf | 2021-01-12 |
| 4 | 202141001296-POWER OF AUTHORITY [12-01-2021(online)].pdf | 2021-01-12 |
| 5 | 202141001296-FORM FOR STARTUP [12-01-2021(online)].pdf | 2021-01-12 |
| 6 | 202141001296-FORM FOR SMALL ENTITY(FORM-28) [12-01-2021(online)].pdf | 2021-01-12 |
| 7 | 202141001296-FORM 1 [12-01-2021(online)].pdf | 2021-01-12 |
| 8 | 202141001296-EVIDENCE FOR REGISTRATION UNDER SSI(FORM-28) [12-01-2021(online)].pdf | 2021-01-12 |
| 9 | 202141001296-EVIDENCE FOR REGISTRATION UNDER SSI [12-01-2021(online)].pdf | 2021-01-12 |
| 10 | 202141001296-DRAWINGS [12-01-2021(online)].pdf | 2021-01-12 |
| 11 | 202141001296-DRAWING [12-01-2022(online)].pdf | 2022-01-12 |
| 12 | 202141001296-CORRESPONDENCE-OTHERS [12-01-2022(online)].pdf | 2022-01-12 |
| 13 | 202141001296-COMPLETE SPECIFICATION [12-01-2022(online)].pdf | 2022-01-12 |
| 14 | 202141001296-STARTUP [29-08-2023(online)].pdf | 2023-08-29 |
| 15 | 202141001296-FORM28 [29-08-2023(online)].pdf | 2023-08-29 |
| 16 | 202141001296-FORM 18A [29-08-2023(online)].pdf | 2023-08-29 |
| 17 | 202141001296-FER.pdf | 2023-12-14 |
| 18 | 202141001296-OTHERS [12-06-2024(online)].pdf | 2024-06-12 |
| 19 | 202141001296-FER_SER_REPLY [12-06-2024(online)].pdf | 2024-06-12 |
| 20 | 202141001296-CORRESPONDENCE [12-06-2024(online)].pdf | 2024-06-12 |
| 21 | 202141001296-CLAIMS [12-06-2024(online)].pdf | 2024-06-12 |
| 1 | SearchStrategyFileE_14-12-2023.pdf |