Description:FIELD OF THE INVENTION
This invention relates to a multimodal sensing and edge-ai framework system for growth and stress analysis in brinjal cultivation.
BACKGROUND OF THE INVENTION
Accurate plant phenotyping in Brinjal cultivation is critical for stress management, disease resistance and improving yield. As it is done largely manually which is time consuming as well as prone to human error. The methods used traditionally lacks off real-time monitoring and also not able to identify early signs of growth irregularities or plant stress effectively. Moreover, in remote areas where there is poor network infrastructure there are limited feasibility of cloud-dependent solutions. Hence, there is a need of an intelligent system which works on edges for analyzing multimodal data including visual inputs as well as the environmental sensors data. Existing models at present fails in adapting the variable conditions like soil properties. changing weather or Brinjal variety differences. Hence there is a need of autonomous, lightweight and scalable AI solution which is capable of delivering the accurate phenotypic insights in real-time.
STATE OF THE ART/ RESEARCH GAP:
Plant phenotyping plays an important role in precision agriculture. It helps in improving crop yield, stress tolerance and disease resistance. Brinjal (Solanum melongena) is widely cultivated and also is significantly economically crop which demands accurate phenotypic assessment for supporting modern breeding programs and on-field crop management. However, the methodologies which are being used currently for Brinjal phenotyping remain largely subjective, manual and labour-intensive. They often depend upon handheld instruments or visual observation, which are not only time consuming but also these are prone to human error and inconsistency. The traditional approaches are insufficient for handling needs of large-scale monitoring or high-frequency data collection required for real-time decision-making.
The existing high-throughput phenotyping platforms are typically limited to the greenhouse environments and depends upon sophisticated, expensive imaging setups and cloud-based centralized analytics. These systems are often inaccessible to resource-constrained farmers or smallholders due to complexity, infrastructure or cost. Furthermore, most of the current researches aims at major cereal crops such as maize, wheat and rice which neglects vegetable crops such as Brinjal and hence it is underrepresented in the development and deployment of smart phenotyping solutions. This leads to a gap in the availability of adaptable, real-time and affordable phenotyping technologies specifically designed for open-field cultivation of Brinjal.
In current phenotyping system the insufficient integration of multimodal data sources is another major research gap. Many other available solutions majorly focus upon visual data only and ignore contextual information such as temperature, humidity, soil moisture and light intensity which plays an important role in plant development and stress response. Without environmental context these purely image-based models often yield predictions which are inaccurate under varying field conditions. Additionally, the data collected through in these existing studies is often stored for the post-hoc analysis and not being processed in real-time for guiding immediate agronomic interventions.
Most of the cloud-based machine learning models used for the phenotypic analysis frequently depends upon consistent connectivity of the network for transmission of the data and inference. This limits their usability in remote and rural areas where there is no or less connectivity of internet. There is a lack of field-deployable, edge-computing-enable systems which can process the image and sensor data at local level using lightweight AI models. Edge-based processing would enhance system responsiveness, reduce latency and ensure the continuity of operation even in offline modes. Despite this, research on deploying quantized, optimised or pruned models for phenotyping tasks on the edge devices remains scarce, particularly in the vegetable crops context.
Another gap exists in the terms of algorithmic capabilities in dynamic modelling of plant growth stages over time. Most of the models focuses upon classification of static image or single-time-point analysis without accounting for the temporal evolution of phenotypic characteristics. Hence, there is a need of incorporating time-series modelling techniques like long short-term memory (LSTM) networks or recurrent neural networks (RNNs) for learning from historical data and predict future growth trends. This can enable the early warnings for stress-induced developmental changes and yield fluctuations. However, the application of such models is underexplored in low resource, edge-computing contexts.
The existing solutions of phenotyping often lacks off generalization across the different environmental conditions or varieties of Brinjal. Transfer learning i.e. a powerful technique which enables model trained on a condition to adapt to new and unseen scenarios which has not been sufficiently applied in phenotyping research of Brijal. This makes the models less difficult and robust to deploy across different contexts of farming. Furthermore, there exist a limited work on integrating self-optimizing learning systems which can adjust their model parameters and predictions automatically as new field data becomes available.
For summarising significant research gaps remains in developing scalable, real-time and low-cost Brinjal phenotyping systems which can integrate multimodal data, work offline through edge devices and support dynamic stress prediction and growth modelling. There is also a critical need of adaptable AI algorithms that can generalise across different Brinjal cultivars and field conditions while delivering consistent performance. Addressing these gaps can help in empowering farmers with timely, actionable insights and advance the broader field of crop informatics and intelligent horticulture.
CN111357009B The present invention relates to a method for data analysis of plant phenotypes of individual plants in a field and a data acquisition and evaluation system for data analysis of plant phenotypes of individual plants in a field. Furthermore, the invention relates to a mobile platform for use in the method and/or the data acquisition and evaluation system, and to the use of the mobile platform in the method and/or the data acquisition and evaluation system. The method comprises the following steps: capturing spectral data via a hyperspectral imaging sensor, capturing image data via an image sensor, capturing geo-reference data via an inertial measurement unit, spatializing the image data to generate geo-reference image data and a digital surface model, spatializing the spectral data, generating geo-reference spectral data based on the spatialized spectral data and the digital surface model; and superimposing the geo-referenced image data and the geo-referenced spectral data with the field map information to generate a high resolution analysis dataset.
RESEARCH GAP: The proposed device differs by enabling real-time, edge-AI-based analysis using low-power, solar-driven multimodal sensors and cameras, whereas the described invention relies on hyperspectral imaging and post-processed spatial data fusion using mobile platforms.
US10803312B2 Inputs from sensors (e.g., image and environmental sensors) are used for real-time optimization of plant growth in indoor farms by adjusting the light provided to the plants and other environmental factors. The sensors use wireless connectivity to create an Internet of Things network. The optimization is determined using machine-learning analysis and image recognition of the plants being grown. Once a machine-learning model has been generated and/or trained in the cloud, the model is deployed to an edge device located at the indoor farm to overcome connectivity issues between the sensors and the cloud. Plants in an indoor farm are continuously monitored and the light energy intensity and spectral output are automatically adjusted to optimal levels at optimal times to create better crops. The methods and systems are self-regulating in that light controls the plant's growth, and the plant's growth in-turn controls the spectral output and intensity of the light.
RESEARCH GAP: The proposed device differs by being designed for open-field crop monitoring (e.g., brinjal) using solar-powered, multimodal edge-AI systems, whereas the described system is optimized for closed-loop control of indoor farming environments via cloud-trained models and IoT-based light regulation.
None of the prior art indicate above either alone or in combination with one another disclose what the present invention has disclosed. This invention relates to a multimodal sensing and edge-ai framework system for growth and stress analysis in brinjal cultivation.
SUMMARY OF THE INVENTION
This summary is provided to introduce a selection of concepts, in a simplified format, that are further described in the detailed description of the invention.
This summary is neither intended to identify key or essential inventive concepts of the invention and nor is it intended for determining the scope of the invention.
To further clarify advantages and features of the present invention, a more particular description of the invention will be rendered by reference to specific embodiments thereof, which is illustrated in the appended drawings. It is appreciated that these drawings depict only typical embodiments of the invention and are therefore not to be considered limiting of its scope. The invention will be described and explained with additional specificity and detail with the accompanying drawings.
The device proposed is an integrated and field-deployable system for real-time plant phenotyping in Brinjal cultivation by combining image hardware, advanced sensors, edge computing and machine learning for monitoring the characteristics of the plant throughout the growth cycle. This smart agricultural device is designed for operating autonomously in open-field conditions and provide timely insights into the physiological as well as morphological state of Brinjal plants, including prediction of growth trends, early detection of stress and yield forecasting. The device’s core lies in its modular architecture, consisting of two primary subsystems i.e. an Imaging unit and a Sensor Unit, each of them is occupied with an independent computing core with solar power supply, wireless communication module and real-time clock.
BRIEF DESCRIPTION OF THE DRAWINGS
The illustrated embodiments of the subject matter will be understood by reference to the drawings, wherein like parts are designated by like numerals throughout. The following description is intended only by way of example, and simply illustrates certain selected embodiments of devices, systems, and methods that are consistent with the subject matter as claimed herein, wherein:
Fig. 1 Overall Architecture
Fig.2 Smart Numeric Sensory Data Collector
Fig 3 Edge Vision Devices
The figures depict embodiments of the present subject matter for the purposes of illustration only. A person skilled in the art will easily recognize from the following description that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles of the disclosure described herein.
DETAILED DESCRIPTION OF THE INVENTION
The detailed description of various exemplary embodiments of the disclosure is described herein with reference to the accompanying drawings. It should be noted that the embodiments are described herein in such details as to clearly communicate the disclosure. However, the amount of details provided herein is not intended to limit the anticipated variations of embodiments; on the contrary, the intention is to cover all modifications, equivalents, and alternatives falling within the scope of the present disclosure as defined by the appended claims.
It is also to be understood that various arrangements may be devised that, although not explicitly described or shown herein, embody the principles of the present disclosure. Moreover, all statements herein reciting principles, aspects, and embodiments of the present disclosure, as well as specific examples, are intended to encompass equivalents thereof.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments. As used herein, the singular forms “a",” “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises,” “comprising,” “includes” and/or “including,” when used herein, specify the presence of stated features, integers, steps, operations, elements and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and/or groups thereof.
It should also be noted that in some alternative implementations, the functions/acts noted may occur out of the order noted in the figures. For example, two figures shown in succession may, in fact, be executed concurrently or may sometimes be executed in the reverse order, depending upon the functionality/acts involved.
In addition, the descriptions of "first", "second", “third”, and the like in the present invention are used for the purpose of description only, and are not to be construed as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Thus, features defining "first" and "second" may include at least one of the features, either explicitly or implicitly.
Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which example embodiments belong. It will be further understood that terms, e.g., those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
The device proposed is an integrated and field-deployable system for real-time plant phenotyping in Brinjal cultivation by combining image hardware, advanced sensors, edge computing and machine learning for monitoring the characteristics of the plant throughout the growth cycle. This smart agricultural device is designed for operating autonomously in open-field conditions and provide timely insights into the physiological as well as morphological state of Brinjal plants, including prediction of growth trends, early detection of stress and yield forecasting. The device’s core lies in its modular architecture, consisting of two primary subsystems i.e. an Imaging unit and a Sensor Unit, each of them is occupied with an independent computing core with solar power supply, wireless communication module and real-time clock.
The Sensor Unit is responsible for capturing contextual and environmental data which influences plant growth. It is equipped with a suit of sensors which includes humidity sensors, pH sensor, light intensity sensor, soil moisture sensor, load cell (fruit weight sensor) and temperature sensor. These sensors provide essential data on root zone hydration, sunlight exposure, microclimatic conditions and soil nutrient availability. All the sensor readings are timestamped using a real-time clock and processed locally using a low-power computing unit. The processed data is then sent to a cloud server using Wi-Fi or LoRaWAN as per depending on the availability of network and also stored in local storage.
The Imaging unit is designed for capturing high-resolution visual data of the Brinjal plant which particularly focuses upon morphological feature like fruit shape, leaf size, canopy structure and stem thickness. It includes a high definition RGB camera as the standard imaging component and a multispectral camera for early disease or stress detection. AN Infrared camera can be incorporated for detecting ripening stages or water stress invisible in visible spectrum. This unit is operated using edge computing device with higher processing capability i.e. Raspberry Pi 4 which handles the initial stage of image preprocessing and lightweight machine learning interference. Like the sensory data collector unit it also operates independently using a rechargeable battery which is attached to solar panel which ensures continuous monitoring even in remote locations.
Both the units are designed such that for seamless synchronization and integration. The real-time clocks are used for ensuring that images and sensor data are accurately aligned for multimodal analysis. The data synchronised is transmitted to the cloud server for advanced processing. This data is also stored at local level for on-edge inference. The device is supported by artificial intelligence algorithms which are optimized for edge computing through pruning techniques and quantization which allows a real-time analysis without heavy computational loads. CNNs are also used on the device which identifies the signs of disease, colour variations or morphological changes at their early stages. The system is also having time-series modelling through LSTM networks which tracks the plant development over the time for prediction of growth future trends.
Also, there exists self-optimized models in the device for prediction of yields which adapts to the environmental conditions changes and characteristics of Brinjal variety. These models are using feature which are image-derived as well as environmental data for estimating the weight of the fruit, their count and their maturity. These models can be adapted for the different conditioned fields due to there transfer learning techniques. The final output which is predicted by the device is visible on the farmer-accessible dashboard or on the mobile interface which offers alerts, visual summaries and recommendations.
This device as whole works as field phenotyping platform which is intelligent, compact and capable in processing, capturing and analysing the real-time data of the Brinjal plant. This device runs on low power and is solar powered. This device also can run on edge computing and is consists of multimodal input handling which makes the device suitable for wide-scale deployment in commercial farms as well as research plots.
Best Method of Working:
Disclosed herein a multimodal sensing and edge-ai framework system for growth and stress analysis in brinjal cultivation comprises of Cloud Server (50), Vision device (60), Sensor data collector device (SDCD) (70), Computing Unit (71), SD card (72), Wifi modem (73), Real time clock (74), Solar panel (75), Battery (76), Power Supply (77), Display (78), Light intensity sensor(79), Temperature & humidity sensor (80), Soil conductivity sensor (81), GPS (82), Soil moisture sensor (83), PH sensor (84), Raspberry Pi (61), Power supply (62), Battery (63), Solar panel (64), RGB Camera (65), IR camera (65a), Keyboard (66), Mouse (67) & Data storage (68).
A device with two independent modules- an imaging unit and a sensor unit, both of them are solar-powered, each integrated with a wireless communication interface, real-time clock and local processing core. The device operates autonomously in the field, timestamping, collecting and transmitting multimodal data to a cloud server or running machine learning inference locally. It supports early stress detection, real-time growth and yield prediction for Brinjal plants.
The unit which runs on sensors includes multiple environmental sensors like humidity, temperature, pH, light intensity and soil moisture which are connected to a microcontroller with real-time clock functionality. Each measurement taken from each sensor is timestamped and wirelessly transmitted to a local or remote analysis system. This unit enables contextual understanding of plant growth and health conditions without manual intervention.
The imaging unit consists with RGB camera and IR camera, configured to capture visual data of Brinjal plant morphology. It uses an edge device to process the images at local level and can perform the real-time feature extraction or anomaly detection using the lightweight models. All image data is timestamped and synchronized with sensor data for integrated phenotyping.
A hybrid AI algorithm utilizing CNNs and LSTM, processes synchronised sensor and image data for predicting growth trends and extracting plant traits. The algorithm is optimised using pruning and quantization techniques to support deployment on low-resource edge devices. It enables early stress detection, automated phenotypic analysis and yield estimation in Brinjal cultivation.
The system uses a multimodal machine learning algorithm which combines image features and sensor data for extracting phenotypic characteristics of Brinjal plants. A convolutional neural network processes multispectral and RGB images for detecting morphological features such as fruit shape, leaf size and colour changes.
Simultaneously, environmental data like humidity, temperature and soil moisture are integrated for contextual analysis. LSTM networks model growth trends over time using sequential image and sensor data. The algorithms are optimized through quantization and pruning for efficient and real-time inference on edge devices. The algorithm used is as follows:
Algorithm Used
BEGIN SYSTEM
// ---------- Initialization ----------
INITIALIZE SensorUnit with:
TemperatureSensor, HumiditySensor, SoilMoistureSensor, LightSensor, PH_Sensor
Optional: LeafWetnessSensor, WeightSensor
RTC_S, WiFi_Module_S, Battery, Solar_Panel
INITIALIZE ImageUnit with:
RGB_Camera, Optional: MultispectralCamera, DepthCamera, IR_Camera
RTC_I, WiFi_Module_I, Edge_Computer, Battery, Solar_Panel
INITIALIZE CloudServer with:
Data_Preprocessing_Module, Phenotype_CNN_Model, LSTM_Growth_Model, Yield_Model
// ---------- Data Collection Loop ----------
LOOP Every T minutes:
// --- Sensor Unit Data Collection ---
SensorData ← {}
SensorData.Timestamp ← RTC_S.GET_TIME()
SensorData.Temp ← TemperatureSensor.READ()
SensorData.Humidity ← HumiditySensor.READ()
SensorData.Moisture ← SoilMoistureSensor.READ()
SensorData.Light ← LightSensor.READ()
SensorData.PH ← PH_Sensor.READ()
SensorData.Optional ← READ_OPTIONAL_SENSORS()
WiFi_Module_S.SEND_TO_CLOUD(SensorData)
// --- Imaging Unit Data Collection ---
ImageData ← {}
ImageData.Timestamp ← RTC_I.GET_TIME()
ImageData.RGB ← RGB_Camera.CAPTURE()
IF MultispectralCamera.ENABLED THEN
ImageData.MS ← MultispectralCamera.CAPTURE()
IF DepthCamera.ENABLED THEN
ImageData.Depth ← DepthCamera.CAPTURE()
IF IR_Camera.ENABLED THEN
ImageData.IR ← IR_Camera.CAPTURE()
WiFi_Module_I.SEND_TO_CLOUD(ImageData)
END LOOP
// ---------- Cloud Server Processing ----------
ON Cloud_Receive(SensorData, ImageData):
SyncedData ← PREPROCESS(SensorData, ImageData)
// --- Morphological Feature Extraction ---
MorphTraits ← Phenotype_CNN_Model.EXTRACT_TRAITS(SyncedData)
// Traits: Leaf_Size, Fruit_Shape, Color, Surface_Health
// --- Time-Series Growth Prediction ---
Growth_Trend ← LSTM_Growth_Model.PREDICT(SyncedData.TimeSeries)
// --- Yield Forecasting ---
Yield ← Yield_Model.ESTIMATE(MorphTraits, SensorData)
SAVE_TO_DATABASE(SyncedData.Timestamp, MorphTraits, Growth_Trend, Yield)
UPDATE_DASHBOARD(MorphTraits, Growth_Trend, Yield)
END Cloud_Receive
// ---------- Edge Optimization ----------
OPTIMIZE_MODELS():
APPLY Quantization TO Phenotype_CNN_Model
APPLY Pruning TO LSTM_Growth_Model
DEPLOY Optimized_Models TO Edge_Computer
END SYSTEM
ADVANTAGES OF THE INVENTION:
• The system captures and analyses plant characteristics continuously in real-time using synchronized image and sensor data. This helps in early detection of stress symptoms like abnormal growth or leaf discoloration. It improves decision-making on fertilization, irrigation and crop management without manual effort.
• All the processing takes places locally on solar-powered edge devices which eliminates its dependency on cloud connectivity. This makes the system ideal for every remote agricultural region which are having poor or no internet access. It ensures uninterrupted data collection and analysis even in offline conditions.
• The combination of environmental sensors along with visual imaging data improves phenotyping precision. Models considers both visible traits as well as external factors for reducing false positives or negatives. The multimodal approach provides robust and reliable plant health assessments.
• Machine learning models forecast plant growth and yield which is based upon real-time and historical data. This helps in planning the harvest schedules and resource allocation with high accuracy. Self-optimizing models adjust predictions over time, improving long-term reliability.
, Claims:1. A multimodal sensing and edge-AI framework system for growth and stress analysis in brinjal cultivation comprising a Cloud Server (50), a Vision Device (60), and a Sensor Data Collector Device (SDCD) (70), wherein the Vision Device (60) and the Sensor Data Collector Device (70) each include a Computing Unit (71), Data Storage (68, 72), Wi-Fi Modem (73), Real Time Clock (74), Solar Panel (64, 75), and Battery (63, 76) for autonomous and continuous operation in open-field conditions.
2. The system as claimed in claim 1, wherein the Sensor Data Collector Device (70) further comprises environmental sensors including Light Intensity Sensor (79), Temperature & Humidity Sensor (80), Soil Conductivity Sensor (81), Soil Moisture Sensor (83), and pH Sensor (84), each connected to the Computing Unit (71) and configured to generate timestamped data.
3. The system as claimed in claim 1, wherein the Vision Device (60) comprises an RGB Camera (65) and an IR Camera (65a) for capturing visual and thermal images of brinjal plant morphology including fruit shape, leaf size, canopy structure, and stem thickness.
4. The system as claimed in claim 3, wherein the Vision Device (60) further comprises a Raspberry Pi (61) as an edge-computing module for preprocessing captured image data and executing lightweight machine learning inference for early stress detection and growth analysis.
5. The system as claimed in claim 1, wherein the Real Time Clocks (74) of the Sensor Data Collector Device (70) and the Vision Device (60) synchronize image and sensor data for multimodal phenotypic analysis.
6. The system as claimed in claim 1, wherein the Solar Panels (64, 75) and Batteries (63, 76) supply uninterrupted power to the Vision Device (60) and the Sensor Data Collector Device (70), enabling field deployment without external power dependency.
7. The system as claimed in claim 1, wherein the Wi-Fi Modems (73) and optional LoRaWAN modules transmit multimodal sensor and image data to the Cloud Server (50) for advanced processing, storage, and visualization.
8. The system as claimed in claim 1, wherein the Cloud Server (50) hosts a hybrid artificial intelligence algorithm comprising Convolutional Neural Networks and Long Short-Term Memory networks for extracting phenotypic traits, detecting stress, and predicting growth trends from synchronized sensor and image data.
9. The system as claimed in claim 8, wherein the artificial intelligence algorithm is optimized using pruning and quantization techniques to enable efficient deployment on low-resource edge devices such as the Raspberry Pi (61).
10. The system as claimed in claim 1, wherein the outputs including stress alerts, growth predictions, and yield estimations are displayed on a farmer-accessible Display (78) or mobile interface with visual summaries and actionable recommendations.