DESC:Field of Invention
The present invention relates to seed classification. More particularly the invention relates to system and process of seed classification. Even more particularly the invention relates to system and process of seed classification through seed vision.
Background of the Invention
In agriculture system examination and investigation of seeds quality and seed classification is necessary step to get high yield cultivation of seed. Computer vision, Image analysis methods, photoelectric detection, X- ray imaging, and color selection technique have also been applied in seed testing and classification. Various methods are used for seed sample testing, some are discussed here.
Reference has been made to CN101905215 titled “Digitalized paddy rice seed testing machine”, by LINGFENG DUAN et al, dated 22-08-2010 which relates to an online non-destructive testing method which is suitable for automatically measuring and recording the total grain number, the filled grain number, the maturing rate, the grain lengths, the grain widths and the length-width ratio of a single plant or single-batch paddy rice in the paddy rice seed testing process for agricultural scientific research personnel. The digitalized paddy rice seed testing machine mainly comprises six functional modules of a bar code recognizer, a grain transfer device, an air separation device, an image acquisition device, a PLC (Programmable Logic Controller) and a computer system. The digitalized paddy rice seed testing machine can be used for separating filled grains and empty and shriveled grains by utilizing the air separation device, dynamically acquiring grain parameters by utilizing a machine vision technology and simultaneously measuring the total grain number and the filled grain number of the paddy rice, thereby overcoming a bottleneck that the traditional testing measure cannot simultaneously acquire the total grain number and the filled grain number and realizing the detection to the grain-size parameters of the paddy rice at the same time of calculating the paddy rice grain number. The invention can be also widely applied to the seed testing work of other grain crops
Reference has been made to WO2017212427 titled “Device and method for classifying seeds”, by LINGFENG DUAN et al, dated 07.06.2017 which relates to a device and method for classifying seeds, for example coffee seeds. The device is characterized by: a seed-feeding mechanism; a seed-containing mechanism connected to the seed-feeding mechanism; an electronic seed-viewing system operationally disposed in the seed-containing mechanism; a seed-ejecting mechanism connected to the outlet of the seed-containing mechanism, the electronic seed-viewing system having a central processing unit that implements methods for classifying seeds. The method is characterized by the steps of: a) obtaining a digital image of the seed; b) storing the RGB components of the image obtained in step a); c) generating a histogram for each colorimetric and luminosity component of the histogram of step b); d) determining the thresholding point according to Otsu's method; e) obtaining a binary image; f) removing areas; g) obtaining the edges; h) obtaining the vectors corresponding to the seeds; i) identifying black seeds; j) identifying seeds with fermentation- and immaturity-related defects; k) identifying seeds with mechanical damage; and l) actuating a seed-ejecting mechanism, activating actuators for black seeds, actuators for seeds with fermentation- and immaturity-related defects and actuators for seeds with mechanical damage.
Reference has been made to CN101929961 titled “Device and method for detecting quality of rice seeds, identifying varieties and grading”, by LINGFENG DUAN et al, dated 18.06.2009 which disclose a device and a method for detecting quality of rice seeds, identifying varieties and grading, wherein the device comprises a rice seed discharging and conveying device, a computer vision identifying and processing device for collecting, analyzing and processing images and an automatic rice seed sorting device, and the three devices are sequentially arranged through lines. The device and the method can overcome the defects of high cost, low efficiency, low accuracy and great difficulty in supplying batch seeds in the prior art to realize the advantages of low cost, high efficiency, high accuracy and easy supply of batch seeds.
Reference has been made to US2014050365 titled “SEED CLASSIFICATION USING SPECTRAL ANALYSIS TO DETERMINE EXISTENCE OF A SEED STRUCTURE”, by COLEMAN MICHAEL DAVID et al, dated 19.07.2011 which disclose a method and system that models a seed structure and uses a spectral analysis to identify which morphological seed structures are existent in the seed/seedling. Additionally, this disclosure relates to a method and system that applies multi-spectral analysis using predetermined models of a seed/seedling to identify which morphological structures are existent in the seed/seedling. The information about the existence or non-existence of structures of the seed/seedling is used to classify the seed as having a specific characteristic, for later commercial use or sale. The seed market determines which specific characteristic the method will use to classify the seed/seedling. The individual seed classification may help determine associated seed lot germination values.
Reference has been made to CN207516257 titled “Wheat seed grain image acquisition platform based on machine vision”, by DU XIONGZI; dated 01.12.2017 which disclose a wheat seed grain image acquisition platform based on machine vision, including the type orifice plate, there is type hole layer board type orifice plate below, type orifice plate and type hole layer board all are provided with the type hole count volume on a plurality of type holes and two boards, the position one -to -one, type hole size on the type orifice plate is greater than the biggest seed grain size of wheat, type hole size on the layer board of type hole is less than the minimum seed grain size of wheat, be equipped with the brush above the type orifice plate, type hole layer board is equipped with the handle, can make wheat seed fall into seed case by height -adjusting, type orifice plate top is provided with light source and CCD camera, type hole layer board below is provided with the scanner, top view that two image acquisition components can acquire wheat seed grain like with the bottom view like, the utility model discloses wheat article outward appearance quality detecting still provides the system, include with image acquisition platform communication connection's image processing platform carries out image data's collection and analysis, the utility model has the characteristics of simple structure, convenient to use, powerful etc.
Reference has been made to US5901237 titled “Method and apparatus for assessing the quality of a seed lot”, by CONRAD ROBERT; dated 17.03.1995 which disclose a method and apparatus for assessing the quality of a seed lot utilizes image analysis equipment to generate an indication of seed quality. In one method, the quality of the seed lot may be determined by generating an image of a plurality of seedlings grown from a plurality of seeds selected from the seed lot, determining the leaf surface area of each of the seedlings from the image, determining the total leaf surface area of the seedlings from the image, determining a surface area threshold relating to the leaf surface areas of a plurality of the seedlings, determining the proportion of the seedlings which have a leaf surface area that exceeds the surface area threshold, and generating an indication of seed quality based upon that proportion.
Reference has been made to WO2021134110 titled “AUTOMATICALLY ASSIGNING HYBRIDS OR SEEDS TO FIELDS FOR PLANTING”, by SURENUT PTY LTD dated 2019-12-29 which disclose a method and system for detecting an aflatoxin on a grain, seed or nut which includes sorting a plurality of the grain seeds in single file, capturing a plurality of shortwave infrared images of each seed, comparing the wavelengths from the captured image with the wavelengths indicative of an aflatoxin presence at a predetermined concentration, and ejecting from a group of the seeds those seeds that have an aflatoxin concentration greater than the predetermined concentration as indicated by the wavelengths from the captured images.
Reference has been made to WO2004066083 titled “SEED IMAGE ANALYZER”, by THE OHIO STATE UNIVERSITY RESEARCH FOUNDATION dated 2003-01-21 which discloses a computer imaging systems are employed (100) to image, analyze, classify and/or sort seeds and other agricultural items (110). The systems may be local and/or remote, serial and/or parallel processing, employing various classification schemes (130) including Fisher Linear Discriminant processing and various hardware including a color, digital scanner (120).
Reference has been made to CA2640639 titled “DEVICE AND METHOD FOR OPTICAL MEASUREMENT OF GRAINS FROM CEREALS AND LIKE CROPS” by FOSS ANALYTICAL AB dated 2006-03-02 which discloses a device (200) for optical measuring of grains for analysis of the quality of said grains, comprises a feeder (102) which is arranged to feed at least one grain (101b) in a direction of transport (107), a light source (105) which is arranged to illuminate said grain (101b) along a line, a detector (108) which is arranged to detect reflection (109) from the surfaces of said grain and an analyzer (111) which is arranged to analyze said detected reflection in order to determine a height profile of the grain along the line and to determine three-dimensional surface topographical information on said grain based on a plurality of determined height profiles as said grain is transported. The device (200) further comprises an arrangement (220) used in generating a two-dimensional image and the analyzer (111) is arranged to determine a quality of said grain based on the three-dimensional surface information and the two-dimensional image of the same grain.
Reference has been made to CN101109743 titled “PORTABLE CEREAL ANALYZER BASED ON DIGITAL PICTURE PROCESSING TECHNIQUE” by UNIV KENT CANTERBURY dated 2007-09-10 which provides a portable grain analyzer that is based on digital image processing technology. The analyzer comprises a mechanic-electric feeding system, a digital imaging system, and an image processing and data analysis system. The grains entered into the mechanic-electric feeding system will be imaged by the digital imaging system, then the imaged signals are transmitted to the image processing and data analysis system. The mechanic-electric feeding system and the digital imaging system operate under control by a computer through an image collecting and a system controlling circuit board. The mechanic-electric feeding system has a funnel, under which, a distributing plate connected with a vibrator is provided; below the distributing plate, a rotary platform for receiving grains is provided; on the rotary platform, a scraping piece is provided; under the rotary platform, a collector is provided; above the rotary platform, the digital imaging system is provided. The analyzer can automatically feed samples, is of small size, easy to operate, of high efficiency and low cost, and will not damage the sample. The invention is applicable for classification, identification, true and false verification, and quality inspection of rice and similar grains and beans.
Reference has been made to EP3038054 titled “GRAIN QUALITY MONITORING” by GRAIN QUALITY MONITORING dated 2014-12-26 which discloses a method and non-transitory computer-readable medium capture an image of bulk grain and apply a feature extractor to the image to determine a feature of the bulk grain in the image. For each of a plurality of different sampling locations in the image, based upon the feature of the bulk grain at the sampling location, a determination is made regarding a classification score for the presence of a classification of material at the sampling location. A quality of the bulk grain of the image is determined based upon an aggregation of the classification scores for the presence of the classification of material at the sampling locations.
Reference has been made to USRE45489 titled “AUTOMATED HIGH-THROUGHPUT SEED SAMPLE HANDLING SYSTEM AND METHOD” by PIONEER HI BRED INTERNATIONAL INC dated 2001-02-02 which discloses a method and apparatus for processing seed or seed samples includes an autonomous sorter which sorts seed by pre-programmed criteria. Optional features can include a counter to autonomously ensure the correct number of seeds to a seed package, a cleaning device, a sheller, and a label applicator. A conveyance path, controlled automatically, can move the seed to appropriate and desired stations during the processing while maintaining the sample segregating from other samples. Validation of the sample can be pre-required and information about the sample can be derived and stored for further use.
Reference has been made to CN101120365 titled “SEED COUNTING AND FREQUENCY MEASUREMENT APPARATUS AND METHOD” by SYNGENTA PARTICIPATIONS AG dated 2005-02-17 which discloses an improved device for measuring the count and frequency of seeds in a stream of seeds is described. The device is useful for measuring the frequency and accuracy of seed planting devices. A seed counting system can include an imaging region, an image sensing device, and a lens between the imaging region and the image sensing device. An optical distance extender between the imaging region and the lens, creates an effective optical distance between the imaging region and the lens that is substantially greater than the physical distance between the imaging region and the lens, thereby providing a substantial depth of field.
Reference has been made to US10244692 titled “METHOD OF IMPROVED PLANT BREEDING” by BASF PLANT SCIENCE GMBH dated 2005-02-17 which discloses an improved plant breeding system for high throughput analysis of plant phenotype and genotype is provided. A method for analyzing the impact of genetic modifications on plants and selecting a plant with a genetic modification of interest is also provided. Also provided is a method for developing marketable information for improved plant breeding and a method for collecting data on a selected plant phenotype for rapid analysis of the effect of a genetic modification on the selected phenotype.
Reference has been made to US2020338599 titled “SYSTEMS AND METHODS FOR SORTING OF SEEDS” by SEEDX TECH INC dated 2017-12-03 which discloses a system for sorting seeds are disclosed, as well as batches of seeds that have been sorted using the systems.
However, none of the above discussed invention discloses the System and process of seed classification through Seed Vision which analyzes different varieties of seeds and classified on the basis of their morphological features via image processing to understand the morphological differences. The present invention also comprises a camera to take images of seeds, a microcontroller to control the working of the conveyor, seed dropper, imaging & lighting units, and synchronizing the work of the components. Here has AI engine which processes and analyzes each seed and makes prediction of seed type, and classified as good, black and off.
Objective of the Invention
The main objective of the invention is to provide the system and process of seed classification.
Another objective of the invention is to analyze different varieties of seeds and classified on the basis of their morphologies.
Another objective of the invention is to provide manually seed sample analyses before the seed vision process.
Another objective of the invention is to provide camera to take images of seeds.
Another objective of the invention is to provide microcontroller to control the working of the conveyor, seed dropper, imaging & lighting units, and synchronizing the work of the components.
Another objective of the invention is to provide AI engine to processes and analyzes each seed.
Summary of the Invention
The present invention relates to the system and process of seed classification through seed vision which is a method to classify and identify seed quality of each type of seed on the basis of their morphological features. Around 10,000 seeds of each type are analyzed. The present invention relates to the analysis of the seed manually via image processing to understand the morphological differences and then performing a feasibility report on whether the seeds can be processed by Seed vision. Seeds are then carefully taken in the funnel and passed through a ramp to a conveyor belt with the help of a seed dropper. A Camera placed over the conveyor belt takes images of seeds passing through it. A microcontroller controls the working of the conveyor, seed dropper, imaging & lighting units. It also helps to synchronize the working of components. Images taken using the camera are sent to Seed Vision AI Engine which processes and analyzes each seed and makes predictions of seed type. Seeds are classified as good, black and off.
Statement of the Invention
The present invention provides system and process of seed classification through seed vision which is a method to classify and identify seed quality of each type of seeds manually via image processing to understand the morphological differences and then seeds are carefully taken in the funnel and passed through the ramp to a conveyor belt; a camera placed over the conveyor belt; microcontroller controls the working of the system to analyze and classify the seeds with better results.
Brief Description of Drawing
Figure 1 shows seed vision system
Figure 2 shows seeds dropper in process
Figure 3 shows conveyor belt
Figure 4 shows Lighting in the system
Figure 5 shows PIC Controller Board
Figure 6 shows seeds collector / exit
Figure 7 shows process flow chart of the system
Figure 8 shows the image data after cleaning
Figure 9 shows images of good seeds
Figure 10 shows workflow of model serving
Figure 11 shows UI
Figure 12 shows image of a particular size and outputs an image of the same size
Figure 13 shows CV2 segmentor
Figure 14 shows mask generated by nerve quality analyzer model.
Figure 15 shows mask generated by Husk Quality Analyzer model
Figure 16 shows JSON response and passing to UI
Figure 17 shows detailed explanation of seedvision AI training pipeline
Figure 18 shows SMC Architecture
Figure 19 shows CNN_2-Base_Model
Figure 20 shows intermediate heatmaps vs original images on sample data
Figure 21 shows model quantization
Figure 22 shows general architecture of the training pipeline
A detailed description of the Invention
It should be noted that the detailed description of features, designs, components, construction, working and embodiments outlined in the specification below exemplify the wide variety and arrangement of instructions employed with the present invention. The present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. All the features disclosed in this specification may be replaced by similar other or alternative features performing similar or same or equivalent purposes. Thus, unless expressly stated otherwise, they are all within the present invention's scope. Various modifications or substitutions are also possible without departing from the scope or spirit of the present invention. Therefore, it is understood that this specification has been described by way of the most preferred embodiments and for illustration, not limitation.
The present invention relates to the system and process of seed classification through seed vision which is a method to classify and identify seed quality of each type of seed on the basis of their morphological features. Around 10,000 seeds of each type are analyzed. The present invention relates to the analysis of the seed manually via image processing to understand the morphological differences and then performing a feasibility report on whether the seeds can be processed by Seed vision. Seeds are then carefully taken in the funnel and passed through a ramp to a conveyor belt with the help of a seed dropper. A Camera placed over the conveyor belt takes images of seeds passing through it. A microcontroller controls the working of the conveyor, seed dropper, imaging & lighting units. It also helps to synchronize the working of components. Images taken using the camera are sent to Seed Vision AI Engine which processes and analyzes each seed and makes predictions of seed type. Seeds are classified as good, black and off.
In the present invention first of all seeds are collected eg paddy, maize etc. then it can be analyzed by manually and image processing to understand the morphological differences and then does a feasibility report on whether the seeds can be processed by seed vision. Seeds to be analyzed are packed into lots and registered in seed vision and passed to seed vision via a funnel. Seeds are carefully taken in and passed through a ramp to a conveyor belt with the help of a seed dropper. A camera placed over the conveyor belt takes images of seeds passing through it. A microcontroller controls the working of the conveyor, seed dropper, imaging & lighting units. It also helps to synchronize the working of components. Images taken using the camera are sent to Seed Vision AI Engine which processes and analyses each seed and makes predictions of seed type. Seeds are classified as good, black and off. Seed counts for each seed type are totaled and evaluated against standard criteria of ODV or corresponding Quality analysis methods. Result is posted in the dashboard with each image labelled with individual predictions, then reports are generated from the calculations and provided for feedback. Provision to update the predictions are also provided via the dashboard.
In the present invention has seeds dropper which is the first part of the system where the seeds will be placed to send to the conveyor belt. The seeds dropper has three parts namely seeds holder, vibrator and slider. The Seeds Holder holds the maximum number of seeds to be tested. The Vibrator helps to give a momentum to the seeds in the chamber so that the seeds will be moved to the Slider. In the Slider, the seeds are controlled through pre-build Channels and safely reach the Conveyor. The Slider is designed such a way that seeds will be dropped slowly which reduces the area of spreading on the conveyor and prevents any physical damage. Different types of Channels can be mounted on to the Slider to provide pathways for different seeds with different sizes.
The present invention comprises conveyor which is second part of the Product where the seeds will be taken to the Imaging Section and to the exit collection. Conveyor is being controlled by a stepper motor, which will move the conveyor either forward and backward. In the present system Conveyor moves in the Forward direction at all the times of the operation. Conveyor belt is made up of a hard rubber sheet for flexibility and long durability. The conveyor runs at a speed where the vibration won’t be transmitted to the belt. So, the seeds on the belt will have very less momentum on the conveyor. The color of the conveyor belt has been replaced with blue color to get better image classification results.
The present invention comprises camera with good low light performance and it has iHDR support also. This is used to take high-definition images of moving seeds with an autofocus mechanism to adjust the focusing of different crop varieties. The system also has full-spectrum LEDs for lighting purposes. This will help to get the maximum data from all frequencies of the visible spectrum as well as near UV and IR regions. Because of this, the captured Seeds images does not have shadows that improves the quality.
The Flow of the System starts from loading the seeds manually on to the Seeds Dropper. Then the seeds will be travelling through the Conveyor to the Collection area. In-between in the Imaging Section, the Camera captures all the seeds moving in the Conveyor and sends the image to the AI System for image processing. Images are captured in the interval equal to the Conveyor Speed. So that there won't be any seed repetition between the frames and also no seeds will be missed in between the frames. Captured Images will be either saved into the local memory to use at the AI Processing side or Images will be directly fed to the AI Processing. Images are Stored as JPEG(.jpg) format which is of very high quality. The Raw images will be stored in the Internal Memory which can be used for creating Seed Models for AI Processing and also it can be used for Testing the Quality of the Build Model. The Stored images can be transferred to another System / Computer using S3 Cloud or using USB Storage.
The present invention provides Artificial Intelligence system which is a way to codify and automate the workflow, it takes to produce a machine learning model. Artificial Intelligence pipelines consist of multiple sequential steps that do everything from data extraction and preprocessing to model training and deployment. The seed vision AI pipeline consists of two blocks, one is model training block and another one is model serving block.
Model training is the process of making highly efficient deep learning models. It is the phase in the data science development lifecycle where practitioners try to fit the best combination of weights and biases to a machine learning algorithm to minimize a loss function over the prediction range. Data collection is the first step in the model training process. It is the process of gathering and measuring information from countless different sources. The system uses Seed Vision hardware module to capture the high-quality images of different seed varieties. After collecting the data, the next step is data cleaning. Data cleaning refers to identifying and correcting errors in the dataset that may negatively impact a predictive model. After cleaning the raw image data, extract the seeds separately from the raw images using some OpenCV image segmentation tools. Then through validation of the data is necessary to confirm that there are no errors in the dataset that may impact the model accuracy.
In the present invention system perform different types of seed quality analysis tests like soak tests, fungal plate tests, mechanical purity test and polymerase chain reaction (PCR) tests. In soak tests, Seeds are soaked overnight in water (nematodes) or in a dilute sodium hydroxide solution (downy mildews, loose smut). Nematodes are concentrated by sieving, followed by microscopic examination. For fungi, seed embryos are stained and examined microscopically. In fungal plate tests, Direct plating of whole seeds onto agar plates (potato dextrose, malt agar, and water agar are common), incubated for 7-14 days and examined for the presence of target fungi. In mechanical purity test, Seed samples are mixed and reduced by use of a mechanical divider or other methods to ensure that the sample is properly sampled in the seed lab as well as by the seed company or other producer at the source of the seed lot. The Polymerase Chain Reaction (PCR) is a high-tech method to extract, increase, detect and identify DNA from biotech traits or pathogens (if present). It is highly sensitive and especially good in determining absence of biotech traits. PCR can give qualitative or quantitative results. Attention to detail and in-depth knowledge of molecular biology and available biotech traits are critical to success in using PCR.
In the system simply model training means learning (determining) good values for all the weights and the bias from labelled examples. Here we’ll train a computer vision deep learning model (CNN) to classify different varieties of seeds with more than 99% accuracy. All models are trained using a training pipeline and are tracked using MLFlow. The trained models are stored in a bucket in S3. Where model validation is performed after training the model. Model validation evaluate the trained model using a product validation dataset that is perform well in the real-world data.
The system also provides model quantization, which is a conversion technique that reduces the model size while improving inference speeds in CPU or accelerated devices like the GPU. The deep learning models used in Seed Vision are trained using tensor flow. The tensor flow models are converted into a TensorRT.
In the figure 10 shows workflow of model serving in the system, where Deep learning inference is performed which is a process that using a trained DNN model to make predictions against live and previously unseen data. First of all, select the crop and variety then camera in hardware module capture high resolution raw images of seeds. These raw images will be transferred to the segmentation module. After collecting the data, extract the seeds separately using OpenCV image segmentation tools. This will contain cut/connected seed images along with the good images. So, the next step is seed merit classification (SMC).
Image segmentation is a method in which a digital image is broken down into various subgroups called Image segments which helps in reducing the complexity of the image to make further processing or analysis of the image simpler and extracting the seeds from the conveyor belt. This is done by two methods one is Unet which is deep learning based and another one is CV2_based. Unet is basically an autoencoder network with skip connections in order to increase robustness. The encoder part of the network finds out the finer features of the objects present in the image and the decoder part is in charge of interpreting the localization patterns. It takes an image of a particular size and outputs an image of the same size, but with pixel wise class predictions (usually into foreground and background). Where Cv2 stands for OpenCV, and this is a segmentation algorithm purely based on classical computer vision algorithms. It involves finding out the location of each seed in the figure 13 and calculating the bounding box coordinates of the same. It works by binary thresholding and then finding the contours. These contour patches are then rotated and brought into a uniform shape. This segmentor algorithm is currently used in production.
The system provides seed nerve quality analyzer that plays an important role in the quality analysis by training to deep convolutional neural network to predict the quality of the seed nerves. By using this deep learning model, we can predict the quality without cutting the seeds which is shown in figure 14. The system also provides seed husk quality analyzer, where Husk is the outer shell or coating of a seed. By knowing the quality and health of a husk we can predict the seed quality. So we trained a husk quality analyzer to predict the seed quality. The system uses both Husk quality analyser and Nerve quality analyser on an ad-hoc basis. The next step is seed classification. For the seed classification, corresponding DNN model has been uploaded to classify the seeds into TRUE type and OFF type seeds. TRUE types are the seeds which the classifier model is confident that it is of the particular variety. And the OFF types are the seeds which the classifier model is getting confused with. After seed classification the final step is making a JSON response and passing it to the product UI. The total number of seeds, number of good seeds, and number of OFF type seeds will be showing into the UI.
All the seed images segregated through segmentation networks are preprocessed to be fed into deep convolutional neural networks (CNNs) that classify the seeds into required categories. In this project, we use three CNNs, first CNN involves differentiating seeds into good seed images and bad seed images, based on complete visibility of the seed in the image. Bad seed images contain cut seeds, where the seed is not completely visible in a single image and the rest are classified as good seeds. Then, the good seed images are fed to a different CNN architecture to train a base model that classifies the seeds of a given crop into all possible subcategories of all its varieties. Finally, in order to classify each seed variety into its sub-category, the base model’s top layers are fine tuned.
The system provides data preparation-1 in which seed images obtained from segmentation network are collected and manually labelled as good and bad images by the data collection team. This data is treated as training data to train CNN-1 SMC model to detect if a seed image is good. This training data is balanced and split into train, validation and test data. System also provides data preprocessing-1 in which All images need to be preprocessed to have a fixed input shape and image data is normalised defore fed into SMC. Initially, only train and validation data are preprocessed in batches with fixed batch size through a custom_data_generator function. This train data generator and validation data generator also generates masked images, which are reduced in size to input images, to act as label data for the decoder. Whereas the test data generator produces only preprocessed input images for testing the Model after the training process. Batch_size is a hyperparameter, chosen to represent distribution of all categories of data, reducing overfitting, improving convergence speed.
In the present system, CNN_1-SMC_Model is each batch of train is fed as input to the SMC model, where each image is passed through an image classification CNN with a decoder arm. Training a batch of images is accompanied by validating a batch of images for early stopping and checkpointing at the convergence. Where SMC is a moderately deep CNN that performs binary classification and auto encoding simultaneously to classify good and bad seed images. Input image is initially passed through a CNN called encoder network containing 14 convolution layers with batch normalization in between to avoid internal covariate shift. Now the encoder output in lower dimensions is fed to the classification network and also to the decoder network. Decoder that has conv layers and up sampling layers, aims to reconstruct the input image. Classification network contains conv layers, dense layers and eventually an output dense layer with 2 neurons for binary output.
The system provide data preparation-2 and preprocessing-2 in which Only good images of seeds are taken to process further. All the images are set to specific resolution to input to Base Model CNN, Image data is normalized and balanced across all categories. Then, the data is split to train, validation and test datasets. A train data generator, validation data generator functions are built to preprocess and augment the data in batches to be fed into Base Model CNN.
In the system, CNN_2-Base_Model is used to capture the finer morphological details of the seeds to distinguish all the variants requires a deeper CNN like ResNet. ResNet contains residual/skip connections to take complete advantage of deeper CNN layers by avoiding vanishing gradient problems. So, all the good images from CNN_1 are classified into all seed varieties at sub-categorical level using a very deep residual network with 50 layers. Final convolution block and dense layer’s weights are re-initialized to construct Seed Vision base architecture. Similar to CNN-1, this Base Model has a decoder arm attached from the last residual block for efficient feature extraction. Hence the model decreases the weighted sum of the compound loss combining the mean squared error with respect to decoder arm and categorical cross entropy with respect to seed classification. Here we are using a bilinear neural network for fine-grained visual recognition and multi-modal deep learning.
In the system, CNN_3-Varietal_Model is used to classify sub variants of a variety. It involves hyperparameter tuning of only the top dense layers in the classification network. In preprocessing, Training loss, validation loss, gradient’s distributions are monitored during training for each epoch through tensor board graphs. Activation maps of seeds, taken from intermediate layers of the network are compared to original seed images to visualize the important morphological features required in classification.
The system also provides model quantization, which is a conversion technique that reduces the model size while improving inference speeds in CPU or accelerated devices like the GPU. The quantization in this project is done using TensorRT by NVIDIA. TensorRT converts the models such that the throughput is maximized with minimal latency. The general methodology of model conversion includes:
a. Selecting a batch size or the maximum number of batches.
b. Selecting the precision.
c. Converting the model based on the above parameters.
Where Batch size is determined by the pipeline architecture and the size of the model. Generally, batch sizes are selected in the multiples of 8 starting from 1. Current pipeline implementation has a batch size of 1 for the Seed Merit Classifier and the Variety classification models. Where precision determines the floating-point precision of the weights in the model. Selecting precision has a tradeoff between inference speed and metric performance. The available precisions in TensorRT are INT8, FP32, FP16 and the mixture of these. Models in Seed Vision have a mixed precision of FP16 and FP32 for maximum performance. Precision also reduces the size of the model.
Once the models are converted into TensorRT engines. The converted model can only be loaded using TensorRT API. The model is directly copied into the Jetson device into a particular folder where the API automatically reads the folder structure and assigns the particular model a name according to the folder and file configuration. The system provides performance measurement for TensorRT is given in the below table which is compared with the tensorflow model. Benchmarks of models are compared which are the Seed Merit Classifier and the variety classifier using 100 data points. The average time taken for 20 runs are taken for comparison. The model and image summaries used for testing are given below.
Table 1: Model and data summary
Model Number of Parameters Image
Height Width Channel
SMC 3,298,208 448 224 3
Variety 62,168,802 448 224 3
Table 2: Test Summary
Test Number Model Time taken (20 run average in sec) % increase Tensorflow TensorRT
1 SMC 36.79 3.41 9.79
2 Variety 17.8 0.38 45.84
All models are trained using a training pipeline and are tracked using MLFlow. The trained models are stored in a bucket in S3. The training starts by moving the data from S3 to a training GPU instance. The training pipeline is invoked in the instance with the right parameters. This will launch the model training. The metrics are tracked remotely in a tracking server using MLFlow which can be viewed in real-time. Additionally, MLFlow tracking server handles model versioning as well. MLFlow is an open-source tracking server for tracking machine learning projects and models. MLFlow is built majorly using python using the flask framework and javascript for the frontend. The MLFlow server is hosted in an EC2 server with a mysql database backend. The purpose of the MLFlow tracking server in SeedVision are the following:
a. Track experiments for various models
b. Track and manage artefacts in S3. Primarily, model files, confusion matrices and classification reports.
The user can use the pipeline by following steps:
a. Clone the repository into the machine (Local machine or an EC2 instance).
b. Create a script for training the model. If the script is already created, go to step c
c. Run the script using the MLFlow command with the respective URI, entry point and parameters (data path, epochs, batches …)
Once the training is complete, the user can access the model and the performance metrics using the MLFlow UI hosted in a separate instance. The user is greeted to the main page of the MLFlow UI when navigating to the URL. The main page consists of a list of experiments and the runs associated with each of the experiments. Every script run is considered as a run. Figure 11 shows the MLflow UI main page. User can add, modify or delete experiments from this page.
The user can click the link under start time to go to the respective run page to view the run parameters, metrics and model hyperparameters. The user can do the following operations from this page:
a. Add the description for the run if any
b. Edit the description for the run
c. View the metrics and hyper parameters.
d. Get the S3 link to the model.
The models tab contains the information about all the models that have been trained. The models that have been logged with the registered name argument appear in this tab. The models that come under the same registered name is versioned in this tab. Where users can check different versions of the same model. The specific version of a model or all the versions of a model can be viewed from the model tab using the model version link. Clicking on the specific version will take you to the run page.
Table 3: Results
Variety Model Test_ type Total_seed Good_seed Detected vs-model Off_type
Paddy Type C Paddy Type F vs Paddy Type C True_type 1707Paddy Type C-475 Paddy Type F-230 0
Paddy Type F Paddy Type F vs Paddy Type F True_type 2940Paddy Type F-1244 Paddy Type C-141 0
Paddy Type B Paddy Type B vs Paddy Type D True_type 6310Paddy Type B-2063 Paddy Type D-751 0
After solving bug on model Paddy Type E
Paddy Type E Paddy Type E vs Paddy Type G True_type 4680Paddy Type E-1495 Paddy Type G-112 0
Paddy Type G Paddy Type E vs Paddy Type G True_type 6230Paddy Type G-2180 Paddy Type E-189 0
Table 4: Versus Model Test Report
S. N. Total images in the Test_ folder Test_ type Time taken (min) Total_ seed count Good_ seed count Off_ type_ seed count Predicted seed count Fault 1 Fault 2 Fault 3 disori ented Black_ seed count
10/11/2021
1 149 Previous _function 8 42 39 0 --- --- 3 --- ---
Rotation _function 3 53 21 0 --- 1 30 --- 1
2 186 Previous _function 6 178 137 5 Paddy Type P count 5 2 33 1 ---
Rotation _function 5 277 137 3 Paddy Type P Count 3 58 112 1 2
3 527 Previous _function 19 78 55 0 --- 1 21 1 ---
Rotation _function 13 194 39 0 --- 100 53 1 1
4 3148 Previous _function 39 172 134 0 --- --- 37 1 ---
Rotation _function 34 214 91 2 Paddy Type P Count 2 10 106 1 4
After adding filtration in the rotation_function
Table 5:
Variety Total No. of seeds used Total photos Empty_ photos
Watermelon Type D 100 223 124 9 133 90 0 4.04
Watermelon Type E 100 105 54 8 62 43 0 7.62
Watermelon Type F 100 104 67 3 70 34 0 2.88
Watermelon Type A 100 188 124 7 131 57 0 3.72
Watermelon Type C 100 105 71 3 74 31 0 2.86
Watermelon Type B 100 117 66 4 70 47 0 3.42
Updated empty checker function
Paddy Type F --- 25 60 6 190 0.00
Paddy Type G --- 50 214 25 250 8.00
Paddy Type E --- 40 241 25 150 2.50
Paddy Type B --- 31 114 15 160 12.90

Watermelon Type D 100 223 1226 128 950 2.69
Watermelon Type E 100 105 549 63 420 8.57
Watermelon Type F 100 104 662 68 360 1.92
Watermelon Type A 100 188 1254 129 590 2.13
Watermelon Type C 100 105 692 71 340 1.90
Watermelon Type B 100 117 617 68 490 5.98

10025 4700 381 5081 49440 3.80
Cotton Type A 1.5 Kg 10174 4398 220 4618 55560 2.16
2438 181 18 24200 0.04
12665 --- --- 5530 71350 ---
Cotton Type B 1.5 Kg 4509 --- --- 1491 30180 ---
3041 --- --- 1053 19880 ---
Cotton Type C 1.5 Kg 5601 2242 188 2430 31710 3.36
15512 --- --- 7795 77170 ---
Cotton Type D 1.5 Kg 6659 --- --- 1445 52140 ---
3040 --- --- 242 27980 ---
Cotton Type E 1.5 Kg 1800 --- --- 234 15660 ---
6675 --- --- 879 57960 ---
Cotton Type F 1.5 Kg 332 3141
640 5405

Chili Type A --- 167 400 40 1270 0.00
Chili Type B --- 332 1721 173 1590 0.30
Chili Type C --- 395 1142 116 2790 0.51
Chili Type F --- 138 730 73 650 0.00
Chili Type A --- 2269 836 26 862 14070 1.15
Chili Type B --- 157 220 22 1350 0.00

Tomato Type E --- 167 974 101 660 2.40
Tomato Type F --- 103 11011 92 0 0.00
Tomato Type G --- 7083 1553 198 1757 53260 2.80
Tomato Type B --- 63 17 26 43 200 41.27
18_133 --- 96 263 29 670 3.13
Tomato Type B --- 3969 570 134 705 32640 3.38
Tomato Type D --- 3658 306 98 404 32540 2.68
18_133 --- 4524 378 105 484 40400 2.32
Tomato Type E 3832 195 35 230 36020 0.91
Tomato Type F 4424 453 56 509 39150 1.27
Maize Type A --- 276 1013 104 1720 1.09
Maize Type B --- 341 2012 203 1380 0.59

Maize Type A --- 238 188 0 118 1200 0.00
Maize Type B --- 728 77 74 77 6510 10.16
Tomato Type 1 Female 6460 1817 73 1890 45700 1.13
Tomato Type 1 Male 4065 1366 66 1432 26330 1.62
Tomato Type 2 6889 --- --- 1985 49040 ---
Tomato Type 3 3646 --- --- 669 29770 ---
Tomato Type 4 6964 1575 69 1644 53200 0.99
Brinjal
Castor
Coated_ maize Sample_ data 29
29 0 0
0 0 0 29 0
0 29 0 0.00
0.00
Raw_ maize 25 0 0 0 25 0 0.00
Soya_ bean 22 0 0 0 22 0 0.00
Abrasion _maize 17 0 0 0 17 0 0.00
Raw_ seed_ maize 500 332 21 3 24 308 0 0.90
Abrasion _seed Sample _data 1349 448 0 448 901 0 0.00
Coated _seed 1440 6240 624 816 0 0.00
Table 6: Empty Image Sorter Test
S N Variety Test_ Type Total Seed Good Seed Off Type Bad Accuracy % Pass / Fail
Chili
1 Chili Type B True_ type 248 157 5 86 96.91 Pass
2 Chili Type B Off_ type 0 144 95 100.00 Pass
3 Chili Type A Off_ type 266 174 13 79 93.05 Pass
4 Chili Type A Off_ type 0 163 84 100 Pass
5 Chili Type A Off_ type 0 1754 --- 100 Pass
6 Chili Type B Off_type 0 131 --- 100 Pass
7 Chili Type A Off_type 0 133 --- 100 Pass
8 Chili Type B Off_type 0 156 --- 100 Pass
Tomato
1

2 Tomato
type F
Tomato
Type B Off_type

Off_type
213

117 13

12 200

105 ---

---
93.90

89.74 Pass

Fail
3

4
Tomato
Type B
Tomato
Type A Off_type

Off_type 1178 2

24 1176

48 ---

---
99.83

66.67 Pass

Fail
5 Tomato Type A Off_type 124 252 --- 67.02 Fail
6 Tomato Type B Off_type 27 208 --- 88.51 Fail
7 Tomato Type A Off_type 33 528 --- 94.12 Pass
8 Tomato Type B Off_type 0 186 --- 100.00 Pass
Table 7: Set 6 Performance Analysis of Chilly and Tomato Seeds
S. N. Variety Test_ Type Total_ Seed Good_ Seed Off_ Type Bad Accuracy % Pass / Fail
Chilly_test_report
1 Chilly Type B True_ type 108 80 3 25 96.38 Pass
2 Chilly Type B Off_ type 109 0 84 25 100 Pass
3 Chilly Type C True_ type 103 67 10 26 87 Fail
4 Chilly Type C Off_ type 50 0 40 10 100 Pass
5 Chilly Type A True_ type 77 63 1 13 98.43 Pass
6 Chilly Type A Off_ type 47 0 34 13 100 Pass
7 Chilly Type D True_ type 149 104 8 37 92.85 Fail
8 Chilly Type D Off_ type 164 0 130 34 100 Pass
9 Chilly Type E True_ type 135 86 21 28 80.37 Fail
10 Chilly Type E Off_ type 76 0 61 15 100 Pass
11 Chilly Type F True_ type 123 92 0 31 100 Pass
12 Chilly Type F Off_ type 56 380 Fail
13 Chilly Type F Off_ type 26 Fail
Maize
1 Maize Type B True_ type 233 100 2 131 98.04 Pass
2 Maize Type B Off_ type 318 12 100 206 89.29 Fail
3 Maize Type A True_ type 322 122 17 183 87.77 Fail
4 Maize Type A Off_ type 237 0 106 131 100.00 Pass
Tomato
1 Tomato Type B Off_ type 208 0 153 55 100.00 Pass
2 Tomato Type B True_ type 319 202 6111 97.12 Pass
3 Tomato Type C Off_ type 383 0 224 159 100.00 Pass
4 Tomato Type C True_ type 420 109 102 209 51.66 Fail
5 Tomato Type D True_ type 270 128 34 108 79.01 Fail
6 Tomato Type D Off_ type 227 2 169 63 98.83 Pass
7 Tomato Type E True_ type 323 162 63 98 72.00 Fail
8 Tomato Type E Off_ type 244 0 174 70 100.00 Pass
9 Tomato Type A True_ type 261 142 6 113 95.95 Pass
10 Tomato Type A Off_ type 317 10 156 151 93.98 Pass
Test
1 Paddy Type C Off_ type 0 406 --- 100.00 Pass
2 Paddy Type M Off_ type 289 1 288 --- 99.65 Pass
3 Paddy Type A True_ type 768 754 12 --- 98.43 Pass
Chilly & Tomato test for demo
1 Tomato Type A True_ type 6965 4977 91 1897 98.20 Pass
2 Tomato Type F True_ type 185 110 5 70 95.65 Pass
3 Tomato Type B True_ type 15379 9342 1071 4966 89.71 Fail
4 Chili Type B True_ type 3882 2126 150 1606 45 93.41 Pass
5 Tomato Type B True_ type 426 214 90.45 Fail
6 Chili Type A True_ type 3653 2244 272 1137 89.19 Fail
Load_test
1 Watermelon Type B No empty_ function load 236 157 10 69 Time taken: 3.48 min Empty images remain with seed images
2 Watermelon Type B Empty_ function load 238 157 10 71 Time taken: 2.30 min Eliminated all empty images
Before Model Update
1 Paddy Type C True_ type 187 84 10 --- 89.36 Fail
2 Paddy Type M Off_type 215 0 85 --- 100.00 Pass
3 Chili Type B True_ type 161 125 6 30 95.42
4 Tomato Type A True_ type 153 93 19 50 83.04
5 Chili Type A Off_ type 133 0 103 30 100.00
6 Chili Type B True_ type 128 100 4 24 96.15
7 Chili Type A True_ type 137 102 1 34 99.03
8 Chili type B Off_ type 134 0 103 31 100.00
Set 1_model testing
1 Paddy Type 5 True_ type 227 40 82 --- 32.79
2 Paddy Type 5 True_ type 107 19 39 32.76
After Model Update
1 Paddy Type D True_ type 170 71 12 --- 85.54
2 Paddy Type A True_ type 842 511 0 --- 100.00
3 Paddy Type D True_ type 375 178 17 --- 91.28
4 Paddy Type D True_ type 1009 490 41 --- 92.28
5 Paddy Type C True_ type 1707 650 55 --- 92.20
6 Paddy Type E 374 0 0 ---

Bug in the result
7 Paddy Type F 297 0 0 ---
8 Paddy Type D 49 0 0
Model improved to solve bug issue
1 Paddy Type E 1532 528 2 99.62

Set_Cotton
1 Cotton Type G female True_ type 203 156 47 (hybrid) --- 76.85 Fail
2 Cotton Type G Hybrid True_ type 165 149 16 (female) --- 90.3 Fail

Table 8: Performance Analysis of Nvidia Board version 1
S. N. Variety Test_ type Total_ seed Good_ seed Off_ type Fault1 Fault2 Fault3 Accuracy % Pass/Fail Raw Folder
Set1_paddy
1 Paddy Type3 True_ type 453 212 25 44 132 40 89.45% Fail Set 1_2
2 Paddy Type3 True_ type 179 73 5 13 68 20 93.59% Pass Set 1_1
3 Paddy Type3 True_ type 1052 489 61 83 292 127 88.91% Nov15 _Set 1_T1
4 Paddy Type 3-OT1 Off_ type 1021 7 463 32 318 201 98.51% Pass Set 1_off1
5 Paddy Type 3-OT2 Off_ type 1236 14 610 132 208 272 97.76% Pass Set 1_ot2
6 Paddy Type 3-OT3 Off_ type 1113 1 471 117 412 112 99.76% Pass Set 1_OT3
7 Mixed (Paddy Type 3=100, OT=60) True_ type 160 55 35 11 36 23 Set 1_mix
Testing after update
1 Paddy Type 3 True_ type 453 226 11 44 132 40 95.36% Pass Set 1_2
2 Paddy Type 3 True_ type 179 75 3 13 68 20 96.15% Pass Set 1_1
3 True_ type 1175 597 27 90 318 143 95.67% Pass Nov15 _Set 1_T1
4 Paddy Type 3-OT1 Off_ type 1021 22 448 32 318 201 95.32% Pass Set 1_off1
5 Paddy Type 3_OT2 Off_ type 1237 44 581 132 208 272 92.96% Fail Set 1_OT2
6 Paddy Type 3_OT3 Off_ type 1113 5 467 117 412 117 98.94% Pass Set 1_OT3
7 Mixed (Paddy Type 3 = 100, OT=60) True_ type 161 57 33 12 36 23 Set 1_mix
Testing after updating smc
1 Cotton_ female hybrid cotton type G True_ type 205 168 0 BAD–37 (no smc update)
Set 1_ cotton
2 Paddy Type 3 True_ type 830 477 13 37 166 136 97.35% T3_ Paddy Type J _true type
3 Paddy Type H (male) True_ type 845 509 1 29 242 64 99.80% Repeated image bug T2_ male_ Paddy type H_ true type
4 Paddy Type H (female) True_ type 361 63 64 20 124 25 49.61% TI_ true_ Paddy type H
5 Paddy Type J True_ type 1077 585 40 71 229 152 93.60% T2_ true_ Paddy Type J
6 Paddy Type J True_ type 996 82 437 61 339 76 84.20 Paddy Type H_true _T1
Issue is solved but results not changed
1 Paddy Type J True_ type 830 477 13 37 166 136 97.35%
2 Paddy Type H (male) True_ type 845 509 1 29 242 64 99.80%
3 Paddy Type J True_ type 1056 434 75 88 266 193 85.27% Fail Paddy Type J_var
4 Paddy Type J Off_ type (OT2) 530 34 201 29 133 133 85.53% Fail Paddy Type J_T1
5 Paddy Type J True_ test 349 188 9 15 83 54 95.43% Pass Paddy Type J_test bug
6 Paddy Type J True_ type 1033 572 28 50 240 143 95.33% Pass Paddy type J_Var
7 Paddy Type J Off_ type (1017) 466 1 239 29 135 62 99.58% Pass Off_ type_ Paddy Type I
8 Paddy Type I True_ type 1018 445 15 83 351 124 96.74% Pass Paddy Type LT1_ true
Set 2-model testing
1 Paddy Type 12 True_ test 398 393 3 99.24% Pass
2 Paddy Type 12 True_ test 40 40 0 100.00% Pass
3 Paddy Type 12 True_ test 136 134 2 98.53% Pass
4 Paddy Type 12 True_ test 456 452 4 99.12% Pass
5 Paddy Type 12 True_ test 412 409 3 99.27% Pass
6 Paddy Type 12 True_ test 306 300 7 97.72% Pass
7 Paddy Type 12 True_ test 200 195 5 97.50% Pass
8 Paddy Type 11 Off_ type (Off_ type1) 108 4 104 96.50 Pass
9 Paddy Type 11 True_ test 203 196 7 96.55 Pass
10 Paddy Type 11 True_ test 2004 1958 46 97.70% Pass
11 Paddy Type 13 (lot1) True_ test 503 490 13 97.42% Pass
12 Paddy Type 13 (lot2) True_ test 305 290 15 95.08% Pass
13 Paddy Type 13 (Off_ lot1) Off_ type_ test 803 93 710 88.42% Fail
14 Paddy Type 13 (Off_ lot2) Off_ type_ test 789 689 94 12.01 Fail
15 Paddy Type 14 (lot1) True_ test 536 521 17 96.84% Pass
16 Paddy Type 14 (lot2) True_ test 904 823 81 91.04% Fail
17 Paddy Type 14 (Off_ lot1) Off_ type_ test 360 0 360 100.00% Pass
18 Paddy Type 14 (Off_ lot2) Off_ type _test 615 28 589 95.46% Pass
19 Paddy Type 11 (lot1) True_ test 553 538 15 97.29% Pass
20 Paddy Type 11 (lot2) True_ test 643 635 8 98.76% Pass
21 Paddy Type 11 (Off_ lot1) Off_ type_ test 523 1 522 99.81% Pass
22 Paddy Type 11 (Off_ lot2) Off_ type_ test 803 47 756 94.15% Pass
23 Paddy Type 12 (lot2) True_ test 718 704 14 98.05% Pass
24 Paddy Type 12 (lot2) True_ test 1060 1018 42 96.04% Pass
25 Paddy Type 12 (Off_ lot1) Off_ type_ test 713 3 710 99.58% Pass
26 Paddy Type 12 (Off_ lot2) Off_ type_ test 569 33 527 94.11% Pass
Set 5_seed
1 Paddy Type 9 True_ test 1103 1082 21 98.10% Pass
2 Paddy Type 10 True_ test 1160 11 0 100.00% Pass Blacks ee d-7
3 Paddy Type 7 True_ test 225 53 0 100.00% Pass Blacks ee d-3
4 Paddy Type 6 True_ test 310 22 2 0 100.00% Pass Blacks ee d-2
5 Paddy Type 6 Off_ type 310 308 0 0.00% Fail Blacks ee d-2
6 Paddy Type 7 Off_ type 1160 308 115 3 0 0.00% Fail Blacks ee d-7
Model_update
Paddy Type 10 model vs all samples
7 Paddy Type 6 Off_ type_ test 328 12 316 96.34% Pass
8 Paddy Type 7 Off_ type_ test 269 12 254 95.49% Pass
9 Paddy Type 8 Off_ type_ test 286 0 286 95.49% Pass
10 Paddy Type 9 Off_ type_ test 403 1 402 99.75% Pass
11 Paddy Type 10 True_ type 644 636 4 99.38% Pass
Paddy Type 8 model vs all samples
12 Paddy Type 6 Off_ type_ test 261 0 261 100.00% Pass
13 Paddy Type 7 Off_ type_ test 269 0 266 100.00% Pass Blacks ee d-3
14 Paddy Type 8 True_ type 286 281 5 98.25% Pass
15 Paddy Type 9 Off_ type_ test 287 1 286 99.65% Pass
16 Paddy Type 10 Off_ type_ test 207 1 206 99.52% Pass
Model_update including discolored seed count
Paddy Type 6 model vs all samples Dis-colored
17 Paddy Type 6 True_ type_ test 328 293 5 98.32% Pass Off-2, Good-5
18 Paddy Type 7 Off_ type_ test 269 34 179 84.04% Fail Off-49, GD-4
19 Paddy Type 8 Off_ type_ test 286 1 280 99.64% Pass Off-5, GD-0
20 Paddy Type 9 Off_ type_ test 403 0 383 100.00% Pass Off-20, GD-0
21 Paddy Type 10 Off_ type_ test 1165 34 985 96.66% Pass Off-133, GD-6
Paddy Type 7 model vs all samples
22 Paddy Type 6 Off_ type_ test 328 8 290 97.32% Pass Off-28, GD-2
23 Paddy Type 7 True_ type_ test 269 196 17 92.02% Fail Off-1, GD-52
24 Paddy Type 8 Off_ type_ test 286 34 247 87.90% Fail Off-5, GD-0
25 Paddy Type 9 Off_ type_ test 403 3 380 99.22% Pass Off-20, GD-0
26 Paddy Type 10 Off_ type_ test 1165 5 1014 99.51% Pass Off-138, GD-1
Paddy Type 9 model vs all samples
27 Paddy Type 6 Off_ type_ test 328 0 298 100.00% Pass Off-30, GD-0
28 Paddy Type 7 Off_ type_ test 269 0 213 100.00% Pass Off-53, GD-0
29 Paddy Type 8 Off_ type_ test 286 0 281 100.00% Pass Off-4, GD-1
30 Paddy Type 9 True_ type_ test 403 381 2 99.48% Pass Off-1, GD-19
31 Paddy Type 10 Off_ type_ test 1165 0 1019 100.00% Pass Off-139, GD-0
Paddy Type 10 model vs mixed blackseeds & discolored seeds
32 Mixed black seeds Smc_ test 106 0 0 Off-55, GD-33
33 Mixed dis-coloured seeds Smc_ test 137 3 4 Off-62, GD-65
In an embodiment, said invention provides system and process of seed classification.
In another embodiment, said seeds (eg: paddy, maize etc.) are collected and send for analysis by manually and via image processing to understand the morphological differences and then does a feasibility report on whether the seeds can be processed by SeedVision.
In another embodiment, said analyzed seeds are packed into lots and registered in SeedVision which passed through funnel to SeedVision.
In another embodiment, said seeds are carefully taken in and passed through a ramp to a conveyor belt with the help of a seed dropper.
In another embodiment, said camera placed over the conveyor belt to take images of seeds which are passing through it.
In another embodiment, said camera has good low light performance and iHDR support which is used to take high-definition images of moving seeds with an autofocus mechanism to adjust the focusing of different crop varieties.
In another embodiment, said microcontroller controls the working of the conveyor, seed dropper, imaging & lighting units, and also synchronise the working of components.
In another embodiment, said images taken using the camera are sent to Seed Vision AI Engine which processes and analyses each seed and makes predictions of seed type.
In another embodiment, said seeds are classified as good, black and off.
In another embodiment, said seed counts for each seed type are totaled and evaluated against standard criteria of ODV or corresponding Quality analysis methods.
In another embodiment, said result is posted in the dashboard with each image labelled with individual predictions, then reports are generated from the calculations and provided for feedback.
In another embodiment, said seeds dropper is the first part of the system where the seeds will be placed to send to the conveyor belt, and it has three parts namely seeds holder, vibrator and slider.
In another embodiment, said seed holder holds the maximum number of seeds to be tested.
In another embodiment, said vibrator helps to give a momentum to the seeds in the chamber so that the seeds will be moved to the Slider.
In another embodiment, said slider control the seeds through pre-build Channels and safely reach the conveyor.
In another embodiment, said slider is designed such a way that seeds will be dropped slowly which reduces the area of spreading on the conveyor and prevents any physical damage, and also different types of channels can be mounted on to the slider to provide pathways for different seeds with different sizes.
In another embodiment, said conveyor is second part of the product where the seeds will be taken to the imaging section and to the exit collection, and it is being controlled by a stepper motor, which will move the conveyor either forward and backward.
In another embodiment, said conveyor moves in the forward direction at all the times of the operation, and it is made up of a hard rubber sheet for flexibility and long durability but not limited to this.
In another embodiment, said conveyor runs at a speed where the vibration won’t be transmitted to the belt so the seeds on the belt will have very less momentum on the conveyor.
In another embodiment, said full-spectrum LEDs for lighting purposes to help to get the maximum data from all frequencies of the visible spectrum as well as near UV and IR regions, Because of this the captured seeds images do not have shadows and that improves the quality.
In another embodiment, said captured images will be either saved into the local memory to use at the AI processing side or images will be directly fed to the AI processing, and images are stored as JPEG(.jpg) format which is of very high quality; the raw images will be stored in the internal memory which can be used for creating seed models for AI processing and also it can be used for testing the quality of the build model.
In another embodiment, said stored images can be transferred to another system/computer using S3 cloud or using USB storage.
In another embodiment, said Artificial Intelligence system is a way to codify and automate the workflow, it takes to produce a machine learning model, and where Artificial Intelligence pipelines consist of multiple sequential steps that do everything from data extraction and pre-processing to model training and deployment, and seed vision AI pipeline consists of two blocks, one is model training block and another one is model serving block.
In another embodiment, said model training is the process of making highly efficient deep learning models, and data collection is the first step in the model training process and it is process of gathering and measuring information from countless different sources; after collecting the data, the next step is data cleaning.
In another embodiment, said data cleaning refers to identifying and correcting errors in the dataset that may negatively impact a predictive model; after cleaning the raw image data, extract the seeds separately from the raw images using some OpenCV image segmentation tools, and then through validation of the data is necessary to confirm that there are no errors in the dataset that may impact the model accuracy.
In another embodiment, said soak tests are soaked seeds overnight in water (nematodes) or in a dilute sodium hydroxide solution (downy mildews, loose smut), where Nematodes are concentrated by sieving, followed by microscopic examination; for fungi, seed embryos are stained and examined microscopically.
In another embodiment, said fungal plate tests are direct plating of whole seeds onto agar plates (potato dextrose, malt agar, and water agar are common), incubated for 7-14 days and examined for the presence of target fungi.
In another embodiment, said mechanical purity test, seed samples are mixed and reduced by use of a mechanical divider or other methods to ensure that the sample is properly sampled in the seed lab as well as by the seed company or other producer at the source of the seed lot.
In another embodiment, said Polymerase Chain Reaction (PCR) is a high-tech method to extract, increase, detect and identify DNA from biotech traits or pathogens (if present); it is highly sensitive and especially good in determining absence of biotech traits; PCR can give qualitative or quantitative results.
In another embodiment, said computer vision deep learning model (CNN) is trained to classify different varieties of seeds with more than 99% accuracy.
In another embodiment, said all models are trained using a training pipeline and are tracked using MLFlow; trained models are stored in a bucket in S3, where model validation is performed after training the model, which evaluates the trained model using a product validation dataset that is perform well in the real-world data.
In another embodiment, said model quantization is a conversion technique that reduces the model size while improving inference speeds in CPU or accelerated devices like the GPU.
In another embodiment, said deep learning models used in Seed Vision are trained using tensor flow, and this tensor flow models are converted into a TensorRT.
In another embodiment, said Deep learning inference is performed which is a process that using a trained DNN model to make predictions against live and previously unseen data:
• select the crop and variety
• camera in hardware module capture high resolution raw images of seeds
• raw images will be transferred to the segmentation module
• After collecting the data, extract the seeds separately using OpenCV image segmentation tools
• seed merit classification (SMC).
In another embodiment, said seed nerve quality analyzer analyses the quality by training to deep convolutional neural network to predict the quality of the seed nerves.
In another embodiment, said husk quality analyzer, where Husk is the outer shell or coating of a seed, by knowing the quality and health of a husk we can predict the seed quality.
In another embodiment, said seed classification, corresponding DNN model has been uploaded to classify the seeds into TRUE type and OFF type seeds, where TRUE types are the seeds which the classifier model is confident that it is of the particular variety, And the OFF types are the seeds which the classifier model is getting confused with.
In another embodiment, said final step is making a JSON response and passing it to the product UI; the total number of seeds, number of good seeds, and number of OFF type seeds will be showing into the UI.
In another embodiment, said seed images segregated through segmentation networks are pre-processed to be fed into deep convolutional neural networks (CNNs) that classify the seeds into required categories.
While particular embodiments of the present invention have been shown and described, it will be evident to those skilled in the art that changes and modifications may be made without departing from this invention in its broader aspects and, therefore, the aim in the present invention is to cover all such changes and modifications as fall within the true spirit and scope of this invention.
,CLAIMS:We Claim:
1. A system and process of seed classification comprising off: method to classify and identify seed quality of each type of seed; analysis of seed manually via image processing; performing a feasibility report; seeds are taken in funnel and passed through a ramp; camera placed over the conveyor belt; microcontroller controls the working of the conveyor, seed dropper, imaging & lighting unit; seed vision AI engine processes and analyses each seed; system perform different types of seed quality analysis tests like soak tests, fungal plate tests, mechanical purity test and polymerase chain reaction (PCR) tests; train a computer vision deep learning model (CNN) to classify different varieties of seeds; system also provides model quantization to reduce the model size; deep learning models used in Seed Vision are trained using tensor flow; trained DNN model to make predictions against live and previously unseen data; system provides seed nerve quality analyzer; system also provides seed husk quality analyzer; seed images segregated through segmentation networks are pre-processed to be fed into deep convolutional neural networks (CNNs).
2. The system and process as claimed in claim 1, wherein said seeds (eg: paddy, maize etc.) are collected and send for analysis by manually and via image processing to understand the morphological differences and then does a feasibility report on whether the seeds can be processed by SeedVision.
3. The system and process as claimed in claim 1, wherein said analyzed seeds are packed into lots and registered in SeedVision which passed through funnel to SeedVision.
4. The system and process as claimed in claim 1, wherein said seeds are carefully taken in and passed through a ramp to a conveyor belt with the help of a seed dropper.
5. The system and process as claimed in claim 1, wherein said camera placed over the conveyor belt to take images of seeds which are passing through it.
6. The system and process as claimed in claim 5, wherein said camera has good low light performance and iHDR support which is used to take high-definition images of moving seeds with an autofocus mechanism to adjust the focusing of different crop varieties.
7. The system and process as claimed in claim 1, wherein said microcontroller controls the working of the conveyor, seed dropper, imaging & lighting units, and also synchronize the working of components.
8. The system and process as claimed in claim 1, wherein said Images taken using the camera are sent to Seed Vision AI Engine which processes and analyses each seed and makes predictions of seed type.
9. The system and process as claimed in claim 1, wherein said Seeds are classified as good, black and off.
10. The system and process as claimed in claim 1, wherein said Seed counts for each seed type are totaled and evaluated against standard criteria of ODV or corresponding Quality analysis methods.
11. The system and process as claimed in claim 1, wherein said Result is posted in the dashboard with each image labelled with individual predictions, then reports are generated from the calculations and provided for feedback.
12. The system and process as claimed in claim 1, wherein said seeds dropper is the first part of the system where the seeds will be placed to send to the conveyor belt, and it has three parts namely seeds holder, vibrator and slider.
13. The system and process as claimed in claim 12, wherein said seed holder holds the maximum number of seeds to be tested.
14. The system and process as claimed in claim 12, wherein said vibrator helps to give a momentum to the seeds in the chamber so that the seeds will be moved to the Slider.
15. The system and process as claimed in claim 12, wherein said slider control the seeds through pre-build Channels and safely reach the conveyor.
16. The system and process as claimed in claim 15, wherein said slider is designed such a way that seeds will be dropped slowly which reduces the area of spreading on the conveyor and prevents any physical damage, and also different types of channels can be mounted on to the slider to provide pathways for different seeds with different sizes.
17. The system and process as claimed in claim 1, wherein said conveyor is second part of the product where the seeds will be taken to the imaging section and to the exit collection, and it is being controlled by a stepper motor, which will move the conveyor either forward and backward.
18. The system and process as claimed in claim 17, wherein said conveyor moves in the forward direction at all the times of the operation, and it is made up of a hard rubber sheet for flexibility and long durability but not limited to this.
19. The system and process as claimed in claim 17, wherein said conveyor runs at a speed where the vibration won’t be transmitted to the belt so the seeds on the belt will have very less momentum on the conveyor.
20. The system and process as claimed in claim 1, wherein said full-spectrum LEDs for lighting purposes to help to get the maximum data from all frequencies of the visible spectrum as well as near UV and IR regions, Because of this the captured seeds images do not have shadows and that improves the quality.
21. The system and process as claimed in claim 1, wherein said captured images will be either saved into the local memory to use at the AI processing side or images will be directly fed to the AI processing, and images are stored as JPEG(.jpg) format which is of very high quality; the raw images will be stored in the internal memory which can be used for creating seed models for AI processing and also it can be used for testing the quality of the build model.
22. The system and process as claimed in claim 1, wherein said stored images can be transferred to another system/computer using S3 cloud or using USB storage.
23. The system and process as claimed in claim 1, wherein said Artificial Intelligence system is a way to codify and automate the workflow, it takes to produce a machine learning model, and where Artificial Intelligence pipelines consist of multiple sequential steps that do everything from data extraction and pre-processing to model training and deployment, and seed vision AI pipeline consists of two blocks, one is model training block and another one is model serving block.
24. The system and process as claimed in claim 1, wherein said model training is the process of making highly efficient deep learning models, and data collection is the first step in the model training process and it is process of gathering and measuring information from countless different sources; after collecting the data, the next step is data cleaning.
25. The system and process as claimed in claim 1, wherein said data cleaning refers to identifying and correcting errors in the dataset that may negatively impact a predictive model; after cleaning the raw image data, extract the seeds separately from the raw images using some OpenCV image segmentation tools, and then through validation of the data is necessary to confirm that there are no errors in the dataset that may impact the model accuracy.
26. The system and process as claimed in claim 1, wherein said soak tests are soaked seeds overnight in water (nematodes) or in a dilute sodium hydroxide solution (downy mildews, loose smut), where Nematodes are concentrated by sieving, followed by microscopic examination; for fungi, seed embryos are stained and examined microscopically.
27. The system and process as claimed in claim 1, wherein said fungal plate tests are direct plating of whole seeds onto agar plates (potato dextrose, malt agar, and water agar are common), incubated for 7-14 days and examined for the presence of target fungi.
28. The system and process as claimed in claim 1, wherein said mechanical purity test, seed samples are mixed and reduced by use of a mechanical divider or other methods to ensure that the sample is properly sampled in the seed lab as well as by the seed company or other producer at the source of the seed lot.
29. The system and process as claimed in claim 1, wherein said Polymerase Chain Reaction (PCR) is a high-tech method to extract, increase, detect and identify DNA from biotech traits or pathogens (if present); it is highly sensitive and especially good in determining absence of biotech traits; PCR can give qualitative or quantitative results.
30. The system and process as claimed in claim 1, wherein said computer vision deep learning model (CNN) is trained to classify different varieties of seeds with more than 99% accuracy.
31. The system and process as claimed in claim 1, wherein said all models are trained using a training pipeline and are tracked using MLFlow; trained models are stored in a bucket in S3, where model validation is performed after training the model, which evaluates the trained model using a product validation dataset that is perform well in the real-world data.
32. The system and process as claimed in claim 1, wherein said model quantization is a conversion technique that reduces the model size while improving inference speeds in CPU or accelerated devices like the GPU.
33. The system and process as claimed in claim 1, wherein said deep learning models used in Seed Vision are trained using tensor flow, and this tensor flow models are converted into a TensorRT.
34. The system and process as claimed in claim 1, wherein said Deep learning inference is performed which is a process that using a trained DNN model to make predictions against live and previously unseen data:
• select the crop and variety
• camera in hardware module capture high resolution raw images of seeds
• raw images will be transferred to the segmentation module
• After collecting the data, extract the seeds separately using OpenCV image segmentation tools
• seed merit classification (SMC).
35. The system and process as claimed in claim 1, wherein said seed nerve quality analyzer analyses the quality by training to deep convolutional neural network to predict the quality of the seed nerves.
36. The system and process as claimed in claim 1, wherein said husk quality analyzer, where Husk is the outer shell or coating of a seed, by knowing the quality and health of a husk we can predict the seed quality.
37. The system and process as claimed in claim 1, wherein said seed classification, corresponding DNN model has been uploaded to classify the seeds into TRUE type and OFF type seeds, where TRUE types are the seeds which the classifier model is confident that it is of the particular variety, And the OFF types are the seeds which the classifier model is getting confused with.
38. The system and process as claimed in claim 1, wherein said final step is making a JSON response and passing it to the product UI; the total number of seeds, number of good seeds, and number of OFF type seeds will be showing into the UI.
39. The system and process as claimed in claim 1, wherein said seed images segregated through segmentation networks are pre-processed to be fed into deep convolutional neural networks (CNNs) that classify the seeds into required categories.