Abstract: A data loading system (104) for loading delta data into a Hive table includes a determination module (122) to determine one or more partition folders of the Hive table to be updated based on the delta data, where a partition folder is indicative of partitions in the Hive table, and where the delta data includes data to be loaded into the Hive table. The data loading system (104) further includes a replacement module (124) to generate one or more temporary folders based on merging the delta data and data in the Hive table, where each temporary folder corresponds to a partition folder. The replacement module (124) may then replace the one or more partition folders with the one or more temporary folders to load the delta data into the Hive table, where each partition folder is replaced with the corresponding temporary folder.
CLIAMS:1. A data loading system (104) for loading delta data into a Hive table, the data loading system (104) comprising:
a processor (114);
a determination module (122) coupled to the processor (114) to determine one or more partition folders of the Hive table to be updated based on the delta data, wherein a partition folder is indicative of partitions in the Hive table, and wherein the delta data includes data to be loaded into the Hive table; and
a replacement module (124) coupled to the processor (114) to:
generate one or more temporary folders based on merging the delta data and data in the Hive table, wherein each temporary folder corresponds to a partition folder; and
replace the one or more partition folders with the one or more temporary folders to load the delta data into the Hive table, wherein each partition folder is replaced with the corresponding temporary folder.
2. The data loading system (104) as claimed in claim 1, wherein the determination module (122) obtains, from a distributed file system (108), the delta data to be loaded into the Hive table.
3. The data loading system (104) as claimed in claim 2, wherein the distributed file system (108) is a Hadoop Distributed File System (HDFS).
4. The data loading system (104) as claimed in claim 1, wherein the partition folder is indicative of one or more partitions of the Hive table and a partition value corresponding to each partition.
5. The data loading system (104) as claimed in claim 1, wherein the Hive table is a partitioned Hive table having a plurality of partitions, and wherein partitioning is based on column fields of the Hive table.
6. The data loading system (104) as claimed in claim 1, wherein the delta data comprises information relating to at least one of a plurality of partitions of the Hive table to be modified, information of at least one row to modify the plurality of partitions, and an indicator which represents an operation to be performed on the at least one row, and wherein the operation is at least one of an update operation, a delete operation, and an insert operation.
7. A method for loading delta data into a Hive table, the method comprising:
obtaining, from a distributed file system (108), the delta data to be loaded into the Hive table, wherein the delta data includes information relating to a plurality of partitions of the Hive table to be modified and information of at least one row to modify the plurality of partitions;
determining one or more partition folders of the Hive table based on the delta data, wherein a partition folder is indicative of partitions in the Hive table;
merging the delta data and data in the Hive table to generate one or more temporary folders, wherein a temporary folder corresponds to a partition folder; and
replacing the one or more partition folders with the one or more corresponding temporary folders to load the delta data into the Hive table.
8. The method as claimed in claim 7, wherein the distributed file system (108) is a Hadoop Distributed File System (HDFS).
9. The method as claimed in claim 7, wherein the Hive table is a partitioned Hive table having a plurality of partitions, and wherein partitioning is based on column fields of the Hive table.
10. The method as claimed in claim 7, wherein the delta data further includes an indicator which represents an operation to be performed on the at least one row, such that each row is represented by a single indicator, and wherein the operation is at least one of an update operation, a delete operation, and an insert operation.
11. A non-transitory computer-readable medium having embodied thereon a computer program for executing a method comprising:
obtaining, from a distributed file system, the delta data to be loaded into the Hive table, wherein the delta data includes information relating to a plurality of partitions of the Hive table to be modified and information of at least one row to modify the plurality of partitions;
determining one or more partition folders of the Hive table based on the delta data, wherein a partition folder is indicative of partitions in the Hive table;
merging the delta data and data in the Hive table to generate one or more temporary folders, wherein a temporary folder corresponds to a partition folder; and
replacing the one or more partition folders with the one or more corresponding temporary folders to load the delta data into the Hive table.
,TagSPECI:As Attached
| # | Name | Date |
|---|---|---|
| 1 | SPEC IN.pdf | 2018-08-11 |
| 2 | FORM 5.pdf | 2018-08-11 |
| 3 | FORM 3.pdf | 2018-08-11 |
| 4 | FIG IN.pdf | 2018-08-11 |
| 5 | ABSTRACT1.jpg | 2018-08-11 |
| 6 | 989-MUM-2014-Power of Attorney-130215.pdf | 2018-08-11 |
| 7 | 989-MUM-2014-FORM 18.pdf | 2018-08-11 |
| 8 | 989-MUM-2014-FORM 1(10-9-2014).pdf | 2018-08-11 |
| 9 | 989-MUM-2014-Correspondence-130215.pdf | 2018-08-11 |
| 10 | 989-MUM-2014-CORRESPONDENCE(10-9-2014).pdf | 2018-08-11 |
| 11 | 989-MUM-2014-FER.pdf | 2019-10-04 |
| 12 | 989-MUM-2014-FORM-26 [17-03-2020(online)].pdf | 2020-03-17 |
| 13 | 989-MUM-2014-OTHERS [26-03-2020(online)].pdf | 2020-03-26 |
| 14 | 989-MUM-2014-FER_SER_REPLY [26-03-2020(online)].pdf | 2020-03-26 |
| 15 | 989-MUM-2014-DRAWING [26-03-2020(online)].pdf | 2020-03-26 |
| 16 | 989-MUM-2014-COMPLETE SPECIFICATION [26-03-2020(online)].pdf | 2020-03-26 |
| 17 | 989-MUM-2014-CLAIMS [26-03-2020(online)].pdf | 2020-03-26 |
| 18 | 989-MUM-2014-Correspondence to notify the Controller [08-10-2020(online)].pdf | 2020-10-08 |
| 19 | 989-MUM-2014-Written submissions and relevant documents [27-10-2020(online)].pdf | 2020-10-27 |
| 20 | 989-MUM-2014-US(14)-HearingNotice-(HearingDate-21-10-2020).pdf | 2021-10-03 |
| 21 | 989-MUM-2014-US(14)-ExtendedHearingNotice-(HearingDate-08-07-2022).pdf | 2022-06-08 |
| 22 | 989-MUM-2014-Correspondence to notify the Controller [23-06-2022(online)].pdf | 2022-06-23 |
| 1 | 10thfilesearchstrategy_04-10-2019.pdf |