Description: TECHNICAL FIELD
[0001] The present invention generally relatestodata cloud. More specifically, the present invention relates tocontext driven domain topology based unified modern advance data analytics management framework and platform for data cloud products.
BACKGROUND ART
[0002] Background description includes information that may be useful in understanding the present invention. It is not an admission that any of the information provided herein is prior art or relevant to the presently claimed invention, or that any publication specifically or implicitly referenced is prior art.
[0003] One of the top strategic goals of many of the firms for quite some time is to be a ‘Data Driven’ company, powered by People and Artificial Intelligence. Becoming ‘Data Driven’ means relying on Data to make decisions and derive actionable insights. They are well aware of the advantages of becoming data empowered, which include, providing the best customer experience at lowering costs and maximizing profits. They've put a lot of resources (human and material) into developing enablers like data and intelligence systems. Because of that, in last two decades IT industry researchers have focussed majority of their time on solving issues from technology point of view. The focus areas have always been around how technology solutions or platforms can be innovated to solve issues around latency, performance, scalability, usability and maintainability of existing technology solutions, arising out of ever changing business scenarios increasing in data volumes, variety, velocity and subsequent complexity. In lieu of that, to support variety of business goals primary data platform technical architecture has undergone lots of innovation. Before 2010, focus was on centralized processing, and, control of legacy proprietary data warehouses. Then for the time period after 2010 Hadoop was adopted. Hadoop, as an open source platform, brought in concept of Data Lake, with support for structured and unstructured data, and power of distributed processing. Somewhere, around 2016, slowly, paradigm shifted towards Cloud Technologies, and ‘Data Cloud’ platforms, like, Snowflake, using cloud computing capabilities of parallel processing with separate compute and storage
[0004] However, despite such increased effort and expense in developing such enabling platforms, companies are finding it difficult to achieve the desired results. We still see problems faced by enterprise business owners, owning data cloud solutions for solving their business problems. Such problems are not technology related or related to technical capabilities of cloud providers or data cloud solution providers. Such problems are centredaround the People, Processes and Architectural Definitions of various implementations with cloud technologies and ‘Data Cloud’ solution providers.
[0005] The problem from business perspective is that monolithic data paradigms still persisted. Cloud providers and ‘Data Cloud’ solution providers have brought in decentralization, distributed and parallel processing with segregated compute and storage infrastructure. However, for business, ‘Data Cloud’ platform built on top of native cloud offerings, is a centralised architecture providing a techno-mechanical function of, ingest, process and serve. For them, the monolithic data platform hosts and owns the data that logically belong to different domains or line of businesses. This has led to centralised data ownership which is domain agnostic but centred around the techno-mechanical functions of ingest, serve and process. Ingest, for them, is to get data from all corners of enterprise, from all line of businesses, from all operational, and transactional systems. Process for them, is to cleanse, enrich, and transform the source data into trustworthy data that, can address the needs of a diverse set of consumers, across all enterprise.
[0006] Finally, serving the datasets, to a variety of consumers with a diverse set of needs. This ranges from, analytical consumption, to, exploring the data looking for insights, ‘Machine Learning’ based decision making, to Business Intelligence reports, that, summarize the performance of the business. The reason for such thinking is that architecture, processes, and people that define data platform are always influenced by these techno-mechanical functions and not by the data, domain, or business goals. This, kind of influenced how teams are formed, and, how tasks are prioritized for delivery. Data requirements are defined around, these mechanical functions. Teams delivering the data use cases, have very little domain knowledge and they see data from technical perspectives like for example whether this is structured data or unstructured data. They are called Ingest Teams, Processing Teams and Serving Teams.
[0007] So, if you really see from a business perspective, it is a black box, where, your data is sourced, and delivered to consumers via centralized platform, with technical distributed processing capabilities around, Ingest, Process, and Serve. While this centralized strategy may work for firms with a simpler domain and fewer different consumption situations, it falls short for businesses with complex domains, a big number of suppliers, and a diverse collection of customers.
[0008] Accordingly, this introduces further pain points. From, different line of business perspective, they need to create different Data Products, using the Data Platform ,and, deliver it to their consumers. However, they have to break out the requirements or deliverables around these techno-mechanical functions of Ingest, Process and Serve, wrapped around a Data Pipeline, serving a lifecycle of a data product belonging to a line of business. Technical Architecture is, orthogonally, biased towards business axis of change. Data Products ,or, Business Requirements are changing horizontally across different line of business' serving different consumers, but data platform is architected vertically, with siloed specialized teams, and, processes defined around it. This creates conflicts, like missing Business-led Data Ownership. Data is owned by siloed specialized technical teams around Ingest, Process and Serve. It disconnects Data Owners, from, Data Consumers. And, as mentioned earlier, it introduces highly coupled data pipeline around ingest, process and serve, which tend to create specialized technical teams. So, business has to work with, the ingest team, the process team, or, the serving team to finish building a data product. The data product delivery is also, impeded by backlogs that are created by technical architecture, and not out of needs of Data, or, changing business requirements.
[0009] Accordingly, the negative outcome, of these pain points are increased Data Redundancy, reduced Data Transparency, more co-ordination among teams, delayed deliveries, reduced test & innovation cycles, and failure to include new use cases, or data sources. Due to such pain points, and, negative outcomes, recent surveys, show that, although the investment in Big Data, or, AI has increased by 66% in 2019, the same time positive business results have reduced by 19%.
[0010] To solve such problems around architecture viewpoints, processes, and people that define Data Platform being influenced by the techno-mechanical functions of Ingest, Serve and Process, and not by the Data, Domain, or Business goals, we are introducing Datageir. Datageir is a Context Driven,Domain Topology based modern advance data analytics management Framework and platform for Data Cloud products built with cloud native services at its core. The sole purpose of Datageir is to solve problems that, business faces which are not due to, technical capabilities of Data Cloud products using cloud native services. It focusses on solving problems that are still persistent from business perspective, irrespective of tremendous technical capabilities of such Data Cloud products. Datageir platform is going to enhance parallel business delivery capabilities of enterprises using Data Cloud products and Cloud-native services. It is going to bring holistic use of such technical capabilities, by aligning Global, Enterprise, and, Domain oriented teams together, aligning towards common business goals, and, objectives. The platform, is going to bring structured approach to solving business problems, focussing on process oriented approach, driven by context aware, workflows. All of this, is going to be managed simplistically, using an interactive Web User Interface. Datageir is also going to bring accelerated technical delivery of enterprise projects by bringing in automation with reusable tools and, utilities.
[0011] Datageir implements processes and frameworks that aligns business axis of change with technological capabilities offered by Data Cloud products and native cloud services. It takes a fresh look on pain areas of architecture viewpoints, people and processes by building context driven frameworks and workflows around technological innovation, that has happened since the year 2016. The Datageir framework and workflows introduces De-centralization and ‘Parallel Business Delivery’ from the perspective of line of business or domains. Overall, Datageir, aligns ‘Architecture Viewpoints’, People, Processes and technology around Loosely Coupled and Highly Cohesive line of businesses.
[0012] Therefore, there is a need in the art for processes and framework, drives towards a Data Architecture that couples Ingest, Process and Serve, into a line of businesses context with architecture viewpoints, processes, and, people supporting it. It actually, aligns business thinking to technical architecture in order to, support independent business evolutions, without any bottlenecks. In other words, business axis of change is in same line with technical architecture, process ,and, team structure. Datageirfuifills the actual gap of process, and, people adhering to such parallel line of business delivery using technology features of Data Cloud products on cloud native service offerings. Datageir intends to bring in architectural viewpoints, people, and, process paradigms to parallel business deliveries.
OBJECTS OF THE PRESENT INVENTION
[0013] Some of the objects of the present invention, which at least one embodiment herein satisfies are as listed herein below.
[0014] It is an object of the present disclosure to provide a system for unified data analytics management framework and platform for data cloud products.
[0015] It is another object of the present disclosure to provide a system for unified data analytics management framework and platform for data cloud products that help enterprise users in using and managing ‘Data Cloud’ products in conjunction with cloud-native services as a Unified Analytics Data Platform
[0016] It is another object of the present disclosure to provide a system for unified data analytics management framework and platform for data cloud products that simplify complexities involved in managing Unified Analytics Data Platform for volume, varieties, and velocity of data with data recency and quality issues on top of Security with Governance.
[0017] It is another object of the present disclosure to provide a system for unified data analytics management framework and platform for data cloud products that can be use by both teams starting fresh on Data Cloud platforms, and teams that are using Data Cloud for shorter and longer periods.
[0018] It is another object of the present disclosure to provide a system for unified data analytics management framework and platform for data cloud products thatiscontexts driven by line of businesses or domains to drive end to end domain data ownership to a single organizational unit.
[0019] It is another object of the present disclosure to provide a system for unified data analytics management framework and platform for data cloud products that provides more control over cost, chargeback models among different organizational units sharing data.
[0020] The foregoing and other objects of the present invention will become readily apparent upon further review of the following detailed description of the embodiments as illustrated in the accompanying drawings.

SUMMARY
[0021] The present invention generally relates to data cloud. More specifically, the present invention relates to context driven domain topology based unified modern advance data analytics management framework and platform for data cloud products.
[0022] An aspect of the present invention provides system for unified data analytics management framework and platform for data cloud products, said system comprising: a web module comprising web user interface components for providing primary mode of input, output and recommendation communication mechanisms for one or more users of the system; a restful service module comprising web services integration interfaces to asynchronous compute intensive operations modules, wherein the web services are restful web APIs that are based on hypertext transfer protocol (HTTP) methods to access resources via uniform resource locator (URL)-encoded parameters and any or combination of JavaScript object notation (JSON) or extensible markup language (XML) to transmit data; a back-end engine module This category of module consists of compute intensive operations modules that require dedicated resources to interact directly with cloud products management components for asynchronously work on computing intensive operations without having to interrupt user workflows via the web user interface; and a unified analytics platform management systems module comprising independent deployable systems that facilitates in governing, monitoring, coding, deploying, securing data residing in data cloud products or flowing through various cloud technologies components.
[0023] In an aspect, the system is domain topology based.
[0024] In an aspect, the web module comprises Login Web Module; Account Creation Web Module; Platform Deployment Web Module; Enterprise Web Module; Project Web Module; Document Web Module; Domain Web Module; and Storage Web Module.
[0025] In an aspect, the restful services module comprises Authentication Restful Services Module; Authorization Restful Services Module; Entity Restful Services Module; Configuration Restful Services Module; Notification Restful Services Module; Deployment Restful Services Module; Context Restful Services Module; Guidance Restful Services Module; Data Governance Restful Services Module; Data Sharing Restful Service Module; Document Integration and Search Restful Service Module; Storage Restful Service Module; and Task Router Restful Service Module.
[0026] In an aspect, the Back-End Engine Modules comprises Infrastructure Deployment Back-End Engine Module; Code Deployment Back-End Engine Module; Context Based ?ML & AI Recommendation Back-End Engine Module; Document Search & Index Back-End Engine Module; Users & Policies Back-End Engine Module; Data Interoperability Back-End Engine Module; Data Storage Backend Engine; Ingest Back-End Engine Module; Process Back-End Engine Module; and Serve Back-End Engine Module,
[0027] In an aspect, the Unified Analytics Platform Management Systems module comprises Computation Management System; Data Catalog Management Systems; Data Incident Management Systems; Testing Management System; Data Events Management System; Query Observer Systems; and Audit Management Systems.
[0028] In an aspect, the system provides domain business operational planes comprising produce; analyse; and consume.
[0029] In an aspect, the produce operational plane facilitates in defining source(s) of data and related artifacts, processes and guidance.
[0030] In an aspect,theanalyse operational plane facilitates in defining cloud warehouses and related artifacts, processes, guidance around data ingestion, processing and serving.
[0031] In an aspect,theconsume operational plane facilitates in defining consumer(s) of data and related artifacts, processes and guidance around that.
[0032] Various objects, features, aspects and advantages of the inventive subject matter will become more apparent from the following detailed description of preferred embodiments, along with the accompanying drawing figures in which like numerals represent like components.
BRIEF DESCRIPTION OF DRAWINGS
[0033] So that the manner in which the above-recited features of the present invention can be understood in detail, a more particular description of the invention, briefly summarized above, may have been referred by embodiments, some of which are illustrated in the appended drawings. It is to be noted, however, that the appended drawings illustrate only typical embodiments of this invention and are therefore not to be considered limiting of its scope, for the invention may admit to other equally effective embodiments.
[0034] These and other features, benefits, and advantages of the present invention will become apparent by reference to the following text figure, with like reference numbers referring to like structures across the views, wherein:
[0035] FIG. 1 illustrates asystem architecture (block diagrams) illustrating all hardware elements connected via network in accordance to an embodiment of the present invention.
[0036] FIG. 2A illustrates steps of software as service deployment in private cloud in accordance to an embodiment of the present invention.
[0037] FIG. 2B illustrates steps of admin (root) user workflow in accordance to an embodiment of the present invention.
[0038] FIG. 2C illustrates steps of enterprise owner workflow in accordance to an embodiment of the present invention.
[0039] FIG. 2D illustrates steps of project owner workflow in accordance to an embodiment of the present invention.
[0040] FIG. 2E illustrates steps of domain owner workflow in accordance to an embodiment of the present invention.
[0041] FIG. 2F illustrates steps of domain user workflow in accordance to an embodiment of the present invention.
[0042] FIG. 3A illustrates steps of software as service deployment on private cloud in module view form in accordance to an embodiment of the present invention.
[0043] FIG. 3B illustrates steps of admin (root) user workflow in module view form in accordance to an embodiment of the present invention.
[0044] FIG. 3C illustrates steps of enterprise owner workflow in accordance to an embodiment of the present invention.
[0045] FIG. 3D illustrates steps of project owner workflow in module view form in accordance to an embodiment of the present invention.
[0046] FIG. 3E illustrates steps of domain owner workflow in module view form in accordance to an embodiment of the present invention.
[0047] FIG. 3F illustrates steps of domain user workflow in module view form in accordance to an embodiment of the present invention.
DETAILED DESCRIPTION OF THE INVENTION
[0048] In the following description, numerous specific details are set forth in order to provide a thorough understanding of embodiments of the present invention. It will be apparent to one skilled in the art that embodiments of the present invention may be practiced without some of these specific details.
[0049] Various methods described herein may be practiced by combining one or more machine-readable storage media containing the code according to the present invention with appropriate standard computer hardware to execute the code contained therein. An apparatus for practicing various embodiments of the present invention may involve one or more computers (or one or more processors within a single computer) and storage systems containing or having network access to computer program(s) coded in accordance with various methods described herein, and the method steps of the invention could be accomplished by modules, routines, subroutines, or subparts of a computer program product.
[0050] If the specification states a component or feature “may”, “can”, “could”, or “might” be included or have a characteristic, that particular component or feature is not required to be included or have the characteristic.
[0051] As used in the description herein and throughout the claims that follow, the meaning of “a,” “an,” and “the” includes plural reference unless the context clearly dictates otherwise. Also, as used in the description herein, the meaning of “in” includes “in” and “on” unless the context clearly dictates otherwise.
[0052] The recitation of ranges of values herein is merely intended to serve as a shorthand method of referring individually to each separate value falling within the range. Unless otherwise indicated herein, each individual value is incorporated into the specification as if it were individually recited herein. All methods described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The use of any and all examples, or exemplary language (e.g. “such as”) provided with respect to certain embodiments herein is intended merely to better illuminate the invention and does not pose a limitation on the scope of the invention otherwise claimed. No language in the specification should be construed as indicating any non-claimed element essential to the practice of the invention.
[0053] Groupings of alternative elements or embodiments of the invention disclosed herein are not to be construed as limitations. Each group member can be referred to and claimed individually or in any combination with other members of the group or other elements found herein. One or more members of a group can be included in, or deleted from, a group for reasons of convenience and/or patentability. When any such inclusion or deletion occurs, the specification is herein deemed to contain the group as modified thus fulfilling the written description of all groups used in the appended claims.
[0054] Exemplary embodiments will now be described more fully hereinafter with reference to the accompanying drawings, in which exemplary embodiments are shown. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. These embodiments are provided so that this disclosure will be thorough and complete and will fully convey the scope of the invention to those of ordinary skill in the art. Moreover, all statements herein reciting embodiments of the invention, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Additionally, it is intended that such equivalents include both currently known equivalents as well as equivalents developed in the future (i.e., any elements developed that perform the same function, regardless of structure).
[0055] The present invention generally relates to data cloud. More specifically, the present invention relates to context driven domain topology based unified modern advance data analytics management framework and platform for data cloud products.
[0056] An aspect of the present invention provides system for unified data analytics management framework and platform for data cloud products, said system comprising: a web module comprising web user interface components for providing primary mode of input, output and recommendation communication mechanisms for one or more users of the system; a restful service module comprising web services integration interfaces to asynchronous compute intensive operations modules, wherein the web services are restful web APIs that are based on hypertext transfer protocol (HTTP) methods to access resources via uniform resource locator (URL)-encoded parameters and any or combination of JavaScript object notation (JSON) or extensible markup language (XML) to transmit data; a back-end engine module This category of module consists of compute intensive operations modules that require dedicated resources to interact directly with cloud products management components for asynchronously work on computing intensive operations without having to interrupt user workflows via the web user interface; and a unified analytics platform management systems module comprising independent deployable systems that facilitates in governing, monitoring, coding, deploying, securing data residing in data cloud products or flowing through various cloud technologies components.
[0057] In an aspect, the system is domain topology based.
[0058] In an aspect, the web module comprises Login Web Module; Account Creation Web Module; Platform Deployment Web Module; Enterprise Web Module; Project Web Module; Document Web Module; Domain Web Module; and Storage Web Module.
[0059] In an aspect, the restful services module comprises Authentication Restful Services Module; Authorization Restful Services Module; Entity Restful Services Module; Configuration Restful Services Module; Notification Restful Services Module; Deployment Restful Services Module; Context Restful Services Module; Guidance Restful Services Module; Data Governance Restful Services Module; Data Sharing Restful Service Module; Document Integration and Search Restful Service Module; Storage Restful Service Module; and Task Router Restful Service Module.
[0060] In an aspect, the Back-End Engine Modules comprises Infrastructure Deployment Back-End Engine Module; Code Deployment Back-End Engine Module; Context Based ?ML & AI Recommendation Back-End Engine Module; Document Search & Index Back-End Engine Module; Users & Policies Back-End Engine Module; Data Interoperability Back-End Engine Module; Data Storage Backend Engine; Ingest Back-End Engine Module; Process Back-End Engine Module; and Serve Back-End Engine Module,
[0061] In an aspect, the Unified Analytics Platform Management Systems module comprises Computation Management System; Data Catalog Management Systems; Data Incident Management Systems; Testing Management System; Data Events Management System; Query Observer Systems; and Audit Management Systems.
[0062] In an aspect, the system provides domain business operational planes comprising produce; analyse; and consume.
[0063] In an aspect, the produce operational plane facilitates in defining source(s) of data and related artifacts, processes and guidance.
[0064] In an aspect, the analyse operational plane facilitates in defining cloud warehouses and related artifacts, processes, guidance around data ingestion, processing and serving.
[0065] In an aspect, the consume operational plane facilitates in defining consumer(s) of data and related artifacts, processes and guidance around that.
[0066] FIG. 1 illustrates asystem architecture (block diagrams) illustrating all hardware elements connected via network in accordance to an embodiment of the present invention.
[0067] In an embodiment,system 100 (alternatively referred to as datageir 100 hereinafter) is divided into 4 categories of modules. Each category of module interact with other category of modules in a way depicted in FIG. 1. Definitions of each of the category of modules are as below.
[0068] Web Module102 : This category of modules consists of the Web User Interface component of Datageir 100. They will be primary mode of input, output and recommendation communication mechanisms for the various users of the platform.
[0069] Restful Services Module 104: This category of modules consists of the Web Services integration interfaces to asynchronous compute intensive operations modules. These web services are Restful web APIs that are typically loosely based on HTTP methods to access resources via URL-encoded parameters and the use of JSON or XML to transmit data.
[0070] Back-End Engine Module 106: This category of module consists of compute intensive operations modules that require dedicated resources. These category of modules would act as major modules to interact directly with Datageir 100 Data Cloud Products management components. Their main jobs would be to asynchronously work on compute intensive operations without having to interrupt user workflows via the Web User Interface.
[0071] Unified Analytics Platform Management Systems module 108: This category of modules consists of independent deployable systems that are going to help in governing, monitoring, coding, deploying, securing data residing in Data cloud products or flowing through various cloud technologies components.
[0072] Web Modules 102 Definitions
Module Name Module Definition
Login Web Module This module primary has web user interface components related to user login and user authorization.
Account Creation Web Module This module primary has web user interface components related to client account creation and account administrator setup.
Platform Deployment Web Module This module primary has web user interface components that managesDatageir 100 platform components deployment for the client account. This will be controlled by account administrator.
Enterprise Web Module This module primary has web user interface components that helps in Enterprise setup and management of client account.
Project Web Module This module primary has web user interface components that helps in Project setup and management of client account. Project is a sub entity of Enterprise.
Document Web Module This module primary has web user interface components that helps in document repository setup and integrating with other external document repositories for an Enterprise or project.
Domain Web Module This module primary has web user interface components that helps in Domain setup and management of client account. Domain is a sub entity of Project.
Storage Web Module This module primary has web user interface components that helps in data storage setup and integrating with other external data storage for a domain.

[0073] Datageir 100Restful Services Modules 104 Definitions
Module Name Module Definition
Authentication Restful Services Module Restful Web API Component for handling user authentication and session management.
Authorization Restful Services Module Restful Web API Component for handling user authorization based on user role.
Entity Restful Services Module Restful Web API Component for handling creation of entities like users, roles, enterprise, projects, domain, business goals and objectives.
Configuration Restful Services Module Restful Web API Component for handling configurations of entities like users, roles, enterprise, projects, domain, business goals and objectives.
Notification Restful Services Module Restful Web API Component for handling notifications or alert via email, sms and phone calls.
Deployment Restful Services Module Restful Web API Component for handling Software as Service Deployment of Datageir 100 Complete Platform.
Context Restful Services Module Restful Web API Component for setting and getting domain, enterprise or project contexts.
Guidance Restful Services Module Restful Web API Component for providing recommendation to end users based contexts.
Data Governance Restful Services Module Restful Web API Component for providing data governance management functionalities to end users based contexts and guidance.
Data Sharing Restful Service Module Restful Web API Component for providing data interoperability and data sharing management functionalities to end users based contexts and guidance.
Document Integration and Search Restful Service Module Restful Web API Component for providing document repositories management and integration functionalities.
Storage Restful Service Module Restful Web API Component for providing data storage registration and management functionalities.
Task Router Restful Service Module Restful Web API Component for routing user actions to proper back-end engine based on the tasks selected.

[0074] Datageir 100Back-End Engine Modules 106Definitions
Module Name Module Definition
Infrastructure Deployment Back-End Engine Module Independent asynchronous processes to handle compute intensive infrastructure deployment task of Datageir 100 Software As Service Platform.
Code Deployment Back-End Engine Module Independent asynchronous processes to handle compute intensive code deployment task of various modules of Datageir 100 management system .
Context Based ?ML & AI Recommendation Back-End Engine Module Independent asynchronous processes to handle compute intensive machine learning and artificial intelligence tasks based on domain. Project or enterprise contexts.
Document Search & Index Back-End Engine Module Independent asynchronous processes to handle compute intensive tasks of keyword based document repository search and indexing keywords of documents in word, excel or text formats.
Users & Policies Back-End Engine Module Independent asynchronous processes to handle compute intensive tasks of associating users with machine executable policies.
Data Interoperability Back-End Engine Module Independent asynchronous processes to handle compute intensive tasks of associating users with machine executable data interoperability standards.
Data Storage Backend Engine Independent asynchronous processes to handle compute intensive tasks of data storage’s data profiling, data cataloguing and metadata capture activities.
Ingest Back-End Engine Module Independent asynchronous processes to handle compute intensive tasks of data ingestion.
Process Back-End Engine Module Independent asynchronous processes to handle compute intensive tasks of data processing.
Serve Back-End Engine Module Independent asynchronous processes to handle compute intensive tasks of data serving or consuming.

[0075] Datageir 100Unified Analytics Platform Management Systems module 108 Definitions
Module Name Module Definition
Computation Management System Independent Deployable System for managing compute for Data cloud product and cloud native technologies
Data Catalog Management Systems Independent Deployable System for cataloguing data residing and being used in Data cloud product and cloud native technologies.
Data Incident Management Systems Independent Deployable System for manging quality incidences of data residing and being used in Data cloud product and cloud native technologies.
Testing Management System Independent Deployable System for manging testing of code used for ingesting, processing and serving of data residing and being used in Data cloud product and cloud native technologies.
Data Events Management System Independent Deployable System for manging events related to data objects for across the domain artifacts communication.
Query Observer Systems Independent Deployable System for observing queries used against Data Storage and finding KPIs around queries.
Audit Management Systems Independent Deployable System for auditing queries, users, policies for data residing and being used in Data cloud product and cloud native technologies.

[0076] Datageir 100 is a Smart, Interactive web User Interface driven platform incorporating workflows and procedural steps for managing Unified Modern Analytics Platform on ‘Data Cloud’ products with cloud-native services for enterprises of various scales. The purpose of the platform is twofold. The platform is going to be used as a course correction mechanism for existing Data Cloud products setups and, also to be used as a guided step by step workflow driven mechanism for fresh Data Cloud products setups. The twofold purpose of Datageir 100 can be translated to two common scenarios of managing data cloud products for an enterprise that one would always encounter while using this platform. The two scenarios will cover clean slate Data Cloud products enterprise setups or existing Data Cloud products warehouse setups that are not clean slate but already in use for a shorter or longer period. All functionalities of Datageir 100 will be developed keeping in mind those two scenarios only.
[0077] With those two-fold purposes and subsequent scenarios in mind, Datageir 100 is going to define a context aware process driven framework-based approach towards managing Unified Modern Analytics Platform on Data Cloud products with cloud native services. The proposed framework is going to determine the context in which Datageir 100 is going to be used for. The framework is going to define contexts in terms of Hierarchy of Enterprise, Projects and Domains. All the framework paradigms and imperatives are delivered to enterprise end-users of Datageir 100 via smart interactive web user interface.
The Global and Domain Context
[0078] Datageir 100 is aimed at deriving context in which Data Cloud products implementations are working or proposed to be working in future. The reason for identifying context is that every organization is different. Their team structure, business compositions and technical processes are tailored for individual needs. Hence, the way organizations want to manage Unified Modern Advance Data Analytics platform using Data Cloud products would vary. Datageir100 being a generic platform to be used by organizations of different nature and scale, needs to keep that kind of flexibility as a core principle. Otherwise, the usage of Datageir 100 would be very limited and cannot be scaled to many organizations. Hence, the workflow driven framework proposed by Datageir 100 is context aware. The Context broadly represents the environment in which the Unified Modern Advance Data Analytics platform using Data Cloud products are going to be managed. The environment represents all factors that are going to enable or constrains resources and users of an organization in managing Unified Modern Advance Data Analytics platform using Data Cloud products. In other words, these factors can be called as the influencers that affect organizations' practical reasoning and choices of how they want to manage Unified Modern Advance Data Analytics platform using data cloud products. Hence, it is imperative for Datageir 100 to capture the context in which organizations are going to use Unified Modern Advance Data Analytics platform using data cloud products. There are two types of contexts that are defined by Datageir 100, the global context and the domain context.
[0079] Global Context defines the environments concerning enterprises and projects that will be commonly applicable across all line of businesses (domains) implementing Unified Modern Advance Data Analytics platform using Data Cloud products. They are related to enterprise global security policies, enterprise global teams and enterprise global data interoperability standards. Domain Context defines the local environments concerning line of businesses implementing Unified Modern Advance Data Analytics platform using data cloud products. They are related to line of business local security policies, line of business local teams and line of business local data modelling standards, data quality standards and many more. One important aspect taken care by Datageir 100 workflow driven framework is that the Global Context are always implemented at domain level by mapping domain level contexts to it. This way global constraints and standards are percolated to granular implementation level of domains.
Web User Interface
[0080] The primary mode of interaction of end users with Datageir 100 will be an interactive web user interface. This interactive user interface will be taking all relevant inputs from organizational end users to determine the context to manage the Unified Modern Advance Data Analytics platform using data cloud products setups for an enterprise. The web UI is going to follow the hierarchical steps defined by Datageir 100 framework. It will be a workflow and rules driven user interface. It will be interacting with a back-end engine whose job will be to map the user inputs from the logical steps of the Datageir 100 core framework to physical implementation at the databases, warehouses and data lakes. The backend engine along with its own dedicated database store will be interacting with databases, warehouses and data lakes APIs to implement various Datageir 100 functionalities.
[0081] Datageir 100 platform will be hosted as Software as Service Application (SaaS) on one of the private cloud providers. As shown in FIG. 2A (and modular implementation of the same in FIG. 3A), after successful login of Deployment Admin, there will be independent automated deployment setup scripts that will setup Datageir 100 web servers on compute instances of private cloud providers and its backend would be cloud managed Relational Database service. There scripts would also be creating data models and base seed data populations for database tables. Admin(root) user would be the highest privilege user created by the scripts and it would have privileges to on board any Enterprises.
[0082] Admin(root) user would be the highest privileged user of Datageir 100 platform. This user would be responsible for onboarding the Enterprises on the Datageir 100SaaS platform. As shown in FIG. 2B(and modular implementation of the same in FIG. 3B), after successful login Admin would create the enterprise and set up enterprise owner role based user with email. Once it is setup, platform is going to send automated email to the enterprise owner to reset its enterprise owner password.
[0083] Enterprise Owner role based user would be the highest privileged user within the realm of Enterprise entity of Datageir 100 platform. As shown in the FIG. 2C(and modular implementation of the same in FIG. 3C), after successful login, the first activity, it is going to perform to set up enterprise level goals and objectives for building Unified Modern Advance Data Analytics on data products implementations using cloud native service offerings. After that, the user will set up global teams of platform admins, security implementors and compliance teams. The three teams are just examples, enterprise owner can setup any teams with users and roles suited to its enterprise level practices using Datageir 100 platform. Enterprise owner after setting up global teams would be setting up enterprise level security policies and global data interoperability standards. Based on these activities, Datageir 100 platform would internally setup enterprise context. This context would be used in smart guidance of the Datageir 100 platform to its project users. Enterprise owner will also initiate a document repository creation on Datageir 100 platform. Datageir 100 guidance system of processes would help enterprise owner setting up document repository with pre-defined document templates and also with different types of document examples related to enterprises implementing Unified Modern Advance Data Analytics. The guidance process would be question and answer based. The sub entity of Enterprise would be project. Enterprise owner is going to setup one or more project and its owner just in similar fashion as Enterprise and its owner is created by Admin (Root) user.
[0084] Project Owner role based user would be the highest privileged user within the realm of Project (sub entity of enterprise) entity of Datageir 100 platform. One of the activities project owner is to align/endorse goals and objectives set by Enterprise Owner. As shown in FIG. 2D(and modular implementation of the same in FIG. 3D), the project owner can also add its own sub goals or objectives based on aligned Enterprise goals. The project owner can create its own project level security policies and map them to enterprise level security policies set by enterprise owner. The project owner can also override the security policies set by enterprise owner. Datageir 100 guidance system of processes would help project owners set security policies based on enterprise context and enterprise security policies set by enterprise owner. Similar, to project security policies, the project owner can create its own project level data interoperability standards and map them to enterprise level data interoperability standards set by enterprise owner.
[0085] The project owner can also override the data interoperability standards set by enterprise owner. Based on these activities, Datageir 100 platform would internally setup project context. The project context would be used in smart guidance of the Datageir 100 platform to its domain users. Datageir 100 guidance system of processes would help project owners set data interoperability standards based on enterprise context and enterprise data interoperability standards set by enterprise owner. Project owner can also upload documents to document repository created by enterprise owner. Datageir 100 guidance system of processes would help project upload documents to document repository based on enterprise context. The guidance process would be question and answer based. The sub entity of project would be domain. Project owner is going to setup one or more domain and its owner just in similar fashion as Project and its owner is created by enterprise owner.
[0086] Domain Owner role based user would be the highest privileged user within the realm of Domain (sub entity of project) entity of Datageir 100 platform. One of the activities that domain owner is going to do is to assign users from global teams, created by enterprise owner, who will take care of align domain activities in terms of global security policies, interoperability standards, business goals and objectives. They will be monitoring and aligning domain team activities towards business delivery.
[0087] As shown in FIG. 2E(and modular implementation of the same in FIG. 3E), the domain owner can create its own domain level security policies and map them to project level security policies set by project owner. The domain owner can also override the security policies set by project owner. Datageir 100 guidance system of processes would help domain owners set security policies based on enterprise context, and project context and also, project security policies set by project owner. Similar, to domain security policies, the domain owner can create its own domain level data interoperability standards and map them to project level data interoperability standards set by project owner.
[0088] The domain owner can also override the data interoperability standards set by project owner. For carrying out activities of Unified Modern Advance Data Analytics platform, domain owner needs can create different domain roles and create users for those roles. Datageir 100 guidance system of processes would help domain owner create other domain roles and users based on enterprise and project context. After creating users and setting up security policies and interoperability standards Another important activity that domain owner is to determine core functions that domain would be implementing for carrying activities of Unified Modern Advance Data Analytics platform built on top of data cloud products using private cloud native service offerings.
[0089] Core functions represent the theme that pervades technical goals of projects using Datageir 100. They represent the different groups of technical activities that a domain would be managing in Unified Modern Advance Data Analytics platform. Datageir 100 is a multi-purpose platform used in managing different aspects of Unified Modern Advance Data Analytics platform built on top of data cloud products using private cloud native offerings and with every different purpose the artifacts, utilities, guidance, and automation scripts provided by it are going to differ. Hence deciding on the core functions are important.
[0090] Examples of Core Functions selections provided by Datageir 100 will be Data Migration, Batch Data Analytics, Real-Time Analytics, Machine Learning Based Advance Analytics. Apart from pre-built Core Functions, Datageir 100 would also provide organization users to create Custom Core Functions that can re-use artifacts, utilities, and automation scripts from multiple pre-built Core Functions, or they can create their own artifacts, utilities, and automation scripts via Datageir 100. Based on the core functions and other activities, Datageir 100 platform would internally setup domain context. The domain context would be used in smart guidance of the Datageir 100 platform to its domain users.
[0091] The domain users (with assigned roles) by domain users would perform various activities as depicted in FIG. 2F (and modular implementation of the same in FIG. 3F). The first activity domain users are going to perform is to select the core functions for which they are going to perform management activities for Unified Modern Advance Data Analytics platform. After selecting the core functions, domain users are going to select warehouse state on which they are going perform management activities.
[0092] The reason for asking warehouse state is that Datageir 100 needs to understand that it being used to set up Unified Modern Advance Data Analytics platform from scratch or to course correct existing Unified Modern Advance Data Analytics platform. Core functions lay down the foundation for artifacts, processes, guidance, and automation scripts that Datageir 100 is going to incorporate to achieve the desired technical goals.
[0093] However, the mechanisms that are going to apply those artifacts, processes, guidance, and automation scripts depends on the state Unified Modern Advance Data Analytics platform is in. Hence after determining Core functions Datageir 100 needs to understand the warehouse State. Next, the domain user needs to select the domain operational plane. Datageir 100 defines three types of domain business operational plane. They bring in perspectives from business. The domain business operational planes are produce, analyse and consume. The “produce” operational plane is going to define source(s) of data and related artifacts, processes and guidance. The “analyse” operational plane is going to define cloud warehouses and related artifacts, processes, guidance around data ingestion, processing and serving. The “consume” operational plane is going to define consumer(s) of data and related artifacts, processes and guidance around that.
[0094] Once the domain business operational plane is selected, the domain user context is setup by Datageir 100 Platform. The domain user context would be used in smart guidance of the Datageir 100 platform in further activities of domain users. Next, the domain user is required to register the data store for the selected operational plane. The data store can be file based, RDBMS or Cloud Warehouse. Once, data store is selected Datageir 100 automated processes would run and test connectivity, get meta data details about the datastore and fetch other related artifacts required for managing Unified Modern Advance Data Analytics platform. After, the data store is registered, domain user needs to select operational data layer. The choice of operational data layer is applicable as per choice of operational plane selected. However, in general there are three options that will be available to the domain user. They are ingest, process and serve. Each layer is responsible for a specific function and interacts with other layers via their well-defined secured interfaces. Next, domain user needs to select the Operational Data Layer Activity. The activity options are plan, build, validate, capture and analyse. These activities are a series of Datageir 100’s Generalized Methodical Workflow-oriented Steps.
[0095] Every one of these activities are synonymous to different phases that any Unified Modern Advance Data Analytics platform would go through while implementation. For every activity (or phase) Datageir 100 would give domain users options to provide relevant inputs alongside presenting relevant guidance in terms of artifacts, utilities, and automation scripts. One important thing to note here is that the sequence in which these activities are going to be executed by Datageir 100 would be directly influenced by the choice of the warehouse state. As shown in FIG. 1, the sequence of steps for clean Slate warehouse state is different from sequence of steps for existing warehouse state. The reason for such difference is for existing warehouses Datageir 100 would start with first analysing the existing warehouse setup via automation scripts and manual interventions. After that it will go ahead with other steps like planning etc. Next step would be to select Operational Data Layer Activity Scope.
[0096] These scopes are going to be Schema, Code, Testing, Security, Data Observability, Alerting, Governance, Orchestration and Alerting & Monitoring. Based on the scope selection domain users are going to be presented with tasks. These tasks granular level implementation activities like Schema Migration Tools/Utilities Analysis and Scope (Cloud Native/ Open Source / Proprietary), Ingestion Data Sources Schema Analysis, Ingestion Data Sources Target Schema Analysis and Mapping, Schema Migration Checklists etc. Datageir 100 guidance system of processes would help domain users with performing these tasks after taking relevant inputs from them. Then, these tasks are applied by Datageir 100 Automation processes to target Unified Modern Advance Data Analytics platform on Data Cloud Products using cloud native service offerings. Datageir100establishes an iterative process of selecting domain operational plane to applying operational data layer activity scope tasks to cover all aspects of managing Unified Modern Advance Data Analytics platform.
ADVANTAGE OF THE PRESENT INVENTION
[0097] The present invention provides a system for unified data analytics management framework and platform for data cloud products.
[0098] The present invention provides a system for unified data analytics management framework and platform for data cloud products that help enterprise users in using and managing ‘Data Cloud’ products in conjunction with cloud-native services as a Unified Analytics Data Platform
[0099] The present invention provides a system for unified data analytics management framework and platform for data cloud products that simplify complexities involved in managing Unified Analytics Data Platform for volume, varieties, and velocity of data with data recency and quality issues on top of Security with Governance.
[00100] The present invention provides a system for unified data analytics management framework and platform for data cloud products that can be use by both teams starting fresh on Data Cloud platforms, and teams that are using Data Cloud for shorter and longer periods.
[00101] The present invention provides a system for unified data analytics management framework and platform for data cloud products thatiscontexts driven by line of businesses or domains to drive end to end domain data ownership to a single organizational unit.
[00102] The present invention provides a system for unified data analytics management framework and platform for data cloud products that provides more control over cost, chargeback models among different organizational units sharing data
, Claims: 1. A system for unified data analytics management framework and platform for data cloud products, said system comprising:
a web module comprising web user interface components for providing primary mode of input, output and recommendation communication mechanisms for one or more users of the system;
a restful service module comprising web services integration interfaces to asynchronous compute intensive operations modules, wherein the web services are restful web APIs that are based on hypertext transfer protocol(HTTP) methods to access resources via uniform resource locator (URL)-encoded parameters and any or combination of JavaScript object notation (JSON)orextensiblemarkup language (XML) to transmit data;
a back-end engine module This category of module consists of compute intensive operations modules that require dedicated resources to interact directly with cloud products management components for asynchronously work on computing intensive operations without having to interrupt user workflows via the web user interface; and
a unified analytics platform management systems module comprising independent deployable systems that facilitates in governing, monitoring, coding, deploying, securing data residing in data cloud products or flowing through various cloud technologies components.

2. The system as claimed in claim 1, wherein the system is domain topology based.

3. The system as claimed in claim 1, wherein the web module comprises Login Web Module; Account Creation Web Module; Platform Deployment Web Module; Enterprise Web Module; Project Web Module; Document Web Module; Domain Web Module; and Storage Web Module.

4. The system as claimed in claim 1, wherein the restful services module comprises Authentication Restful Services Module; Authorization Restful Services Module; Entity Restful Services Module; Configuration Restful Services Module; Notification Restful Services Module; Deployment Restful Services Module; Context Restful Services Module; Guidance Restful Services Module; Data Governance Restful Services Module; Data Sharing Restful Service Module; Document Integration and Search Restful Service Module; Storage Restful Service Module; and Task Router Restful Service Module.

5. The system as claimed in claim 1, wherein the Back-End Engine Modules comprises Infrastructure Deployment Back-End Engine Module; Code Deployment Back-End Engine Module; Context Based ?ML & AI Recommendation Back-End Engine Module; Document Search & Index Back-End Engine Module; Users & Policies Back-End Engine Module; Data Interoperability Back-End Engine Module; Data Storage Backend Engine; Ingest Back-End Engine Module; Process Back-End Engine Module; and Serve Back-End Engine Module.

6. The system as claimed in claim 1, wherein the Unified Analytics Platform Management Systems module comprises Computation Management System; Data Catalog Management Systems; Data Incident Management Systems; Testing Management System; Data Events Management System; Query Observer Systems; and Audit Management Systems.

7. The system as claimed in claim 1, wherein the system provides domain business operational planes comprising produce; analyse; and consume.
8. The system as claimed in claim 7, wherein the produce operational plane facilitates in defining source(s) of data and related artifacts, processes and guidance.

9. The system as claimed in claim 7, wherein the analyse operational plane facilitates in defining cloud warehouses and related artifacts, processes, guidance around data ingestion, processing and serving.

10. The system as claimed in claim 7, wherein the consume operational plane facilitates in defining consumer(s) of data and related artifacts, processes and guidance around that.