COMPLEX DATA ACCESS
Technical Field
The present invention relates generally to the field of software applications More particularly, the present invention relates to software applications that store data and report data, such as databases More particularly still, aspects of the present invention relate to storing and querying data in such software applications
Background of the Invention
In order to manage large quantities of data, computer software applications, such as spreadsheet and database applications, have been developed to organize and store the data in a logical manner Typical spreadsheet and database applications comprise a large number of records of information, wherein each record compnses a predetermined number of fields In the context of a database, a database management system is typically used to provide the software tools to manipulate the database more simply Example database management systems include Microsoft® Access and Microsoft® SQL Server, among others Databases generally allow users to establish complex data interrelationships that are not available in spreadsheets, which increases their power while also making database applications even more difficult for new users to master
A typical database management system provides the user the ability to add, modify or delete data, query data using filters, and the ability to report records in the database Data in databases is typically represented using rows and columns A field is an intersection of a row and a column In general, any given field may only contain a single, scalar data value Data queries typically take the form of a query language, such as Structured Query Language (SQL) Such queries may be performed on locally held data, or submitted to a database server for execution
In some cases, a database may contain a hierarchy of data rather than a flat table of values For example, in an issue tracking application, each issue may be assigned to one or more people A user must therefore model this type of hierarchy using multiple linked tables, and query such
tables using relatively complex queries that join data from multiple tables More specifically, there might be one table of issues, and another table of people containing all the possible people who could have an issue assigned to them, and finally, a junction table where issues are assigned to people Setting up such a hierarchy of tables can be both cumbersome and confusing to a user, especially a user new to database applications Querying across such a hierarchy can be similarly problematic due the complex nature of multi-table queries, especially if a user lacks in-depth knowledge of the query language
It is with respect to these considerations and others that the present invention has been made
Summary of the Invention
In accordance with the present invention, a computer-implemented method and computer-readable medium are provided for adding complex data to a conceptual table A conceptual table is a table that contains at least one complex data type A complex data type is one or more scalar columns and/or other complex types that are "grouped" together to form a single, new data type First, a signal is received to add a complex data type to a table One or more data tables are created Additionally, links to the one or more data tables are created in the conceptual table
In accordance with other aspects, the present invention relates to a computer-implemented method and computer-readable medium for modeling data in a conceptual table First, a signal to store one or more data values is received The structure of the conceptual table is analyzed, and one or more mappings to one or more data tables are determined Finally, data is stored in the one or more data tables
In accordance with yet other aspects, the present invention relates to a computer-implemented method and computer-readable medium for the expansion of a query of a conceptual table First, a query of a conceptual table is received Next, the structure of the conceptual table is analyzed, and one or more expansion rules are applied to the query Finally, the expanded query is issued
In accordance with still other aspects, the present invention relates to a computer-implemented method and computer-readable medium for cursonng in a conceptual table First, a signal to access complex data in the conceptual table is received One or more mappings associated with the complex data are read Finally, scalar data from one or more data tables is read using the mappings
In accordance with additional, other aspects, the present invention relates to a system for conceptual table modeling A data modeling module adds complex data to the table, and stores complex data to one or more data tables A query expansion module expands conceptual table quenes to include one or more data table quenes A cursonng module retrieves complex data in a conceptual table
The invention may be implemented as a computer process, a computing system or as an article of manufacture such as a computer program product or computer readable media The computer readable media may be a computer storage media readable by a computer system and encoding a computer program of instructions for executing a computer process The computer program readable media may also be a propagated signal on a earner readable by a computing system and encoding a computer program of instructions for executing a computer process
These and vanous other features as well as advantages, which charactenze the present invention, will be apparent from a reading of the following detailed descnption and a review of the associated drawings
Brief Description of the Drawings
Fig 1 shows a database environment in which an embodiment of the present invention may be implemented
Fig 2 illustrates an example of a suitable computing system environment on which an embodiment of the present invention may be implemented
Fig 3 illustrates the modules present in one embodiment of the present invention
Fig 4 illustrates the operational flow of the operations performed in accordance with an embodiment of the present invention
Fig 5 illustrates the operational flow of the operations performed in accordance with an embodiment of the present invention
Fig 6 illustrates the operational flow of the operations performed in accordance with an embodiment of the present invention
Fig 7 illustrates the operational flow of the operations performed in accordance with an embodiment of the present invention
Detailed Description of the Invention
Fig 1 illustrates a screenshot of a tabular view 102 of a product order database within a database management system The table 102 includes a table comprised of multiple rows and columns ot data Each row of data generally comprises a single data record Generally, each column of data m a database can be counted on to include data elements of a homogenous type For example, the Order ID column 104 includes data elements in numeric format, the Customer column 106 includes data in the form of alphanumeric strings, the Order Date column 108 includes data in date format, and so on The Order ID column of a single record corresponds to that Order ID field of that record A collection of fields thus may comprise a column One skilled in the art will appreciate that many other types of data can be kept in a database, and displayed using a table within a database management system The present invention allows a complex hierarchy of data to be modeled using the conceptualization of a simple database table (similar in appearance to database table 102, but structurally different), relieving users of the burden of forming sophisticated queries across multiple tables, and flipping back and forth between multiple tables to view and manipulate data
Given that the present invention may be implemented as a computer system, Fig 2 is provided to illustrate an example of a suitable computing system environment on which embodiments of the invention may be implemented In its most basic configuration, system 200
includes at least one processing unit 202 and memory 204 Depending on the exact configuration and type of computing device, memory 204 may be volatile (such as RAM), non-volatile (such as ROM, flash memory, etc ) or some combination of the two This most basic configuration is illustrated in Fig 2 by dashed line 206
In addition to the memory 204, the system may include at least one other form of computer-readable media Computer-readable media can be any available media that can be accessed by the system 200 By way of example, and not limitation, computer-readable media might comprise computer storage media and communication media
Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data Memory 204, removable storage 208, and non-removable storage 210 are all examples of computer storage media Computer storage media includes, but is not limited to, RAM, ROM, EPROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by system 200 Any such computer storage media may be part of system 200
System 200 may also contain a communications connection(s) 212 that allow the system to communicate with other devices The communications connection(s) 212 is an example of communication media Communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a earner wave or other transport mechanism and includes any information delivery media The term "modulated data signal" means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media The
term computer readable media as used herein includes both storage media and communication media
In accordance with an embodiment, the system 200 includes peripheral devices, such as input device(s) 214 and/or output device(s) 216 Exemplary input devices 214 include, without limitation, keyboards, computer mice, pens, or styluses, voice input devices, tactile input devices and the like Exemplary output device(s) 216 include, without limitation, devices such as displays, speakers, and printers For the purposes of this invention, the display is a primary output device Each of these devices is well know in the art and, therefore, not descnbed in detail herein
With the computing environment in mind, the following figures are descnbed with reference to logical operations being performed to implement processes embodying vanous embodiments of the present invention These logical operations are implemented (1) as a sequence of computer implemented steps or program modules running on a computing system and/or (2) as interconnected machine logic circuits or circuit modules within the computing system The implementation is a matter of choice dependent on the performance requirements of the computing system implementing the invention Accordingly, the logical operations making up the embodiments of the present invention descnbed herein are referred to vanously as operations, structural devices, acts or modules It will be recognized by one skilled in the art that these operations, structural devices, acts and modules may be implemented in software, in firmware, in special purpose digital logic, and any combination thereof without deviating from the spint and scope of the present invention as recited within the claims attached hereto
Fig 3 illustrates modules in accordance with one embodiment of the present invention A user interface module allows users to view, modify, query, and otherwise manipulate data User interaction with data modeling module 304, query expansion module 306, and cursor modeling module 308 therefore takes place through user interface module 302 Likewise, data from said modules 304, 306, and 308 is displayed via user interface module 302
User interface 302 passes table hierarchy parameters to data modeling module 304, and may, additionally, receive feedback from data modeling module 304 to aid a user with further table definition or modification User interface module 302 passes quenes supplied by a user to query expansion module 306, and likewise receives query results from query expansion module 306 User interface module 302 passes requests from a user for data to cursonng module 308 for data retrieval, and likewise receives retrieved data from cursonng module 308
In one embodiment, user interface module 302 directly renders data returned from one or more of modules 304, 306 and 308 on a display In an alternate embodiment, user interface module 302 uses Application Programming Interfaces (APIs) to update a display
Data modeling module 304 allows creation and modification of a conceptual table A conceptual table is a data table which may contain complex (non-scalar) data in the form of another table, as well as complex data A data table typically includes only scalar data, and may include numbers, alphanumeric strings, fractions, and the like Complex data may include one or more nested tables, which may themselves include additional levels of nested tables A conceptual table can contain arbitranly many such hierarchically related tables A given field in a conceptual table may therefore be physically mapped to a field in another table altogether Further, this physical mapping and its consequences may be entirely transparent to a user Data modeling module 304 may receive parameters specified by a user for a conceptual table from user interface 302
In one embodiment, a conceptual table is stored as one or more data tables containing scalar data Scalar data values that exist only in the conceptual table are stored in a data table specifically allocated for non-mapped scalar values One or more data values that physically map to a data table are "shredded" by data modeling module 304 such that those data values are physically stored in that data table 310 As a result, the user need only deal with a single conceptual table, while the complexities of translating the conceptual table to multiple hierarchical tables for storage are transparent to the user Likewise, the implementation of
shredding does not necessarily involve any added complexity in the database storage engine or database query processor
In an embodiment, data values in a conceptual table are shredded to the appropriate data tables when a complex column is added to a table In an alternative embodiment, data values in a conceptual table are shredded to the appropriate data tables when a save command is received from a user In yet another embodiment, data values in a conceptual table are shredded to the appropnate data tables whenever a value change is detected
Query expansion module 306 receives a conceptual table query from user interface module 302 and expands the query based on the hierarchical structure of the conceptual table In one embodiment, query expansion module 306 expands a query based on the hierarchical structure of the conceptual table, using a set of rules defined by a developer An exemplary set of query expansion rules is set forth below in Tables 1-12
In an alternate embodiment, query expansion module 306 expands a query by joining (a term of art in the database field, analogous to combining) a subset of relevant data tables together into a single data table, and then executes the conceptual table query on that single data table In another alternate embodiment, query expansion module 306 expands a query by joining all data tables 310 linked by a conceptual table, regardless of relevance, into a single data table, and then executes the conceptual table query on that single data table
In an embodiment, query expansion module 306 performs quenes directly on data tables 310 In an alternate embodiment, query expansion module 306 submits the expanded query to a database server for execution Such an embodiment does not necessarily require any changes to the query processor or database server
Cursonng module 308 enables reading and navigation of a conceptual table When a
signal is received via user interface 302 to open a conceptual table, cursonng module 308
analyzes any mappings to data tables that exist Scalar data values from those data tables are
retrieved using the mappings, and the scalar data values are returned to user interface 302 for
display
If a user selects a conceptual table field containing a data table, the table mapping in the field is referenced A snapshot of the mapped-to data tables is taken, and die results are displayed in the conceptual table In one embodiment, the mapped-to data tables are also locked for editing so that their contents cannot change In an alternate embodiment, the mapped-to data tables are not locked, and instead, the mapped-to data tables are periodically checked for updates to be propagated to the displayed conceptual table
One skilled in the art will recognize that while the exemplary embodiments presented in conjunction with Fig 3 involve a conceptual table that may include data tables, arbitrarily many levels of indirection may be used without departing from the scope of the claimed invention For example, a conceptual table may include one or more conceptual tables, which may themselves include still other conceptual tables In this situation, care must be taken to prevent an infinite loop, namely a conceptual table referring (either directly or indirectly) to itself, thus creating an infinite number of levels of indirection
In an embodiment, data modeling module 304 may perform data stores atomically This means that in any given data store attempt, either all or none of the data tables are updated As such, reading data while the data tables are in a transitional state may be avoided Atomic data table writes may be performed via semaphore, file locking, or other method known to those skilled in the art, and may additionally incorporate rollback (undoing one or more pending writes)
Fig 4 illustrates an operational flow in accordance with one embodiment of the invention, in which a conceptual table is created Receive operation 402 receives a signal from a user to add a complex data type to an existing table or conceptual table This signal may be received from a user via a user interface, from an automated computer process, or other source of input This receipt triggers creation operation 404 to create at least one data table
Creation operation 404 creates at least one data table to contain any scalar values
associated with the complex data Data entered into the complex data field will be shredded into
the corresponding data table when the conceptual table is stored In one embodiment, each
created data table takes the form of one or more database files In another embodiment, each created data table takes the form of a data structure within one or more database files In yet another embodiment, creation operation 404 additionally initializes one or more scalar values to a default value within each created data table In one embodiment where a suitable data table already exists, create operation 404 may be omitted and the data table may simply be linked to
Following create operation 404, link operation 406 adds one or more links to the conceptual table, corresponding to the one or more data tables created by creation operation 404 A link may be used to locate a data table when a complex data field is selected via cursonng (described above, in conjunction with Fig 3, and below, in conjunction with Fig 7) A link may also be used to locate a data table when a scalar value within a complex data field is added or modified (described above, in conjunction with Fig 3)
Determine operation 408 determines whether additional tables are required to accommodate the complex data If more tables are needed, flow branches YES back to create operation 404 If no more tables are needed, flow branches NO to end of flow 410
Once end of flow 410 has been reached, complex data may be viewed or manipulated using cursonng (discussed above in conjunction with Fig 3, and below in conjunction with Fig 7), and stored via shredding (discussed above in conjunction with Fig 3, and below in conjunction with Fig 5)
In an embodiment, an existing data table may be inserted into a conceptual table In such an embodiment, a verification operation (not pictured) would verify the existence of the data table, and creation operation 404 would be omitted
Fig 5 illustrates an operational flow in accordance with one embodiment of the present invention in which data from a conceptual table is shredded and stored to one or more corresponding data tables
Receive operation 502 receives a signal to store one or more complex data values
associated with a conceptual table The signal may be the result of a user explicitly saving data
within a database management application Alternatively, the signal may be received from an
automated process that synchronizes modified data Receipt of a signal by receive operation 502 triggers analyze operation 504
Analyze operation 504 analyzes the conceptual table through which the complex data values are stored, and determines the structure of the conceptual table Based on this structural data, determine operation 506 determines one or more mappings to locations in data tables These mappings may be in the form of a set of data table coordinates, a pointer to a memory location containing a data table, a data table file name, unique record identification number, or other means by which data may be addressed, or any combination thereof
Store operation 508 stores one or more data values from the conceptual table into the data value's respective data table using these mappings, at which time the conceptual table data has been successfully shredded The data may later be "unshredded" via cursonng (descnbed above, m conjunction with Fig 3 and below, in conjunction with Fig 7) In an embodiment, store operation 508 performs data stores atomically In another embodiment, store operation 508 can roll back incomplete stores
Fig 6 illustrates an operational flow in accordance with one embodiment of the present invention in which a query of a conceptual table is expanded to process shredded data Receive operation 602 receives a query of a conceptual table In one embodiment, such a query may be entered in text form by a user In another embodiment, said query may be fully or partially constructed using a graphical user interface In still another embodiment, said query may be generated automatically by a computer
Analyze operation 604 analyzes the conceptual table through which the complex data is to be quened, and determines the structure of the conceptual table along with any associated mappings to data tables These mappings may take on any of several different forms (as discussed previously m conjunction with Fig 5)
Determine operation 606 uses the structural data and mappings from analyze operation
604, in conjunction with a set of expansion rules, to determine if any rules are appropnate for the
query and whether those rules call for an expansion If no rules are appropnate for the query, or
if an applicable rule dictates that the expansion is complete, flow branches NO to issue operation 610 If one or more rules are appropriate for the query, flow branches YES to apply operation 608
If one or more expansion rules are appropriate to apply to a query, apply operation 608 expands the query according to those rules In one embodiment, apply operation may create a new query as part of expanding the query Some exemplary expansion rules are set forth below in Tables 1-12, including expanding using such SQL terms as JOIN to combine data tables together before evaluating data One skilled in the art will appreciate that determine operation 606 and apply operation 608 may take place repeatedly before a query is fully expanded and thus ready to be issued by issue operation 610 More than one rule can be applied in a vanety of orders, and a single rule may be applied more than once m the expansion of a query In an embodiment implementing incremental expansion of a query, query expansion rules may be kept orthogonal to each other, and thus kept relatively simple In an embodiment performing incremental expansion ot a query, a data structure may be necessary to store intermediate results of incremental expansion rule application In an alternative embodiment, a plurality of query expansion rules may be applied by apply operation 608
Issue operation 610 issues a query expanded by determine and apply operations 606 and 608 In an embodiment, issue operation 610 queries database data directly In an alternative embodiment, issue operation 610 sends the query to a database server
Fig 7 illustrates an operational flow m accordance with one embodiment of the present invention, in which cursonng enables data to be retrieved from a shredded conceptual table Receive operation 702 receives a signal to open complex data m a conceptual table Such a signal may result from a user opening an existing conceptual table containing complex data that must be displayed, from a user moving the cursor to a conceptual table field containing complex data, or via other user-generated or computer-generated inputs
Read operation 704 reads one or more mappings to locations in data tables that are
associated with the conceptual table As discussed previously, these mappings may take a vanety
of forms Retrieve operation 706 then retrieves scalar data values from the data tables indicated by the mappings The scalar values retrieved by retrieve operation 706 are used to populate a conceptual table, which is displayed by display operation 708
Display operation 708 displays the conceptual table containing the scalar values retrieved by retrieve operation 706 Display operation 708 may directly render the conceptual table on the screen in one embodiment, may rely on Application Programming Interfaces (APIs) to perform rendering in another embodiment, or may use other methods for displaying data known to those skilled in the art in still other envisioned embodiments
One skilled in the art will appreciate that the embodiments claimed herein may be used not only in the context of database and spreadsheet applications which use complex data types, but also in the context of collaborative service suites such as Microsoft® Sharepoint Such suites may incorporate their own complex data types containing multiple values (e g, multiple choices, attached files, etc) These complex data types may be handled substantially the same as in the exemplary embodiments discussed herein
In one particular embodiment, a set of SQL query expansion rules is used by query expansion module 306 (Fig 3), and by determine operation 606 and apply operation 608 (Fig 6) An exemplary set of several SQL query expansion rules is presented in Tables 1-11 These rules deal with situations of varying complexity In an example (Table 1), a relatively simple SELECT query is expanded A simple SELECT query is defined as a query that has no grouping or aggregates When there are no restrictions on complex scalar fields, only the parent table needs to be referenced in the expanded query When there is a restriction on a complex scalar in the WHERE clause of a query, a join between a table and complex data type must be performed (see Table 1 for a basic example) Filters in the WHERE clause must be modified as necessary to reference a complex data type instead of a complex column in a table Fields in SELECT clauses must be similarly modified using proper table references When a non-left join on a complex scalar must be performed between a table and another table, the expanded query will include a
subquery to join the parent table and its complex table (see Table 4 for a basic example) If a
NOT appears in the WHERE clause of an unexpanded query, the NOT will be translated into [Table] [Complex Column] NOT IN (SELECT [Complex Column] FROM [Table]JOIN [Flat Table] ON [Table] [Complex Column] = [Flat Table] [Foreign Key] WHERE [Flat Table] [Complex Scalar] = value)
Tables 4-11 illustrate expansion rules for queries with aggregates One such rule is that aggregates of complex types are executed against a LEFT JOIN of the table and complex data type Another such rule is that when there are aggregates of different complex types in the unexpanded query, each aggregate will be computed using a subquery Also, occurrences of the HAVING clause (Table 10) will be converted to a WHERE clause in an outer query If a HAVING clause references a complex type that is not otherwise referenced in an aggregate SELECT list, a new subquery may be created to join the table and the complex type

(Table Removed)

Table 1 Example Filter on a Complex Scalar

(Table Removed)
Table 2 Example Filter on multiple Complex Scalars

(Table Removed)

Table 3 Example Subqueries when joining on a Complex Scalar

(Table Removed)

Table 4 Example Aggregate on a Complex Type

(Table Removed)

Table 5 Aggregate on a Complex Type with a Restriction on the same Complex Scalar

(Table Removed)

Table 6 Example Aggregate on a Complex Type with a Restriction on a different Complex Scalar

(Table Removed)
Table 7 Example Aggregate Complex Type 1 with a Restriction on Complex Scalar 1 and Complex Scalar 2

(Table Removed)

Table 9 Example Aggregates on Complex Type 1 and Complex Type 2 with Restrictions on Complex Scalar I and Complex Scalar 2

(Table Removed)

Table 10 Example Complex Type in the HAVING Clause

(Table Removed)

Table 11 Example Aggregate on Complex Type I with Complex Type 2 in the HAVING Clause
The vanous embodiments described above are provided by way of illustration only and should not be construed to limit the invention Those skilled in the art will readily recognize vanous modifications and changes that may be made to the present invention without following the example embodiments and applications illustrated and descnbed herein, and without departing from the true spint and scope of the present invention, which is set forth in the following claims.

We Claims :-
1. A computer-implemented method for adding complex data to a conceptual table
comprising:
receiving a signal to add complex data to a table to the conceptual table; creating one or more data tables; and creating links to said one or more data tables.
2. A computer-implemented method as defined in claim I, wherein said complex data includes a data table.
3. A computer-implemented method as defined in claim 1, wherein said complex data includes a conceptual table.
4. A computer-implemented method as defined in claim 1 further comprising modeling data in the conceptual table, wherein the modeling data comprises:
receiving a signal to store one or more data values;
analyzing a structure of the conceptual table;
determining one or more mappings to one or more data tables; and
storing data in the one or more data tables.
5. A computer-implemented method as defined in claim 1 further comprising:
expanding a query of me conceptual table, wherein the step of expanding the
query comprises:
receiving a query of a conceptual table; analyzing a structure of the conceptual table; applying one or more expansion rules to the query; and issuing the query.
6. A computer-implemented method as defined in claim 5, wherein said applying one or more expansion rules to the query is performed incrementally.
7. A computer-implemented method as defined in claim 5, wherein a plurality of
said one or more expansion rules may be applied in the same operation..
8. A computer-implemented method as defined in claim 5, wherein said applying one
or more expansion rules to (he query further comprises:
combining a plurality of data tables into a model of the conceptual table; and
applying the query to the model of the conceptual table.
9. A computer-implemented method as defined in claim 5, wherein said issuing the query further comprises submitting the query to a database server for execution.
10. A computer-implemented method as defined in claim 5, wherein said issuing the query further comprises accessing the database data directly.
11. A computer readable medium accessible to a computing system and encoding a computer program for performing the method defined in claim 1.
12. A computer-implemented method as defined in claim 1, further comprising cursoring in the conceptual table wherein cursoring comprises:
receiving a signal to access complex data in the conceptual table; reading one or more mappings associated with said complex data; and retrieving scalar data from one or more data tables using said mappings.
13. A computer-implemented method as defined in claim 12, further comprising displaying the scalar data in the conceptual table.
14. A system for conceptual table modeling comprising:
a data modeling module for adding complex data to a table, and storing complex data to one or more data tables;
a query expansion module for expanding a conceptual table query to include one or more data table queries; and
a cursoring module for retrieving complex data in a conceptual table.
15. A system as defined in claim 14, further comprising a display module for
displaying results from said query expansion module and said cursoring module
16. A computer readable medium accessible to a computing system and encoding a
computer program for modeling data in a conceptual table comprising:
receiving a signal to store one or more data values;
analyzing a structure of the conceptual table;
determining one or more mappings to one or more data tables; and
storing data in the one or more data tables.
17. A computer readable medium as defined in claim 16 wherein the computer
program further comprises expanding conceptual table queries, wherein expanding comprises:
receiving a query of a conceptual table; analyzing a structure of the conceptual table; applying one or more expansion rules to the query; and issuing the query.
18. A computer readable medium as defined in claim 16 wherein the computer
program further comprises cursoring in a conceptual table, wherein cursoring comprises:
receiving a signal to access complex data in a conceptual table; reading one or more mappings associated with said complex data; and retrieving scalar data from one or more data tables using said mappings.