Determining active real objects for running a software application
The present invention relates to the identification of a software
application using real objects that have a communication interface with a
communication network.
It concerns the object Internet, and more specifically, the Object Web".
These relatively new concepts have been the subject of various initiatives. The
work described on the site www.webofthings.com started by two researchers
from the University of Zurich can be cited as an example.
The "web of things" is also described in the article by D. Guinard and V.
Trifa, "Towards the Web of Things: Web Mashups for Embedded Devices" in
Proceedings of International WWW (World Wide Web) Conferences, Madrid,
Spain, 2009.
The trend toward connecting physical objects to communication networks
is also reported in the press release from IMS Research dated 9 August
201 0, "Internet Connected Devices About to Pass the 5 Billion Milestone". This
press release is available at http://imsresearch.com/news-events/presstemplate.
php?pr_id= 1532&cat_id = 108
It is possible to find the results of the work from these authors in other
articles, such as, for example, "Supporting Device Discovery and Spontaneous
Interaction with Spatial References", by Gellersen, Fischer, Guinard, Gostner,
Kortuem, Kray, Rukzio and Streng, or, "Using Spatial Conditions for Proactive
Computing and Interaction Metaphors" by Streng, Guinard and Gellerson, or
"Design of a Web-based Distributed Location-aware Infrastructure for Mobile
Devices" by Trifa, Guinard, Bolliger and Wieland.
These works on the Web of things aim to transform real life objects into
resources available on the Web that may potentially communicate among
each other through the invention: lamps, televisions, communication
terminals, small household appliances, etc. may interface with the Internet,
and the software applications available over it, and thus open up new
possibilities.
Current solutions do not however allow the user to identify which real
objects around him or her are connected to the Internet, thus making them
unusable for running software applications.
Therefore, it is also not possible for the user to determine which software
applications he or she may be able to use.
The present invention aims to resolve this technical problem.
Additionally, it is important that the solution does not inundate the user
with information when he or she is surrounded by too large a number of
active real objects. The present invention also takes this additional problem
into account.
The invention's first object is a method to run a software application
accessible from a communication terminal using at least one real object,
comprising
• A step of describing a curve (C) in space with said communication
terminal
• A step of identifying a geographic zone (ZG) based on this curve,
• A step of determining a set of active real objects located within this
geographic zone having a communication interface with a
communication network
· A step of identifying an application from among these active real
objects
• A step of interfacing the application with the previously determined
active real objects.
According †o the embodiments of the invention, the delimitation step
consists in automatically determining the position of the communication
terminal using a locating device and determining the geographic zone in
relation to this position.
The communication terminal can display the geographic zone being
determined on the screen in real time.
The delimitation and determination steps can be carried out
progressively and in parallel, so that the terminal displays the active real
objects in the geographic zone being determined on the screen.
The determination step may consist of querying a first database
associating real objects and geographic positions, and determining the set of
active real objects by comparing the associated geographic positions in the
database with the geographic zone.
The terminal may display at least part of the geographic zone on the
screen and show the active real objects from among the set of active real
objects corresponding to at least that one part using an augmented reality
mechanism.
The identification step may include re-querying a second database
associating the applications available with the required conditions, and
determining a set of applications for which the associated required conditions
correspond to the abilities offered by the real objects in the set.
It is possible to use an ontology to determine the required conditions and
the applications corresponding to the abilities of the objects.
The set of applications may be provided to the terminal user so that he
or she can select an application from this set of applications. This set of
applications may be displayed on the terminal screen.
The second database may be created from the set of software
applications actually installed on the terminal.
It may also include applications available from at least one application
server.
The application may be run by using a subset of the set of active real
objects, this subset being determined by a criterion of distance from the
position of the communication terminal.
The application may be run by using a subset of the set of active real
objects, this subset being determined by a criterion linked to the profile of the
user of the communication terminal.
The delimitation step may be carried out by determining a path, and the
determination, identification, and interface steps will then be carried out in
relation to the location of the communication terminal along this path.
It is also an object of the invention to have a communication terminal
equipped with processing means designed to allow the use of software
applications and having an interface for communicating with a
communication network, these means of processing comprising at least:
• A first module designed to determine a geographical zone based on
a curve in space described by the communication terminal,
• A second module designed to determine a set of active real objects
located in the geographic zone and having an interface with the
communication network,
• A third module designed to identify an application in relation to the
active real objects
• A fourth module designed to interface the application with the
previously determined active real objects.
The third object of the invention is to have a man-machine interface for a
communication terminal comprising a screen having the means to display a
list of available applications that may be run on active real objects within a set
of active real objects located in a geographic zone determined from the
location of the communication terminal and the curve described by the
communication terminal, and a means of interaction to allow a user to select
an application from said list and to start the interface between the application
and the previously determined active real objects.
The invention and its benefits will become more clearly apparent in the
following description, with reference to the attached figures.
Figure 1 shows a general architecture within which the invention may be
incorporated.
Figure 2 diagrams one implementation for the invention in which the
User U is located outdoors.
Figures 3a and 3b diagram the description of a curve in space by user
U's mobile terminal.
Figure 4 diagrams an implementation in which user U uses a mapping
application to define his or her path.
The communication terminal T shown in figure 1 has a means of
processing TRT comprising several modules MZG, MIA, MOR, and MDA.
Traditionally, these means of processing may include electronic circuits and an
operating system designed to run software modules consisting of computer
code.
The communication terminal also has an interface INT for
communicating with a communication network N. The communication
terminal T is preferably a mobile terminal and the interface is then a radio
interface.
The communication network N is traditionally a composite network. It
may be a radio access network (using 2G/3G/4G, Wi-Fi or other
technologies), a private wired network, a public network, etc. The invention
may be applied independently of the architecture of the communication
network and the technologies used.
The real objects O l , 0 2 and 0 3 are connected to communication
network N and have appropriate interfaces Intl , ln†2, and ln†3 respectively to
allow them to communicate with network N. This communication may
potentially take place through an adapter that may be embedded in the real
objects themselves or may be a gateway between the real objects and the
communication network.
These real objects may be of various types. They may be real objects
available in a private home, in a work space (office, factory, workshop, etc.),
in a public space (street, restaurant dining room, bar, etc.), etc. They may be a
lamp, television screen, telephone, speaker, digital photo frame, media player
(DVD, etc.), radio, clock, etc.
Some are complex real objects offering many functionalities (or abilities);
for example, a television may offer a very large array of abilities on its
interface with communication network N allowing complete remote control
(channel changing, configuring brightness, contrast, adjusting volume,
selecting a video signal enhancement, etc.)
Others are much simpler and may only offer very simple functionalities:
for example, a lamp may offer an interface that only allows turning it on or
off.
Delimitation of the geographic zone
The inventive method comprises a first step consisting of delimiting a
geographic zone ZG. This step may be carried out by a software module MZG
embedded in communication terminal T itself.
Preferably, the position of the communication terminal is automatically
determined by a locating device. This locating device may be a GPS (Global
Positioning System) system embedded in the terminal itself, but other
techniques may also be foreseen. For example, a software module may also
be designed to calculate a location from the position of known base stations
and a triangulation algorithm.
The geographic zone may then be determined in relation to this position
and a curve described by the communication terminal.
According to one implementation, the user uses his or her mobile
terminal to describe a curve in space. The geographic zone is determined by
this curve. Such an implementation is depicted in Figure 3a. The user U
describes a curve C with his or her terminal T. The curve may be determined
by the software module MZG using various technologies. In particular, the
mobile terminal may have a motion sensor to determine a good estimate of
the curve actually described in space.
It is also possible to consider only the location of the terminal at the start
and end of the movement to determine an approximation C of the real curve.
This curve C may then be a straight line linking these two locations or
potentially a curve C determined from these two end points and a model of a
typical hand movement.
From this curve C, a geographic zone may be determined.
In this example, the geographic zone ZG is a volume found between two
cylindrical sections. The two cylinders CI and C2 are centred on the centre of
curve C. The diameter of the interior cylinder CI may be the distance
calculated or estimated between the centre of the curve C and the geographic
position of the mobile terminal T. The diameter of the exterior cylinder C2 may
be configurable.
Other geographic zone shapes are of course possible, and the shape
used may be potentially selected by the user.
For usage in a limited space, the geographic zone ZG may have no
other markers than the physical limit of the space. In the example in figure 3a,
the geographic zone ZG may not be restricted by the exterior cylinder C2 but
by the space in the room (which is to say by the furniture or the wall that one
can imagine behind the user).
In order to help the user to capture the geographic zone, the mobile
terminal, may display the geographic zone being determined on the screen in
real time. Thus, the user can accurately know which part of the space he or
she has captured with his or her motion. Figure 3b shows a communication
terminal T equipped with a screen E showing the same scene as the one in
Figure 3a. This scene was captured using a video camera integrated into
terminal T. By overlaying this scene, the geographic zone ZG is also shown,
for example, in the shape of a greyed-out area, thus allowing the user to see
whether or not the geographic zone ZG actually corresponds to his or her
intention. Thus he or she can correct his or her motion and know when to stop
it to capture the ideal zone.
The geographic zone ZG may be memorised in a memory linked to
communication terminal T or user U. In this way, it is possible to reuse the
same geographic zone ZG when the terminal and/or user encounters the
same situation, without having to carry out the steps described above again.
Determining a geographic zone may thus consist of searching the memory for
one that corresponds to the current situation.
For example, when the user sits at his or her desk, always in the same
place with more or less the same real objects present around him or her, the
geographic zone ZG may be directly recovered from the user profile.
Geographic zones ZG may be indexed by the position of the
communication terminal (thus, one knows that the user is at his or her desk),
or by other information in the user profile, and in particular by his or her
status as it may appear in the presence database on the communication
network.
As for the active real objects, they may be re-determined in the sense
that their status may have changed between two visits to the same geographic
area (objects turned off, battery spent, or possibly the installation of a new
object).
Determining a set of active real objects
The inventive method then comprises a second step consisting of
determining a set of active real objects located in the previously determined
geographic zone. These active real objects have a communication interface
with a communication network.
This second step may be carried out by software module MOR
embedded in terminal T itself.
Here an "active" object is a real object actually able to communicate with
the communication network. To do this, some objects may need to be
connected to a power supply, turned on, or appropriately configured.
The steps of determining active real objects and delimiting the
geographic zone may be carried out progressively. As the geographic zone is
delimited, the corresponding active objects are determined and may thus be
presented to the user so that he or she can dynamically act upon the
determination of the geographic zone. If there are a sufficient number of real
objects "captured", he or she may terminate delimitation of the geographic
zone, or if there are not a sufficient number, he or she may continue
delimitation until achieving a satisfactory result. This implementation can thus
help save time in a situation in which the user does not necessarily wish to
cover all of his or her environment, but rather to rapidly obtain a sufficient
environment to run certain applications.
In the example in Figure 3a, he or she may describe curve C until a
sufficient number of active real objects are detected.
These real objects may be presented to the user in various ways.
For example, they may be presented in the form of a list showing an
identifier (which could be a description configured by the user), a type,
potentially a graphic icon, etc. This list may be updated dynamically as mobile
terminal T travels along curve C.
It may also display at least part of the geographic zone ZG on the screen
E of terminal T, indicating the active objects in this part using an augmented
reality mechanism. Concretely, in the example in Figure 3b, the screen E
shows the environment as captured by a video camera (or potentially a digital
camera) integrated into terminal T. As an overlay on this realistic view, the
man-machine interface may show the geographic zone ZG (or a part of this
geographic zone) and indicator signs SI showing the locations of the active
real objects detected.
These indicator signs SI may simply show that an active real object is
present, or provide additional semantic information, for example by indicating
the type of object. These indicator signs SI may be icons or other graphic
symbols.
In order to determine the active real objects present, several
implementations are possible.
A database DB1 may be provided to reference the available real objects.
This database may link the real objects with their geographic position. It may
for example contain records linking real object identifiers with characteristics
(types, abilities offered, etc.), states (active / inactive), and geographic
positions.
The geographic position may consist of a longitude / latitude pair. It may
also comprise an altitude.
One can then determine the set of active real objects by sending a query
to database DB1 containing the characteristics of geographic zone ZG. A
search engine may for example compare the geographic positions of the real
objects present in database DB1 and geographic zone ZG.
If geographic zone ZG is determined by a curve C traced by terminal T,
such a query may be sent after the end of the movement is detected and
geographic zone ZG is completely determined. Alternatively, a query may be
sent at each time quantum in order to show the detected real objects to the
user in real time.
For the sake of optimisation, it may also be designed to carry out a first
query of a general database DB1 at the start of the process based on a global
geographic zone forming a superset of possible geographic zones. In the case
of the example in Figure 3a, this global zone may be the entire room, as
regardless of curve C, the geographic zone ZG may never exceed the limits of
the room. Then, the search engine can carry out its queries only on the results
of this first query. This implementation is especially interesting when the active
real objects detected are shown in real time on the screen E.
Identifying an application
The inventive method comprises a third step consisting of identifying an
application in relation to the active real objects that were determined. This step
may be carried out by a third module MIA embedded in communication
terminal T.
This application identification module MIA may query a second database
DB2. This database can link the available applications with required
conditions.
Database DB2 may cover only the software applications already installed
on communication terminal T. In this configuration, database DB2 may be
embedded in terminal T itself. This implementation allows the user to
implement his or her own well identified applications depending up on the
context in which he or she finds herself at a given time.
Alternatively, database DB2 may be separate from communication
terminal T and may contain a larger set of available software applications.
This is the implementation depicted in Figure . The objective is thus to offer
the user software applications that he or she does not necessarily use
habitually (and of which he or she may not be aware) but which can be run
with the detected real objects.
More concretely, database DB2 may contain descriptions of software
applications and in particular the conditions that they require.
In one embodiment, these required conditions may be of two types:
interface input points that require an incoming information flow, and interface
output points that require the ability to transmit an outgoing information flow.
Alternatively, the required conditions may be types of objects that must
be present (or potentially as options) to run the application.
These descriptions may also comprise other information about the
applications. Some of this information may be designed to be shown to the
user U to help him or her identify the application or to have some information
to decide whether to install or select it.
The descriptions may for example be in XML language (Extensible
Markup Language) as in the simplified example below.

disable
no
Lift Application
appli_lift.png
The Lift Application allows you
to prompt on a screen the caller name and to flash a lamp when your phone
rings
Monique
4.5
$2
3