Abstract: In various embodiments, the size, shape, and arrangement of keys on a virtual keyboard may be determined based on touchscreen contacts made by the user. Further, the actual contact patch made by the user may be analyzed to interpret which point of contact was intended, and other factors such as spelling and context may also be considered. These factors may be determined based on a calibration session and/or on continuing inputs during operation of the keyboard, and applied to future operational interpretations of the touchscreen contacts FIG. 1
WO 2010/147611 PCT7US2009/069931
This Divisional Application is out of Indian National Phase Application is 8403/DELNP/2011
ADAPTIVE VIRTUAL KEYBOARD FOR HANDHELD DEVICE
BACKGROUND
5 As multi-purpose wireless devices become too small for standard
mechanical/electronic keyboards, virtual keyboards are increasing being used as a primary
input device by displaying an arrangement of keys on a touchscreen. The user enters a
keystroke by simply touching the screen with a fingertip at the location where the desired
key is displayed. Because of the small size and hand-held nature of these devices, many
10 users typically use only their thumbs to enter the data. However, most of these virtual
keyboards arrange the keys in either a rectangular matrix of keys, or in the standard
QWERTY format. The linear nature of the rows in such arrangements makes them ill
suited for used by the thumbs. Specifically, to move his thumb along the length of a row
of keys or along the height of a column of keys, a user must articulate the several joints of
15 his thumb in a relatively unnatural manner. Indeed, becoming accustomed to such
arrangements can require extensive proprioceptive development from the user. While the
designs of some physical keyboards on multi-purpose wireless devices do provide
improved ergonomics (compared to a rectangular matrix of keys), the degree of to which
the ergonomics can be tailored to a particular individual remains limited. Further, such
20 designs do not account for the fact different users have different sizes of hands, fingers
and thumbs, so a keyboard that is properly sized for one user may be more difficult for
another user.
BRIEF DESCRIPTION OF THE DRAWINGS
25
Some embodiments of the invention may be understood by referring to the
following description and accompanying drawings that are used to illustrate embodiments
of the invention. In the drawings:
Fig. 1 shows a multi-function handheld user device, according to an embodiment
30 of the invention.
Fig. 2 shows a virtual keyboard configured for two-handed operation, according to
an embodiment of the invention.
1
WO 2010/147611 PCT7US2009/069931
Fig. 3 shows a flow diagram of a method of calibrating the size of the keyboard to
the individual user, according to an embodiment of the invention.
Fig. 4 shows a flow diagram of a method for an initial contact patch calibration
sequence, according to an embodiment of the invention.
5 Fig. 5 shows a flow diagram of a method of adaptively interpreting keystrokes,
according to an embodiment of the invention.
DETAILED DESCRIPTION
10 In the following description, numerous specific details are set forth. However, it is
understood that embodiments of the invention may be practiced without these specific
details. In other instances, well-known circuits, structures and techniques have not been
shown in detail in order not to obscure an understanding of this description.
References to "one embodiment", "an embodiment", "example embodiment",
15 "various embodiments", etc., indicate that the embodiment(s) of the invention so described
may include particular features, structures, or characteristics, but not every embodiment
necessarily includes the particular features, structures, or characteristics. Further, some
embodiments may have some, all, or none of the features described for other
embodiments.
20 In the following description and claims, the terms "coupled" and "connected,"
along with their derivatives, may be used. It should be understood that these terms are not
intended as synonyms for each other. Rather, in particular embodiments, "connected" is
used to indicate that two or more elements are in direct physical or electrical contact with
each other. "Coupled" is used to indicate that two or more elements co-operate or interact
25 with each other, but they may or may not be in direct physical or electrical contact.
As used in the claims, unless otherwise specified the use of the ordinal adjectives
"first", "second", "third", etc., to describe a common element, merely indicate that
different instances of like elements are being referred to, and are not intended to imply that
the elements so described must be in a given sequence, either temporally, spatially, in
30 ranking, or in any other manner.
Various embodiments of the invention may be implemented in one or any
combination of hardware, firmware, and software. The invention may also be
implemented as instructions contained in or on a computer-readable medium, which may
2
WO 2010/147611 PCT7US2009/069931
be read and executed by one or more processors to enable performance of the operations
described herein. A computer-readable medium may include any mechanism for storing
information in a form readable by one or more computers. For example, a computerreadable
medium may include a tangible storage medium, such as but not limited to read
5 only memory (ROM); random access memory (RAM); magnetic disk storage media;
optical storage media; a flash memory device, etc.
Various embodiments of the invention relate to a configuration of virtual keys on
the touch-screen of a virtual keyboard. Rather than being arranged in straight horizontal
rows, the keys may be arranged in arcs that are conveniently reached by the user's
10 thumb(s) when the device is held in the user's hand(s). In some embodiments, the
placement of the keys may be customized to fit the individual user's thumb and/or
personal preferences. In some embodiments, adaptive sensing may be used to compensate
for the contact surface of the user's thumb being off-center from a key and/or being larger
than the key.
15 Fig. 1 shows a multi-function handheld user device, according to an embodiment
of the invention. The illustrated device 110 is shown with a touchscreen 120 for
displaying information to the user and receiving tactile inputs from the user when the user
touches the screen at one or more particular locations. Three hard buttons are also shown
above the display. Other physical buttons, sensors, features, etc. may also be included but
20 are not shown to avoid excessive clutter in the drawing. Within the context of this
document, 'hard' buttons are so called because they are physical buttons, permanently
located in specific areas. But the device may also contain 'soft' buttons, each consisting
of an image on the touch-sensitive display screen, denoted herein as a touchscreen. When
the user touches a soft button, the device may sense that touch and perform whatever
25 function is associated with that soft button. The term 'key' is used in this document to
denote a soft button that represents an individual key on a virtual keyboard shown on the
touchscreen.
Although the illustrated device 110 is depicted as having a particular shape,
proportion, and appearance, with buttons located in particular locations, this is for example
30 only and the embodiments of the invention may not be limited to this particular physical
configuration. For example, in some embodiments various features may be located
elsewhere on the same side or on different sides of the device. In some embodiments the
overall shape of the device 110 may be different than shown.
3
WO 2010/147611 PCT7US2009/069931
Device 110 may also include functionality for wireless communication, for various
visual, audio, and physical inputs, and for various visual, audio, and physical outputs that
are not specifically described herein. In some embodiments, the device may use this
functionality in different ways depending on which mode it is in.
5
Virtual Keyboard with Tiered Arcs
Fig. 1 also shows a virtual keyboard on the touchscreen display. In a virtual
keyboard, each key on the keyboard is implemented as a soft button on the touchscreen.
When the user touches a particular key with his/her thumb (or finger, or stylus, or other
10 object), the device 110 senses that touch, determines where on the screen the touch
occurred, determines which key is associated with that location, and interprets this touch
as a keystroke of the selected key. In some embodiments, a hysteresis effect may be used
in which the user must remove his finger from the key for a minimum amount of time and
touch the key again before a second touch of that key will be registered.
15 In this example, the keys on the keyboard are arranged in three rows that each
follow an arc shape. These rows are positioned for ease of reach by the user's thumb.
Because of the configuration of the human thumb, the arcs may not be perfectly circular,
but rather each arc might have a variable rate of curvature. For this reason, the terms
'circular' and 'concentric' are not used to describe them here, although in some instances
20 the arcs might be circular and/or concentric. These arcs are described herein as 'tiered'
arcs because each arc has a pivot point (the pivot point of the user's thumb) that is in
approximately the same place, and each arc has a similar shape, with each arc being
approximately the same distance from the next adjacent arc throughout the length of those
arcs, when measured radially from the pivot point. Ease of reach by the user's thumb,
25 rather than a rigid geographical shape, may be the guiding principle when determining the
curvature and location of each arc.
The example of Fig. 1 shows three rows of keys, but other embodiments may have
one, two, four, or more rows. The keys are shown the same size in all rows, but in some
embodiments, some keys may be larger or smaller than others. For example, the inner row
30 may have smaller keys than the outer row. Not only does this allow more keys to be
placed on the inner row, which has less room for keys, but it also recognizes that the user
is likely to touch keys on the inner row with the end of his thumb, which presents a
smaller touch area than is felt by keys on the outer row, which are touched with the thumb
4
WO 2010/147611 PCT7US2009/069931
in an extended position. The illustrated example also shows that the three rows are spaced
the same distance from each other, but other embodiments may differ. Again, the
mechanics and flexibility of the human thumb may determine this spacing.
Each key is shown with a somewhat rectangular shape, but the soft keys may be
5 displayed with any convenient shape. In some embodiments, different keys may have
different shapes to provide addition information to the user (for example, a square shape
for upper case, and a round shape for lower case). Different colors may also be used to
denote additional information about that key. Each key is shown labeled with the
character it represents. These labels are all shown oriented with respect to the bottom of
10 the device (for easy reading by the user) but other embodiments may orient the label with
respect to the radial center of the arcs, or some other definable reference point. In some
embodiments, the displayed character will be shown as upper- or lower-case to indicate
whether the upper- or lower-case version of the character is represented by that key. .
In some embodiments, whenever a key touch is registered by the device, the
15 symbol represented by the key will be shown in an enlarged version to provide positive
feedback to the user, and the position of this enlarged key may be shifted so that it won't
be obscured by the user's thumb. In the example of Fig. 1, the user touches the 'M' key
(which is hidden from view by the user's thumb), and an enlarged version of the 'M' key
is shown just beyond the user's thumb, temporarily overlaying whatever other keys are
20 located there. Different color, style, shape, etc. may also be used to distinguish this touch
indicator from the remainder of the keys.
The example of Fig. 1 shows particular characters assigned to particular key
positions, but this is for example only. Other embodiments may assign characters to key
positions in any desirable arrangement, such as QWERTY, Dvorak, etc. In some
25 embodiments, the key assignments may be programmable.
Because the configuration shown in Fig. 1 is designed for one-handed operation,
and the keyboard is therefore limited to the space reachable by a single thumb, there may
not be enough space to simultaneously represent all the characters that he user wants to
type. To compensate for this, all or a portion of the key positions may be reassigned to
30 represent other characters, and new labels representing the new characters may be
produced on the touchscreen for those keys. This change may be initiated in any
convenient manner, such as but not limited to: 1) touching one of the keyboard keys
5
WO 2010/147611 PCT7US2009/069931
assigned to this function, 2) touching a particular soft key outside the keyboard area, 3)
pressing a hard button, 4) dragging the thumb along a substantial part of an arc, 5) etc.
In some embodiments, the full available keyboard may be thought of as an
approximate wheel with four quadrants, with each quadrant having a separate set of
5 characters, and with only one quadrant visible on the touch screen at a time (or two
quadrants visible for the two-handed operation described later). The user may then call up
whichever quadrant he wants displayed. For example, one quadrant might contain keys
with letters, another quadrant contain keys with numbers and punctuation marks, another
quadrant contain keys representing pictures, icons, letterheads, etc., that the user likes to
10 insert into documents, and the fourth quadrant contain commonly used phrases, sentences,
paragraphs, etc. Of course, more or fewer than four quadrants may also be used, since this
is a virtual concept and is not constrained to an actual physical circle.
The keyboard configuration shown in Fig. 1 is designed for right-handed operation
by making the rows of keys concentric about the lower right-hand corner. By making the
15 rows of keys concentric about the lower left hand corner, the device may be suitable for
left-handed operation. The illustrated keyboard is also shown configured for vertical
operation, i.e., the long side of the display is vertical. Some embodiments may operate
with a horizontal operation, i.e., the long side of the display is horizontal. In some
embodiments, the right/left handed configuration and/or the vertical/horizontal operation
20 is selectable by the user. In some embodiments, these configurations may be
automatically selected by the device (e.g., sensing gravity to select the vertical/horizontal
operation, and sensing which part of the display is touched by the user's thumb to select
right- or left-handed operation).
Fig. 2 shows a virtual keyboard configured for two-handed operation, according to
25 an embodiment of the invention. The primary difference between this two-handed
operation and the one-handed operation of Fig. 1 is that there are two portions to the
virtual keyboard, one configured for operation with the right thumb and the other
configured for operation with the left thumb. More keys, and therefore more characters,
can be simultaneously displayed with this configuration, as compared with the one-handed
30 configuration of Fig. 1. The two keyboard portions may have the same or a different
number of rows, the same or a different number of keys in each row, different spacing, etc.
In some embodiments, the assignment of characters to the individual keys may be
switched between the left and right areas for the convenience of users that have a left- or
6
WO 2010/147611 PCT7US2009/069931
right-handed preference. All the same features, techniques, choices, etc. that are available
for one-handed operation may also be applied to this two handed operation, and in some
embodiments may be applied separately for each portion of the keyboard. In some
embodiments the user may manually select either one- or two-handed operation. In some
5 embodiments, the device may automatically select one- or two-handed operation, based on
some automatically sensed criteria, such as device orientation or sensing touches on both
sides of the touchscreen.
Fig. 3 shows a flow diagram of a method of calibrating the size of the keyboard to
the individual user, according to an embodiment of the invention. In flow diagram 300, at
10 310 the calibration process may be initiated by the user, or started automatically based on
some pre-determined criteria, such as but not limited to: a) powerup of the device, b)
creation of a new user account, c) changing the font size of the characters on the virtual
keyboard, d) etc. The user may then be prompted at 320 to draw an arc with his thumb
on the touchscreen surface, with his hand in its normal data-entry position. This arc may
15 be referred to as a 'calibration arc' because its purpose is to calibrate the location of a
keyboard row so the keys on that row will be at a convenient position for the user's thumb.
In some embodiments, this arc will be visible on the touchscreen after being drawn by the
user, but other embodiments may not display the calibration arc.
In either case, the location of this arc on the display screen may be recorded at 330.
20 This location may be used to determine where the corresponding row of keys will be
placed on the screen. In some embodiments, the user may be prompted to enter more than
one arc. For example, to calibrate the device for two-handed keyboard operation, the user
may be prompted to draw separate arcs, one with each thumb. If more than one row is to
be calibrated, the user may be prompted to draw a middle arc, an outer arc, and/or an inner
25 arc. In some embodiments, the user may be prompted to retrace the same arc more than
once, so the device can determine an average position for that arc. Any or all of these
options may be accommodated with the operations at 320-330-340.
In some embodiments, the arc for only one row of keys is input by the user's
thumb, and the other arcs for the other rows for that thumb are automatically located with
30 respect to that arc. In such a case, the locations of the other arcs are determined at 350. In
one embodiment, the user may have been prompted to draw an arc with the thumb in a
mid-position, neither fully extended nor fully retracted, and the other rows placed in larger
and smaller arcs with respect to that one. In another embodiment the user may draw an
7
WO 2010/147611 PCT7US2009/069931
arc with the thumb fully extended to mark the location of the outer row of keys, with the
other arcs created inside that one. Conversely, the user may draw an arc with the thumb
fully retracted to mark the location of the inner row of keys, with the other arcs created
outside that one.
5 Once the arc locations have been determined, at 360 the device may assign the
position of each key along each arc, with each arc representing a row of keys. In some
embodiments, a predetermined spacing between adjacent keys on the same row may be
assumed. In other embodiments, the spacing may vary. For example, if the calibration
process was able to determine the width of the user's thumbprint in various locations, this
10 information may be used to decide the space between keys. A wide thumb print may lead
to a wider spacing between keys, to reduce the chance of touching multiple keys in a
manner that could lead to error.
At 370, the various characters (letters, numerals, punctuation marks, etc.) may be
assigned to the various key positions on the different rows. In some embodiments, this
15 arrangement may be predetermined. In other embodiments, this arrangement may be
customized, based on various criteria. For example, the most frequently used characters
may be placed on the middle row, to reduce thumb travel. The next most frequently used
characters may be placed on the outer row, since extending the thumb is generally
considered easier than retracting the thumb. Commonly occurring sequences of characters
20 (e.g., digraphs) can be ordered sequentially along an arc to facilitate a more natural
'inward sweep' motion of the thumb. For two-handed operation, commonly occurring
sequences can be alternated between the two thumbs to facilitate alternating thumb action.
Other considerations may also be used to assign characters to key positions. In some
embodiments, the user may be able to assign characters to specific key positions. At 380,
25 the device may generate the completed keyboard on the display based on these character
position assignments, and exit the calibration sequence at 390. By virtue of these and
other features, various embodiments of the invention may provide a virtual keyboard that
is well suited to the natural motion of the user's thumb(s) and is customized to
characteristics and preferences of the user. The invention may thereby improve the ease,
30 speed, and accuracy with which the user can enter text on the device.
Due to the close spacing of the keys, and the relatively large contact area of the
user's thumb, it is likely that the user will frequently contact more than one key
simultaneously. Various approaches may be taken to reduce the negative effects of this by
8
WO 2010/147611 PCT7US2009/069931
interpreting which key the user intended to touch. Some approaches involve interpreting
the area contacted by the user's thumb, while others are based on context and repetitive
errors. Both approaches are described below.
5 Contact Patch Adjustments
The 'contact patch' is the area in which the user contacts the touchscreen when
attempting to touch a key or other icon. If a stylus is used, the contact patch may be very
small and well-defined in shape. If the user's fingertip is the instrument of contact, the
contact patch may be somewhat larger, and the size may vary depending on the pressure
10 which the user applies with that finger. The size and shape of the contact patch may both
vary depending on the angle with which the finger contacts the touchscreen. If the thumb
is used as the instrument of contact, the same considerations apply, but the size is likely to
be even larger than with the fingertip, due to the thumb's generally greater cross section,
and the shape and size may vary considerably depending on the contact angle of the
15 thumb, which may generally be shallower than the contact angle of a finger.
Since the contact patch may be even larger than the key on the touchscreen that is
being touched, the sensing device may have to interpret the sensor information and
determine a point at which the touch was intended. Within this document, this point is
called the 'active point', which may or may not be the centroid (the geographical center)
20 of the contact patch. These calculations may be complicated by the fact that the size and
shape of the contact patch varies with: 1) which row the key is on (which affect the angle
of the thumb), and 2) whether the user is entering data with a one-handed or two-handed
keyboard operation (the position of the hand, and therefore the angle of the thumb, may
generally be different in a two-handed operation than it is in a one-handed operation).
25 Further, the actual contact patch and/or the actual centroid of that contact patch may be
different than the contact patch and centroid as perceived by the user.
In some embodiments the device may assume that a particular row of keys will
experience an elliptical contact patch with the ellipse oriented in a particular direction,
while a different row of keys may experience an elliptical contact patch with the ellipse
30 oriented in a different direction, or even a circular contact patch. These assumptions may
be taken into consideration when calculating the active point. Assumptions made about a
contact patch for a middle row may be interpolated between assumptions made about
contact patches for the inner and outer rows. These are only examples; the actual
9
WO 2010/147611 PCT7US2009/069931
assumptions may vary from this based on actual experience with the user and/or previous
studies based on the general population.
As an example of user perception, for some rows the user may perceive the active
point to be near the end of the thumb tip, and the device may move the active point away
5 from the centroid to accommodate this perception.
Since the angle of the thumb, and therefore the shape of the contact patch, is
generally different during two-handed operation than during one-handed operation, the
device may sense which mode of operation is being currently used, and adjust the active
point calculations accordingly. For example, the device may assume a two-handed
10 operation is being used if a two-handed keyboard is displayed on the touchscreen. Or the
device may infer the same by sensing contact in both parts of the screen. In another
embodiment, the device may assume one-handed operation if the device is being held in a
vertical orientation (the short dimension is horizontal), and two-handed operation if the
device is being held in a horizontal orientation (the long dimension is horizontal), based on
15 accelerometers or other sensors. However, the user may be able to manually override
these assumptions if desired.
In some embodiments, the type of instrument being used to contact the screen may
be inferred by the device based on the size and shape of the contact patch. If the contact
patch is relatively large, the device may assume the thumb is being used and adjust the
20 active point accordingly. If the contact patch is smaller, the device may assume that a
finger is being used, and adjust the active point based on that assumption. If the contact
patch is very small, the device may assume a stylus is being used, and not make any
adjustments.
Regardless of the type of adjustments being made, in some embodiments these
25 adjustments may be pre-defined, either based on a standard set of parameters, or based on
the results of one or more calibration sessions. In other embodiments, the adjustments
may be continually or frequently modified, based on the recent history of operation. In
some embodiments, both initial calibration and ongoing adjustments may be incorporated.
Fig. 4 shows a flow diagram of a method for a contact patch calibration sequence,
30 according to an embodiment of the invention. The method of flow diagram 400 may be
used to pre-calibrate the device to a particular user's characteristics. Operations 430 - 460
may also be used for ongoing adjustments during normal operation of the virtual
keyboard. After entering the contact patch calibration sequence at 410, at 420 the device
10
WO 2010/147611 PCT7US2009/069931
may prompt the user to press a selected key. In some embodiments, the keyboard (e.g.,
the keyboard generated by the keyboard calibration sequence of Fig. 3) may be fully
displayed, but in other embodiments the key may be shown in isolation in the same
location it will occupy in that keyboard.
5 When the user touches the selected key, the device determines relevant information
on the contact patch at 430. Based on readings from the individual contact sensors in the
touchscreen, the device may determine the size and shape of the contact patch, and in
some embodiments may record the contact readings for the different areas of the contact
patch at 440. At 450 the device may determine the centroid for the contact patch, i.e., the
10 geometric center of the contact patch. In some procedures, the centroid may be considered
the initial active point for the contact patch, but may then be relocated based on other
factors, such as those previously described. In some embodiments, the touchscreen may
provide the device with only a calculated centroid position for the contact patch, rather
than providing multiple contact sensor readings and having the device calculate the
15 centroid.
At 460, the device may then return to 420 to calibrate the contact patch for another
key on the keyboard. In some embodiments this process may be repeated for every key on
the keyboard. In other embodiments, only some keys or touchscreen locations may be
used in this calibration procedure, and calibration data for the rest may be interpolated
20 from the results of those keys. In some embodiments, the same key may be calibrated
more than once, to obtain an average of the readings for that key. After all the selected
keys or areas of the touchscreen have been calibrated, either directly or through
interpolation, the calibration sequence may be exited at 470.
25 Keyboard Adaptation
When more than one key is simultaneously touched by the user, the device may
use various techniques to determine which key the user intended to touch. Some
techniques are contextual in nature, and may not be performed immediately. For example,
the device may compare the multiple contacted characters to the rest of the word to
30 determine which character forms a real word, based on a spelling dictionary. If more than
one of the possible characters forms a real word, the context of the sentence may be
examined to select which word was most likely intended, and therefore which character
was intended.
11
WO 2010/147611 PCT7US2009/069931
In some cases, there may be a previous history of striking this same combination of
adjacent keys, when a particular one of those keys was usually intended, and this history
may be taken into account.
If the touchscreen provides local pressure measurements, the key receiving the
5 highest pressure from the thumb may be assumed to be the intended key. Alternately, the
different pressure measurements for the different contacted keys may be multiplied by a
probability factor, based on predictive text analysis, to determine which key was most
likely intended.
The 'hot spot', i.e., the spot that the device considers the center of a key, may be
10 shifted if the user consistently misses the hot spot in the same direction. For example, if
the user consistently types below the hot spot of the ' s ' key, the device may move the hot
spot of the ' s ' key downward. Determining that the user consistently misses in the same
direction may be determined in various ways. For example, localized pressure sensing at
various points in the contact patch may be used to determine that the highest pressure
15 point in the contact patch consistently misses the center of the key in the same direction.
In another example, it may be determined that the centroid or active point consistently
misses the key or its hotspot in the same direction.
In another example, words with that particular character may be frequently
misspelled by substituting the same adjacent character. This may be verified, for example,
20 when the user either manually corrects such misspellings, and/or when the user accepts
automatic spelling corrections. In some embodiments, the displayed position of the key
may remain unchanged, even though the hotspot for that key is relocated. In other
embodiments, the displayed position of the key may be moved so that the new hotspot is
centered within the displayed key. In some embodiments, the device may attempt to
25 differentiate between simply typing the wrong character (e.g., none of the contacted
characters spell a word, or maybe dyslexic letter inversion frequently occurs), and errors
due to missing the hotspot, and adjust the hotspot only in response to the latter.
Fig. 5 shows a flow diagram for a method of adaptively interpreting keystrokes,
according to an embodiment of the invention. In flow diagram 500, at 510 the device may
30 receive input from the touchscreen indicating a key has been touched, and determine the
centroid of the contact patch. In some embodiments this input may comprise readings
from multiple contact sensors that define the contact patch, which may then be converted
12
WO 2010/147611 PCT7US2009/069931
to the location of the centroid. In other embodiments, the input from the touchscreen may
simply represent the location of the centroid, as determined by the touchscreen logic.
In either case, the active point may be determined at 520 based on the location of
the centroid and on previously determined differences between the centroid and active
5 point for that part of the touchscreen. If the intended key is obvious based on the location
of the active point, at 535 the device may assume that key was intended. However, if the
active point is sufficiently close to multiple keys to cause uncertainty as to which key was
intended, as determined at 530, the device may examine the hotspots of those multiple
keys at 540. At 550, the device may determine the probability that each of these keys
10 represents the correct key, based on various criteria. A significant criteria may be the
previous history with this same grouping of keys.
Based on these and other considerations, at 560 the device may select which key
was most likely intended by the user, and enter that character into the typing sequence.
At this point it may not be feasible to look at spelling and context considerations for that
15 particular keystroke because the rest of the word or sentence has not been completed. The
device may then return to 530 to process more keystrokes. At each pass through 565, the
device may determine if it is finished with spelling/context considerations for that
particular keystroke. If it is, it may possibly change the previously chosen key at 570
based on those factors.
20 Once the chosen character and associated key have been finalized, the lessons
learned from this process may be incorporated for future processing. For example, at 580
the location of the hotspot for that key may be adjusted. This information may be
recorded for future use at 590.
The foregoing description is intended to be illustrative and not limiting. Variations
25 will occur to those of skill in the art. Those variations are intended to be included in the
various embodiments of the invention, which are limited only by the scope of the
following claims.
13
Claims:We claim:
1. A apparatus comprising:
a handheld electronic device having a touchscreen for displaying a virtual keyboard having three arc-shaped tiered rows of keys, including keys associated with respective characters, wherein the device is configured to:
enable a user to enter text content via the virtual keyboard with one hand while holding the handheld electronic device in the same hand; and
perform predictive text analysis to determine characters intended to be entered by the user as the user enters the text content via the virtual keyboard.
2. The apparatus of claim 1, wherein to perform predictive text analysis the device is configured to:
in response to a user selection of a key that is part of a word in a sentence being entered by the user via the virtual keyboard,
compare characters for each of one or more adjacent keys proximate to a sensed input coordinate to a remaining portion of the word to determine whether using the character forms a real word; and
if more than one of the characters that are compared forms a real word, examine a context of a sentence that would be formed using that word to select which word and character was most likely intended.
3. The apparatus of claim 2, wherein the apparatus is configured to determine a key intended to be selected by the user based, in part, on a history of previous determinations of intended key selections.
4. The apparatus of claim 1, wherein the apparatus is configured to:
in response to a user selection of a key that is part of a word in a sentence being entered by the user via the virtual keyboard,
sense an area of the touchscreen a user’s finger or thumb is in contact with corresponding to the key selection,
detect that the user intended to select one of multiple adjacent keys proximate to the area of the touchscreen that is sensed; and
determine a key intended to be selected by the user based on at least one of the area of the touchscreen that the user’s finger or thumb is sensed to be in contact with and an active point detected via the touchscreen or determined as a function of the area; and
predictive text analysis that considers at least one of a context of the word and a context of the sentence.
5. The apparatus of claim 1, wherein the apparatus is configured to adapt active points relative to areas of the touchscreen that the user’s finger or thumb are sensed to be in contact with based on observation of intended keys over time.
6. The apparatus of claim 1, wherein the three arc-shaped tiered rows of keys are arranged to be selectively activated via a user’s thumb, and wherein the apparatus is configured to determine what key is selected is based on detecting areas of the touchscreen the user’s thumb is sensed to be in contact with corresponding to the key selection.
7. The apparatus of claim 1, wherein the three arc-shaped tiered rows of keys are arranged to be selectively activated via a user’s thumb when the user is holding the apparatus using one hand including the user’s thumb.
8. The apparatus of claim 1, wherein the apparatus is configured to enable a user to adjust a layout of the keyboard.
9. The apparatus of claim 1, wherein the apparatus is configured to:
detect whether a user is selecting keys using a finger or using a thumb; and
adjust a location of an active point relative to a detected area of the touchscreen display that the user’s finger or thumb is sensed to be in contact with based on whether use of a finger or thumb for selecting keys is detected.
10. A method for implementing a virtual keyboard on a handheld electronic device having a touchscreen for displaying the virtual keyboard, the method comprising:
displaying three arc-shaped tiered rows of keys on the touchscreen display;
in response to a user selection of a key that is part of a word in a sentence being entered by the user via the virtual keyboard,
sensing an area of the touchscreen a user’s finger or thumb is in contact with corresponding to the key selection,
detecting that the user intended to select one of multiple adjacent keys proximate to the area of the touchscreen that is sensed; and
determining a key intended to be selected by the user based on at least one of the area of the touchscreen that the user’s finger or thumb is sensed to be in contact with and an active point detected via the touchscreen or determined as a function of the area;
performing predictive text analysis including,
comparing characters for each of the multiple adjacent keys to a remaining portion of the word to determine whether using the character in forms a real word; and
if more than one of the characters that are compared forms a real word, examining a context of a sentence that would be formed using that word to select which word and character was most likely intended.
11. The method of claim 10, wherein determining a key intended to be selected by the user further includes considering a history of previous determinations of intended key selections.
12. The method of claim 10, further comprising adapting active points relative to contact patches based on observation of intended keys over time.
13. The method of claim 10, wherein the three arc-shaped tiered rows of keys are arranged to be selectively activated via a user’s thumb, further comprising implementing logic to determine what key is selected by detecting areas of the touchscreen the user’s thumb is sensed to be in contact with corresponding to the key selection.
14. The method of claim 10, wherein the three arc-shaped tiered rows of keys are arranged to be selectively activated via a user’s thumb when the user is holding the apparatus using one hand including the user’s thumb.
15. The method of claim 10, further comprising enabling a user to adjust a layout of the keyboard.
16. The method of claim 10, further comprising:
detecting whether a user is selecting keys using a finger or using a thumb; and
adjusting a location of an active point relative to a detected area of the touchscreen display that the user’s finger or thumb is sensed to be in contact with based on whether use of a finger or thumb for selecting keys is detected.
| # | Name | Date |
|---|---|---|
| 1 | 201618042316-IntimationOfGrant26-03-2024.pdf | 2024-03-26 |
| 1 | Priority Document [12-12-2016(online)].pdf | 2016-12-12 |
| 2 | 201618042316-PatentCertificate26-03-2024.pdf | 2024-03-26 |
| 2 | Form 5 [12-12-2016(online)].pdf | 2016-12-12 |
| 3 | Form 3 [12-12-2016(online)].pdf | 2016-12-12 |
| 3 | 201618042316-FER.pdf | 2021-10-17 |
| 4 | Form 20 [12-12-2016(online)].pdf | 2016-12-12 |
| 4 | 201618042316-FORM 13 [28-09-2021(online)].pdf | 2021-09-28 |
| 5 | Form 18 [12-12-2016(online)].pdf_410.pdf | 2016-12-12 |
| 5 | 201618042316-PETITION UNDER RULE 137 [20-03-2021(online)].pdf | 2021-03-20 |
| 6 | Form 18 [12-12-2016(online)].pdf | 2016-12-12 |
| 6 | 201618042316-RELEVANT DOCUMENTS [20-03-2021(online)].pdf | 2021-03-20 |
| 7 | Drawing [12-12-2016(online)].pdf | 2016-12-12 |
| 7 | 201618042316-ABSTRACT [19-03-2021(online)].pdf | 2021-03-19 |
| 8 | Description(Complete) [12-12-2016(online)].pdf_409.pdf | 2016-12-12 |
| 8 | 201618042316-CLAIMS [19-03-2021(online)].pdf | 2021-03-19 |
| 9 | 201618042316-COMPLETE SPECIFICATION [19-03-2021(online)].pdf | 2021-03-19 |
| 9 | Description(Complete) [12-12-2016(online)].pdf | 2016-12-12 |
| 10 | 201618042316-DRAWING [19-03-2021(online)].pdf | 2021-03-19 |
| 10 | abstract.jpg | 2017-01-21 |
| 11 | 201618042316-FER_SER_REPLY [19-03-2021(online)].pdf | 2021-03-19 |
| 11 | Form 26 [11-03-2017(online)].pdf | 2017-03-11 |
| 12 | 201618042316-FORM 3 [19-03-2021(online)].pdf | 2021-03-19 |
| 12 | Other Patent Document [17-03-2017(online)].pdf | 2017-03-17 |
| 13 | 201618042316-OTHERS [19-03-2021(online)].pdf | 2021-03-19 |
| 13 | 201618042316-Power of Attorney-200317.pdf | 2017-03-22 |
| 14 | 201618042316-OTHERS-200317.pdf | 2017-03-22 |
| 14 | 201618042316-PETITION UNDER RULE 137 [19-03-2021(online)].pdf | 2021-03-19 |
| 15 | 201618042316-Correspondence-200317.pdf | 2017-03-22 |
| 15 | 201618042316-Proof of Right [19-03-2021(online)].pdf | 2021-03-19 |
| 16 | 201618042316-RELEVANT DOCUMENTS [19-03-2021(online)].pdf | 2021-03-19 |
| 16 | Other Patent Document [04-04-2017(online)].pdf | 2017-04-04 |
| 17 | 201618042316-OTHERS-170417.pdf | 2017-04-19 |
| 17 | 201618042316-FORM 3 [20-01-2021(online)].pdf | 2021-01-20 |
| 18 | 201618042316-Correspondence-170417.pdf | 2017-04-19 |
| 18 | 201618042316-Information under section 8(2) [20-01-2021(online)].pdf | 2021-01-20 |
| 19 | 201618042316-FORM 3 [13-11-2017(online)].pdf | 2017-11-13 |
| 19 | Form 3 [20-04-2017(online)].pdf | 2017-04-20 |
| 20 | 201618042316-FORM 3 [23-10-2017(online)].pdf | 2017-10-23 |
| 21 | 201618042316-FORM 3 [13-11-2017(online)].pdf | 2017-11-13 |
| 21 | Form 3 [20-04-2017(online)].pdf | 2017-04-20 |
| 22 | 201618042316-Correspondence-170417.pdf | 2017-04-19 |
| 22 | 201618042316-Information under section 8(2) [20-01-2021(online)].pdf | 2021-01-20 |
| 23 | 201618042316-FORM 3 [20-01-2021(online)].pdf | 2021-01-20 |
| 23 | 201618042316-OTHERS-170417.pdf | 2017-04-19 |
| 24 | Other Patent Document [04-04-2017(online)].pdf | 2017-04-04 |
| 24 | 201618042316-RELEVANT DOCUMENTS [19-03-2021(online)].pdf | 2021-03-19 |
| 25 | 201618042316-Proof of Right [19-03-2021(online)].pdf | 2021-03-19 |
| 25 | 201618042316-Correspondence-200317.pdf | 2017-03-22 |
| 26 | 201618042316-OTHERS-200317.pdf | 2017-03-22 |
| 26 | 201618042316-PETITION UNDER RULE 137 [19-03-2021(online)].pdf | 2021-03-19 |
| 27 | 201618042316-OTHERS [19-03-2021(online)].pdf | 2021-03-19 |
| 27 | 201618042316-Power of Attorney-200317.pdf | 2017-03-22 |
| 28 | 201618042316-FORM 3 [19-03-2021(online)].pdf | 2021-03-19 |
| 28 | Other Patent Document [17-03-2017(online)].pdf | 2017-03-17 |
| 29 | 201618042316-FER_SER_REPLY [19-03-2021(online)].pdf | 2021-03-19 |
| 29 | Form 26 [11-03-2017(online)].pdf | 2017-03-11 |
| 30 | 201618042316-DRAWING [19-03-2021(online)].pdf | 2021-03-19 |
| 30 | abstract.jpg | 2017-01-21 |
| 31 | 201618042316-COMPLETE SPECIFICATION [19-03-2021(online)].pdf | 2021-03-19 |
| 31 | Description(Complete) [12-12-2016(online)].pdf | 2016-12-12 |
| 32 | 201618042316-CLAIMS [19-03-2021(online)].pdf | 2021-03-19 |
| 32 | Description(Complete) [12-12-2016(online)].pdf_409.pdf | 2016-12-12 |
| 33 | 201618042316-ABSTRACT [19-03-2021(online)].pdf | 2021-03-19 |
| 33 | Drawing [12-12-2016(online)].pdf | 2016-12-12 |
| 34 | 201618042316-RELEVANT DOCUMENTS [20-03-2021(online)].pdf | 2021-03-20 |
| 34 | Form 18 [12-12-2016(online)].pdf | 2016-12-12 |
| 35 | 201618042316-PETITION UNDER RULE 137 [20-03-2021(online)].pdf | 2021-03-20 |
| 35 | Form 18 [12-12-2016(online)].pdf_410.pdf | 2016-12-12 |
| 36 | 201618042316-FORM 13 [28-09-2021(online)].pdf | 2021-09-28 |
| 36 | Form 20 [12-12-2016(online)].pdf | 2016-12-12 |
| 37 | Form 3 [12-12-2016(online)].pdf | 2016-12-12 |
| 37 | 201618042316-FER.pdf | 2021-10-17 |
| 38 | Form 5 [12-12-2016(online)].pdf | 2016-12-12 |
| 38 | 201618042316-PatentCertificate26-03-2024.pdf | 2024-03-26 |
| 39 | Priority Document [12-12-2016(online)].pdf | 2016-12-12 |
| 39 | 201618042316-IntimationOfGrant26-03-2024.pdf | 2024-03-26 |
| 1 | searchE_21-09-2020.pdf |