WORD OR CHARACTER BOUNDARY-BASED SCRATCH-OUT GESTURE
RECOGNITION
FIELD OF THE INVENTION
The pieseni: invention relates generally to computing devices, and more particularly to user interaction with c o m p u t. i n q c i e v . L c e s .
BACKGROUND OF THE INVENTION
Contemporary computing devices allow users to input information .in a number of ways, including via a keyboard, by or more types of pointing devices, and dedicated hardware button:; (typically on portable devices). With respect to pointing devices, a computer system may receive user input from a mouse and/or a touchpad, and if configured with a digitizer, by sensing pen and touch data, e.g., entered via a st y1us ( p en) or a finger.
s'oine pei.sonal computers, such as tablet-based personal computers, have the digitizer built into the display screen. This is hiqhly beneficial for many users, because such a computing device can be operated with or without a keyboard. Tablet. -basi--d personal computers and similar handwriting-receptive computing devices allow users to enter handwritten data such ,-is printed characters and cursive words. One valuable aspect of being able to enter handwriting into a
oomput in-r': may attempt some form of scratch-out gesture,
gesture is neither intuitive nor natural
Wh.-jt J. .-•; needed is a more natural scratch-out gesture (or .set. of gestures) that, works with most users' personal styles o[ crose.inq oot text... Any such scratch-out gesture should be i.n t.ui t. L vi- tor users as well as inherently discoverable, and gene ra 1 1 y should increase user satisfaction with hand-entered deletion op^r -tions . At the same time, the rate of correctly different ir.it i.i,g scrat;ch-out gestures from other ink input-should In- impi oved .
SUMMARY OF THE INVENTION
Briefly, the present invention provides a system and method i. ha I. differentiates certain electronic ink in the form of .'i seo..tt' h-out. gest.ure from other ink, by evaluating the posit.. Jon and horizontal width of a potential scratch-out gesture ag,.i inet the boundaries of words or characters that may be scratched out . Threshold amounts and ratios are used to decide whet he:1 a gesture is a scratch out or further ink.
A:.; j use: enters electronic ink data corresponding to stroke.-'-., a sc; at.oh-out gesture detector applies various t hresho 1 d-ha.sed rules to differentiate between scratch-out gesture.-: and >ther ink input. Scratch outs are used to erase existing ink, while non-scratch outs add ink strokes to other
strokes representing characters or words. Because the scratch-on*, qesture detector uses word and character boundaries i:i.. r its decision making, scratch-out is not limited to any predefined scratch-out gesture. Thus, any of several 3ci at .c'n - on:. styles may be used, including a strikethrough gesture, .1 Z-shaped gesture, a vertical scratch-out gesture, a c i r euiai sor.-i tch-out. qesture, an angled scratch-out gesture, and crossing out, including an X-shaped gesture or double s tr i ketn rouq:.- gesture .
In general, any tingle stroke may be recognized as a scratch-out gesture. Example criteria for differentiating a .sci a t ch-ou t .. troke fr\>m an inking stroke may include that a scratch-out seeds t:o tie on top of a previously recognized word or character (intersect its bounding box), and/or that a scratch-out stroke needs to have a certain ratio of its width (h.ori/,onl ,1 i 1 y) to the width of the previously recognized word or character. To allow for scratch-out styles such as crossing out. a word (an X-out or a double-strikethrough approach), a stroke meeting timing and/or proximity criteria relat i v.:- i i ,! previous scratch-out gesture is considered part ot the same cratch-out gesture.
i'n on- implementation, in a free-formed lined writing area, ,-i si roke qualifies as a scratch-out gesture relative to a word or wotds when the scratch-out gesture stroke intersects
one or mox e f: rev iously recognized words, and has a bounding box width i hat; achieves at Least a threshold percentage of the width of i he bounding box of the previously recognized word or word tse.it i h*~ stroke Intersects. The stroke also needs to intersect, w.i t n i tie word's bounding box by more than some amount, (e.g., one-half.) of the width of the word's bounding box, and . i pert ion of the scratch-out stroke needs to be above the baseline for: the line on which the word or words sit. Different thresholds may be used based on the actual measured length of i in- word, in distance (which may include relative distances -;ueh as pixels) .
Another type of writing entry surface comprises a boxed writing a:ea, where a user enters a character per box. In one iffiplemetit-.it i, n, criteria for a scratch-out gesture includes that i he .;t n ke needs to intersect one or more previously recognised o:atncters, and nas a bounding box width that is at least some- ttreshold percentage of the width of the box of the pievioiudy recognized character or characters that the stroke Intersects. Further, the stroke needs to cross a point such •is i he center vertical axis (half the width) of the box or boxes ; e.r i he character or characters it intersects. Also, some pei I i. on of the scratch-out stroke needs to be above the baseline for the 1Lne on which the character or characters
Lii fh-eeiorrii lined areas and a boxed writing areas, a ::, i!;q l.e ;;L: ")k« may .scratch-cut multiple boxes, including boxes that .'ire uor.'SS multiple lines. When a single stroke is used to scratch 01,t multiple words or characters on a line, the intervening words or characters between the first and last woid '••>, ch,i;:. ict.er scratched out are deleted, even if the suoki-- does not inner sect one or more of their bounding boxes, or dot--:, rv •; uua.'l Li:y as an scratch-out with respect to one or more 01 them. Similarly, when a single stroke is used to .scratch out v-'ords or characters on multiple lines, the words 01 oharacier between the first and last word or character scratched out are deleted, even if the stroke does not intersect on--' or more of them or, if it does intersect, does not oilier.wi.3f-1 qualify as an scratch-out with respect to one or
When i v.'ord is scratched out and it is not the last word, t.he nexi word is also scratched out if there is no space between i i. and the previous word, and the word is one ohctraoier Long. Thus, punctuation such as a period is serai ohed ,ariable media. Computer-readable media can be any available meoia that can be accessed by the computer 110 and includes bctc volatile and nonvolatile media, and removable and rion-rernov ab] e media. By way of example, and not limitnt ion, • omputer-readable media may comprise computer ,-.;t c-racjc medi,-.. and communication media. Computer storage media include.-^ volatile and nonvolatile, removable and non-removable medi.a implemented in any method or technology for storage of inioovit ion : uch as computer-readable instructions, data ct rue! ice:;, proqram modules or other data. Computer storage media includes, but ic not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (i-'VP) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage device;;, or . «ny otner medium which can be used to store the desired in.'.otmat ion arid which can accessed by the computer M C. (v.mmun.; cat ion media typically embodies computer-readable instruo.i rens, ddta structures, program modules or other data
in a nuxiu I.ated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The term "modulated data signal" means a signal that has one or mure oE its characteristics set or changed in such a manner as lo encode information in the signal. By way of example, and not limitation, communication media includes wired media : uch as a wired network or direct-wired conned ion, and wireless media such as acoustic, RF, infrared and othor wireless media. Combinations of the any of the above should also be included within the scope of computer-readable rn> -d.i a .
The system memory 130 includes computer storage media in the form <.>!; 'volatile and/or nonvolatile memory such as read only memo:y (ROM) 131 and random access memory (RAM) 132. A basic j nput./output system 133 (BIOS), containing the basic routines ihat help to transfer information between elements witiiir: computer. 110, such as during start-up, is typically stored in ROM 131. RAM 132 typically contains data and/or program modules I.hat are immediately accessible to and/or present ly being operated on by processing unit 120. By way of example, ot,d .not limitation, FIG. 1 illustrates operating system L34, application programs 135, other program modules 136 and p:oq::arn data 137.
Tin. .•onifuter L10 may also include other removable/nonremovable, volatile/nonvolatile computer storage media. By w.viy oL exampi.e only, FIG. 1 illustrates a hard disk drive 141 (.hat: reads f: om or writes to non-removable, nonvolatile magnetic media, a magnetic disk drive 151 that reads from or wr.ii.es ID a removable, nonvolatile magnetic disk 152, and an opt ical disk drive 15b that reads from or writes to a removable, nonvolatile optical disk 156 such as a CD ROM or other optioaj media. Other removable/non-removable, volatile/nonvolatile? computer storage media that can be used in the exemplary operating environment include, but are not limited to, ;ragnet io tape cassettes, flash memory cards, digital versatile disks, digital video tape, solid state RAM, solid state F-.OM, and the like. The hard disk drive 141 is t. ypieaJiy connected to the system bus 121 through a non-removribJe nemory interface such as interface 140, and magnetic disk dr i vi-: I'M ana opt ical disk drive 155 are typically connected to the system bus 121 by a removable memory inter;nee, sech as interface 150.
The drives and their associated computer storage media, discussed ab>>ve and illustrated in FIG. 1, provide storage of compute..;.- roacsabl e instructions, data structures, program modules and -.ther data for the computer 110. In FIG. 1, for example, haru disk drive 141 is illustrated as storing
opera i. i ng sy... tern 144, application programs 145, other program modules 1 -1 n .;.nd proqram data 147. Note that these components can t-;-i. t iif-: tv the samo as or different from operating system 134, appli.eai ion programs 135, other program modules 136, and program da la 137. Operating system 144, application programs 14S, ot.hei progiam modules 146, and program data 147 are given di : :..e rent numbers nerein to illustrate that, at a minimum, they ate d.i. llerent copies. A user may enter commands and informal, ion inlo the computer 20 through input devices such as a tablet, (electronic: digitizer) 164, a microphone 163, a keyboard In2 and pointing device 161, commonly referred to as mouse, i ta.-kiiail or touch pad. Other input devices (not shown) may include a joystick, game pad, satellite dish, scanner, ;;; i he like. These and other input devices are often connected to I:ho processing unit 120 through a user input Inter I'aee I b: that, is coupled to the system bus, but may be connected :>y other .interface and bus structures, such as a parallel por1, game port or a universal serial bus (USB). A rnonito! 1 :U <.r other type of display device is also connected to the ;:y,;teir bus 121 via an interface, such as a video interface 19i'. The monitor 191 may also be integrated with a touch-seri-.''-:! panel 193 or the like that can input digitized input .M.icii a; handwriting into the computer system 110 via an interface, such as a touch-screen interface 192. Note that
the monitor and/or touch screen panel can be physically coupJen (.•, a housing in which the computing device 110 is inoorpi;:-ated, r:,uch as in a tablet-type personal computer, wherein I he touch screen panel 193 essentially serves as the LabLei !b4. In addition, computers such as the computing device IK. m.iy also include other peripheral output devices such as speaker:-; 195 and printer 196, which may be connected through an output peripheral interface 194 or the like.
Tnt-.- computer 110 may operate in a networked environment using jog Lea'I connections to one or more remote computers, such cis a 'remote computer 180. The remote computer 180 may be a personal computer, ..-i server, a router, a network PC, a peer device or other common network node, and typically includes many or. a i. i ,.:f the elements described above relative to the compute.) 1.1.0, although only a memory storage device 181 has been .;.liust r-.ted in FIG. 1. The logical connections depicted i.n Fit;, l include a local area network (LAN) 171 and a wide area network (WAN) 17.':;, but may also include other networks. Such networking environments are commonplace in offices, enterprise-wide computer networks, intranets and the Internet.
When usf-d .in a LAN networking environment, the computer 11C is oonne< ted to the LAN 171 through a network interface or adapter I''.). When used in a WAN networking environment, the computer ) LO typically includes a modem 172 or other means for
establishing comrnuniCr.it tons over the WAN 173, such as the Internal.. The modem .172, which may be internal or external, may be conne ted to (.be system bus 121 via the user input in t er Lrn 'e LD:. or ot.hei appropriate mechanism. In a networked environment, program modules depicted relative to the computer 11C, (',' po:'L ions (.hereof, may be stored in the remote memory storage d.j-/l.-e. By way of example, and not limitation, FIG. 1 i 1 1 list; d t ,:••; remote application programs 185 as residing on memory -'lev i o<- 181.. It will be appreciated that the network connect Jon ; shown are exemplary and other means of establishing a communications link between the computers may be used.
T:>- present invention Is primarily directed towards user input '..inta entered via electronic inking, generally computer pen (stylus) input received as electronic ink at a pen i.i i g i t i ..>.- r . in general, the ink may be received at a freeform input area, wnere it is recognized into words, where as used herein, -a "wf-rd" may l>e any combination or permutation of one or mor<> elu racte rs, symbols and the like, even if not an actual wo.rd, that is logically separable in some way from other word-:. Ink may also be received at a boxed writing aiea, in wnicn (typically) one character is entered per box.
In addition 10 word arid character entry, the present invention del ert;: and ,: L1:1 'erenti ates certain electronic ink in the form oi a ,-:r.r a i;. ,'h •- out gesture, which the user enters when intending to erase previously entered words or characters, such as by diawing a lire through a set of one or more previously entered words o: characters, or through typed (or recognized) text that, is to b>- deleted. The differentiation evaluates the position and horizontal width of the gesture against the boundaries o! words or characters that may be scratched out, using thresh- Id amounts and ratios to decide whether a gesture is a scia!ch out , or further ink.
As will be understood, numerous ways to implement the present invention are feasible, and only some of the al t ei. n.-"i l iv<->s are described herein. For example, in one implementai i< n, word boundaries are determined by a nandwr.it ing-: o~t ext. recognizer. However, other ways to deiine-ite word boundaries may be employed, such as one that evalu.it.---s white space between written input, such as when a r. eoi >qii i ..-:e i is not available, or is not keeping up with the user's tat*-' ,-. f input. Indeed a recognizer is not even necessary to the present invention, e.g., a user can scratch out hcindwi M j ng input that is then erased, independent of whether the system even has a recognizer. Further, while F.nglish l.-mgu-age words are used in the example, the present
invent ion i r; 1 .anguage independent. As such, the present invent i...n i.:.; not Limited to any particular examples used herein, bu: , at. tier may be used various ways that provide benefit:- and advantages in computing in general.
Tinning to FIG. 2, there is shown an example architecture in which o pt n digitizer 202 and a touch digitizer 204 couples through a suitable driver 206 and 208 to an operating system Level oompom-n!., referred to as the tablet input panel 210. Note thai for purpose::: of example, each input device is shown a 5; having it:-, own driver, however one driver may handle the input oi no>r> than one- device.
AM represented in FIG. 2, the tablet input panel 210 may comprise an object or the like that includes a handwriting surface 21.2 on which the user enters electronic ink data, cor respond i.rv.i to strokes. As is known, such strokes may be sent; i. n a '.recognizer 216, which returns text based on those :"A i. oker;, ..:-. g . , one or more words. If more than one word is returned, trie words are typically associated with some probalro i io.y oaf a or ranking, with the most probable word automat i C.YJ i Ly chosen .for: the user by default. Because reoognit ion is not instantaneous, at any given time there may be recognized stroke data (text) 220 and unrecognized stroke
T'ne lab let Input panel 210 supports different types of handwrit .ma . u ri an.-;?- . Surfaces include a freeform writing area j/ij (!;'ic;. 3), which may be set (e.g., lined) as a free-form I ext : erogn i t. ion panel with editing capability for character input, a soit (e.g., QWERTY) keyboard, and a boxed input -ue,i 4-0 (FIG. 4), in which handwriting is entered as individual characters, such as for completing formatted fields. Mote that although the ink may be received in a freeforrn inking area, one implementation constrains the user to freetoirn writ ing in lined input areas in which the user enters ink one line at a time. This makes it easier to determine ihe boundaries between words, which is otherwise more di t f. i'M.i.l t when users can write in any direction and/or
or.iance wit h various aspects of the presenting invent i.•.,'[!, a sc catch-out; det.ector 224 applies various t.hreshoid-based rules to differentiate between scratch-out gestures and >ther ink input, and also applies threshold-based rules to determine to which words or boxes a scratch-out gesture is directed. To this end, the surface 212 provides trie un;vo(-gni '.ed stroke data to the scratch-out gesture detector lr. 4, which then compares the stroke data to the recogni.i'Ki st : oke dato (text word or words or boxed character or ehai a. :t e rs i 220 to determine if the stroke comprises a
scr atch - on!, gesture. The scratch-out gesture detector 224 returns results to the surface 212; for a scratch-out gesture, the results include t: se words and stroke or strokes to erase. For a non-'.-so r at ch-out gesture, the result is such that the stroke is provided as part of the ink data sent to the recognise.' 216. Note that this is only one example mechanism Lor processing strokes.
F<>r. >.-i.-idii'.pie, an alternative mechanism may handle situations in which the recognizer gets behind the user and processes strokes Cor recognition slower than the user can go back and scratch out those strokes. An optional (as indicated by the dashed box and lines in FIG. 2) boundary detector 226 l hat can r-s I. j mate word boundaries, e.g., based on spacing rat net. L.han recognition, may be used by the scratch-out gesture r>." :og ni t ion detector 224 for differentiating scratch-out f 1:0m -'I her gestures. Note that such a boundary detector 22h m.-jv hi- used at", other times, such as in conjunction with recognii ion results. In any event, it is straightforward to alternatively implement scratch-out detection on non-recogn .i rei i strokes, as long as there is some pre-recognition bounds r y ;'io Li ni t. Lori.
As described above, and in contrast to other (e.g., neutal network-based) systems, the scratch-out gesture detector. •'.! 4 ;f the present invention uses word and character
boundaries f< L i LS decision making, whereby scratch-out is not limited to any p redef i ned scratch-out gesture. For example, there .-u e :io constraints on direction of strokes, order of strokes, o i. cumber of zigzags. Instead, the scratch-out detector ,:.-.! 4 essentially assumes that when in a lined writing rirea j.-'i;, ' ho user is only writing text (in contrast to drawing pictures or: diagrams) in order to distinguish scratch-out gesvtnos from words or boxed letters. Thus, any of several sc>at ,;h-oui. styles may be used. Some styles that will be recognised as scratch-out gestures include a strikethrough gesture, .1 Z-shaped gesture, a vertical scratch-out gesture (e.g., in 'he general pattern of an M or W), a circular scratch-out gesture, an angled scratch-out gesture, and crossing out, including an X-shaped gesture or double st. r i ke i nrough gesture. A user's personal scratch-out style will a i. so wor k, ur be configurable to work, as a scratch-out gest ure-.
In g<-sner. il, any single stroke may be recognized as a soratoh-out gesture based on established criteria, such as predefined or based on a user training the system. In one imp lernen i t the second stroke relative to the scratch-out st..;oke, a.", d' scribed below. Thus, two (and possibly more) rapid -Hid eb se scratch-out strokes will be considered a single scrat: <. h-out, wrule one scratch-out stroke followed con.sid>.~.r at' I y later by another stroke, and/or far away from the scratch-oat .'-troke, will be considered new ink. In this way, a u:.:;et >viti enter ink somewhat shortly after scratching out othir-r ink, yi--t;: s Li 1.1 use multiple rapid scratch out strokes.
Exarnp i e criteria for differentiating a scratch-out stroke :. rorn an inking stroke may include that a scratch-out needs to be on top i,i ,-i previously recognized word or character, and/or ih.ii. ,-i so: ,-i i. rh-out stroke needs to have a certain ratio of its width (th.-it !•;, liOtizontal length, not thickness) to the width (ho i i ..'.on i .a i l-.Migth) 01 the previously recognized word or chd :. ciri..e i. . B.Hjndinq boxes may be computed for the comparison.
In :>iif implementation, in a free-formed lined writing aiea, .'inch as the area 320 represented in FIG. 3, a stroke quaiiL.it'.; .-,;; ( ,sr:ratch-out gesture relative to a word or words when .i. t meets a number of criteria. Criteria for free-formed word em : y ineludes t.hat a scratch-out gesture stroke Lni er str-'M :.: ont or mor'e previously recognized words (comprising at. least k.ne -'h,-:i i aeter i , and has a bounding box width that nt least a threshold percentage (e.g., eighty
percent ) >;f i he width of the bounding box of the previously reroqn j ,:ed w^rd or word that the stroke intersects. This is gener a 1 1 y represented in FIG. 5A, where the long stroke substant i a 1.1 y over the handwritten word "cream" is considered a scratch >ui gesture, compared to FIG. 5B, where the stroke is no i 1 ong .: nough .
Further, in this implementation, the stroke needs to intersect, with the word's bounding box by more than some amount (e.g., one-half) of the width of the word's bounding box, ctnri a portion of the scratch-out stroke needs to be above the baseline for the line on which the word or words sit, e.g., S'.« a,:> to not confuse a scratch-out gesture with an underline and/or a character's descender. Thus, the gestures in FI(-J:;. >>:\ and 7A are scratch-out gestures, while those in FIGS. t)£ ,md IE are not, wherein in FIGS. 6A and 6B the dashed lines indicate the centers width-wise, and in FIGS. 7A and 7B the dashed 1 i ue.s indicate the baselines.
For words that are short in width as measured against some t.h re; noi d distance unit; such as inches or pixels, e.g., a one or t' wo--eharacter word such as "it" or the like, a stroke is required to have a bounding box width of a larger threshold percentage than previously described, typically greater than one-hundreii percent 01 the word width. For example, an implementation may require that for any word that does not
ach i.evr s < '-;\\t^ i.n i es "10 J d length, such as a half-inch (on an appcopr iatcl>-si zed tablet), to qualify as a scratch-out ge;"-t..u/e, i.ne stroke width needs to be some other percentage value, <•-.']., at Least 120 percent (as opposed to the 80 percent v.i 1 u>- i. n the example above) of the width of the box of the previously recognized word or words the stroke intersects. Thus, in i !ie example of FIG. 8A, the stroke would meet this eiit.e: In, :vht- reas Ln FIG. 83, it would not. Note that in FIG. 8B, il appears Lo an extent that the short stroke may be an attempt i.; > convert the lowercase "1" character to a "T" or "t" character:, such as in a user "fix-up" operation following Incorrect recognition. As can be readily appreciated, other size orite;:i;i may have other percentage values. For example, in order i •. •< scratch out a word that is greater than one-hall inch in widt.'i and less than or equal to two-thirds of an inches in wrath, to qualify as a scratch-out gesture a stroke is required t. 3 nave a bounding box width that is one-hundred percent- o •• more of the width of the box of the previously r. eoi >gr. i zed word that the stroke intersects, as opposed to an BO per'.-t-ni threshold example of FIGS. 6A and 6B or the 120 percent (-h; esno Id example of FIGS. 8A and 8B) . Other criteria may be employed, such as for words less than a half-inch in wrdth trie :;ci i ten-out stroke may need to be generally centered (>n the word';- center vertical access.
A,-:-1 deoci. ibed above, another type of writing entry surface eompr J .-'.i ''S i I oxed writing area, such as represented in FIG. 4. In a boxed writ ing area, a stroke is defined as a scratch-out gesture re! at i.ve to a box or boxes if the stroke meets certain ot.her orri.iorii. In one implementation, such criteria includes t rial t. he :::l roke needs to intersect one or more previously recognized characters, and has a bounding box width that is at least, .some: threshold percentage (e.g., 120 percent) of the width of the box ot the previously recognized character or character:: that the stroke intersects. Further, the stroke needs to <:;. os ; the center vertical axis (half the width) of the box or. boxes for trie character or characters it., intersects. A portion of the scratch-out stroke needs to be above the baseline for the .line on which the character or
trid! a scratch-out deletes the entire content of a box. Therefore, when a box in the boxed input area contains multiple characters, the scratch-out gesture deletes all chaiacters In the box. It is possible for a single stroke to scratch-out multiple boxes, including boxes that are across mul i iple 1 i ru:e: .
Certain rules may apply to a freeform lined area and a
boxed wr.it ing area. For example, in order to allow for
scratch out styles such as crossing out a word (an X-out or a
double-sL,1 ike through approach), a stroke meeting certain criteria qualifies as a scratch-out gesture even if it does not iniers.'ct another ink stroke. Thus, if a stroke is added no nioi.t- than i snort, amount of time (e.g., one second) after the pen up ot the previous stroke when that previous stroke was a serai oh'-out gesture1, the stroke may be considered part of the :;arnt- ::; :r a t. ch-ou t gesture. Other criteria that may be evaluated .includes that the stroke intersects with the union of the oounding boxes previously scratched-out by more than some percentage (e.g., 80 percent) of the width of the union-ed bounding b^xes, and the stroke meets other criteria for being a scratch-out, gesture, using the bounding box of the previously de.eted ink in place of an intersected ink stroke.
When a wi.'rd is scratched out and it is not the last word, the next word is deleted as well if there is no space between i l ,-md tne previous word, and the word is one character long. Thus, a per ion is scratched out at the end of the sentence even though the scratch out gesture deletes the previous word but. does rv'i. necessarily intersect with the period.
When a singJe stroke is used to scratch out multiple words or characters on a line, the words or characters between the first -nid last word or character scratched out are deleted, even if the st roke does not intersect one or more of I'heir bound i.nq boxes, or does not qualify as an scratch-out
_
wit h r. espeol to one or more of them. Thus, for example, in Fin, ''), \ ne .'enter word "Ice" would still be scratched out, even I iiouqn t he stroke is below the baseline of the word, and dJso eve-n though L tie stroke does not intersect very much of the bounding box because the word is between two words so:at shod out with a single gestures. Similarly, when a sj.uqle stsokv is used to scratch out words or characters on muJ 1.1 |j.l (- ii.rifS, the words or characters between the first and last word or character scratched out are deleted, even if the stroke does i.ot intersect one or more of them or, if it does Interned., di'es not otherwise qualify as an scratch-out with resped to one or more of them.
Turning to an explanation of the operation of the scratch out detect ior- mechanism, FIG. 10 shows an example process when a new st rsKe is received, as represented by step 1002. Step lOCs'l i.est.s whether the stroke is a second stroke of a previous scr.-i ts;h-out gesture, which as described above, occurs when a user m-jkes a first scratch out gesture, lifts the pen, and makes ,-4 second gesture. As described above, considerations including in used at. step 1004 to differentiate between whet.he; t. MI- user intended a multi-stroke gesture or a new s t. roke !'ol lowing a scratch-out gesture. In the event that the st roke was a second st roke of a scratch-out gesture, the first stroke alieady has been determined to have been a scratch-out
gesture, and I.lie word or words (or box or boxes) to scratch our are know:,, whereby the new stroke may be automatically erased it step 1006. The process then ends until another new si i oke is del ect ed .
Returning lo step 1004, if not the second stroke of a sc:: dtch--oui.., step LOOK is instead executed, essentially represent ing the generation of a list of words or boxes to which trie possible scratch-out gesture applies, namely those i.hat die .intersected by the stroke. As described below with reference 10 FIGS. 11A and 11B for freeform (e.g., lined surface) input, and FIGS. 13 for boxed input, various evaluation,:, .ire performed to determine whether the stroke is a scr d I cri-ou: gesture, and if so, to what word(s) or box(es) the stroke- applies. When invoked via step 1008, which inherently includes .-i determination of whether the user is writing on a ireefor.n i. i. boxed input surface, FIGS. 11A and 11B generate and return a .List of zero or more words. When alternatively invoked vi.i step 1008, FIG. 13 generates and return a list of
After invoking FIGS. 11A and 11B or FIG. 13, as appropriate, step 1010 evaluates whether the list of words or boxes is empty. If so, step 1010 branches to step 1012 to handle the st ,:oke as an ink stroke, otherwise step 1014 is
executed, which erase;;, the stroke and the words or boxes in trie list . The process then ends.
A. mcnt i oiifd above, FIGS. 11A and 11B represent the part o-.' ! he-- prcioe, s that i.- invoked when the user enters a new st.toke on .1 :reeform surface, to determine whether the stroke is a serai :h-out gesture, and if so, to generate the list of words u< dels--•te. To this end, FIGS. 11A and 11B evaluate the stroke against various rules, including word rules based on threshold evaluations, as generally described below with reference < o FIGS. 12A and 1.2B.
FIG. ilA begiris its evaluation by determining at step 1102 whether t:he st.roke intersects with more than one line. If so, the Line boundary is changed from one line to multiple lines at r. iep 1104, essentially by extending a virtual ink bounding U>x tor all words to the top of the uppermost and the bot t orr: .:•'. ' l"u- lowermost line that is intersected by the strike. Is not i nl e rsect.ing more than one line, step 604 is bypassed.
Siep NO) then adds all the words in the line having a bounding bc.-x that intersects with the stroke with the normal t'hreshold -int. •.; tne list, at least for now. The intersection of i.he word with the stroke requires the bounding box to be intersect ing wil.n the stroke by a constant threshold amount.
.'it'-p i p.-rf tests whether the list contains only one word. If not., the j.rocess continues to FIG. 11B, as described below. Otherwise, there is only one word in the list, and step 1110 is execut. t-d which tests the stroke length and whether it extend:- t. n a> least the center of the bounding box, as described above and with reference to FIGS. 12A and 12B,

It ni;i, there are no words to scratch out, and the r.-linns t.o FIG. 10 where it ends. Otherwise, the oont i uues to FIG. 11B. FfCi. ! IB tests for an exception, namely whether the last word in t:x-> 1 ist is followed by the single character word, without a t railing space. Similarly, the first word is evaluated lor a preceding character. As described above, this handles putic:t Mat ion (before and after words) such as a period, question m^rk, exclamation point, apostrophe, comma, quotation marks and .-.o :orth, as well as other characters. Note that handling preceding and trailing single character words also nelps w i. t h language-Independence, e.g., a Spanish-language quest.] on mark character appears upside-down preceding the sentence feiai ivc to how the question mark character ("?") •appears endLn'.: I:he sent:ence. This also handles scratch-out operations in right-to-left (RTL) languages, such as Arabic-based .languages or Hebrew. Since it is likely that the single charade; wor>; is part of a word, step 1120 looks for this
5i t uat. i i Ji, .1; c.l j i. Lour.d, adds the preceding and/or trailing .•;:in>.il|J on-ira ter word (or words) to the list at step 1122.
Stop i 1/4 looks for a multi-line scratch-out, (but could also I-,.«.-k for wnet.her there are any intervening words between the- first: ,-tnc last word on a single line), and if so, adds any 'n\ ( er \/'-'i: i ii \ words between the first and the last word sci at. rh^-d '>ui into the list. At this time, the list of words to delete is known, and the process returns to step 1010 of FIG. .1 iJ .
FIGS. I/A and 12B represent an example of the lined, singlr.--wor-d / .::ra tch-ou t evaluation in more detail. In FIG. 12, the !'."'-' Width =- The minimum width of the scratch-out stroke. Word Tl The threshold value of the word's ink bounding box
SC T.1 - The threshold value of the scratch out stroke width reldiiv.^ i.o Word Width Tl (in %).
SC '12 - The i.hreshold value of the scratch out stroke width reldt iv- I.o Word Width T2 (in %, SC Tl > SC T2) .
WBBox - The i.nk bounding box of a word.
SC Width - T'ie width of the scratch-out stroke
SC TO - Threshold value of the scratch out stroke width
reiaiiv^ to word width larger than T2 (in %, SC T2 > SC TO)
Fie. L2A, step 1202 evaluates the ink bounding box of
i. a minimum, e.g., a half-inch as described above. I: be-low the minimum, step 1202 branches to step 1204 where t.he .set .itch-out .stroke width is compared to the threshold value of the scratch out stroke width, relative to word widt.n, i:--). g . , in percent) . If greater, the process continues to l;'IG. J2B, as described below. Otherwise step 1204 branches to step 1210, where the word is not included in the 1 LSI .
ReM.ur.niiq to step 1202, if the minimum is achieved, step 1206 i ;; executed, which evaluates the ink bounding box of the word against a tnreshold value of the word's ink bounding box. If less than the threshold, step 1206 branches to step 1208 where the wi scratch out. If so, the word is added to the se:.it.i'h ou! 1 1st via step 1226, and the process returns, essential;-/ ;o FIG. 11A step 1110. If not, step 1224 determine:: whether the scratch-out stroke was centered (e.g., some pa it ..-, £ it crossed the vertical axis) relative to the word':; bounding box. If so, the word is added to the scratch out. 1.1..; I via step 1226; if not, the word is not included via step 12^8. As can be seen, free-form entered words are thus handled b-ived on various threshold rules related to the
bound i nq b->x and the possible scratch-out stroke or strokes;
thus values styles of. strokes are acceptable as scratch-out
st coke.;-' .
Returning to step 1008 of FIG. 10, instead of processing t.eps 1 .LA, lor boxed input areas the list is generated via 1 -i. ,;tep 1302 represents adding the boxes with text that
intersect with the stroke, with the normal threshold, into the
list, ris <.j.>nr- rally determined via FIG. 14. In FIG. 14, the
following .ibbrex/iat.ions are used:
lax T. 1 - The threshold value of the wide box
50 Tl The ihreshold value of the scratch-out stroke width
reiat'ivt- to wide box (in %) .
|_SC_ Width • Tr.e width of the scratch-out stroke
BeqinnHV) at step 1402, the process evaluates whether the width ••>! (t\e box exceeds the threshold value of the wide box. If so, step 1-104 compares the stroke width against the box width time;, tne percentage threshold value of the scratch-out stroke widt n e.lative to normal box. If the stroke width is Long enough a!, step L4U4, step 1404 branches to step 1410 to add the box t<> the list. Otherwise, the box is not included in the i is I .
If instead at step 1402 the width of the box did not exceed t tv- threshold value of the wide box, step 1402 branches to step 1-106 where the stroke width is compared against the product o' the width of the box times the percentage threshold value of t ;ie scratch-out stroke width relative to the wide box. If t :ie stroke width achieves this threshold, the box is ridded ro I he list at step 1410, otherwise it is not included via step i -'10.° .
Her u r i) i ng to FIG. 13, step 1304 determines whether the stroke intersects with more than one line. If so, step 1306 is executed t o includes the Intervening boxes into the list. Otherwise, the step is bypassed. In any event, the list of (zero or iriMre) boxes to delete is now complete, as represented at step l.r.i'8, and the process returns to FIG. 10, where the list i:'.; urfd is described above. In this manner, scratch-out gestures in boxed writ ing areas are likewise handled in a bounda i y •'i n re 5hoJ d-based way that allows various scratch-out gestures i (.• be detected.
As ••MI, be seen from the foregoing detailed description, there is provided a method and system that uses bounding boxes ami thresholds to evaluate a user's intent with respect to a scratch-out gesture. Styles of scratch-out gestures are mostly unrons" rained, and users are better able to manipulate
a pen when writing to differentiate a scratch-out gesture from ot he r ink i nput .
While I tie invention is susceptible to various mod i £ if vj t. i.• >ns ami alternative constructions, certain illust rat<-d embodiment, s thereof are shown in the drawings and have been described above in detail. It should be understood, however, that there is no intention to limit the invention to the spec i lie Lorm or torms disclosed, but on the contrary, the intention i.s ;.o cover all modifications, alternative const ruct j t>n.s, and equivalents falling within the spirit and

WE CLAIM:
1. A computer-implemented method for intuitively applying scratch-out gestures in a computing device that comprises a handwriting surface component that receives electronic ink from a digitizer, as well as a component that provides boundary information corresponding to boundaries of words or characters and a scratch-out gesture detector coupled to the handwriting surface component and the component that provides the boundary information, wherein the scratch-out gesture detector also evaluates electronic ink received at the handwriting surface against criteria based on the boundary information to determine whether the electronic ink corresponds to any of a plurality of different acceptable scratch-out gesture types, the method comprising:
the scratch-out gesture detector associating a plurality of different acceptable types of scratch-out gestures with a scratch-out gesture process that erases previously-entered ink;
the computing device receiving user input corresponding to inking data entered via a digitizer at a handwriting surface component of the computing device;
the scratch-out gesture detector evaluating the inking data based on boundary threshold data of at least one corresponding word or character displayed by the computing device to determine whether the user input is a scratch-out gesture corresponding to at least one of the plurality of different acceptable types of scratch-out gesture types and that would cause the scratch-out process to erase

previously-entered ink; and wherein
a) if the inking data is a scratch-out gesture, erasing previously-entered ink corresponding to the scratch-out gesture, as well as the inking data corresponding to the scratch-out gesture; and
b) if the inking data is not a scratch-out gesture, processing the inking data as ink stroke data of at least part of a word or character.

2. The method of claim 1 wherein receiving user input corresponding to inking data entered via a digitizer corresponds to receiving computer pen-entered data.
3. The method of claim 1 wherein the inking data is received at a freeform writing area as a possible scratch-out gesture, wherein the boundary threshold data is based on a bounding box of a previously-entered word or character, and wherein evaluating the inking data comprises, comparing a width of the gesture relative to a width of the bounding box.
4. The method of claim 1 wherein evaluating the inking data based on boundary threshold data comprises determining whether the gesture extends into a bounding box of a previously-entered word or character beyond a certain point.
5. The method of claim 1 wherein the inking data is received at a freeform writing area as a possible scratch-out gesture, wherein the boundary threshold data is based on a bounding box of a previously-

entered word or character, and wherein evaluating the inking data comprises, comparing a width of the bounding box to a minimum size, and if the width of the bounding box is below the minimum size, selecting a first threshold for a comparison with the width of the gesture, and if the width of the bounding box is above the minimum size, selecting a different threshold for a comparison with the width of the gesture.
6. The method of claim 1 wherein evaluating the inking data comprises, determining whether at least part of the gesture is above a baseline of a previously-entered word or character.
7. The method of claim 1 wherein the inking data is determined to be a scratch-out gesture that applies to at least two separated words or characters, and further comprising, erasing previously-entered ink corresponding to the separated words or characters, and erasing any intervening word or character between the separated words or characters.
8. The method of claim 1 wherein the gesture comprises at least one additional separate gesture stroke, and wherein evaluating the inking data comprises evaluating a first stroke of the gesture to determine whether the inking data is a scratch-out gesture, and if so, processing each additional stroke of the gesture as part of the first stroke of the gesture, including erasing each additional stroke.
9. The method of claim 1 wherein processing the inking data as

ink stroke data comprises sending the inking data to a recognizer.
10. The method of claim 1 wherein the inking data is received at a boxed writing area as a possible scratch-out gesture, and wherein the boundary threshold data is based on a width of a box of a previously recognized character that the stroke intersects.
11. The method of claim 1 wherein evaluating the inking data based on boundary threshold data comprises determining whether the gesture extends a threshold distance into the box.
12. A computer-readable storage medium having computer-executable instructions, which when executed perform the method of claim 1.
13. The computer-readable medium of claim 12 wherein evaluating the inking data determines whether the criteria is met based on a ratio of a horizontal stroke width of the inking data to a horizontal width of the bounding container.
14. The computer-readable medium of claim 13 wherein the bounding container comprises a bounding box logically around a word entered via inking in a freeform writing area, and wherein evaluating the inking data comprises a selecting a threshold ratio based on the horizontal width of the bounding box versus a fixed value corresponding to a distance.
15. In a computing device, a system, comprising:

a handwriting surface component that receives electronic ink from a digitizer;
a component that provides boundary information corresponding to boundaries of words or characters; and
a scratch-out gesture detector coupled to the handwriting surface component and the component that provides the boundary information, wherein the scratch-out gesture detector associates a plurality of different scratch-out types with a scratch-out process that erases previously entered ink that is identified by the scratch-out process and that also erases the electronic ink comprising the scratch-out gesture, wherein the scratch-out gesture detector also evaluates the electronic ink received at the handwriting surface against criteria based on the boundary information to determine whether the electronic ink corresponds to any one of the plurality of different acceptable scratch-out types associated with the scratch-out process.
16. The system of claim 15 wherein the component that provides boundary information comprises a handwriting recognizer.
17. The system of claim 15 wherein the criteria includes at least one criterion of a set, the set containing, whether the electronic ink intersects one or more previously recognized words or characters, whether the electronic ink has a bounding box width that is at least a threshold percentage of the width of a bounding box of at least one previously recognized word or character with which the electronic ink

intersects, whether the electronic ink extends into a bounding box of at least one previously recognized word or character with which the electronic ink intersects beyond an threshold extent, and whether at least some portion of the electronic ink is above a baseline of at least one previously recognized word or character with which the electronic ink intersects.
18. The system of claim 15 wherein the handwriting surface component corresponds to a freeform writing area or to a boxed writing area.
19. The computer-readable medium of claim 12, wherein evaluating the inking data determines whether the criteria is met based on whether a horizontal stroke of the inking data extends into the bounding container beyond a certain horizontal extent.
20. The computer-readable medium of claim 12, wherein evaluating the inking data determines that the criteria for a scratch-out gesture is met, and wherein erasing previously-entered ink comprises erasing at least one word or character between two other words or characters that are also erased, and/or erasing a single character word that directly precedes or trails a word that is also erased.