Title INTERACTIVE EXPLODED VIEWS FROM 2D IMAGES
TECHNICAL FIELD
Th~srn ventlon IS lelhted to d sofhvate tool, and more speafically, an mteractive
softwd~eto ol that takes A static d~agrdma nd makes ~t inte~active
BACICGROWND OF THE INVENTION
Interactivity 1s one of the key capab~lltleso fcompute~sth at sets them apait fiom
othei types of information displays such as books, television, and rddlo Yet, most ofthe
lnfoimatlon vlewed on computeis vzci the Inteinet does not take advantage of th~s
~ntelactivity
Dinglams ale extlemely effectlve COI commun~catingth e structu~eo f complex 3D
objects that ale composed of many subpa~tss, uch ds mecl~anicala ssemblies, aich~tectural
environments, and biological organisms To eluclddte the composrte structule of such
objects, 111ustiators commonly use diagrammatic techniques such as exploded views that
educe or elim~nateo cclusion and expose lntelnal parts
Hoiveva, because exploded vtews ale usually desrgned as static rllustrdt~onsfo l
ynnt publications, they often suffei fiom two important duwbacks
Ambiguous spatial ieldtionshlps A static diagtam can only show a fixed set of
spatla1 relatlonsh~psb etween paits Fot complex objects, tt may not be clear fiom a statlc
exploded view how all the palls fit together, interact with, and constram one anothel
Vlsual clutte~ Static dldglams ale usually desl9ed to include all the lnfo~mation
the viewer might need about the object As a i esult, they are o flen visually clutteled,
makrng ~td ifficult to extract specrfrc lnfolmdt~ona bout a particular pdrt 01 subset of parts
wrthout catefully peruslng the entile ~llustration
In contrast, exploded vlew diag~amvs lecved through a compute1 can alleviate
both of these problems by allowng vlewers to lnteractlvely man~yulateth e parts and
thereby dynamically filtei the lnfoimdt~onp ~esentedin the dugram For example, a
vlewel mlght interact~velye xpand and collapse only the wheel assembly of a cal dlagrm
to better understand how the pals of that dssembly lnv :act w~tho ne another On the
othel had, a static, general-purpose c a dlagram cvould have to show all of the paits In an
exploded state, making ~td ~fficultto focus on the heel assembly In genelal, lntelactive
d~agi~mcasn be fa1 mole clear, mfo~matlvea, nd compeIling than the11 statlc
counterparts
In add~tiont,~ adltlonals ystems do not produce Intel actlve illust~a tlons that allow
users to directly manipulate the palts of the diagram
What is needed is a tool thdt levelages the ~nteractlvec apabihttes ofthe compute1
as applied to stat~dcr aglams
SUMMARY OF THE INVENTlON
The following plesents 3 simplified summdly of the lnventron ~n older to plovide
a basic unde~stdndlngo f some aspects of the Invention T ~ IsSu mmay IS not an extensive
overvlew ofthe rnvent~on It IS not Intended to 1dent1Fy keylcntical elements of the
~nventiono r to delineate the scope of the invention Its sole purpose IS to present some
concepts of the ~nventlonIn srmpl~fiedfo rm ds ii prelude to the mole detailed
desc~~pt~thoant 1 s p~esentedla te1
The p~esenitn vention disclosed and cla~medh eiem, In one aspect thereof,
compnses a~chltectulef or c~eatlnga nd vlewing lnte~act~evxep loded vlew d~aglamsm
wh~chv iewels can dl~ectlym an~pulatep d~tsa, nd the~ebyd ynamically filter the
infoimat~onp resented by expdndlng and collapsing the exploded view to seach for
rndlvldual pats The p~esenItn vention ovetcomes the defic~enciesa nd shoitfalls of the
p~lot at by malung the static exploded mew dldglams mtelacttve, wh~ch ale typically
vlsuatly clutteled oi unclea Furtheimoie, whe~edps noi systems ale amed at plovtdlng
completely automated p~epackagedd esigns and theleby et~rnlnat~nthge need fo~a human
deslgnei, the p~esenltn ventlon piov~dess emi-automatic h~gh-leveln tetactrve desrgn
tools that enable human designers to quickly produce the deslied illushatlon
The fiamewo~k compnses two maln components, a sulte of semi-automat~c,
sketch-based aulhonng tools that allows a use1 to qulcldy cleate mte~dc tive d ~ aagm~s
using 2D Images as Input, dnd, d vlewlng system thdt allows the UXI to dnectly expand
and collapse the exploded view, md seaich f o ~ln dlv~duapl aits
In another aspect of the plesent invention, a classifier 1s p~ovidedto automate
featm es by maktng in fee~n ces based on data assoc~atedw ith the authoring component
and the vlewlng component
To the accornpl~shment o f the fotegorng and related ends, certain lllustt atlve
dspects of the invenuon a e described heiein rn connection with the folIowing descnptton
and the annexed drawings These aspects ale mdicative, howevei, ofbut a few of the
vanous ways In whlch the pnncrples of the Invention can be employed and the present
lnventron 1s m tended to lnclude all such aspects and the11 equivalents Other advantages
and novel featuies of the invent~onR 111 become dppdient fiom the following debiled
descnptlon of the lnventlon when consldeied in conjunction w~thth e Qwrngs
BRIEF DESCRIPTION OF TI-W DRAWINGS
FIG 1 illushates a system m accotdance with the piesmt inventlon
FIG 2 ~llustratesa flow chait of a tendering piocess m accordance unth the
present t nvention
FIG 3A illustrates a collapsed view of the master cyllnder In accordance wlth the
present r nvention
FIG 3B ~llustratesa partially expanded vlew of the master cylindei, \nth its
ieseivolr cover and leseivoir dtaphcctgm exposed vertically
FIG 3C rllustrates a hrther expanded mew oftbe mastei cyllnder m accotdance
wth the piesent Invention
FIG 4 illustrates a flow chait of an duthonng process for converting a statlc 2D
exploded vtew of the diagram into an interactive dlagam In accordmce wth the present
inventlon
FIG 5 ~llustrates a flowchart for creatlng pdts in accordance wth the p~esent
Invention
FIG 6 ~llusndtesa flow chat? for one ~mplementdt~oonf a stack~ngp rocess in
accoldance with the plesent inventron
FIG 7A ~Hustratesth e rnitldl, posltron of the ieservoir cove1 lelatlve to a center
potnt of the loot part that 1s a housing of the mastel cyllndet of FIG 3A
FIG 7% illustlates the maxlrnum offset stack yamnetel using the expanded
mastei cyl~ndeor f FIG 3B
FIG 7C illustrates a stack hieiaichy fol an expanded masta cylinde~
FIG 8A illusbates an assembly bteakout of an dutomatlc t~ansmisslonp art
FIG 8B illustrates the free-form slroke diawn fiom a bottom part to a top part
806
FIG 8C ~Ilusttatesth at the system plocesses the stloke as a sttalght line explos~on
AXIS fiom an ayyioximate centei of the bottom pdlt to an apptoxlmate centei of the top
Pat
FIG 8D illusbdtes that the usel cm then lntetdct~velya djust tlle exp~ostona xis, if
necessaly
FIG 9 illustiates a flow chait ofone plocess for layenng in acco~dancew rth the
pi esent lnventlon
FIG 10 lllustrdtes a geometric ieptesentation of the seml-automatic fidgrnentatlon
tool In acco~dancew tth the piesent rnventron
FIG 1 I illustlates a lestr-lcbont hat can be employed ln dccoidance w~thth e
pl esent invention
FIG 12 illustiates the seml-automdtrc fragmentahon plocess for a cover of the
bansmlssron assembly of FIG 8A
FIG 13 lllust~atesa n example of a cav~tyw ~tha non-planar openlng
FIG 14 illusttales a flow chdt of one imylementatron of the seml-automdtlc
depth assignment tool in accordance w~thth e plesent lnventron
FIG 15A illust~at es thee 1nte1l ocklng parts that wee xpanded along an explos~on
dXlS
FIG 15B ~llustratesth e partla1 colldpse of the interlocking p a s
FIG 15C ~llushdtesto tdl collapse ofthe intellockrng parts In a port ofthe hous~ng
of the mastel cylinde~
FIG 16 ~llustratesa flow chart of one implemcntatlon for p~ocesslngn on-ad~acent
parts that ove~lapIn acco~da nce wlth the PIe sent mvent~on
FIG 17 ~llusttatesa flow chat of one lmplementatlon of the authoring plocess
whele one 01 mole xsumpuons a e v~oldted

executable, th~eado f execution, a yrogam, andlor d computel By way of illustrdtion,
both an apphcdt~onr unnmg on a, sei-ver and the selvei cdn be a component One or mote
compoilents can leslde within a plocess and/oi thlead of executlon, and a component can
be locallzed on one computel and/o~d lsb~butedb etween two or mole computers
AS used herein, the tetrn to ''infel" 01 "rnfeience" iefer generally to the pxocess of
reasoning about ot ~nfeinngs tates of the system, environment, and/or usel fiom a set of
observations as captu~edv m events dnd/or data hfelence can be employed to ~dentifya
specific context or actton, 01 can generdte a ptobdbil~tyd istnbut~ono vei states, fot
example The rnference can be probabllrst~c-that IS, the computation of a probdbrlity
drstnbution over states of lnte~esbt ased on a cons~derattono f data and events Inference
cdn also refer to techn~quese mployed fo~c omposing hlgher -level events fiom a set of
events and/or ddta Such mfeience lesults in the construction of new events 01 actlons
fiom set of obse~vede venls andlol stored event data, wherhet 01 not the events are
conelated m close ternpoldl ploxlmity, and whethe1 the events dnd data come fiom one
or seve~ael vent and datd soul ces
INTERACTIVE EXPLODEVDI EWSF ROM~IDM AGES
The disclosed archlrecluie IS a novel fiamewolh for cieattng and vlewlng
~nteiactrvee xploded vlew diagrms of complex mechdnlcal dssemblles Rather than
using 3D models as mput, dynamtc rllust~& ons die constructed fiorn 2D images,
lesult~ngrn a layeled 2 5D draglam rep~esentdtlon This tmage-based strategy has sevetdl
key benefits rt is easy to support a1 b~tlaryt e ndenng styles by simply finding or cleating
plctules of each pat of the object m the deslred style, ~t obvlates the need fot 3 D models,
wh~chA le, In geneid, much mote difficult to acqulre oi bu~ldth dn Images, and, finally,
usrng 2D ~magesd llows leve~aglngth e abundance of exrsttng statlc exploded vlews
commonly found ~n textbooks, iepall manuals, dnd olhei educattonal matei~al
One of the maln featules is d 2 5D ieylesentatton f o ~m telacbve diagt ams that
consists of layels of Images To fac~l~tatthee cieatron of dlaglarns In this format, set of
2 5D authollng tools rs ptov~ded Although ldyei-based 1 ept esentatlons are not new In
computer g~aph~cmso, st ofthls previous wolk on 2 5D duhonng h a f ocused pnmanly
on cleating layered dnlmatlons Trdd~tlonallyt,o ols cm be p~ovldedth at select, bend,
and evan delete entire objects, idthe1 than p~xelsi,n digital photographs In contrast to
these genelal-purpose systems, the disclosed archltectut e focuses on the specific
authonng Issues ~nvolvedin cleating Intel dctlve ~mage-basede xploded wew diagrams
Refetlrng now to FIG 1, thele is lllustlated a system 200 1n accoldance with the
present lnventlon The system 100 includes an autllonng component 102 that receives as
an Input one or mole 2D Images 104 foi piocessmng One challenge in tul-nmg a 2D
image Into an interactive exploded vlew dldgram is speclfjlrng how parts ~nteracwt lth
one another The authoring component 102 is a sutte of semi-automat~tco ols that
fac~l~tactoen stranrng the motron of parts as the p d sa ee xpanded and collapsed, for
Iayelmg pal* to plopeily occlude one another as the pMs move, and for annotating the
dlagram wrth part labels and guidehnes that expl~c~tslhyo w how parts move rela~veto
one another The authoi~ngc omponent 102 allows a user to quickly create compelling
intelactive d~agramsv ia slmple, sketch-based inte~actions
The authoring component 102 ~ n tfeac~e s to an ~nteractivev lewing component 106
that helps a vlewel dynamically filter the information plesented m a d~agram The
viewlng component 106 suppo~tas number of useful interacbons Specifically, the
v~ewingc omponent 106 fdcilitats an inteldct~veI mage output 208 that allows the user to
di~ectlye xpand md collapse the exploded view, dnd sedlch f o ~ln d~viduapl arts These
~nte~ctiohnesl p Ihe vlewer undelstdnd the spatial telationships between parts and the
overall structuie of the object
The ~mage-baseds ystem 100 enabIes ditect support of atb~traryle nderrng styles,
eliminates the need fo~bu ~ldlng rT 3D models, and dlows leverage of the abundance of
exlsttng static dmgiams of complex objects
'I f PC
Refmng now to FIG 2, thele is illustrated a flow chart of a ~endenngp rocess m
accordance wth the present lnventlon While, f o ~pu rposes of simphcity of explanat~on,
I
the one or more methodolog~ess hown heieln, e g ,m the form of a flow chart, are shown
and descnbed as a sefres of acts, ~t 1s to be undeistood and appleclated that the present
invention is not l~m~tebyd the older of acts, ds some acts may, in accoldance wrth the
present lnventlon, occui in d dribent older andjol concunently wlth other acts fiom that
shown and descnbed heteln Foi exdmyte, those shilled in the art wlll understand md
appreciate that a methodology could altelnatively be lep~esenteda s a senes of
intellelated states or events, such as In a state d~agtam Moleover, not all 11lustr;ited dcts
may be requited to implement a methodology In accoldance wlth the piesent mventlon
At 200, 2D lmdge 1s ~ecelveda s dn input to the system At 202, the use!
spec~fiehso w image pats ~nte~acAt t 204, parts motion IS consbarned dunng expanslon
and collapse At 206, pms ate layeted to plope~lyo cclude during movement relatrve to
one anothel At 208, the ~nteldcbve~ mageis output The plocess then redches a Stop
block
F~gures3 A-3C illustiate a 2D image that has been ptocessed In acco~dancew ~th
the ylesent ~nvent~oton plov~dea n rnlet dct~vev ersion showlng partral expanslon of an
auromobile mastel cyllndet FIG 3A illustrates a collapsed vrew 300 of the master
cyllndet rn accoldance wlth the piesent invention FIG 3B rllusttates a pat~dlly
expanded v~ew30 2 of the mastel cylinder, wtth its lestrvoil 303, teselvotl cover 304,
and reservoit d~aphragm3 06 exposed m veitical alignment The usel need only select
and hag the covet 304 to expand and expose the undellylng parts such as the lesexvorr
303 and dldphiagm 306 FIG 3C lllust~atesa further exphnded vlew 308 of the master
cylmdet honzontdlly rn accoldance wlth the present lnventlon Here, the user can select
(01 gi ab) and d~aga dlstd end 3 10 push rod 3 12 to expand and expose an assocrated
dust boot 3 14 and push I od tetstner 3 16 The levelse IS also true, whele the usel can
select and d~agth e push iod 3 12 towards the cover 304 to collapse the parts as vlewed m
boih FIG 3A and FIG 3B Note that the usel IS not restricted to gtabbmg the end part
3 10, but can gab any lntetvenrng pdrt and dlag ~tf o~ex panslon 01 collapse FOI
example, the user can gab the dust boot 3 14 and d~agit horizontally towatd the cove!
304 to collapse it, ~vhilele av~ngth e push ~ o 3d1 2 exposed m expanded v~ewT he Ieverse
1s hue for gldbblng the dust boot 3 14 wh~lern the collapsed mode, and dragging lt
outwald (awdy fiom the cove1 304) to expose the letdfinei 3 16 and boot 3 14 sqalately
1
tiom t11e covet 304 Howevel, the push lod 3 12 would ~ema~conll apsed ~nwadto the
boot 3 14 and !etainel 3 16 In dn dlternative ~mylemenlal~ownh, en the usel grdbs an
~nteivenwgp art for an expanslon pt ocess away ffiorn a ientel palnt, a11 patts on the same
dxls w111 dIso automdt~callye xpdnd In yet another implementation, only the first part
next to the dl ag point, but dwdy fiom centel porn t, will expand out with the othel
expanded parts It IS to be appreciated by one sk~lledm the art thdt other vdlrations on
the expansion and collapse ofparts assoc~atedw ith drag yolnts can be Implemented
Refelnng now to FIG 4, the~eis illusttdted a flow chart of an authonng process
for conveltmg static 2D exploded vlew of the draglam Into an lnteractrve dlagram in
accordance wth the present invention Several steps ae lnvolved m creatlng an
mte~active~ mage-basedd ~aglam At 400, a 2D ~npuItm age IS recaved at the Input As
input, the system accepts elthe1 a slngle rmage of an cioject with all of its constituent
yleces visible ( I e , In d fully exploded state), 01 d set of ~mageso, ne pel piece At 402,
the input dlagram is segmented ~ntopd its [n one ~mplementatrona, n assumption IS made
that the object IS lendeled uslng an orthographic project~ona, s IS typlcal m technical
lllustratlons However, peispectlve projecttons can also be piocessed In accordance w~th
the present invention to p~ov~dexdp hnsion and coIlapse of the parts If needed, lmages
of peispect~vep lojectlons can be warped applop~~dtelIyn the case where a srngle Image
1s used as input, the statrc didgram IS fils1 segmented Inlo parts conespondrng to the
const~tuenpt ieces of the depicted object At 404, the parts are organ~zedln to stacks
Stacks define how parts move reldtlve to one dnothei as the object IS expanded and
collapsed At 406, the parts aeb ~o ken down ~ntol~ agmentsIn preparat~onf o~la yenng
At 408, the parts and fiagments ate layeled by ass~gnrngd epth values to each part such
that the parts pioperly occlude one mothel At 4 10, the user adds deslred annotabons to
the pa*, such as labels and guldelrnes The process then reaches a Stop block
As piev~ouslym dlcated, a diaglam, ds processed in accotdance wlth the piesent
invention, conststs of p arts and stacks Each part ~ncludesd n Image of its correspond~ng
component, d~ well as an alpha mask thctt defines Its L~undmgsr lhouette To achleve the
correct implesslon of lelat~ved epth between the vanous poi-bons of the object, parts are
also asslgned depth values thdt determrne how they ale layered When two or more palls
interlock such that they canno$be conectly rendeled (usmg, eg , a ''painter's dgonthm")
I
11 1s Insufficient to asslgn a single depth value to each pat To solve th~sp ~oblemp, arts
are d~vlded~ ntofi agments By specifying the apptopnate depth value for each fragment,
the conect occlus~onte latlonslzip can be achieved between parts that overlap in complex
ways
To enable parts to expdnd dnd colldpse dyndmrcdlly, they ale oiganized Into
sidcks lhat deiine how the prls die dlowed to move in lelatlon to one another More
p~eclsely,a stack IS an o~daedse quence of pam parlshat shale the same explosion ax~sT he
explosion axls IS a vectoi that specifies d l~ndel ong which stack parts can move The first
part in a stack 1s refened to as ~ llsoo t In one rmplementatton, each part can be a
non-loot membei of only one stack However, the same part can be the root for any
number of stacks Thus, a cotlectlon of stacks forms a ti ee, as is illustrated heleinbelow
with respect to FIG 7A-C
Refemng now to FIG 5, there 1s illust~ateda flowchat for credtlng pa~tsm
accoldance wlth the present ~nvent~onA t 500, the stalrc d~agiamI S~ ecelveda t the lnput
of the system To help the use1 segment s~nglest atic lllust~at~o~nn top ats, the
autho~~nsygs tem Includes an intelligent cutting tool that mdkes it easy to cut out the
~nd~vtducaolm ponenb of the depicted object The user simply loads the Input Image into
the interface and then ovetsketches the appiopnate part boundmes uslng cutting tool
Thus, at 502, pats are defined by oversketchlng pa11 boundalles At 504, the pats can
then be separated at the sketched boundaries In some cases, a component that is partially
occluded in the input illustiat~onm lght have holes m 11 that need to be filled At 506, the
holes ate filled, as needed Th~csa n ertllel be done mllnually using a conventional
graphrcs package, oi vm automatic hole-fill~ng techniques The ptocess then leaches a
Stop block
Refeinng now to FIG 6, theie 1s lllusbated a flow chalt for one lrnplementatlon
of a stacklng piocess in dccoidance wlth the plesent invention Aftel the parts have been
created, they can be otgan~zedI nto stack viu d sketch-based itltei&ctlon At 600, pd~ts
a e~ ecelvedm an exploded view To cieate d new stack, the usel connects the
appiopl-late set of pats by d~dwlnga free-folm stioke, as lndlcated dt 602 At 604, these
\ '
components a* then olgan~zedln to a stack, presaving the part 01 der defined by the
st~okeA t 606, the system assumes that the specified parts ae cuilently In the~fru lly
1
exploded configuiat~on,a nd then lnfels an explos~ona xls To detamine the explosion
axls, the system connects centels of a bound~ngb ox of edch of the filst and last stack
components wrth a stlalght llne At 608, the system defines the initld pos~t~ocnw, nent
offset, and rndx~mumo ffset fo~th e new slack that ale consistent with thls layout The
rnltial positlon for each part IS set by default to be a small ofiet fiom its predecessor part
along the explos~ona xls Slnce the pd1-b start out in the11 fully exploded layout, the
system sets the maxlmum offset fol each part to be the d~stancefi om the part's in~t~dl
posrtron to its cumnt, fully exploded posltlon
At 610, the use1 can then manually tweak the stack parametas vm a number of
dlrect-mautpul~trono perations once the new stack ts cleated For example, to modify the
explos~ona xls, the user drags out a llne anchored at the stack's loot, and then adjusts th~s
vector to the deslted dliectron The stack's axts updates lnte~act~vedlyu nng thts
opelation so that the usel can easlly see how the parts align To change a pat's lnitlal
posrhon dnd rnaxlrnum offset,t he usel swltches to a duect-rnbnipuldt~onm ode, and then
dlags the component to ~ tasp pioynate fully collapsed and expanded positrons
Figuies 7A-7C ~llusbatea bee ofvatlous stacks and parametes that are generdted
for some assembl~eso f the master cyl~ndesi ystem 300 of FIG 3A-3 C FIG 7A
illustrates the ~n~tipdols itron of the 1 eseivorr covet 304 relative to a center pornt 700 of
the loot part that IS a houslng 702 ofthe mate1 cyllndel 300 of FIG 3A The in~tral
position specifies the posil~ono f 3 part tn ~ tfsu lly collapsed stab with respect to ~ t s
predecessor part FIG 7B illust~atesth e maxrmum offset stack pawmeter using the
expanded mastel cylrnder 302 of FIG 3B The maximum offset is the furthest dlstance a
pa11 can move w~thre spect to ~ tpste decesso~ Here, the cover 304 IS expanded to
Illustrate ~ t sm axlrnum offset (MaxOcp) fiom the dlaphlagm 306, and the dtaphrdgm 306
IS expanded to ~llustiate~ t ms aximum offset (MaxQm) fiom the housing 702, both aIong
an explosion axis 704 defined by both comspondlng vectois
FEG 7C illust~atesa stack hie~archyfo r an expanded mastel cyl~nde7r 06 FOI
each of 11s consutuent parts, the stack stoles thee paiametels the mltral posltlon, the
cunent offset, that heeps hack ofthe part's cunent displacement fiom ~ tlsn lt~apl oslt~on,
and, the maxlmum offset, which ind~catesh ow far a part can posslbly move away from
- 1 4 (
the piecedtng part tn the stack 91ven these stack parameteis, the positlon of each part
\? r 3
depends only on the posltion of@ piedeces&i.
There aef ive stacks (STK 1- STK5) ~llustlatedw, here arrows rndicate the
ordenng of the pa* wwlthln each stack A filst stack (STKI) 1s defined from the hous~ng
702 (r e ,t he loot pat) to the ~eservoc~olv e] 304 A second stack (STIC2) is defined
fiom the houslng 702 to a check valve 708 A thi~dst ack (STK3) 1s defined fiom the
hous~ng7 02 to a second check valve 7 10 A foulth stack (STK4) 1s defined fiom the
hous~ng 702 to d secondary cup 712 A fifth stack (STKS) ts defined fiom the teselvolr
cover 304 to the push ~ o3d 1 2
F~guies8 A-8D ~llust~a\ter m oa~sp~ec ts of deftnlng parts otdet and an explosion
XIS using fiee-fo~ms hoke m acco~dancew tth the pesent ~nventlon FIG 8A illustrates
;m assembly bleakout 800 of a cdtdlyt~cc onvertel FIG 8B ~llustratesth e free-form
stroke 802 dla\vn from a bottom pal 804 to a top part 506 The Gee-form stroke IS
drawn to olganlze a set ofparts into a stack FIG 8C lndrcates that the system processes
the stloke 802 as a stla~ght-Irne xplos~ond x~8s0 8 fiom the centel of the boundlng box
of the bottom pat 804 to the centel of the bounding box of the top palt 806 The sboke
dlrectly ind~catesth e oldel of the parts In the stack, as kvell as the explos~ona xls 808
FIG 8D tllustrates that the usel can then interactively adjust the explosion axis 808, if
necessary Hete, the use1 adjusts the axls 808 such that the cenlei pant of the top part
806 moves left of the centel of the bottom part 804
Refeimng now to FIG 9, the~eIS illustl ated a Cow chart of one process fo1
ldyenng 1n dccoidance wlth the piesent lnvent~on Afie~a ll ofthe stacks have been
cleated, pats are fiagmented, ~f necessaly, and then Idyeled to ploduce the correct
impression of ielat~ved epth between them At 900, the stacks have been generated At
902, fiagmentdhon IS perfolmed, and depth ass~gnrnents hte made fol interlocking p a s
The outer part IS then spllt Into fiont dnd back fiagrnents At 904, the outet part IS
fiagmented by ovelsLetchlng the closed boundary to genelate a cuive C At 906, a rdy IS
cast from each p~xeol n culve C towald any axis away tiom the vlewer At 908, a11 rays
that Intersect culve C twice ale added to the occluding aotfron of the occluding cuwe Co
At 910, the system extrudes cuwe CO along the explos~ona xrs At 912, p~xelsw lthrn the
extruded reglon ale assigned to the fiont fiiigment All other pixels ae assrgned to the
back fragment At 9 14, the innel p d ~ItS layeted to pass between the outel fiont fiagment
and the outel back fi agment by setting con espond~ngd epth values The piocess then
leaches a Stop block
The user can manudly partlt~ona pat Into fiagments with the cutt~ngto ol, and
then expl~cltlya sslgn ;1 depth vdlue to each pal 01 iiagment In the dtagram However,
for objects wlth mole than d few components, this lype of manudl laya speafication can
be tedious To educe the authoring butden, the disclosed dchltecture system provldes
seml-automatic frdgmentat~onm d depth assignment t.~otsth at can be used for a large
class of rnterlocklng parts
*
Referring now to FIG 10, there rs lllustrdted a geometilc representabon of the
semr-automat~cfl agmentation tool in dccoladancew lth the pl sent mventron Typically,
when two pats ~nterlocko, ne component fits loughly inslde the other In this case, the
conect ldyenng can usually be achieved by spl~ttn~thge outel part Into front and back
fragments, ad then laye~lnglh e lnnei pall to pass between them To fiagment the outel
part, the user oversketches (w~thle~lp fiom the cutllng tool) the closed boundary 1000 of
the cav~tyo r openlng that encloses the ~nnper~ec e As shown m FIG 10, the 3D
boundary of the cdvity 1s lefel-red to ds B A curve C that the user draws IS also B's
piolechon onto an Image plane 1002 Gwen cuive C, the system computes the occluding
portion Co of thls cumC , which occlud~ngp ortion Co 1s defined to be between a filspow 1004 on the bounday B, dnd a second polnt 1006 on the boundary B Th~asl so
defines whe~e an inner part ZOO8 passes behlnd an outer part 10 10 The system then uses
the occluding cu~veC Ot o d~videth e enclosrng component rnto two kagments (a front
fiagment and a bdck fiagment)
The system extracts curve Co by detenn~nmgf, or any 3D po~ntpth at extends
thlough the openmg, whe~eyp dsses behind B (1 e ,o ut of the viewer's sight) Slnce prts
ale constr alned to move w~thlnth e11 stacks, only polnls that go th~oughth e openlng whlle
I
traveling In the explosion direct~onr ae cons~de~ed
Refemng bnefly to FIG 1 1, thele IS ~llustrateda iestnctlon that can be employed
m accordance with the ptesent ~nventlon The system can assume that curve C, here,
cuive 1 100 does not self-inteisect, and that any lrne 1102 palallel to an explosion axis
1104 ~ntersectsc urve C no mote $an tw~ce A curve 1106 does not meet thrs restnctron,
I
In that, d line 1108 paallel to the explos~ona xls 1104 rntelsects the curve 1106 four
tlmes, as lndlcated at intelsect~onp olnts 11, Iz, 13, and I4
Refeinng dgdln to FIG 10, glven t h ~ sie stnct~ono n the shape of curve C, and
~gnonngta ngent cdses, the plojection of I onto the image plane 1002 will ~nteisecct urve
C exactly twlce (at polnts Cf and C2) Let CI be the fit st intasection polnt following r
away horn the viewei By default, ~t is assumed thatp passes In fiont ofB at C, and
behlnd 3 at C2,w h~chc otresponds to the common case in whlch 1) enters the opentng
defined by B as rt moves away from the viewel Giver this assumption, FIG 10 deplcts
the basic steps fo~co mputing occludlng culve Co The usel speclfies whlch end of the
explos~ona xis poin& away fiom the vrewei Considei the path of every yo~ntth at passes
th~ough3 by caslrng alay fiom evely prxel on Cmt he explosion dilect~on lf the ray
Intersects C again, the pixel coi~espondingto th~sse cond ~ntersectlonp oint 1s added to
occludlng cwve Co Once culve yr ocesstng has completed, occlud~ngc urve Co 1s
extruded by I asteiizlng a llne ofp ~xelsIn the explosion dtrectton, stat~ngfi om each p~xel
on the boundaiy B Every prvel encounteied IS added to the part's fiont fragment 1012,
and all 1e matnlng pixels colnyl~seth e back fiagment 1014 Extrus~on1s stopped once the
boundary of the image IS leached
Refelnng now to FIG 12, there 1s lllust~atedth e seml-automat~cf ragmentation
process foi a cove1 1200 of the tldnsmlsslon assembly of FIG 8A The usel first
overskelches the boundaly cdvlty w~tha curve C (also denoted curve 1202) uslng the
cuttlng tool to generate the cutve 1202 The system then casts lays 1204 fiom each plxel
1206 on the curve C in the d~tectlono f the explosion ax~tsh at polnts away from the
vlewel Fol all lays 1206 that intersect c u t e C tivlce, the second points of lntel-sect~on
1208 die added to the occludlng poitlon Co of the culv.: C The system extludes
occlud~ngc urve COa long Ihe exploston axls A11 ptxels ly~ngw ithtn the extruded reglon
121 0 ale claslfied as the fiont fidgment 12 12 and the lemdrnlng plxels are classified as
the back fidgment 12 14 The system cm now set the depth values of the turbme 1216 to
Ite between the depth values oithe front fidgment 1212 md back fragment 1214 to
produce the collect occIus~onie ldt~onship Note that the assumptions p~oduceco llect
fi.agmentat~onsfo l a whole class of enclosing cavtttes of different shapes and
o~lentatlons Syectficdlly, the assurnpt~onsd o not lestnct B to Ile m a plane that IS
orthogonal to the explos~ond ~erc tton
Refemng now to FIG 13, thele IS illustrated dr! exdrnple of a cdvity 1300 w~tha
non-plmr open~ng h thls situation, the system computes the conect fragmentation
because all of the assumptlons hold The openlng defined by B has the notch 1302 that
causes B to be non-pldndr As long asp always passes in fiont of 3 at C,f o ~al l pairs of
lnteisectron potnts CI and C2, the ftagmentation dgoilthm obtarns the correct ~esult
The default fiagmentatlon assumptlons ale val~dfo r a large class of ~nterlocking
parts Howevel, if B IS o~renteds uch thdtp emages fiom behind CI and passes m fiont
of C2,t hen the fiagmentatlon assurnpt~onsd o not hold Wlthout any user intervent~on,
the system computes an lncotzect ftdgmentat~on A top-down vlew of the scene would
clexly ~llust~atthea t 3 is ~nf iont of r at C, To obmn the coi-rect results, however, the
user can manually rndlcttte to the system to invert the fiagmcntatlon computat~onw, htch
can be done simply by revasing the explosron dnectlon in the fiagmentatlon algorithm
SEMI-AUTOMADTEICP THA SSIGNMENT
Refemng now to FIG 14, the~eIS ~llustrateda flow char&o f one implementat~on
of the seml-automatic depth asslgnment tool ~n accoldmce wlth the present Invention In
general, non-adjacent parts in a stdck can ove~laps,u ch as in FIG 15A-C As a result,
depth values cannot be piopagated outwalds &om the root In a single pass Depth
asslgnment then becomes constrant sdlisfdctlon pioblem Once all of the appropnate
parts have been fragmented, the use! can iequest that the system lnfer pat layers At
1400, the system recelves completed fragments for plocessing Fot all pails of parts, to
detelrnlne whether the pa~rsin terlock, the system employs a heuristic At 1402, a check
is made to determine if the cioss-sectlon of one part (with respect to the explosion
d~rection)f its withln the cross-section of the cuive that defines the cavity opening ( ~ f
there is one) in anothel part At 1404, the system detnmmes ~f theie IS d fit m the part
cross-sections A number of constmnt ptocesses can be employed to address fit
situations At 1406, for my lntellocklng parts, a constrant is added that the Inner part is
layeled in front of the back frdgment and behind the fiont fiagment At 1408, a
detenn~narionis made if the mtetlock~ng e la at tons hips ale consjstent If YES,a t 1410,
the system of ~nequalltyc osstrd~ntsis solved using local constrdrnt piopagatlon
techniques The piocess then I eaches d Stop block If, however, the lelationships are not
consistent, flow IS fiom 1408 to 14 12, to Inform the use1 of the rncons~stency
At 1404, tfthere is not a fit, flow 1s to 1414 to conslder any two non-intalochng
pdrts For non-tnterlockmg parts, it is dssumed that the~dre pth values are e~therst nctly
jncleasing or decreasing when consldei~ngth em In sta~k~onlgde l Thus, fo~an y two
non-intellocklng parts In a slngle stack, the constlaint IS added to layer the pan at the neai
end ofthe stack m fiont of the other pa3 Flow is then to 1408 to agaln pel form the
consistency checks, as p~ev~ousdlyes cnbed
As ind~cdtedp tevlously, the user can ~equestth at the system rnfei part layers horn
fidgmented pdt'ts Although lhls heunstlc wo~ksIn most cases, the~ear e sltuat~onsIn
whlch it could fat1 Foi example, the heunstlc wuId fall wheie an Innel part contains a
bulbous end that does not fit wlthm an outei pdlt In oldel to handle such situations, the
user can manually tntelvene to spec@ the conect mte. iochng t elatlonshlps, as ind~cated
in flow from 1404 to 14 16 Foi exdrnple, the use1 could fiagrnent a bulbous pat so t h ~ t
the aoss-section dssumpt~onh olds for the fiagrnent lhat actually fits rnto the enclosing
component Flow is fiom 141 6 to 1408 to again peilblm the consistency checks, as
pevmusly desclrbed
FIG 15A-C lllustlate that mult~plep ats in a stack can rntetlock FIG 15A
~llust~attehs ~eern terlocking palis 1500, 1502, and 1504 that ale expanded along an
exploston axis FIG 15B ~llust~dttehse pdrtld collapse of the ~nte~lockinpgar ts FIG
15C ~llustratesto tal collapse of the interlock~ngp arts 1.1 a port 1506 ofthe housing 702 of
the mastel cylinder 300 In thls case, the algolithm co~rectlyln fas the depth for all of
the parts, slnce all of the assumptions ofthe semr-automat~cd epth assignment algorithm
hold Namely, the c~oss-sectlond ssumptlon holds, lr 1s true that all non-mte~lockmgp arts
In the stdck cdn be layeted in oldel wrth the parts close1 to the near end of the stack on
top wlth no vlsudl aitifacts, and ~tI S also true that an Inner part that inte~locksw lth an
outel part cm be layeled between the outel parts fiont md back fiagments, agarn wlth no
vlsual a ~ tfal cts
Refemng now to FIG 16, there is illustlated a ..low chat of one tmplementation
of the authoi~ngp locess lvhe~eo ne 01 mole dssumpt~onsa te vlolated S~nceth e system IS
organtzed as a colIectlon of semr-automatrc tools, it does not folce the user to choose
either a filly automatlc piocess ot a completely manual interaction Instead, the system
can fluidly accept manual guldance at any stage in the authonng plucess At 1600, the
system ncelves the authonng fi3rnewolk At 1602, the system processes cases according
to predetermined assumpt~ons At 1604, the system deteimines if a v~olationh as
occulred If Yes, at 1606, the system plompts the user for manual gutdance For
instdnce, lf the system ts unable to find the pioper fragments because one of the
fragmentation assumpt~onsis rnvahd, the use1 can manudlly d~videa part into front and
back pieces and then use the autornatlc depth asslgnm at tool to infer a fayenng
Slmdarly, rfthe system guesses incorrectly whether or not two parts fit together, the user
can fiist explicitly specify the correct inte~Iochng~ elationshiya nd then use the system's
constrant solve] to asslgn depth values At 1608, the system determines if manual
processing should contlnue If Yes, flow is back to 1606 If No, flow is to 1610 to return
to automatlc pl-ocessing At 1612, the system determines if piocessmg has completed for
that case If No, flow IS back to 1602 However, ~f p~vcessingh as completed for that
case, flow IS to a Stop block If no violat~onh as occutred, flow is from 1604 to 1612 to
detemlne if process~ngh as completed
Refe~lmgn ow to FIG 17, the~e1s lllustlated a flow chart for one ~mplementation
for annotation m accordance wlth the present invention As an optronal final step, the
user can annotate individual parts wlth labels and add gu~dellnesth at indicate expllcltly
how parts move m relation to one anothel At 1700, for each part that reqmi-es a label,
the user specifies the appropnate label text To lndlcate where a label should be placed,
the usel clicks to set an anchor point (typ~callyo n 01 neai the part being labeled), and
then d~agtsh e label to the desned posttion, as Indicated at I702 When the d~agram1 s
la~do ut, the system ensules the offset between the lab i and its anchor remans constant
so thdt the label moves ngldiy w~thit s correspondrng part, as lndlcated at 1704 At 1706,
the system renden a llne fiom the centel of the label to ~tsan chol point to make the
assoclatron between a part and ~ tlsab el explrclt At 1708, the use1 creates a guideline by
select~ngt wo parts and then dlagg~ngo ut a line that connects the two pill* rn the desired
fash~on Each endpoint of the lrne IS t~eatedas dn ancho~th at sticks to its corresponding
part As a result, the guldellne dddpts dpp~opnatelya s the parts move By default,
guidelines ate lende~edd s dotted l~nes The process then reaches a Stop block
Refe~mgno w to FIG 18, the~eIS ~llustldteda flow chart of one piocess for
vlewlng a layout m accoidance with the piesent ~nventlon To d~spIayd ynamlc exploded
view ~llusttat~onass,o ftwale tool IS provrded that supr7rls a number of usehl
tntelactlons to help the human viewel extract ~nformatlon fiom the diagiam A local
propagation algoit~h m is employed to lay out the parts of the dragram At 18 00, the
algolrrhm woiks by t~ave~sinihge stack hie~archyIn topologtcd1 order At 1802, the
algorithm then success~velyc ~lculdtesa nd updates the posltlon of each part based on its
pledecessol and cunent offset Although locdl p~opagationc annot handle cycles, th~iss
not an Issue because the stack hlelachres folm a tree, as ment~onh eremabove Once all
part positions have been calcuhted, the system xende~se ach part and ~ tfsra gments at
the11 specified depths, as ind~cdtedd t 1804 At 1806, the use1 can also enable label and
guldellne ~endel~nIng t he vlewlng interface to display annotdtlons The ptocess then
leaches a Stop block To plevent v~suacl lutter, the system only I enders labels and
guidelines whose ancho~p o~ntsa1 e unoccluded by othe~p arts
Refenlng now to FIG 19, rhele IS ~llust~dtedd f low chart of one process For
animation expansion and coII~pse The vlewrng progim suppo~tsa slmple but useful
lnteiaction that allows the v~ewei to expand or collapse the entlie draglam with the cllck
of a bulton Thus, at 1900, the usel perfoims this functlon accord~ngto Input device
conbol To p~oduceth e desiled mimatron, the system ymoothly ~nte~poldttehse cu~yent
offset of each part eithei to its hlly expanded 01 hlly coIlapsed state, depending on
whlch dnirnatlon the user selects, as lndicaled at 1902 The ptocess then reaches a Stop
block
Refemng now to FIG 20, there 1s illust~dteda flow chdrt of a direct rnan~pulat~on
plocess in dcco~dancew ~thth e ptesent Invention To enable the human viewer to focus
on the intelactions and spatla1 lelat~onshlpsb etween d spec~ficse t of pdlts w~thoust eelng
all of an object's components m exploded fom, the disclosed a~chitecturea llows the user
to select~velye xpmd and collapse polllons of the dlag un via consttained drreet
manlpulatlon At 2000, the user selects component fot manlpulatlon Afin selecting a
component, the wewe1 can ~nte~dct~vmeloyd ify ~ tcsu nent offset by dlagg~ngth e part
outwards or ~nwaldsw ith~n1t. s stack The mm~pularionis constlalned because a part can
only move along ~ tesx plosion axis, no mattel whele the user diags When the user
initiates thms mnterdctlon, the system filst iecords where the selected part is glabbed, as
~ndlcatedd t 2002 At 2004, as the use1 diags, the system calculates the projection of the
cullent porntei locatlon onto the explosion axls At 2006, the system then sets the culrent
oBet of the selected part such that the g~abbedp oint sltdes towards th~psr ojected point
At 2008, the system dete~mmesrf the user has exceeded the drag capab~htles If the user
drags d part beyond ~ t sfu lly collapsed 01 expdnded state, the system tnes to modifjr the
current offsets of the p~decessotp diis to accommoddte the lntelactron Each
predecessoi 1s cons~deredm older unt~tlh e root 1s leached, as Indicated at 20 10 For each
pedecessor part, at 2012, the system moves the p~edecessolp art to accommodate the
user's dlaggzng, as long as the pledecessor part has not exceeded its dtag capabllitles If
the predecessor part has exceeded its diag capabilrties and the gabbed polnt on the
selected part has still not leached the ylojected point specified by the usel's dtagging,
then the next predecessor put IS consldned Thus, a predecessor part w~lol nly be moved
fall of ~ tdse scendants (up to the manlyulated component) ale elther hlly collapsed or
expanded The process then reaches a Stop block Thus, the user can effecavely push a
set of parts togethel or pull them dpdrt one by one If no excessive drag has been
detected, flow 1s from 2008 to 2014 to plocess the pats acco~dingly Flow IS then to the
Stop block
Refmng now to FIG 2 1, theie 1s illustrated a flow chart of a process for d part
seach in acco~ddncew ~thth e y~esenmt vmt~on In some cases, the mewel may want to
locate a part thdt IS hidden from view when the object is m rts fully collapsed state, as
ind~cateda t 2 100 At 2 102, znstedd of expdnd~ngth e etltlre diagram and then searching
v~suallyfo l the component In questron, the disclosed drchltecture allows the user to
seach foi d part by look~ngu p its name 01 plctuze In a l~sotf all the object's components
At 2 104, the vlew~ngs oftwale expands the object to reveal the requested part as well as
the paljs ~mmediatelys urround~ngrt In the stack In oldel to piovtde the applopnate
context The process then leaches a Stop block
Referring now to FIG 22, thele IS illustrated a part search operaon In accoldance
w~thtl le plesent tnventlon The system ylovldes a sealch wlndow 2200 to fdc111tate
findlng the hidden component Her,e, the usel chooses to find a hldden pat that IS a
speaka 2202 of d device 2204 Aftel entenng labels fot all components, the system
stoles the labels in a format thdt is easily presented and sedrchable Here, the use1 enters
the tem "spedker" mto a seatch fietd and selects a Sedtch button 2206 The system
g~~phlcalpliye sents the speakel part in an uppel portton 2208 of the seaich window
2200 Alternatively, the w~ndotv2 200 yiov~desa means wheteby the user can sc~oll
down a l~s2t 21 0 of ldbels (thdt cm be sorted in alphabettcal order, for example) to the
label "spellker", and select ~t In tesponse thereto, the system automat~callye xpands the
devrce 2204 to plovide an exploded view 22 1 2 along an exploston dxls to show the
ieIative posttlon of the hldden spedkei 2202 among the dev~cpea rts
Refeimg now to FIG 23, Chete 1s ~Hustrated the system of FIG 1 further
ernployrng di~~ficr~anl tellrgencem nccoldance wlth one lrnplementatlon of the present
lnventlon The subject Invention cdn employ t &YOUS ai3ificldI intelligence based schemes
fol calrylng out vd~~oaussp ects of the subject Invention For example, a ptocess for
detemln~ngd epth vdlues for parts can be fdcrlitaled vla dn automatic classifiei system
and plocess 2302
A classrfiet 1s d functlon that maps an ~npuat ttr~butev ector, x = (xl, x2, x3, x4,
xn), to a confidence that rhe Input belongs to A class, that is, f(x) = confidence(c1ass)
Such cldss~ficatianc m employ plobdb~lrslrca nd/ol stattstlcal-based dnalysls (e g ,
factonng ~ntoth e andlysis utlfrtles dnd costs) lo p~ognose01 rnfer an actton that d use1
desiles to be automdtlcally pel formed
A suppoit vectoi machine (SVM) IS an example of a classifier that can be
employed The SVM operates by finding a hypersurfdce In the space of poss~bleI nputs,
whtch hypersul face attempts to splrt the tryggeimg cntetla fiom the non-tnggenng events
Intult~velyt,h is mdkes the classificdtlon collect foi testrng data that is neat, but not
~denbcalto tlantng data Olhet dilected and undliected model classlficatron appiodches
include, e g ,n ave Bdyes, Bdyesran netwoiks, decls~ontt ees, and piobabllistlc
class~ficationm odels providrng d~ffeienpt atterns of ~ndependencec m be employed
Classificat~ona s used heieln dlso IS lncluslve ofstatrst~cale gresslon that 1s utll~zedto
deveIop models of pnonty
As will be redd~lya ppteciated fiom the subject spec~ficat~otnh,e subject mvention
can employ class~fiersth at are explrcrtly t~alned( e g , vzn a genenc tldfnlng data) as well
as rmplrc~tlytt drned (e g ,k zcr observing use1 behavlo~l,e celvrng extl-lns~icn fomatlon)
For example, SVM's ale configured via a leam~ngo r tlalnmg phase wthin a classifier
constructor dnd feature select~onm odule Thus, the class~fiel(sc)a n be used to
automatically perfolm a numbel of functions, rncludlng but not limited to determining
according to a p~edetermmedc nteltd 01 lealn~ngp locesses how parts aet o be stacked,
what parts are part of a stack, what and how a part IS to be itagmented, what annotation 1s
associated with a part, how the pxt IS annotated, employing and updahng user
preferences when worlung w~thth e authonng and vlewing components, processing
~nteilockrngp arts, and so on
In one ~mplementat~otnh,e arch~tectu~oef the plesent invention can be configured
to accommodate arb~trarye xploslon paths To achieve a more compact exploded vlew
layout, illustlators sometimes anange p a s using non-lmea~ exploslon paths that are
often lndlcated wth guidel~nes The disclosed constramt-based layout framewo~k
facilitates extending suppoit to ~ rti abry ~,u s a -specified explosron paths
In anothel ~mplementatlond, ynarnrc annotahons can be employed In support
the~eoft,h e system determines how to anange th~ms eta-mfo~matlond ynamically to take
~ntoac count the changing layout of dn inte~dctlved ~agram
The atch~tectureo f the piesent lnventlon suppolls parts ernphasrs and
de-emphasis In anothet ~mplementat~o~nt, I S usefuf to povlde dlagrdm authors with
~rnage-basedto ols foi emph~~zlnangd deemphasizing p&Ilcular pdrts of the depicted
obJect These tools atle slm~larto intellrgent filterst hat take Into dccount the ps~ceptual
effect of pel fo~mingp a~ticual1 rmage trans formdtions The emphasls operahons can be
used at d~splayti me to highlight lmpoitdnt paits
In anothel ~mplementat~osne,r nantlc zoomlng ISs upported For exti*emely
compl~catedo bjects, ~tI S usefbl lo mt~oducem ult~plele vels ofdetad that allow the
vlewer to ~nte~act~vceolnyt lol how much info~matloni s piesented for particular poltions
of the subject matte1
Depth cues can be suppotted Interactrve drdg~amsc leated from 2D images can
sometimes have a "flattened" appedlance where layels overlap It is possrble to
automat~callyre ndei.slmyIe deplh cues (e g , dlop shadows) when viewlng the d~dgramto
clarify the spatial I elatlonshtps between these layel s
In the context of this descnytion, "dlrect ~nteiactiono r manipulat~on"r efers to any
usa ~nteractlonth at involves dlleci 01 lmmed~ateu se1 contiol Usudlly, thls involves the
usel causlng an object 01 pat to be dldgged, ~esrzed0,1 drawn wlth an input mechan~sm,
for example, a mouse, tiackball, 01 othe~in put means In one ~mplernentat~otnh,e system
facilitates both d~recmt anlpulat~on( where the use1 can drdg parts iuound), and Indirect
uset ~nterdctton( whete the user rnanlyulates 01 exposes ot hides parts or objects with the
semh ~ntefta ce and animated expand/colldpse)
It ~sto be apprec~atedth at dtrect and tndllect Leer inteiactlon 1s not lrmtted to
common Input devlce mechanisms such as a mouse, keyboa~da, nd thumb pad, for
example, but can also employ volce controls to lnltrate expanston orobjects, collapse of
such objects, and sea~chesfo i such ydrts 01 object5 When uslng volce control, the user
wtll ham the system acco~dingto use1 commands, and Input such commands Into a
miclophone f o ~pr ocessing and execution
In bnef summay, exploded vlews ate cruc~alf oi explalnrng the ~nternasl tructure
of compl~catedo bjects Inteldctrve dtgibl dlagldms ate cspecrally important for altourlng
the vlewel to extiact specific ~nfo~matiofino m dn 1 1 1n~a tion by dynamically modify~ng
the way In whlch the subject mane1 is plesented The disclosed arch~tectured esc~rbesa
novel fiamewoik fol cleating and vrewrng ~nte~acte~xvpelo ded vlew diagiams uslng
stattc Images js Input Mote spec~iicaIlyd, set of autho~~ntogo ls IS prov~dedt hat
facrlitdtes the task of cteatlng such diagrams, and a vlewlng piograrn IS provlded that
enables useis to bettel understand spatldl ~elationsh~byest ween paits and the overall
structure of the object
Refeinng now to FIG 24, there IS illustrated a block dldgram of computer
ope~ableto execute the disclosed arch~tectute Tn o~dtt o p~ov~added ~tionacl ontext fo~
vanous aspects of the present Invention, FIG 24 and the following dlscusslon are
tnlended to ptovlde a bnef, genela1 descnytlon of a sultable computing environment 2400
In which the vanous aspects of the present invention can be implemented W ~ lthee
lnventlon ha been described above m the gene14 context of computer-executable
rnstn~ctionst hdt mdy run on one 01 mole c.omputels, those sk~lledIn the art w~lrle cognrze
that the tnventlon dlso can be ~rnplemenedin cornblnatron wrth olher ploglam modules
anandlo1 as a combmatlon of hardwale md softwale
Generally, piogldm modules Include loutlnes, progiams, components, data
structures, etc , t hat peifo~mp articulal tdsks 01 ~mplenienpt a~t~culaabrs tract data types
Moreover, those skilled m the a? will appteciate that the lnventlve methods can be
practiced wlth other compute1 system configu~ationsi,n cluding single-processor or
mu1t iprocesso~c omputer systems, mln~compute~ms,d in frame computels, as well as
pelsonal computers, hand-held cornpuung devices, mico~p ~ocesso-ib ased or
progmnmable consumer elecbon~csa, nd the I~kce, ach ofwh~chca n be operatively
coupled to one 01 mole associdted devlces
The lllusttated aspects of the invention may also be practrced in dlsttlbuted
computing envl~onmentsw heie cnta~nta sks a ep el fo~medb y I emote processing devrces
that iu e I~nkedr h~ou gh cornmunlcatlons netwo~k In a d~stnbutedc ornputlng
envnonment, progrm modules can be located in both local and jemote memory storage
devices
A computer typ~cdllyIn cludes a variety of computei-ieadable med~a
Cornputel-readable media can be any dvd~ldblem edid that can be accessed by the
compute1 and lncludes bath volatlle and nonvolat~lem edla, lemovable dnd nonlemovable
medm By way of example, md not limltat~onc, omputer readable med~aca n
complrse compute1 stoldge med~aan d communication media Computer stoidge media
~ncludesb oth volatlle and nonvolat~lei,e movable and non-1 emovable medr a implemented
In dny method 01 technology fot stoidge of ~nforrndtrons uch as computer i~adable
~nstluctlons,d ata sti*uctutes,p ~ogrmm odules 01 other data Compute1 stolage media
mcludes, but IS not llmlted lo, RAM, ROM, EEPROM, fldsh memory or othel memory
technology, CD-ROM,d lgrtd v~deod ~sk(D VD) 01 other optlcal d~skst orage, magnetlc
cassettes, magnetlc tape, mdgnetlc disk stolage or other magnetlc storage devices, or any
other medlum whlch can be used to stole the destted lnformatlon and which can be
accessed by the computer
Communrcat~onm ed~aty plcatly embod~esc ornputel-readable msbuctions, data
shuctures, program moduIes 01 other data In a moduldted data s~gnaslu ch as a caner
wave 01. otliel twnspoit mechanism, and ~ncludesa ny lnfonnatlon delrvery media The
term "modulated data signal" means a slgnd that has one or more of its chdlacte~~stlscest
or changed m such a mdnnei as to encode ~nfomatlonIn the stgnal By way of example,
and not Iimltat~onc, ommunlcatIon med~aln cludes wiled med~as uch as a w~redn etwork
01 dnect-wlied connection, and wl~elessm edia such as acoust~cR, F, ~nfraleda nd other
wl~elessm edia Comblnatrons of the any of the above should also be Included wtthtn the
scope of cornputel-leadable media
Wtth refelence agan to FIG 24, the~eis lllustlated an exemplaiy environment
2400 fol lmplemenbng vanous aspects of the invention that lncludes a computer 2402,
the colnputei 2402 Including 3 plocessing unlt 2404, a system memory 2406 and a
system bus 2408 The system bus 2408 couples system components includmg, but not
llmlted to, the system memoly 2406 to the plocessing unit 2404 The ytocesslng unit
2404 can be any of viurous comrnel c~allyd valIable processors Dual mlclopl ocessols
and othei rnulh-p~ocessora rchitectules may also be employed as the plocesslng unrt
24 04
The system bus 2408 cdn be any of seveial types of bus shucture that may further
intelconnect to a memory bus (w~th01 wrlhout a memory cont~ollel)a, peripheral bus,
and a local bus using any of a vanety of coinme~cldIlya vatlable bus archttectures The
system memory 2406 lncludes I ead only memory (ROM) 24 10 and I andorn access
memory (RAM) 2412 A baslc mput/output system (BIOS) is stoied m a non-volaole
memory 2410 such as ROM, EPROM, EEPROM, whlch BIOS contalns the baslc
routmes that help to tiansfer info~mationb etween elements wlthln the compute1 2402,
such cis dunng start-up The RAM 24 12 c ma lso rnclude a h~gh-speedR AM such as
stattc RAM fol cdching data
The computer 2402 futlhet ~ncludesa n ~nterndh ard dlsk dnve (HDD) 24 14 (e g ,
EIDE, SATA), wh~ch intel-ndl hard dlsk dnve 24 14 mdy dlso be configuied foi external
use in suitable chassts (not sho\ia), a magnetrc floppy dlsk dnve (FDD) 241 6, (e g to
leiid fiom orwnte to a ~emovdbled lskette 24 18) and an optical dlsk dnve 2420, (e g ,
ieddlng a CD-ROM drsk 2422 01, to read fiom or wlrte lo other high cdpactty optlcal
medla such as the DVD) The hard drsh dlrve 24 14, magnetlc dlsk dnve 2416 and optleal
dlsk dnve 2420 cm be connected to the system bus 2408 by a hard disk dnve Interface
2424, a magnet~cd tsk dnve mlet fdce 2426 and dn opt~cadl nve ~nterface2 428,
respect~vely The inteiface 2424 fot external dnve trnplementat~onsln cludes at least one
or both of Universal Sendl Bus (USB)a nd IEEE 1394 mte~fa ce technolog~es
The dt~vesa nd the11 dssoc~atedc omputel -1 eadable media provlde nonvolat~le
sto~ageo f data, data sh-uctures, cornputel-executable mstructlons, and so forth For the
computer 2402, the dnves and medla accommodate thz storage of any data In a su~table
dlgtal format Although the descnption of computer-leadable medla above refers to a
HDD,a ~emovablem agnetic diskette, md a ~emovableo ptlcal media such as a CD or
DVD, rt should be dppleciated by those shlIled In the at that other types of media whlch
are readable by a computel, such as zlp dilves, magnetlc cassettes, flash memory cards,
cartridges, and the like, may also be used In the exempldty operattng envnonment, and
further, that any such medld mdy contain cornputel-executable ~nstrucbonsf o~
pe~fo~minthge methods of the yr esent Invention
A number ofytoglam modules can be stofed n- the dnves and RAhd 2412,
lncludtng an opelating system 2430, one ot mote appllcalion ploglams 2432, other
ptogram modules 2434 and plogtarn data 2436 All OL poitlons of the opet atlng system,
appl~cat~onmso, dules, and/ol data can also be cached In the RAM 2412
It IS apprec~atedt hat the ptesent lnventron can be ~mplementedw lth vmous
commercially available opeldtlng systems or combm~tlonso f ope~ahngsy stems
A usel can entel commands and infonnatlon Into the computer 2402 thiough one
or more wtted/w~relessr nput devlces, e g ,a keyboard 2438 and a poIntlng devlce, such
as a mouse 2440 Othel Input devices (not shown) ma--l nclude a m~crophonea, n IR
remote conbol, a joysttck, a game pad, a stylus pen, touch scteen, or the like These and
other rnput devlces are often connected to the yiocesslng unit 2404 through an rnput
dev~celn teiface 2442 that 1s coupled to the system bus 2408, but can be connected by
othermterfaces, such as a paallel port, an IEEE 1394 senal port, a game poi-t, a USB
port, an IR mteiface, etc
A monitor 2444 or other type of dlsplay device 1s also connected to the system
bus 2408 vui an ~ntefta ce, such as a mdeo adaptei 2446 In add~honto the monltor 2444,
a computer typically includes other penpheral output czvlces (not shown), such as
speakets, pilnte~se tc
The cornputel 2402 may opeiate in a netwo~kede nvrionment us~ngI og~cd
connect~onvs mw lred andlo1 w~lelessc ommunicabons to one or mole remote cornputen,
such as a remote computel.(s) 2448 The lemote compukr(s) 2448 can be a wolkstatron,
a sewer computer, a routel, a pelsonal computel, pol-tdble compute~,
mrcroprocessoi-baed entellalnment dppliance, a peer device or other common netwolk
node, and typically includes many or dl of the elements descnbed relatlve to the
computer 2402, although, fol purposes of btevlty, only a memory storage device 2450 IS
illuslrdted The logrcal connections dep~cted~ ncludew uedhneless connechmty to a
local atea netwolk (LAN) 2452 and/ol ldiger network, e g , a wde area network (WAN)
2454 Such LAN and WAN netwoiklng enviionments a e commonplace m ofices, and
companies, and facilitate enteipnse-wide coinputel networks, such as inti.anets, all of
wh~chm ay connect to a globdl communication nehvo~ke, g ,t he htelnet
When used in a LAN netwoik~nge nmtonment, the computel 2402 IS connected to
the local network 2452 th~ougha wlied andlor wileless commun~cationn etwolk interface
01 adapter 2456 The adapt01 2456 may fdc111tdle wued 01 wlreless communlcatron to the
LAN 2452, which may also tnclude a wtreless access point dtsposed thereon foi
communtcat~ngw ith the wie~le ss adapt01 2456 Whei- used m a WAN nehvolk~ng
envnonment, the compute~2 402 can include a modem 2458, or IS connected to d
communicatlons seiver on the LAN, 01 has othei means for establishing communlcatlons
ove~th e WAN 2454, such as by way of the Inremet The modem 2458, which can be
lnteinal orexternd1 and a wlied 01 WIJ eless devrce, 1s connected to the system bus 2408
via the senal poit ~ntelfdce2 442 In a netwolked envlionment, piogram modules
dep~ctedle lattve to the computer 2402,ol portions thaeof, can be stored in the remote
memo~y/stoi.aged evice 2450 It will be appiecldted that the netwolk connections shown
Ale exemplary and othei means ofestabllshing a comr..unlcdtions link between the
compute s can be used
The computer 2402 IS ope1 able to communicate wlth any wileless demces or
enbtles opelatlvely disposed In wileless communication, e g , a pnnter, scanner, desktop
md/or porzable comyutet, poltable data assistant, communicatlons sdtellite, any plece of
equipment ot Zocat~ona ssocldted wth a wl~elesslyd etectable tag (e g , a kiosk, news
stand, reshoom), and telephone This ~ncludesh t least Wi-Fi and BluetoothTM wtteless
technologies Thus, the cornmunicatton cdn be d pr edefincd stluctu~ea s wlth
convent~onaln etwork or slmply an ad hoc communica Ion between dt least two devlces
Wi-Fi, or Wueless Fidelity, allows connectlor, x, the Internet from a couch at
home, a bed in a hotel loom 01 a confe~encelo om at work, wlthout wres WI-FI1 s a
wlzeless technology llke a cell phone that enables such devices, eg , computers, to send
and lecelve data ~ndoorsa nd out, mywhere wilhm the iange of a base station Wi-FI
networks use radlo technologies called IEEE 802 11 (d, b, g, etc ) to p~ovldese cule,
reliable, fast w~relessc onnectlvlty A WI-Fi nebvo~kc an be used to connect computers
to each other, to the Internet, and to wlred netwo~ks (whlch use lEEE 802 3 or Ethernet)
WI-Fi netwoiks operate In the unhcensed 2 4 and 5 GHz ladlo bands, with an I 1 Mbps
(802 11b ) 01 54 Mbps (802 1 I a) datd rate 01 w~thp ~oc'r cts that contatn both bands (dual
band), so the networks can provlde leal-woild perforn~ances ~rnlldto~ the bas~clO BaseT
wired Etflelnel netwolks used In inany ofiices
Refemng now to FIG 25, theie 1s illust~ateda schematzc block dlagrdm of m
exemplary computlng envnonment 2500 m accordance with the plesent invention The
system 2500 Includes one 01 moie cl~ent(s2) 502 The cllent(s) 2502 can be hardware
and/oi software (e g , theads, processes, computing devlces) The cllent(s) 2502 can
house cookle(s) andlot assocldted contextudl lnfonnatlon by employing the present
mvention, fol exmple The system 2500 also includ~ one or moie seivei(s) 2504 The
serva(s) 2504 can also be hadwale andiol software (e g , threads, processes, computlng
devrces) The servers 2504 can house th~eadtso per foim tmsforrnatrons by en~ploylng
the present tnvenhon, fol example One poss~blec ornmunlcatlon between a dlent 2502
and a server 2504 cnn be In Ihe foim of a data pdcket adapted to be transm~ttedb etween
two or more computa ptocesses The data packet may ~ncludea cook~ea nd/or
associated contextual informdt~onf,o i exampIe The system 2500 lncludes
communicat~on fiamewolk 2506 (e g , a global cornmunlcatlon netwolk such as the
Internet) thdt can be employed to fdcllitate communlcc' Ions between the cllent(s) 2502
and the sewel(s) 2504
1
Communications can be fac~litatedv ln a wiled (includmg opt~calf iber) andlor
> -
w~relesste chnology The c\~ent(s2)5 02 ale ope~abvelyc onnected to one or more client
data sto~e(s2) 508 that cdn be employed to store ~nformdt~olonc al to the client(s) 2502
(e g ,c oo kle(s) andfor assocd~t ed contextual infonnat~on) S ~m~larltyh,e server(s) 2504
are ope~at~vecloyn nected to one 01 mole sewer data stoie(s) 2510 that can be employed
to st01e ~nfolmat~olonc al to the seivels 2504
What has been described above Includes examples of the present invent~on It IS,
of couae, not posslble to descnbe eveiy conceivdble L ~mbmahono f components or
methodologies fot purposes of descilbing the p~esentn ventlon, but one of oldlnary skill
rn the art may lecognlze that many furthei comb~natlonsa nd peimutatlons of the present
~nventiona le possible Accoldmgly, the ytesent mvention 1s Intended to emblace all
such altelatlons, modifications and vamtlons that fdll wth~nth e splnt and scope of the
appended claims Fuitheimoie, to the extent thdt the term "lncludes" IS used In elther the
detailed desc~~ptio01n t he cldims, such term IS Intended to be tnclus~veIn a manner
slrnilar to the term "compns~ng"as "compnsrng" is ~nte~yietewdh en employed as a
bdnsitlonal wold In a cla~m

What is claimed is:
1. A system that facilitates the generation of an exploded view diagram,
comprising, an authoring component that ttceives an image or set of images and
facilitates processing of the image or set of images Into the exploded view diagram for
user interaction therewith.
2. The system of claim 1, the authoiing component allows for layering in the
exploded view diagrani that indicates how parts of an object in the image or set ofimages
expand, collapse, and occlude one another.
3. The system of claim 1, the authoring component is semi-automatic such
that some user intervention is accommodated.
4. The system of claim 1, the image or set of images that are an input to the
authoring component ~ I 2ED .
, 5 . The system of claim 1, the autho~ingc omponent facilitates processing an
object ofthe image or set of images into at least one of parts, stacks, and fragments.
6. The system of claim 1, the authoiing component facilitates assigning a
depth value to at least one part in the exploded vieu, diagram.
7. The system of claim 1, the authoring companent facilitates layering of
parts by allowing the user to assign a depth value to each fragment of a part.
8. The system of claim 1, the authoring component facilitates associating a
plurality of pals with a stack, which stack defines how the pats are allowed to move in
~aelationto one another.
9. The system of claim 8, the stack shaurs acornmon explosion axis.
10. The system of claim 1, the authoring component facilitates the generation
of a stack tree.
1 1. The system of claim I , the authoring component facilitates associating a
plu~alityo f pat% with a stack, which stack stores an initial position parameter, a cuimt
offiet parameter, and a maximum offset y arameter for each constituent part.
12. The system of claim 1 1, the autho~ingc omponent facilitates associating a
plurality of parts with a stack, where each pal of the plurality of parts depends on a
position of a predecessor patt.
13. The system of claim I, the authoring component facilitates the creation of
a stack by connecting related palls via a he-fo~mst roke.
14. The system of claim 13, the stack preselves an order of the related pa~ts
associated with hef ree-fo~ms troke. 15. The system of claim I, fullher comp~isinga cutting tool that allows a user
to define individual parls of an object in the image or set of images.
16. The system of claim 1, further compt-ising an annotation tool that allows
annotation of a part with a label, and specification of a guideline between any two Pam.
17. The system of claim 1, further comp:-'ing a viewing component that
dynamically filters diagram information in response to a direct user action.
18. The system of claim I, the viewing component facilitates user interaction
that is at least one of,
direct, by allowing the user to cause An object or part in the exploded view
diagram to be dragged, resized, or drawn with an input device that includes at least one of
a mouse, a key-based device, and voice control; and
indirect, by allowing the user to manipulate an object via at least one of a
search interface and animated expand/collapse.
19. A system that facilitates user interaction with an exploded view diagram,
comprising, a viewing component that dynamically filters diagram information in
response to the user interaction.
20. The system of claim 19, the viewing component accommodates the direct
user interaction by allowing the user to directly expand and collapse the exploded view
diagram, and indirect user interaction by allowing the user to search for individual pat-ts
of the exploded view diagram.
21. The system o f claim 19, the exploded view diagram is a 2.5D layer-based
representation of an input 2D image or set of images in at least one of an orthogonal view
and a perspective view.
22. The system of claim 19, the viewing component facilitates the
presentation of animation of at least one of expansion and collapse of the exploded view
diagram in response to an input device signal.
23. The system of claim 19, the viewing component facilitates the
presentation of labels and guidelines whose anchor points are unoccluded by other parts.
24. A system that facilitates the creation of an interactive exploded view
diagram from an image or set of images, complising:
an autho~ingc omponent that receives the image or set of images and
facilitates processing of the image or set of images into the exploded view diagrm; and
a viewing component that facilitales dynamic filtering of diagram
info~mationo fthe exploded view diagram associat 1w ith at least one of di~ccatn d
indirect user interaction.
25. The system of claim 24, the exploded view diagram is a 2.5D layer-based
representation of tlle image or set of images.
26. A computer readable medium having stored thereon computer executable
inst~uctionsfo r casrying out the system of claim 24.
27. A computer that employs the systerr JF claim 24.
28. A system that facilitates the creation of an interactive exploded view
diagram from an image or set of images, comprising:
an authoring component that receives a 2D image or set of images and
facilitates processing of the 2D image or set of images into an exploded view diagram,
the authoring component fullher comprising,
a cutting tool that allows a user to manually separate an object ofthe
2D image or set of images into constituent pals;
a stacking component that allows the user to associate the constituent
parts along an explosion axis using a ti-ee-for,-. stroke;
a layering component that allows the assignment of a depth
parameter h r each of the constituent parts; and
an annotation component that allows the addition of a label for any
part and the specification of a guideline between any two parts;
and
a viewing component that facilitates dynamic filtering of diagram
information of the exploded view diagram associated with direct and indirect user
inteiaction.
29. The system of claim 28, further comprising a fragmentation tool that
facilitates fragmenting palls that interlock.
30. The sysEm of claim 28, further comprising a depth tool that at least one of
automatically infers the depth parameter for assignment to one of the constituent parts
and pmmpts the user to assign lhe depth parameter to the part.
3 1. The system of claim 28, the authorin. component automatically calculates
and assigns depth values outward fiom a root pal, in a single pass.
32. The system of claim 28, the autholing component utilizes a constraint
propagation algorithm to automatical1y choose the depth parameter ofa part to ensure
that the part is pl-operly layend relative to other pa~ts.
33. The system ofclaim 28, h-ther cornpising an annotation tool that allo~vs
annotation of a part with a Iabel, and specification cC a guideline between any two parts.
34. The system of claim 28, fu~thecro mprising a classifier that automates a
feature by making an inference based on one or more data associated with at least one of
the authoring component and the viewing component
35. A computer-readable medium having computer-executable instructions for
perfotming a method of generating an interactive exploded view diagram fiom an image
or set of images, the method compiising:
receiving ihe image or set of images 'hat contains an object;
segmenting the object into pa*; 'I
organizing the parts into a stack;
reducing at least one of the parts into fragments;
layering each part; and
presenting the image as the exploded view diagram,
36. The method of claim 35, the act of organizing is yafo~medw ith a
free-form stroke.
37. The method of claim 35, fui-ther comprising at least one ofthe acts of:
animating at least one of expansion and collapse of the exploded view
diagram with an input device signal; and
dinectly manipulating a poltion of the exploded view diagram via
constrained direct manipulation.
38. The method of claim 35, further comprising at least one of the acts of:
searching for a hidden part by inputting the part name into a search
engine;
dir-ectly manipulating a portion of the stack of parts by selecting one of the
parts of the stack; and .
annotating a pal? with at least one of label and a guideline between any
two parts.
39. The method of claim 35, further comprising at least one of the acts of
automatically exposing a hidden part with the viewing component in
response to initiating a search of the hidden part;
manually resolving a violation during the act of fragmenting; and
providing manual interaction and automatic processing.
40. The method of claim 35, further con.,irising at least one of the acts of:
traversing a stack hierxrchy in a topological order,
successively computing and updating a position of each part based on a
predecessor part and cument offset.
4 1. The method of claim 35, runher comprising rendering labels and
guideIines whose anchor points are unoccluded by other parts.
42. A system that facilitates the amtion of an exploded view diagmm from an
image, comprising:
means for receiving the image that contains an object;
means for segmenting the object into parts;
means for organizing the pa* into a stack;
means for reducing at least one of the parts into fragments;
means for layering each pan
means for labeling each part;
means for adding guidelines betwea any two parts; and
means for presenting the image as an exploded view of a 2.5 D intpctive
layer-based diagram. -