US20190339848A1 - Bi-directional search and sorting - Google Patents

Bi-directional search and sorting Download PDF

Info

Publication number
US20190339848A1
US20190339848A1 US16/513,448 US201916513448A US2019339848A1 US 20190339848 A1 US20190339848 A1 US 20190339848A1 US 201916513448 A US201916513448 A US 201916513448A US 2019339848 A1 US2019339848 A1 US 2019339848A1
Authority
US
United States
Prior art keywords
objects
computer
search
list
selecting
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
US16/513,448
Inventor
Jay Allan Behr
Aaron D. Munter
Michael William Lollar
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Autodesk Inc
Original Assignee
Autodesk Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Autodesk Inc filed Critical Autodesk Inc
Priority to US16/513,448 priority Critical patent/US20190339848A1/en
Assigned to AUTODESK, INC. reassignment AUTODESK, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: BEHR, JAY ALLAN, LOLLAR, MICHAEL WILLIAM, MUNTER, AARON D.
Publication of US20190339848A1 publication Critical patent/US20190339848A1/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/04845Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/40Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
    • G06F16/44Browsing; Visualisation therefor
    • G06F16/444Spatial browsing, e.g. 2D maps, 3D or virtual spaces
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/04815Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/0482Interaction with lists of selectable items, e.g. menus
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/048Indexing scheme relating to G06F3/048
    • G06F2203/04806Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/04842Selection of displayed objects or displayed text elements

Definitions

  • the present invention relates generally to three-dimension (3D) modeling applications, and in particular, to a method, apparatus, and article of manufacture for interacting with and searching a 3D model.
  • Embodiments of the invention provide the ability for a user to easily/visually identify 3D objects/data that the user is searching for. Further, embodiments of the invention provide the ability to utilize found objects as a means for initiating or refining a query, and quickly associating a textual and visual representation of data, with the ability to toggle between views.
  • FIG. 1 is an exemplary hardware and software environment 100 used to implement one or more embodiments of the invention
  • FIG. 2 schematically illustrates a typical distributed computer system using a network to connect client computers to server computers in accordance with one or more embodiments of the invention
  • FIG. 3 is a block diagram that illustrates the components of the graphics program (of FIG. 1 ) in accordance with one or more embodiments of the invention
  • FIG. 4 is a block diagram that illustrates the structure of an object list maintained by the 3D databases in accordance with one or more embodiments of the invention
  • FIG. 5 illustrates the selection of an item/object in a list/palette view and the association of such a selected item/object with the object's location in the 3D model in accordance with one or more embodiments of the invention
  • FIG. 6 illustrates a resulting view once a deep zoom operation is performed on a selected item in accordance with one or more embodiments of the invention
  • FIG. 7 illustrates the logical flow for performing an automatic zoom operation in accordance with one or more embodiments of the invention
  • FIG. 8 illustrates the multiple selection of items in accordance with one or more embodiments of the invention.
  • FIG. 9 illustrates the narrowed search based on the addition of the selected properties from FIG. 8 in accordance with one or more embodiments of the invention.
  • FIG. 10 illustrates the logical flow for quickly amending search queries in accordance with one or more embodiments of the invention
  • FIG. 11 illustrates the different options available from the context of an item the user has navigated to in accordance with one or more embodiments of the invention
  • FIG. 12 illustrates search results conducted as a function of a selected object in accordance with one or more embodiments of the invention
  • FIG. 13 illustrates the logical flow for performing searches as a function of an object in accordance with one or more embodiments of the invention
  • FIG. 14 illustrates a comparison view of multiple selected elements/items/parts in accordance with one or more embodiments of the invention
  • FIG. 15 illustrates the logical flow for interacting with multiple objects via rotationally synchronized viewports in accordance with one or more embodiments of the invention.
  • FIG. 16 illustrates the logical flow for conducting a search based on a change in search results in accordance with one or more embodiments of the invention.
  • FIG. 1 is an exemplary hardware and software environment 100 used to implement one or more embodiments of the invention.
  • the hardware and software environment includes a computer 102 and may include peripherals.
  • Computer 102 may be a user/client computer, server computer, or may be a database computer.
  • the computer 102 comprises a general purpose hardware processor 104 A and/or a special purpose hardware processor 104 B (hereinafter alternatively collectively referred to as processor 104 ) and a memory 106 , such as random access memory (RAM).
  • processor 104 a general purpose hardware processor 104 A and/or a special purpose hardware processor 104 B (hereinafter alternatively collectively referred to as processor 104 ) and a memory 106 , such as random access memory (RAM).
  • RAM random access memory
  • the computer 102 may be coupled to, and/or integrated with, other devices, including input/output (I/O) devices such as a keyboard 114 , a cursor control device 116 (e.g., a mouse, a pointing device, pen and tablet, touch screen, multi-touch device, etc.) and a printer 128 .
  • I/O input/output
  • computer 102 may be coupled to, or may comprise, a portable or media viewing/listening device 132 (e.g., an MP3 player, iPodTM, NookTM, portable digital video player, cellular device, personal digital assistant, etc.).
  • the computer 102 may comprise a multi-touch device, mobile phone, gaming system, internet enabled television, television set top box, or other internet enabled device executing on various platforms and operating systems.
  • the computer 102 operates by the general purpose processor 104 A performing instructions defined by the computer program 110 under control of an operating system 108 .
  • the computer program 110 and/or the operating system 108 may be stored in the memory 106 and may interface with the user and/or other devices to accept input and commands and, based on such input and commands and the instructions defined by the computer program 110 and operating system 108 , to provide output and results.
  • Output/results may be presented on the display 122 or provided to another device for presentation or further processing or action.
  • the display 122 comprises a liquid crystal display (LCD) having a plurality of separately addressable liquid crystals.
  • the display 122 may comprise a light emitting diode (LED) display having clusters of red, green and blue diodes driven together to form full-color pixels.
  • Each liquid crystal or pixel of the display 122 changes to an opaque or translucent state to form a part of the image on the display in response to the data or information generated by the processor 104 from the application of the instructions of the computer program 110 and/or operating system 108 to the input and commands.
  • the image may be provided through a graphical user interface (GUI) module 118 .
  • GUI graphical user interface
  • the GUI module 118 is depicted as a separate module, the instructions performing the GUI functions can be resident or distributed in the operating system 108 , the computer program 110 , or implemented with special purpose memory and processors.
  • the display 122 is integrated with/into the computer 102 and comprises a multi-touch device having a touch sensing surface (e.g., track pod or touch screen) with the ability to recognize the presence of two or more points of contact with the surface.
  • multi-touch devices include mobile devices (e.g., iPhoneTM, Nexus STM, DroidTM devices, etc.), tablet computers (e.g., iPadTM, HP TouchpadTM), portable/handheld game/music/video player/console devices (e.g., iPod TouchTM, MP3 players, Nintendo 3DSTM, PlayStation PortableTM, etc.), touch tables, and walls (e.g., where an image is projected through acrylic and/or glass, and the image is then backlit with LEDs).
  • mobile devices e.g., iPhoneTM, Nexus STM, DroidTM devices, etc.
  • tablet computers e.g., iPadTM, HP TouchpadTM
  • portable/handheld game/music/video player/console devices e.g., iPod TouchTM, MP3 players, Nintendo 3
  • Some or all of the operations performed by the computer 102 according to the computer program 110 instructions may be implemented in a special purpose processor 104 B.
  • the some or all of the computer program 110 instructions may be implemented via firmware instructions stored in a read only memory (ROM), a programmable read only memory (PROM) or flash memory within the special purpose processor 104 B or in memory 106 .
  • the special purpose processor 104 B may also be hardwired through circuit design to perform some or all of the operations to implement the present invention.
  • the special purpose processor 104 B may be a hybrid processor, which includes dedicated circuitry for performing a subset of functions, and other circuits for performing more general functions such as responding to computer program 110 instructions.
  • the special purpose processor 104 B is an application specific integrated circuit (ASIC).
  • ASIC application specific integrated circuit
  • the computer 102 may also implement a compiler 112 that allows an application or computer program 110 written in a programming language such as COBOL, Pascal, C++, FORTRAN, or other language to be translated into processor 104 readable code.
  • the compiler 112 may be an interpreter that executes instructions/source code directly, translates source code into an intermediate representation that is executed, or that executes stored precompiled code.
  • Such source code may be written in a variety of programming languages such as JavaTM, PerlTM, BasicTM, etc.
  • the application or computer program 110 accesses and manipulates data accepted from I/O devices and stored in the memory 106 of the computer 102 using the relationships and logic that were generated using the compiler 112 .
  • the computer 102 also optionally comprises an external communication device such as a modem, satellite link, Ethernet card, or other device for accepting input from, and providing output to, other computers 102 .
  • an external communication device such as a modem, satellite link, Ethernet card, or other device for accepting input from, and providing output to, other computers 102 .
  • instructions implementing the operating system 108 , the computer program 110 , and the compiler 112 are tangibly embodied in a non-transitory computer-readable medium, e.g., data storage device 120 , which could include one or more fixed or removable data storage devices, such as a zip drive, floppy disc drive 124 , hard drive, CD-ROM drive, tape drive, etc.
  • a non-transitory computer-readable medium e.g., data storage device 120 , which could include one or more fixed or removable data storage devices, such as a zip drive, floppy disc drive 124 , hard drive, CD-ROM drive, tape drive, etc.
  • the operating system 108 and the computer program 110 are comprised of computer program 110 instructions which, when accessed, read and executed by the computer 102 , cause the computer 102 to perform the steps necessary to implement and/or use the present invention or to load the program of instructions into a memory 106 , thus creating a special purpose data structure causing the computer 102 to operate as a specially programmed computer executing the method steps described herein.
  • Computer program 110 and/or operating instructions may also be tangibly embodied in memory 106 and/or data communications devices 130 , thereby making a computer program product or article of manufacture according to the invention.
  • the terms “article of manufacture,” “program storage device,” and “computer program product,” as used herein, are intended to encompass a computer program accessible from any computer readable device or media.
  • FIG. 2 schematically illustrates a typical distributed computer system 200 using a network 204 to connect client computers 202 to server computers 206 .
  • a typical combination of resources may include a network 204 comprising the Internet, LANs (local area networks), WANs (wide area networks), SNA (systems network architecture) networks, or the like, clients 202 that are personal computers or workstations (as set forth in FIG. 1 ), and servers 206 that are personal computers, workstations, minicomputers, or mainframes (as set forth in FIG. 1 ).
  • networks such as a cellular network (e.g., GSM [global system for mobile communications] or otherwise), a satellite based network, or any other type of network may be used to connect clients 202 and servers 206 in accordance with embodiments of the invention.
  • GSM global system for mobile communications
  • a network 204 such as the Internet connects clients 202 to server computers 206 .
  • Network 204 may utilize ethernet, coaxial cable, wireless communications, radio frequency (RF), etc. to connect and provide the communication between clients 202 and servers 206 .
  • Clients 202 may execute a client application or web browser and communicate with server computers 206 executing web servers 210 .
  • Such a web browser is typically a program such as MICROSOFT INTERNET EXPLORERTM, MOZILLA FIREFOXTM, OPERATM, APPLE SAFARITM, GOOGLE CHROMETM, etc.
  • the software executing on clients 202 may be downloaded from server computer 206 to client computers 202 and installed as a plug-in or ACTIVEXTM control of a web browser.
  • clients 202 may utilize ACTIVEXTM components/component object model (COM) or distributed COM (DCOM) components to provide a user interface on a display of client 202 .
  • the web server 210 is typically a program such as MICROSOFT'S INTERNET INFORMATION SERVERTM.
  • Web server 210 may host an Active Server Page (ASP) or Internet Server Application Programming Interface (ISAPI) application 212 , which may be executing scripts.
  • the scripts invoke objects that execute business logic (referred to as business objects).
  • the business objects then manipulate data in database 216 through a database management system (DBMS) 214 .
  • database 216 may be part of, or connected directly to, client 202 instead of communicating/obtaining the information from database 216 across network 204 .
  • DBMS database management system
  • client 216 may be part of, or connected directly to, client 202 instead of communicating/obtaining the information from database 216 across network 204 .
  • COM component object model
  • the scripts executing on web server 210 (and/or application 212 ) invoke COM objects that implement the business logic.
  • server 206 may utilize MICROSOFT'STM Transaction Server (MTS) to access required data stored in database 216 via an interface such as ADO (Active Data Objects), OLE DB (Object Linking and Embedding DataBase), or ODBC (Open DataBase Connectivity).
  • MTS Transaction Server
  • these components 200 - 216 all comprise logic and/or data that is embodied in/or retrievable from device, medium, signal, or carrier, e.g., a data storage device, a data communications device, a remote computer or device coupled to the computer via a network or via another data communications device, etc.
  • this logic and/or data when read, executed, and/or interpreted, results in the steps necessary to implement and/or use the present invention being performed.
  • computers 202 and 206 may be interchangeable and may further include thin client devices with limited or full processing capabilities, portable devices such as cell phones, notebook computers, pocket computers, multi-touch devices, and/or any other devices with suitable processing, communication, and input/output capability.
  • computers 202 and 206 may be used with computers 202 and 206 .
  • FIG. 3 is a block diagram that illustrates the components of the graphics program 118 (of FIG. 1 ) in accordance with one or more embodiments of the invention.
  • There are three main components to the graphics program 118 including: a Graphical User Interface (GUI) 302 , an Image Engine (IME) 304 , and a DataBase (DB) 306 (e.g., the database 216 of FIG. 2 ) for storing objects in Drawing (DWG) files 308 .
  • GUI Graphical User Interface
  • IME Image Engine
  • DB DataBase
  • DWG Drawing
  • the Graphical User Interface 302 displays information to the operator and provides the functionality for the operator's interaction with the graphics program 118 .
  • the Image Engine 304 processes the DWG files 308 and delivers the resulting graphics to the monitor 122 for display.
  • the Image Engine 304 provides a complete application programming interface (API) that allows other computer programs to interface to the graphics program 118 as needed.
  • API application programming interface
  • the Database 306 is comprised of two separate types of databases: (1) a 3D database 310 known as the “3D world space” that stores 3D information; and (2) one or more 2D databases 312 known as the “2D view ports” that stores 2D information derived from the 3D information.
  • FIG. 4 is a block diagram that illustrates the structure of an object list 400 maintained by the 3D databases 310 in accordance with one or more embodiments of the invention.
  • the object list 400 is usually comprised of a doubly linked list having a list head 402 and one or more objects 404 interconnected by edges 406 , although other structures may be used as well.
  • an object 404 may be a member of multiple object lists 400 in the 3D databases 310 .
  • Such an object list 400 may represent the structure maintained by a 3D modeling system and/or computer aided drafting (CAD) system.
  • objects 404 may represent geometric objects, parts of an assembly model, a collection of points in 3D space (that may or may not be connected by various geometric entities), etc.
  • Embodiments of the invention may be implemented in a solid modeling application, a CAD application, or any other computer graphics application capable of processing 3D objects.
  • Embodiments of the invention provide the capability to interact with and search a 3D model for one or more 3D objects.
  • a design phase of a project e.g., a hospital
  • an architect may need to find all of the instances of swing doors across the project and ensure that such doors are replaced with consistent choices.
  • Prior art systems fail to provide such capability.
  • Various techniques and methods may be used to enable such features.
  • a list palette may be displayed that contains the search results.
  • Such a list palette may be placed anywhere desired within a graphical user interface.
  • the list palette may be placed off to the side in a 3D viewer (viewport) of a 3D model and may also show an icon/glyph/image of the 3D objects/geometry in 3D.
  • Embodiments of the invention provide the ability to coordinate between the selection in the list view and a selection indication in the viewport.
  • FIG. 5 illustrates the selection of an item/object in a list/palette view and the association of such a selected item/object with the object's location in the 3D model in accordance with one or more embodiments of the invention.
  • the 3D model 502 is displayed in viewer 500 .
  • the search window 504 indicates that a search for doors has been conducted and the results of the search are displayed in list view/palette 506 .
  • the location of that item is highlighted 510 in the viewer 500 .
  • all instances of the selected design item/object 508 may be highlighted 510 in the viewer 500 (i.e., on model 502 ).
  • properties 512 for the selected item 508 may be displayed within the list/palette view 506 .
  • Selection of a different item/object from list 506 may change the view (e.g., zoom, angle, etc.) to the appropriate view for that item.
  • the user may also have the option of viewing any particular item closer.
  • the user can opt to perform a deep zoom of a particular item by selecting an appropriate function (e.g., by selecting the magnifying glass 514 ).
  • a deep zoom operation not only reorients the model but also the zoom of the selected item 508 .
  • FIG. 6 illustrates a resulting view once a deep zoom operation is performed on a selected item in accordance with one or more embodiments of the invention.
  • the user has selected the magnifying glass 514 associated with “Door 2.”
  • the resulting view in the view window 500 illustrates a zoomed view 602 of “Door 2” within viewer 500 .
  • the model 502 has been rotated and zoomed to the point that the selected item (i.e., “Door 2”) consumes a majority of the viewport 500 .
  • any occlusion from the reoriented view of the selected object is removed such that the entirety of the selected object can be seen in the viewport 500 .
  • Such a removal of occlusions does not actually edit the model 502 .
  • all occluding objects are made transparent/translucent.
  • embodiments of the invention will automatically zoom to the object and remove all occluding objects.
  • FIG. 7 illustrates the logical flow for performing an automatic zoom operation in accordance with one or more embodiments of the invention.
  • a 3D model is acquired that consists of two or more 3D objects.
  • the 3D objects in the model are searched based on an identified search parameter.
  • the search parameter specifies a property value for a property of the 3D objects to be searched.
  • a list palette is displayed.
  • the list palette includes the results of the searching and is a list of each 3D object that has the identified property value/search parameter.
  • the list palette may also include a 3D geometric view of each found 3D object.
  • one of the 3D objects in the list palette (i.e., from the result list) is selected.
  • Such a selecting may comprise a single click of a user input device.
  • a zoomed view of the 3D model is displayed (e.g., in response to a single click of the user input device).
  • the zoomed view zooms in on the selected 3D object and removes any of the other 3D objects that occludes the selected 3D object without modifying the 3D model.
  • various functionality may be triggered from an item in a list of search results.
  • Such functionality includes a change in the view orientation and controlling the visual representation of objects in a 3D scene in order to isolate an instance in a larger, occluding model.
  • embodiments of the invention provide the ability to quickly identify a found element/object, and provide an optimal, in-context view of a selected item.
  • embodiments provide the ability to control a list of objects from the object(s) in the model.
  • Such a list can be reoriented (or repopulated) as a function of the orientation/zoom extents of a camera.
  • Result sets can also be filtered as a function of the zoom operation.
  • a result list can be scrolled based upon the orientation of the view.
  • Embodiments of the invention provide the ability to quickly amend search queries with property values based on multiple selected items/objects.
  • the multi-selection of objects in a 3D canvas shows common or differing properties and allows the further quick revision of the search query based upon the addition of one or more of these properties.
  • FIG. 8 illustrates the multiple selection of items in accordance with one or more embodiments of the invention.
  • the user has selected two items 802 A and 804 A from the list palette 506 .
  • the locations of the selected items 802 A and 804 A are highlighted 802 B and 804 B in the viewer 500 .
  • the selection of multiple items 802 and 804 enables various features including the display of properties in common/different between the selected items 802 and 804 .
  • FIG. 8 illustrates a display of shared properties 806 between objects 802 and 804 .
  • the parameters in the parameters pane 806 reduces to just the parameters that are in common to the two selected items 802 and 804 .
  • the user has the option to “show me all objects similar to these.” Such an option may be invoked by selecting the “+” button adjacent any parameter or the “All” text to select all of the parameters.
  • the search may be automatically updated based on the added properties/parameters. In other words, the search query/filter conducted on the search is updated based on the newly selected parameter/properties.
  • FIG. 8 the initial search for “doors” retrieved twenty-three (23) results. Two of the doors 802 A and 804 A were selected. Out of the shared properties, the “swing” and “steel” parameters were added as filters to the query/search.
  • FIG. 9 illustrates the narrowed search based on the addition of the selected properties from FIG. 8 in accordance with one or more embodiments of the invention. The name/value pairs added to the query are shown at 902 . The search results have been reduced/narrowed from 23 to 6 (as shown at 904 ). The user also has the ability to compare selected items (e.g., by selecting “compare” button 906 ).
  • FIG. 10 illustrates the logical flow for quickly amending search queries in accordance with one or more embodiments of the invention. Steps 1002 - 1006 are the same as steps 7002 - 7006 of FIG. 7 .
  • step 1008 two or more of the 3D objects in the list palette are selected.
  • a list of the shared properties (or different properties that distinguish/differentiate the selected items/objects) for the selected two or more objects are displayed.
  • one or more of the shared/differing properties are selected (e.g., using a single click of an icon such as a “+” button adjacent a parameter or “All” to select all shared/differing properties).
  • the search is updated/amended based on the selected properties.
  • the 3D objects in the model are searched based on the selected properties.
  • the updated/amended search may widen the scope of the search or may narrow the scope of the search.
  • objects/elements within a drawing/model may be searched based on attributes and a tolerance within those attributes. For example, a search may be conducted for doors of a particular dimension, as well as those doors within a 6′ height or width of the particular dimension (or from a particular manufacturer, or made from a certain material).
  • FIG. 11 illustrates the different options available from the context of an item the user has navigated to in accordance with one or more embodiments of the invention.
  • the user has navigated to door 1102 and right clicked the mouse button/cursor control device to activate the contextual radial menu 1104 for the instance of the door 1102 .
  • various icons/buttons/glyphs allow the user to perform various functions.
  • a search on the selected item 1102 may be conducted. Such a search may extend beyond a mere simple search. The search is based on the properties of the selected object 1102 (both within and between documents) as well as orthogonal searches by object properties within a document. Such a search may perform multiple searches with the results sorted in groups. For example, the activation (via a single click) of the search option may perform three or more different searches that may include one or more of the following:
  • FIG. 12 illustrates search results conducted as a function of a selected object in accordance with one or more embodiments of the invention.
  • the first row 1202 of the search results shows instances of the selected object within this model/design.
  • the second row 1204 of the search results shows instances of the selected object in a larger domain than the model (e.g., within the project, hub, or other larger construct).
  • three (3) instances of the selected door were found in the design (row 1202 ) and 23 instances of the door were found in the project (e.g., the city).
  • rows 1206 - 1210 illustrate an automatic search that is performed orthogonally on the properties of the selected object.
  • Row 1206 illustrates search results based on the manufacturer property (e.g., Trustle)
  • row 1208 illustrates search results based on the material property of the selected object (e.g., Steel)
  • row 1210 illustrates search results based on the fire rating property of the selected object (e.g., 2 hrs).
  • One or more properties of the selected object may be searched. Alternatively, all objects of the selected property may be searched. Determining which properties should be searched and/or displayed may be based on a variety of factors. Such factors are used to determine which properties are important to the user. Importance is often determined in two different ways: (1) based on prior usage/conduct; and (2) based on user role/permissions.
  • embodiments of the invention may longitudinally study how a user has used the search tools in the past over time. For example, if the user has only carried about materials in the past, default search parameters may be based on material. Accordingly, initial/early usage by the user of the search tool may commence by showing all properties/parameters. Over time, the search parameters may become more focused/customized based on the user's conduct (e.g., if always select materials, future searches may limit search results to such materials).
  • the user's role may be used to search/display particular parameters. For example, if the user role is identified as an industrial engineer, parameters such as the structure and materials may be searched/displayed while the manufacturer information may not be searched/displayed.
  • the domain searched may also be selected based on importance to the user (e.g., a domain smaller than the model such as the single floor the user has access to, or a project based domain, city based domain, etc.).
  • FIG. 13 illustrates the logical flow for performing searches as a function of an object in accordance with one or more embodiments of the invention.
  • a 3D model (that consists of two or more 3D objects) is displayed.
  • one of the 3D objects is selected.
  • a search option is selected based on the selected object (e.g., right clicking on an object and single clicking a search option).
  • the 3D model is searched for all instances of the selected 3D object.
  • a domain containing the 3D model may also be searched.
  • an orthogonal search of various properties may be searched. Such a search includes identifying/determining the properties of the selected 3D object to be searched, and searching the model for objects having the identified/determined properties. The properties to be searched may be identified/determined based on importance to the user as determined by prior user interaction and/or user role.
  • the results of the search are displayed (e.g., on a monitor/display device). All found instances of the selected 3D object are displayed together in a group(ing). Further, if the domain is also searched, the search results display instances found within the domain in a second group(ing). In addition, if an orthogonal search of identified properties is conducted, the search results may include a third group(ing) in which the objects having such properties are listed.
  • Embodiments of the invention provide the ability to select multiple found elements (e.g., from search results), select a comparison/viewing option, and display the selected elements/items/objects isolated and laid out in multiple viewports with synchronized camera angles/views. Once displayed in the comparison view mode, all of the viewports may be synchronized such that the rotation of one element/item/object rotates all of the other elements as well. However, such a rotation does not rotate the elements within the model but merely the individual isolated viewport containing the item. Accordingly, the data of the object itself is not modified.
  • a user may be interested in multiple parts of an assembled engine.
  • the user can maintain the assembled parts in the engine but view each of the parts in multiple orderless viewports such that each part may be individually examined and compared to other parts.
  • the camera angle with which each part is viewed may be synchronized such that when one part is rotated, the other parts are rotated as well.
  • the synchronization feature may be disabled for one or more of the individual viewports. When disabled, synchronized rotation may be not be performed and a single object may be rotated individually without rotating the other parts. Thereafter, synchronization can be enabled again such that any future rotations are synchronized.
  • FIG. 14 illustrates a comparison view of multiple selected elements/items/parts in accordance with one or more embodiments of the invention.
  • Each of the selected objects found in the search results are displayed separately in synchronized viewports 1402 A- 1402 F.
  • Such a search may be saved and the user may be notified when a new item/object has been added that matches the search criteria (e.g., via an activity stream, email, SMS message, etc.).
  • FIG. 15 illustrates the logical flow for interacting with multiple objects via rotationally synchronized viewports in accordance with one or more embodiments of the invention.
  • the 3D model (comprised of two or more 3D objects) is displayed.
  • step 1504 two or more of the 3D objects are selected.
  • each of the selected 3D objects is displayed in a separate viewport.
  • the view, in the separate viewport, of one of the selected 3D objects is rotated.
  • Such a rotation synchronously rotates, in the separate viewports, all of the selected 3D objects without modifying the 3D objects.
  • the synchronized rotation option may be turned off/disabled such that when one of the object is rotated, the other objects are not rotated.
  • embodiments of the invention provide the ability to isolate multiple 3D objects into an array of synchronized viewports for visual comparison.
  • Objects can be selected from a textual list (e.g., a list of search results) or from multiple selected objects in a 3D scene.
  • One trigger action isolates the objects each in their own viewport.
  • Each viewport is controlled synchronously from one camera interface (e.g., a viewcube).
  • the view angle and visual representation of all of the viewports can be controlled together (or separately).
  • This enables very detailed and dynamic visual comparison of 3D graphical elements.
  • Such capability may be viewed as each part is being viewed with a separate camera, wherein when one camera moves, all of the other cameras are moved simultaneously/synchronously with respect to their respective object/part. However, the actual object is not being rotated within the model and that data for the part is not being modified.
  • Users may have the ability to search for particular elements/objects and save those searches. Searches may be conducted (automatically on a scheduled trigger or manually) on a regular basis (e.g., once a day/week) with differing search results. Embodiments of the invention may provide the capability to determine what caused the change/delta in the search results and utilize such a delta to conduct additional searches. Such an additional search may be associated with/related to the original search (e.g., as a child).
  • search A returns 5 results (e.g., 5 doors) for three weeks in a row. However, in week 4, search A only returns 2 results.
  • Embodiments of the invention determine what caused the difference in the search results (e.g., one door was changed to wood and the other three doors were walled up).
  • a new/amended search may then be conducted for all objects having that delta (e.g., all doors that were walled up and/or changed to wood).
  • a report may be generated that identifies what caused the delta and/or what other objects have the same/similar delta (e.g., within a threshold).
  • FIG. 16 illustrates the logical flow for conducting a search based on a change in search results in accordance with one or more embodiments of the invention.
  • the 3D model (comprising two or more 3D objects) is acquired.
  • a first search of the 3D objects is conducted.
  • the first search is based on an identified search parameter (or set of parameters) that specifies a property value for a property of the 3D objects.
  • a second search of the 3D object is conducted based on the same identified search parameters (i.e., the first search is repeated).
  • a difference in the results of the two searches is identified.
  • the cause for the difference in the search results is determined.
  • a cause may be a change in a property of one or more of the 3D objects (e.g., from a first value to a second value) (e.g., a change in a material of an element).
  • a third search is conducted of the 3D objects based on the determined cause.
  • the search may comprise searching for objects having a property whose value changed from a first value to a second value.
  • the search may be conducted for any objects whose had a change in a particular property (e.g., a manufacturer or material has changed, regardless of the particular value).
  • any type of computer such as a mainframe, minicomputer, or personal computer, or computer configuration, such as a timesharing mainframe, local area network, or standalone personal computer, could be used with the present invention.

Abstract

A method, apparatus, system, and computer program product provide the ability to search for and interact with three-dimensional (3D) objects in a 3D model/drawing. Searches may be conducted on a per object basis (based on properties of the object) and search queries may be amended based on shared/common attributes between objects found in an initial search. Differences in search results can be identified and the causality may be used to perform additional searches. Search results may be listed in a list view and a correspondence between objects in the list view and the modeling view may be visibly discernible. Further, selection of a single object from a search result may focus the view on such an object while removing any occluding objects. Multiple selected objects can be viewed in an array of rotationally synchronized viewports.

Description

    CROSS-REFERENCE TO RELATED APPLICATIONS
  • This application is a divisional application of U.S. patent application Ser. No. 14/811,622, filed on Jul. 28, 2015, with inventor(s) Jay Allan Behr, Aaron D. Munter, and Michael William Lollar, entitled “Bi-Directional Search and Sorting,” (attorney's docket number 30566.0517USU1) which application is hereby incorporated by reference herein, and which application claims the benefit under 35 U.S.C. Section 119(e) of the following U.S. provisional patent application(s), which is/are incorporated by reference herein: Provisional Application Ser. No. 62/032,382, filed on Aug. 1, 2014, by Jay Allan Behr, Aaron D. Munter, and Michael William Lollar, entitled “Bi-Directional Search and Sorting,” attorneys' docket number 30566.0517USP1.
  • BACKGROUND OF THE INVENTION 1. Field of the Invention
  • The present invention relates generally to three-dimension (3D) modeling applications, and in particular, to a method, apparatus, and article of manufacture for interacting with and searching a 3D model.
  • 2. Description of the Related Art
  • Many 3D modeling systems provide for and maintain 3D objects and 3D data in single and/or multiple files. However, prior art systems fail to easily and quickly provide efficient mechanisms for searching and finding particular objects/data.
  • SUMMARY OF THE INVENTION
  • Embodiments of the invention provide the ability for a user to easily/visually identify 3D objects/data that the user is searching for. Further, embodiments of the invention provide the ability to utilize found objects as a means for initiating or refining a query, and quickly associating a textual and visual representation of data, with the ability to toggle between views.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • Referring now to the drawings in which like reference numbers represent corresponding parts throughout:
  • FIG. 1 is an exemplary hardware and software environment 100 used to implement one or more embodiments of the invention;
  • FIG. 2 schematically illustrates a typical distributed computer system using a network to connect client computers to server computers in accordance with one or more embodiments of the invention;
  • FIG. 3 is a block diagram that illustrates the components of the graphics program (of FIG. 1) in accordance with one or more embodiments of the invention;
  • FIG. 4 is a block diagram that illustrates the structure of an object list maintained by the 3D databases in accordance with one or more embodiments of the invention;
  • FIG. 5 illustrates the selection of an item/object in a list/palette view and the association of such a selected item/object with the object's location in the 3D model in accordance with one or more embodiments of the invention;
  • FIG. 6 illustrates a resulting view once a deep zoom operation is performed on a selected item in accordance with one or more embodiments of the invention;
  • FIG. 7 illustrates the logical flow for performing an automatic zoom operation in accordance with one or more embodiments of the invention;
  • FIG. 8 illustrates the multiple selection of items in accordance with one or more embodiments of the invention;
  • FIG. 9 illustrates the narrowed search based on the addition of the selected properties from FIG. 8 in accordance with one or more embodiments of the invention;
  • FIG. 10 illustrates the logical flow for quickly amending search queries in accordance with one or more embodiments of the invention;
  • FIG. 11 illustrates the different options available from the context of an item the user has navigated to in accordance with one or more embodiments of the invention;
  • FIG. 12 illustrates search results conducted as a function of a selected object in accordance with one or more embodiments of the invention;
  • FIG. 13 illustrates the logical flow for performing searches as a function of an object in accordance with one or more embodiments of the invention;
  • FIG. 14 illustrates a comparison view of multiple selected elements/items/parts in accordance with one or more embodiments of the invention;
  • FIG. 15 illustrates the logical flow for interacting with multiple objects via rotationally synchronized viewports in accordance with one or more embodiments of the invention; and
  • FIG. 16 illustrates the logical flow for conducting a search based on a change in search results in accordance with one or more embodiments of the invention.
  • DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
  • In the following description, reference is made to the accompanying drawings which form a part hereof, and which is shown, by way of illustration, several embodiments of the present invention. It is understood that other embodiments may be utilized and structural changes may be made without departing from the scope of the present invention.
  • Overview
  • Embodiments of the invention provide various features including:
      • Bi-Directional Interaction Between List View and Model Views;
        • including deep zoom functionality with the removal of occlusions;
      • Search Query Amendments Based on Shared Properties;
      • Searching as a Function of a Selected Object;
        • including one-click orthogonal searching and increased scope;
      • Comparison View Mode (e.g., multiple camera/viewport synchronization; and
      • Reverse Search
  • Each of these features are described in detail herein.
  • Hardware Environment
  • FIG. 1 is an exemplary hardware and software environment 100 used to implement one or more embodiments of the invention. The hardware and software environment includes a computer 102 and may include peripherals. Computer 102 may be a user/client computer, server computer, or may be a database computer. The computer 102 comprises a general purpose hardware processor 104A and/or a special purpose hardware processor 104B (hereinafter alternatively collectively referred to as processor 104) and a memory 106, such as random access memory (RAM). The computer 102 may be coupled to, and/or integrated with, other devices, including input/output (I/O) devices such as a keyboard 114, a cursor control device 116 (e.g., a mouse, a pointing device, pen and tablet, touch screen, multi-touch device, etc.) and a printer 128. In one or more embodiments, computer 102 may be coupled to, or may comprise, a portable or media viewing/listening device 132 (e.g., an MP3 player, iPod™, Nook™, portable digital video player, cellular device, personal digital assistant, etc.). In yet another embodiment, the computer 102 may comprise a multi-touch device, mobile phone, gaming system, internet enabled television, television set top box, or other internet enabled device executing on various platforms and operating systems.
  • In one embodiment, the computer 102 operates by the general purpose processor 104A performing instructions defined by the computer program 110 under control of an operating system 108. The computer program 110 and/or the operating system 108 may be stored in the memory 106 and may interface with the user and/or other devices to accept input and commands and, based on such input and commands and the instructions defined by the computer program 110 and operating system 108, to provide output and results.
  • Output/results may be presented on the display 122 or provided to another device for presentation or further processing or action. In one embodiment, the display 122 comprises a liquid crystal display (LCD) having a plurality of separately addressable liquid crystals. Alternatively, the display 122 may comprise a light emitting diode (LED) display having clusters of red, green and blue diodes driven together to form full-color pixels. Each liquid crystal or pixel of the display 122 changes to an opaque or translucent state to form a part of the image on the display in response to the data or information generated by the processor 104 from the application of the instructions of the computer program 110 and/or operating system 108 to the input and commands. The image may be provided through a graphical user interface (GUI) module 118. Although the GUI module 118 is depicted as a separate module, the instructions performing the GUI functions can be resident or distributed in the operating system 108, the computer program 110, or implemented with special purpose memory and processors.
  • In one or more embodiments, the display 122 is integrated with/into the computer 102 and comprises a multi-touch device having a touch sensing surface (e.g., track pod or touch screen) with the ability to recognize the presence of two or more points of contact with the surface. Examples of multi-touch devices include mobile devices (e.g., iPhone™, Nexus S™, Droid™ devices, etc.), tablet computers (e.g., iPad™, HP Touchpad™), portable/handheld game/music/video player/console devices (e.g., iPod Touch™, MP3 players, Nintendo 3DS™, PlayStation Portable™, etc.), touch tables, and walls (e.g., where an image is projected through acrylic and/or glass, and the image is then backlit with LEDs).
  • Some or all of the operations performed by the computer 102 according to the computer program 110 instructions may be implemented in a special purpose processor 104B. In this embodiment, the some or all of the computer program 110 instructions may be implemented via firmware instructions stored in a read only memory (ROM), a programmable read only memory (PROM) or flash memory within the special purpose processor 104B or in memory 106. The special purpose processor 104B may also be hardwired through circuit design to perform some or all of the operations to implement the present invention. Further, the special purpose processor 104B may be a hybrid processor, which includes dedicated circuitry for performing a subset of functions, and other circuits for performing more general functions such as responding to computer program 110 instructions. In one embodiment, the special purpose processor 104B is an application specific integrated circuit (ASIC).
  • The computer 102 may also implement a compiler 112 that allows an application or computer program 110 written in a programming language such as COBOL, Pascal, C++, FORTRAN, or other language to be translated into processor 104 readable code. Alternatively, the compiler 112 may be an interpreter that executes instructions/source code directly, translates source code into an intermediate representation that is executed, or that executes stored precompiled code. Such source code may be written in a variety of programming languages such as Java™, Perl™, Basic™, etc. After completion, the application or computer program 110 accesses and manipulates data accepted from I/O devices and stored in the memory 106 of the computer 102 using the relationships and logic that were generated using the compiler 112.
  • The computer 102 also optionally comprises an external communication device such as a modem, satellite link, Ethernet card, or other device for accepting input from, and providing output to, other computers 102.
  • In one embodiment, instructions implementing the operating system 108, the computer program 110, and the compiler 112 are tangibly embodied in a non-transitory computer-readable medium, e.g., data storage device 120, which could include one or more fixed or removable data storage devices, such as a zip drive, floppy disc drive 124, hard drive, CD-ROM drive, tape drive, etc. Further, the operating system 108 and the computer program 110 are comprised of computer program 110 instructions which, when accessed, read and executed by the computer 102, cause the computer 102 to perform the steps necessary to implement and/or use the present invention or to load the program of instructions into a memory 106, thus creating a special purpose data structure causing the computer 102 to operate as a specially programmed computer executing the method steps described herein. Computer program 110 and/or operating instructions may also be tangibly embodied in memory 106 and/or data communications devices 130, thereby making a computer program product or article of manufacture according to the invention. As such, the terms “article of manufacture,” “program storage device,” and “computer program product,” as used herein, are intended to encompass a computer program accessible from any computer readable device or media.
  • Of course, those skilled in the art will recognize that any combination of the above components, or any number of different components, peripherals, and other devices, may be used with the computer 102.
  • FIG. 2 schematically illustrates a typical distributed computer system 200 using a network 204 to connect client computers 202 to server computers 206. A typical combination of resources may include a network 204 comprising the Internet, LANs (local area networks), WANs (wide area networks), SNA (systems network architecture) networks, or the like, clients 202 that are personal computers or workstations (as set forth in FIG. 1), and servers 206 that are personal computers, workstations, minicomputers, or mainframes (as set forth in FIG. 1). However, it may be noted that different networks such as a cellular network (e.g., GSM [global system for mobile communications] or otherwise), a satellite based network, or any other type of network may be used to connect clients 202 and servers 206 in accordance with embodiments of the invention.
  • A network 204 such as the Internet connects clients 202 to server computers 206. Network 204 may utilize ethernet, coaxial cable, wireless communications, radio frequency (RF), etc. to connect and provide the communication between clients 202 and servers 206. Clients 202 may execute a client application or web browser and communicate with server computers 206 executing web servers 210. Such a web browser is typically a program such as MICROSOFT INTERNET EXPLORER™, MOZILLA FIREFOX™, OPERA™, APPLE SAFARI™, GOOGLE CHROME™, etc. Further, the software executing on clients 202 may be downloaded from server computer 206 to client computers 202 and installed as a plug-in or ACTIVEX™ control of a web browser. Accordingly, clients 202 may utilize ACTIVEX™ components/component object model (COM) or distributed COM (DCOM) components to provide a user interface on a display of client 202. The web server 210 is typically a program such as MICROSOFT'S INTERNET INFORMATION SERVER™.
  • Web server 210 may host an Active Server Page (ASP) or Internet Server Application Programming Interface (ISAPI) application 212, which may be executing scripts. The scripts invoke objects that execute business logic (referred to as business objects). The business objects then manipulate data in database 216 through a database management system (DBMS) 214. Alternatively, database 216 may be part of, or connected directly to, client 202 instead of communicating/obtaining the information from database 216 across network 204. When a developer encapsulates the business functionality into objects, the system may be referred to as a component object model (COM) system. Accordingly, the scripts executing on web server 210 (and/or application 212) invoke COM objects that implement the business logic. Further, server 206 may utilize MICROSOFT'S™ Transaction Server (MTS) to access required data stored in database 216 via an interface such as ADO (Active Data Objects), OLE DB (Object Linking and Embedding DataBase), or ODBC (Open DataBase Connectivity).
  • Generally, these components 200-216 all comprise logic and/or data that is embodied in/or retrievable from device, medium, signal, or carrier, e.g., a data storage device, a data communications device, a remote computer or device coupled to the computer via a network or via another data communications device, etc. Moreover, this logic and/or data, when read, executed, and/or interpreted, results in the steps necessary to implement and/or use the present invention being performed.
  • Although the terms “user computer”, “client computer”, and/or “server computer” are referred to herein, it is understood that such computers 202 and 206 may be interchangeable and may further include thin client devices with limited or full processing capabilities, portable devices such as cell phones, notebook computers, pocket computers, multi-touch devices, and/or any other devices with suitable processing, communication, and input/output capability.
  • Of course, those skilled in the art will recognize that any combination of the above components, or any number of different components, peripherals, and other devices, may be used with computers 202 and 206.
  • Computer-Implemented Graphics Program
  • FIG. 3 is a block diagram that illustrates the components of the graphics program 118 (of FIG. 1) in accordance with one or more embodiments of the invention. There are three main components to the graphics program 118, including: a Graphical User Interface (GUI) 302, an Image Engine (IME) 304, and a DataBase (DB) 306 (e.g., the database 216 of FIG. 2) for storing objects in Drawing (DWG) files 308.
  • The Graphical User Interface 302 displays information to the operator and provides the functionality for the operator's interaction with the graphics program 118.
  • The Image Engine 304 processes the DWG files 308 and delivers the resulting graphics to the monitor 122 for display. In one or more embodiments, the Image Engine 304 provides a complete application programming interface (API) that allows other computer programs to interface to the graphics program 118 as needed.
  • The Database 306 is comprised of two separate types of databases: (1) a 3D database 310 known as the “3D world space” that stores 3D information; and (2) one or more 2D databases 312 known as the “2D view ports” that stores 2D information derived from the 3D information.
  • Object List
  • FIG. 4 is a block diagram that illustrates the structure of an object list 400 maintained by the 3D databases 310 in accordance with one or more embodiments of the invention. The object list 400 is usually comprised of a doubly linked list having a list head 402 and one or more objects 404 interconnected by edges 406, although other structures may be used as well. There may be any number of different object lists 400 maintained by the 3D databases 310. Moreover, an object 404 may be a member of multiple object lists 400 in the 3D databases 310.
  • Such an object list 400 may represent the structure maintained by a 3D modeling system and/or computer aided drafting (CAD) system. In this regard, objects 404 may represent geometric objects, parts of an assembly model, a collection of points in 3D space (that may or may not be connected by various geometric entities), etc. Embodiments of the invention may be implemented in a solid modeling application, a CAD application, or any other computer graphics application capable of processing 3D objects.
  • Software Embodiment Details
  • Embodiments of the invention provide the capability to interact with and search a 3D model for one or more 3D objects. As an example, in a design phase of a project (e.g., a hospital), an architect may need to find all of the instances of swing doors across the project and ensure that such doors are replaced with consistent choices. Prior art systems fail to provide such capability. Various techniques and methods may be used to enable such features.
  • Bi-Directional Interaction Between List and Model Views
  • When conducting a search of a 3D model, various 3D objects/geometry may be found. A list palette may be displayed that contains the search results. Such a list palette may be placed anywhere desired within a graphical user interface. For example, the list palette may be placed off to the side in a 3D viewer (viewport) of a 3D model and may also show an icon/glyph/image of the 3D objects/geometry in 3D. Embodiments of the invention provide the ability to coordinate between the selection in the list view and a selection indication in the viewport.
  • FIG. 5 illustrates the selection of an item/object in a list/palette view and the association of such a selected item/object with the object's location in the 3D model in accordance with one or more embodiments of the invention. The 3D model 502 is displayed in viewer 500. The search window 504 indicates that a search for doors has been conducted and the results of the search are displayed in list view/palette 506. Upon selection of an item/object 508 from the list view 506, the location of that item is highlighted 510 in the viewer 500. In one or more embodiments, all instances of the selected design item/object 508 may be highlighted 510 in the viewer 500 (i.e., on model 502). In addition, properties 512 for the selected item 508 may be displayed within the list/palette view 506.
  • Selection of a different item/object from list 506 may change the view (e.g., zoom, angle, etc.) to the appropriate view for that item. The user may also have the option of viewing any particular item closer. The user can opt to perform a deep zoom of a particular item by selecting an appropriate function (e.g., by selecting the magnifying glass 514). A deep zoom operation not only reorients the model but also the zoom of the selected item 508.
  • FIG. 6 illustrates a resulting view once a deep zoom operation is performed on a selected item in accordance with one or more embodiments of the invention. As illustrated, the user has selected the magnifying glass 514 associated with “Door 2.” The resulting view in the view window 500 illustrates a zoomed view 602 of “Door 2” within viewer 500. The model 502 has been rotated and zoomed to the point that the selected item (i.e., “Door 2”) consumes a majority of the viewport 500. In addition, any occlusion from the reoriented view of the selected object is removed such that the entirety of the selected object can be seen in the viewport 500. Such a removal of occlusions does not actually edit the model 502. Instead, within the viewport 500, all occluding objects are made transparent/translucent. Thus, based on the single selection of an object/zoom feature, embodiments of the invention will automatically zoom to the object and remove all occluding objects.
  • FIG. 7 illustrates the logical flow for performing an automatic zoom operation in accordance with one or more embodiments of the invention.
  • At step 702, a 3D model is acquired that consists of two or more 3D objects.
  • At step 704, the 3D objects in the model are searched based on an identified search parameter. The search parameter specifies a property value for a property of the 3D objects to be searched.
  • At step 706, a list palette is displayed. The list palette includes the results of the searching and is a list of each 3D object that has the identified property value/search parameter. The list palette may also include a 3D geometric view of each found 3D object.
  • At step 708, one of the 3D objects in the list palette (i.e., from the result list) is selected. Such a selecting may comprise a single click of a user input device.
  • At step 710, simultaneously with the list palette, a zoomed view of the 3D model is displayed (e.g., in response to a single click of the user input device). The zoomed view zooms in on the selected 3D object and removes any of the other 3D objects that occludes the selected 3D object without modifying the 3D model.
  • In view of the above, various functionality may be triggered from an item in a list of search results. Such functionality includes a change in the view orientation and controlling the visual representation of objects in a 3D scene in order to isolate an instance in a larger, occluding model. Thus, embodiments of the invention provide the ability to quickly identify a found element/object, and provide an optimal, in-context view of a selected item. In addition, embodiments provide the ability to control a list of objects from the object(s) in the model. Such a list can be reoriented (or repopulated) as a function of the orientation/zoom extents of a camera. Result sets can also be filtered as a function of the zoom operation. Alternatively, a result list can be scrolled based upon the orientation of the view.
  • Search Query Amendments Based on Shared Properties
  • Embodiments of the invention provide the ability to quickly amend search queries with property values based on multiple selected items/objects. The multi-selection of objects in a 3D canvas (or from a list) shows common or differing properties and allows the further quick revision of the search query based upon the addition of one or more of these properties.
  • FIG. 8 illustrates the multiple selection of items in accordance with one or more embodiments of the invention. In FIG. 8, the user has selected two items 802A and 804A from the list palette 506. The locations of the selected items 802A and 804A are highlighted 802B and 804B in the viewer 500. The selection of multiple items 802 and 804 enables various features including the display of properties in common/different between the selected items 802 and 804. FIG. 8 illustrates a display of shared properties 806 between objects 802 and 804. In other words, the parameters in the parameters pane 806 reduces to just the parameters that are in common to the two selected items 802 and 804.
  • Based on the shared properties 806, the user has the option to “show me all objects similar to these.” Such an option may be invoked by selecting the “+” button adjacent any parameter or the “All” text to select all of the parameters. Once the appropriate selection of parameters has been conducted, the search may be automatically updated based on the added properties/parameters. In other words, the search query/filter conducted on the search is updated based on the newly selected parameter/properties.
  • As an example, in FIG. 8, the initial search for “doors” retrieved twenty-three (23) results. Two of the doors 802A and 804A were selected. Out of the shared properties, the “swing” and “steel” parameters were added as filters to the query/search. FIG. 9 illustrates the narrowed search based on the addition of the selected properties from FIG. 8 in accordance with one or more embodiments of the invention. The name/value pairs added to the query are shown at 902. The search results have been reduced/narrowed from 23 to 6 (as shown at 904). The user also has the ability to compare selected items (e.g., by selecting “compare” button 906).
  • FIG. 10 illustrates the logical flow for quickly amending search queries in accordance with one or more embodiments of the invention. Steps 1002-1006 are the same as steps 7002-7006 of FIG. 7.
  • At step 1008, two or more of the 3D objects in the list palette are selected.
  • At step 1010, a list of the shared properties (or different properties that distinguish/differentiate the selected items/objects) for the selected two or more objects are displayed.
  • At step 1012, one or more of the shared/differing properties are selected (e.g., using a single click of an icon such as a “+” button adjacent a parameter or “All” to select all shared/differing properties).
  • At step 1014, the search is updated/amended based on the selected properties. In other words, the 3D objects in the model are searched based on the selected properties. The updated/amended search may widen the scope of the search or may narrow the scope of the search.
  • As an alternative, or in addition to the above, objects/elements within a drawing/model may be searched based on attributes and a tolerance within those attributes. For example, a search may be conducted for doors of a particular dimension, as well as those doors within a 6′ height or width of the particular dimension (or from a particular manufacturer, or made from a certain material).
  • Search as a Function of the Object
  • Just as users can navigate to an item from the context of a query, users can also query from the context of an item they have navigated to. FIG. 11 illustrates the different options available from the context of an item the user has navigated to in accordance with one or more embodiments of the invention. The user has navigated to door 1102 and right clicked the mouse button/cursor control device to activate the contextual radial menu 1104 for the instance of the door 1102. Based on the contextual radial menu 1104, various icons/buttons/glyphs allow the user to perform various functions.
  • By selecting the search icon 1106, a search on the selected item 1102 may be conducted. Such a search may extend beyond a mere simple search. The search is based on the properties of the selected object 1102 (both within and between documents) as well as orthogonal searches by object properties within a document. Such a search may perform multiple searches with the results sorted in groups. For example, the activation (via a single click) of the search option may perform three or more different searches that may include one or more of the following:
  • (1) for all instances of the selected object 1102 within the model;
  • (2) for all instances of the selected object 1102 within the larger domain that contains the model; and
  • (3) based on properties of the selected object 1102, search orthogonally of the properties that are important to the user (with such importance computed based on various methodologies).
  • FIG. 12 illustrates search results conducted as a function of a selected object in accordance with one or more embodiments of the invention. The first row 1202 of the search results shows instances of the selected object within this model/design. The second row 1204 of the search results shows instances of the selected object in a larger domain than the model (e.g., within the project, hub, or other larger construct). In the example illustrated in FIG. 12, three (3) instances of the selected door were found in the design (row 1202) and 23 instances of the door were found in the project (e.g., the city).
  • In addition, rows 1206-1210 illustrate an automatic search that is performed orthogonally on the properties of the selected object. Row 1206 illustrates search results based on the manufacturer property (e.g., Trustle), row 1208 illustrates search results based on the material property of the selected object (e.g., Steel), and row 1210 illustrates search results based on the fire rating property of the selected object (e.g., 2 hrs).
  • One or more properties of the selected object may be searched. Alternatively, all objects of the selected property may be searched. Determining which properties should be searched and/or displayed may be based on a variety of factors. Such factors are used to determine which properties are important to the user. Importance is often determined in two different ways: (1) based on prior usage/conduct; and (2) based on user role/permissions.
  • When determining importance based on prior usage/conduct, embodiments of the invention may longitudinally study how a user has used the search tools in the past over time. For example, if the user has only carried about materials in the past, default search parameters may be based on material. Accordingly, initial/early usage by the user of the search tool may commence by showing all properties/parameters. Over time, the search parameters may become more focused/customized based on the user's conduct (e.g., if always select materials, future searches may limit search results to such materials).
  • When determining importance based on role (e.g., the user's identification, role, title, permission level, user level authorization, etc.), the user's role may be used to search/display particular parameters. For example, if the user role is identified as an industrial engineer, parameters such as the structure and materials may be searched/displayed while the manufacturer information may not be searched/displayed.
  • Further to the above, in addition to selecting particular properties to be searched/displayed, the domain searched may also be selected based on importance to the user (e.g., a domain smaller than the model such as the single floor the user has access to, or a project based domain, city based domain, etc.).
  • FIG. 13 illustrates the logical flow for performing searches as a function of an object in accordance with one or more embodiments of the invention.
  • At step 1302, a 3D model (that consists of two or more 3D objects) is displayed.
  • At step 1304, one of the 3D objects is selected.
  • At step 1306, a search option is selected based on the selected object (e.g., right clicking on an object and single clicking a search option).
  • At step 1308, in response to the selection of the search option the 3D model is searched for all instances of the selected 3D object. In addition, in response to the selection of the search option, a domain containing the 3D model may also be searched. Also in response to the selection of the search option, an orthogonal search of various properties may be searched. Such a search includes identifying/determining the properties of the selected 3D object to be searched, and searching the model for objects having the identified/determined properties. The properties to be searched may be identified/determined based on importance to the user as determined by prior user interaction and/or user role.
  • At step 1310, the results of the search are displayed (e.g., on a monitor/display device). All found instances of the selected 3D object are displayed together in a group(ing). Further, if the domain is also searched, the search results display instances found within the domain in a second group(ing). In addition, if an orthogonal search of identified properties is conducted, the search results may include a third group(ing) in which the objects having such properties are listed.
  • Comparison View Mode
  • Embodiments of the invention provide the ability to select multiple found elements (e.g., from search results), select a comparison/viewing option, and display the selected elements/items/objects isolated and laid out in multiple viewports with synchronized camera angles/views. Once displayed in the comparison view mode, all of the viewports may be synchronized such that the rotation of one element/item/object rotates all of the other elements as well. However, such a rotation does not rotate the elements within the model but merely the individual isolated viewport containing the item. Accordingly, the data of the object itself is not modified.
  • As an example, a user may be interested in multiple parts of an assembled engine. By the selection of a single button, the user can maintain the assembled parts in the engine but view each of the parts in multiple orderless viewports such that each part may be individually examined and compared to other parts. The camera angle with which each part is viewed may be synchronized such that when one part is rotated, the other parts are rotated as well. Alternatively, the synchronization feature may be disabled for one or more of the individual viewports. When disabled, synchronized rotation may be not be performed and a single object may be rotated individually without rotating the other parts. Thereafter, synchronization can be enabled again such that any future rotations are synchronized.
  • FIG. 14 illustrates a comparison view of multiple selected elements/items/parts in accordance with one or more embodiments of the invention. Each of the selected objects found in the search results are displayed separately in synchronized viewports 1402A-1402F. Such a search may be saved and the user may be notified when a new item/object has been added that matches the search criteria (e.g., via an activity stream, email, SMS message, etc.).
  • FIG. 15 illustrates the logical flow for interacting with multiple objects via rotationally synchronized viewports in accordance with one or more embodiments of the invention.
  • At step 1502, the 3D model (comprised of two or more 3D objects) is displayed.
  • At step 1504, two or more of the 3D objects are selected.
  • At step 1506, each of the selected 3D objects is displayed in a separate viewport.
  • At step 1508, the view, in the separate viewport, of one of the selected 3D objects is rotated. Such a rotation, synchronously rotates, in the separate viewports, all of the selected 3D objects without modifying the 3D objects. Further, as described above, the synchronized rotation option may be turned off/disabled such that when one of the object is rotated, the other objects are not rotated.
  • In view of the above, embodiments of the invention provide the ability to isolate multiple 3D objects into an array of synchronized viewports for visual comparison. Objects can be selected from a textual list (e.g., a list of search results) or from multiple selected objects in a 3D scene. One trigger action isolates the objects each in their own viewport. Each viewport is controlled synchronously from one camera interface (e.g., a viewcube). The view angle and visual representation of all of the viewports can be controlled together (or separately). This enables very detailed and dynamic visual comparison of 3D graphical elements. Such capability may be viewed as each part is being viewed with a separate camera, wherein when one camera moves, all of the other cameras are moved simultaneously/synchronously with respect to their respective object/part. However, the actual object is not being rotated within the model and that data for the part is not being modified.
  • Reverse Search
  • Users may have the ability to search for particular elements/objects and save those searches. Searches may be conducted (automatically on a scheduled trigger or manually) on a regular basis (e.g., once a day/week) with differing search results. Embodiments of the invention may provide the capability to determine what caused the change/delta in the search results and utilize such a delta to conduct additional searches. Such an additional search may be associated with/related to the original search (e.g., as a child).
  • For example, suppose that search A returns 5 results (e.g., 5 doors) for three weeks in a row. However, in week 4, search A only returns 2 results. Embodiments of the invention determine what caused the difference in the search results (e.g., one door was changed to wood and the other three doors were walled up). A new/amended search may then be conducted for all objects having that delta (e.g., all doors that were walled up and/or changed to wood). Alternatively, or in addition, a report may be generated that identifies what caused the delta and/or what other objects have the same/similar delta (e.g., within a threshold).
  • FIG. 16 illustrates the logical flow for conducting a search based on a change in search results in accordance with one or more embodiments of the invention.
  • At step 1602, the 3D model (comprising two or more 3D objects) is acquired.
  • At step 1604, a first search of the 3D objects is conducted. The first search is based on an identified search parameter (or set of parameters) that specifies a property value for a property of the 3D objects.
  • At step 1606, a second search of the 3D object is conducted based on the same identified search parameters (i.e., the first search is repeated).
  • At step 1608, a difference in the results of the two searches is identified.
  • At step 1610, the cause for the difference in the search results is determined. Such a cause may be a change in a property of one or more of the 3D objects (e.g., from a first value to a second value) (e.g., a change in a material of an element).
  • At step 1612, a third search is conducted of the 3D objects based on the determined cause. For example, the search may comprise searching for objects having a property whose value changed from a first value to a second value. Alternatively, the search may be conducted for any objects whose had a change in a particular property (e.g., a manufacturer or material has changed, regardless of the particular value).
  • Conclusion
  • This concludes the description of the preferred embodiment of the invention. The following describes some alternative embodiments for accomplishing the present invention. For example, any type of computer, such as a mainframe, minicomputer, or personal computer, or computer configuration, such as a timesharing mainframe, local area network, or standalone personal computer, could be used with the present invention.
  • The foregoing description of the preferred embodiment of the invention has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form disclosed. Many modifications and variations are possible in light of the above teaching. It is intended that the scope of the invention be limited not by this detailed description, but rather by the claims appended hereto.

Claims (10)

What is claimed is:
1. A computer-implemented method for interacting with a three-dimensional (3D) model, comprising:
acquiring, in a computer, the 3D model comprised of two or more 3D objects;
searching, in the computer, the two or more 3D objects based on an identified search parameter, wherein the identified search parameter specifies a property value for a property of the two or more 3D objects;
displaying, on a display device via the computer, a list palette comprised of results of the searching, wherein the list palette comprises a list of each 3D object of the two or more 3D objects having the property value;
selecting, in the computer, two or more of the 3D objects in the list palette;
displaying, on the display device via the computer, a list of shared properties for the selected two or more objects;
selecting, in the computer, one or more of the shared properties; and
searching, in the computer, the two or more 3D objects in the model based on the selected shared properties.
2. The computer-implemented method of claim 1, wherein:
the selecting one or more shared properties comprises a single click of an icon.
3. The computer-implemented method of claim 1, wherein:
the selecting one or more shared properties widens a scope of the search.
4. The computer-implemented method of claim 1, wherein:
the selecting one or more shared properties narrows a scope of the search.
5. The computer-implemented method of claim 1, wherein:
the selecting one or more shared properties comprising selecting all of the one or more shared properties; and
a scope of the searching is narrowed based on the selected shared properties.
6. A computer-implemented system for interacting with a three-dimensional (3D) model, comprising:
(a) a computer having a memory;
(b) a processor executing on the computer;
(c) the memory storing a set of instructions, wherein the set of instructions, when executed by the processor, cause the processor to perform operations comprising:
acquiring the 3D model comprised of two or more 3D objects;
searching the two or more 3D objects based on an identified search parameter, wherein the identified search parameter specifies a property value for a property of the two or more 3D objects;
displaying a list palette comprised of results of the searching, wherein the list palette comprises a list of each 3D object of the two or more 3D objects having the property value;
selecting two or more of the 3D objects in the list palette;
displaying a list of shared properties for the selected two or more objects;
selecting one or more of the shared properties; and
searching the two or more 3D objects in the model based on the selected shared properties.
7. The computer-implemented system of claim 6, wherein:
the selecting one or more shared properties comprises a single click of an icon.
8. The computer-implemented system of claim 6, wherein:
the selecting one or more shared properties widens a scope of the search.
9. The computer-implemented system of claim 6, wherein:
the selecting one or more shared properties narrows a scope of the search.
10. The computer-implemented system of claim 6, wherein:
the selecting one or more shared properties comprises selecting all of the one or more shared properties; and
a scope of the searching is narrowed based on the selected shared properties.
US16/513,448 2014-08-01 2019-07-16 Bi-directional search and sorting Pending US20190339848A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US16/513,448 US20190339848A1 (en) 2014-08-01 2019-07-16 Bi-directional search and sorting

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US201462032382P 2014-08-01 2014-08-01
US14/811,622 US10402067B2 (en) 2014-08-01 2015-07-28 Bi-directional search and sorting
US16/513,448 US20190339848A1 (en) 2014-08-01 2019-07-16 Bi-directional search and sorting

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US14/811,622 Division US10402067B2 (en) 2014-08-01 2015-07-28 Bi-directional search and sorting

Publications (1)

Publication Number Publication Date
US20190339848A1 true US20190339848A1 (en) 2019-11-07

Family

ID=55180042

Family Applications (2)

Application Number Title Priority Date Filing Date
US14/811,622 Active 2037-08-24 US10402067B2 (en) 2014-08-01 2015-07-28 Bi-directional search and sorting
US16/513,448 Pending US20190339848A1 (en) 2014-08-01 2019-07-16 Bi-directional search and sorting

Family Applications Before (1)

Application Number Title Priority Date Filing Date
US14/811,622 Active 2037-08-24 US10402067B2 (en) 2014-08-01 2015-07-28 Bi-directional search and sorting

Country Status (1)

Country Link
US (2) US10402067B2 (en)

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10782850B2 (en) * 2014-11-16 2020-09-22 Yowza LTD. User interface and a method for searching a model
US10726464B2 (en) * 2014-12-18 2020-07-28 Ebay Inc. Expressions of user interest
EP3040945B1 (en) * 2014-12-30 2019-06-19 Dassault Systèmes Creation of bounding boxes on a 3d modeled assembly
US10719193B2 (en) * 2016-04-20 2020-07-21 Microsoft Technology Licensing, Llc Augmenting search with three-dimensional representations
US20180232125A1 (en) * 2017-02-10 2018-08-16 Adobe Systems Incorporated Contextual Linking of Digital Representations of Objects with Documents
US10496239B2 (en) * 2017-05-01 2019-12-03 Microsoft Technology Licensing, Llc Three-dimensional model look-at-point rotation and viewport modes
CN115407676A (en) * 2018-06-15 2022-11-29 谷歌有限责任公司 Smart home device placement and installation using augmented reality visualization
US11222020B2 (en) * 2019-08-21 2022-01-11 International Business Machines Corporation Deduplicated data transmission

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130007061A1 (en) * 2011-06-29 2013-01-03 Nokia Corporation Apparatus and associated methods

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2004068300A2 (en) * 2003-01-25 2004-08-12 Purdue Research Foundation Methods, systems, and data structures for performing searches on three dimensional objects
US7688318B2 (en) * 2007-02-02 2010-03-30 Dassault Systemes Solidworks Corporation Reusable data constructs for a modeling system
US9454283B1 (en) * 2008-01-07 2016-09-27 The Mathworks, Inc. Three-dimensional visualization
US20090307255A1 (en) * 2008-06-06 2009-12-10 Johnson Controls Technology Company Graphical management of building devices
US9213785B2 (en) * 2009-02-03 2015-12-15 Thomas Plewe Systems and methods for component-based architecture design
US20120246148A1 (en) * 2011-03-22 2012-09-27 Intergraph Technologies Company Contextual Display and Scrolling of Search Results in Graphical Environment

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130007061A1 (en) * 2011-06-29 2013-01-03 Nokia Corporation Apparatus and associated methods

Also Published As

Publication number Publication date
US20160034150A1 (en) 2016-02-04
US10402067B2 (en) 2019-09-03

Similar Documents

Publication Publication Date Title
US20190339848A1 (en) Bi-directional search and sorting
US10719220B2 (en) Dynamic scrolling
US11809681B2 (en) Reality capture graphical user interface
EP3019930B1 (en) Interactive digital displays
US10359905B2 (en) Collaboration with 3D data visualizations
US10275122B2 (en) Semantic card view
US10768421B1 (en) Virtual monocle interface for information visualization
US20160203194A1 (en) User terminal for displaying image and image display method thereof
EP2708984A2 (en) Display Apparatus for Multiuser and Method Thereof
US20130232200A1 (en) User and content recommendation and discovery application
US10001907B2 (en) Flow fields
US20100077333A1 (en) Method and apparatus for non-hierarchical input of file attributes
US20090293008A1 (en) Information Processing Device, User Interface Method, And Information Storage Medium
CN104583923A (en) User interface tools for exploring data visualizations
US20170109020A1 (en) Interactive presentation system
US20170212906A1 (en) Interacting with user interfacr elements representing files
US9405446B1 (en) Efficient and interactive presentation of item images
US11334582B1 (en) Mobile explorer
US10523523B2 (en) Infographic search
US11409406B2 (en) User interface for mobile device to navigate between components
Jianu et al. Visual integration of quantitative proteomic data, pathways, and protein interactions
JP6063153B2 (en) Information processing apparatus, information processing method, and program
US9324046B2 (en) Enterprise ecosystem
KR20110038596A (en) Ui of search
JP5615654B2 (en) Information processing apparatus, information processing method, and program

Legal Events

Date Code Title Description
AS Assignment

Owner name: AUTODESK, INC., CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:BEHR, JAY ALLAN;MUNTER, AARON D.;LOLLAR, MICHAEL WILLIAM;REEL/FRAME:049769/0393

Effective date: 20150727

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: FINAL REJECTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE AFTER FINAL ACTION FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: ADVISORY ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: FINAL REJECTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: FINAL REJECTION MAILED