WO2019007372A1 - 模型显示方法和装置 - Google Patents

模型显示方法和装置 Download PDF

Info

Publication number
WO2019007372A1
WO2019007372A1 PCT/CN2018/094524 CN2018094524W WO2019007372A1 WO 2019007372 A1 WO2019007372 A1 WO 2019007372A1 CN 2018094524 W CN2018094524 W CN 2018094524W WO 2019007372 A1 WO2019007372 A1 WO 2019007372A1
Authority
WO
WIPO (PCT)
Prior art keywords
target
model
coordinates
target model
screen
Prior art date
Application number
PCT/CN2018/094524
Other languages
English (en)
French (fr)
Inventor
吕晓磊
刘享军
赵刚
Original Assignee
北京京东尚科信息技术有限公司
北京京东世纪贸易有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京京东尚科信息技术有限公司, 北京京东世纪贸易有限公司 filed Critical 北京京东尚科信息技术有限公司
Publication of WO2019007372A1 publication Critical patent/WO2019007372A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/04845—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/048—Indexing scheme relating to G06F3/048
    • G06F2203/04806—Zoom, i.e. interaction techniques or interactors for controlling the zooming operation

Definitions

  • the present application relates to the field of computer technology, and in particular to the field of Internet technologies, and in particular, to a model display method and apparatus.
  • Augmented Reality (AR) technology is a technique for calculating camera position and attitude in real time and superimposing images, videos and 3D rendering models in real world scenes.
  • AR Augmented Reality
  • the application of augmented reality is more and more extensive.
  • Model display using augmented reality technology in mobile devices has become a hot spot.
  • the existing method usually uses a default distance as the distance between the camera and the model, and scales the model to display the initial model.
  • a default distance as the distance between the camera and the model
  • scales the model to display the initial model when the mobile device loads the model in different postures and shifts to the same posture, the phenomenon that the displayed model size is inconsistent occurs, and thus, there is a problem that the accuracy of the model display is low.
  • the purpose of the embodiments of the present application is to provide an improved model display method and apparatus to solve the technical problems mentioned in the above background art.
  • an embodiment of the present application provides a model display method for a mobile device, where the mobile device is equipped with an imaging device and an inertial measurement device, and the method includes: loading in response to detecting a loading operation of a target model performed by a user. a model file for constructing a target model, wherein the model file records size information of the target model; extracts posture information of the camera device collected by the inertial measurement device, and determines a scaling ratio of the target model based on the posture information and Rotation angle; determining the initial display size of the target model based on the scaling and size information; rotating the target model by the rotation angle, and presenting the target model in the display screen of the mobile device in the initial display size.
  • the method further comprises: responsive to detecting the user's first sliding of the display screen Operation, determining the number of horizontal pixels involved in the first sliding operation; determining the target rotation angle of the model based on the number of horizontal pixels and the total number of horizontal pixels in the display screen; using the vertical axis of the target model as the rotation axis, The target model is rotated in the display screen after rotating the target rotation angle.
  • rotating the target model according to a rotation angle and presenting the target model in a display screen of the mobile device in an initial display size includes determining that an intersection of the optical axis of the imaging device and the target plane is in the world coordinate system Coordinates, wherein the target plane is a ground plane or a ceiling plane; the determined coordinates are used as the original coordinates of the target model in the world coordinate system, and the screen coordinates in the screen coordinate system corresponding to the original coordinates are determined, wherein the screen coordinate system is A plane rectangular coordinate system pre-established based on the display screen; the target model is presented at the position indicated by the screen coordinates.
  • the method further comprises: responsive to detecting a second sliding of the display screen by the user Operation, determining a starting position and an ending position of the second sliding operation, mapping the ending position to a near clipping plane of the camera coordinate system to determine coordinates of the mapped point in the camera coordinate system, wherein the camera coordinate system is a camera
  • the optical center of the device is the origin, and the optical axis of the imaging device is a rectangular coordinate system of one of the coordinate axes; the coordinates of the mapped point in the camera coordinate system are used as mapping coordinates, based on the mapping coordinates, the original coordinates, and the ground plane method.
  • a vector determining a first target coordinate of the target model in the world coordinate system; using coordinates in a screen coordinate system corresponding to the first target coordinate as the first target screen coordinate, moving the target model from the position indicated by the screen coordinates to The position indicated by the first target screen coordinates.
  • the method further comprises: determining the acquired posture information as initial posture information. Extracting current posture information of the imaging device collected by the inertial measurement device; determining the posture change based on the comparison between the current posture information and the initial posture information, determining the target model in the world coordinate system based on the initial posture information, the current posture information, and the original coordinates a second target coordinate; a coordinate in a screen coordinate system corresponding to the second target coordinate as a second target screen coordinate, moving the target model from a position indicated by the screen coordinates to a position indicated by the second target screen coordinate .
  • determining a zoom ratio and a rotation angle of the target model based on the posture information includes: extracting a preset zoom ratio and a preset rotation in response to determining that an angle between an optical axis of the imaging device and the target plane is not greater than a preset angle Angle, the preset zoom ratio is determined as the zoom ratio of the target model, and the preset rotation angle is determined as the rotation angle of the target model.
  • an embodiment of the present application provides a model display device for a mobile device, where the mobile device is equipped with an imaging device and an inertial measurement device, the model display device includes: a loading unit configured to respond to detecting the user a loading operation of the target model, loading a model file for constructing the target model to construct a target model, wherein the model file records size information of the target model; and the first extracting unit is configured to extract the image captured by the inertial measurement device a posture information of the device, and determining a scaling and a rotation angle of the target model based on the posture information; the first determining unit configured to determine an initial display size of the target model based on the scaling and size information; the first rendering unit configured to be used The target model is rotated by the angle of rotation and the target model is presented in the display screen of the mobile device in the initial display size.
  • the model display device further includes: a second determining unit configured to determine a number of horizontal pixels involved in the first sliding operation in response to detecting a first sliding operation of the display screen by the user; a determining unit configured to determine a target rotation angle of the model based on the number of horizontal pixels and the total number of horizontal pixels in the display screen; the second rendering unit configured to use the vertical axis of the target model as a rotation axis to target The model rotates the target rotation angle and appears in the display screen.
  • the first rendering unit includes: a first determining module configured to determine coordinates of an intersection of an optical axis of the camera device and a target plane in a world coordinate system, wherein the target plane is a ground plane or a ceiling plane; a second determining module configured to determine, as the target model, the original coordinates of the target model in the world coordinate system, the screen coordinates in the screen coordinate system corresponding to the original coordinates, wherein the screen coordinate system is based on the display screen A planar rectangular coordinate system is established; a rendering module configured to present the target model at a position indicated by screen coordinates.
  • the model display device further includes: a fourth determining unit configured to determine a start position and an end position of the second slide operation in response to detecting a second sliding operation of the display screen by the user, which will end The position is mapped to the near clipping plane of the camera coordinate system to determine the coordinates of the mapped point in the camera coordinate system, wherein the camera coordinate system is based on the optical center of the imaging device, and the optical axis of the imaging device is one of the coordinates a Cartesian coordinate system of the axis; a fifth determining unit configured to use the coordinates of the mapped point in the camera coordinate system as mapping coordinates, and determine the target model in the world coordinate system based on the mapping coordinates, the original coordinates, and the normal vector of the ground plane a first target coordinate; a first moving unit configured to use a coordinate in a screen coordinate system corresponding to the first target coordinate as the first target screen coordinate, and move the target model from the position indicated by the screen coordinate to the first The position indicated by the target screen coordinates.
  • the model display device further includes: a second extracting unit configured to determine the acquired posture information as initial posture information, and extract current posture information of the imaging device collected by the inertial measurement device; a determining unit configured to determine a posture change in response to the comparison based on the current posture information and the initial posture information, and determine a second target coordinate of the target model in the world coordinate system based on the initial posture information, the current posture information, and the original coordinates; And a moving unit configured to use the coordinates in the screen coordinate system corresponding to the second target coordinates as the second target screen coordinates, and move the target model from the position indicated by the screen coordinates to the position indicated by the second target screen coordinates.
  • a second extracting unit configured to determine the acquired posture information as initial posture information, and extract current posture information of the imaging device collected by the inertial measurement device
  • a determining unit configured to determine a posture change in response to the comparison based on the current posture information and the initial posture information, and determine a second target coordinate of the target model in the world coordinate system based on
  • the first extracting unit is further configured to: in response to determining that an angle between an optical axis of the imaging device and the target plane is not greater than a preset angle, extract a preset zoom ratio and a preset rotation angle, and preset the zoom The scale is determined as the zoom ratio of the target model, and the preset rotation angle is determined as the rotation angle of the target model.
  • an embodiment of the present application provides a mobile device, including: one or more processors; a storage device for storing one or more programs; an imaging device for acquiring an image; and an inertial measurement device for The gesture information is acquired; when one or more programs are executed by one or more processors, such that one or more processors implement a method as in any of the model display methods.
  • an embodiment of the present application provides a computer readable storage medium having stored thereon a computer program, which when executed by a processor, implements a method such as any one of the model display methods.
  • the model display method and apparatus load a model file after detecting a loading operation of a target model performed by the user after the camera device is turned on to construct a target model, and then extract posture information to determine a scaling of the target model. Proportion and rotation angle, then determine the initial display size of the target model based on the scaling and the size information in the model file, and finally rotate the target model according to the determined rotation angle and display it in the initial display size, so that the current posture can be based on the current posture
  • the information determines the scaling scale and the display size, and then displays the model.
  • FIG. 1 is an exemplary system architecture diagram to which the present application can be applied;
  • FIG. 2 is a flow chart of one embodiment of a model display method in accordance with the present application.
  • FIG. 3 is a schematic diagram of an application scenario of a model display method according to the present application.
  • FIG. 4 is a flow chart of still another embodiment of a model display method according to the present application.
  • FIG. 5 is a schematic structural view of an embodiment of a model display device according to the present application.
  • FIG. 6 is a block diagram showing the structure of a computer system suitable for implementing the mobile device of the embodiment of the present application.
  • FIG. 1 illustrates an exemplary system architecture 100 in which a model display method or model display device of the present application may be applied.
  • system architecture 100 can include mobile devices 101, 102, 103, network 104, and server 105.
  • Network 104 is used to provide a medium for communication links between mobile devices 101, 102, 103 and server 105.
  • Network 104 may include various types of connections, such as wired, wireless communication links, fiber optic cables, and the like.
  • the user can interact with the server 105 over the network 104 using the mobile devices 101, 102, 103 to receive or send messages and the like.
  • Various devices, sensors, and the like can be mounted on the mobile devices 101, 102, and 103, such as an imaging device, an inertial measurement unit (IMU), and the like.
  • the inertial measurement device is also referred to as an inertial measurement unit, and is a device for measuring an attitude (or angular velocity) and acceleration of an object, and can be used to acquire attitude information of the imaging device.
  • the device may include an accelerometer, a magnetometer, a gyro sensor, and the like.
  • various communication client applications such as a shopping application, a web browser application, a search application, an instant communication tool, a mailbox client, a social platform software, and the like, may also be installed on the mobile devices 101, 102, and 103.
  • the mobile devices 101, 102, 103 can be various electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop portable computers, desktop computers, and the like.
  • the server 105 may be a server that provides various services, such as a cloud server for storing model files.
  • the cloud server can send the corresponding model file to the mobile device 101, 102, 103 when the mobile device 101, 102, 103 loads the model.
  • the mobile device 101, 102, 103 may also store the model file locally, in which case the network 104 and the server 105 may not be included in the exemplary system architecture 100.
  • model display method provided by the embodiment of the present application is generally performed by the mobile devices 101, 102, and 103. Accordingly, the model display device is generally disposed in the mobile devices 101, 102, and 103.
  • a flow 200 of one embodiment of a model display method for a mobile device in accordance with the present application is illustrated.
  • the above mobile device is equipped with an imaging device and an inertial measurement device.
  • the model display method includes the following steps:
  • Step 201 In response to detecting a loading operation of the target model performed by the user, loading a model file for constructing the target model to construct the target model.
  • the electronic device (such as the mobile device 101, 102, 103 shown in FIG. 1) on which the model display method runs is responsive to detecting a loading operation of the target model performed by the user (eg, clicking on the electronic device Displaying the "Load Model” button presented in the screen or clicking on the icon for indicating the loading of the model, etc., can load the model file used to build the above target model to build the target model.
  • the model file may record size information of the target model, and the size information may include an actual size of each component of the target model preset by a technician.
  • the size information may include the dimensions (eg, length, height, thickness, diameter, etc.) of the various components of the table.
  • the above model file may also record other information of the target model, such as images, materials, lights, geometric networks, bones, scenes, and the like.
  • the format of the above model file may be a file format for constructing a three-dimensional model such as 3ds format, MD2 format, obj format, bsp format, x format, dae format, or fbx format.
  • the above target model may be a three-dimensional model of an item involved in the page when the user browses the page.
  • the above electronic device may be installed with a shopping application or a search application.
  • the above shopping application or search application can display the item on the page.
  • the user can click the "Load Model” button on the page or click an icon for indicating the model loading of the item, etc., to implement the loading operation of the model.
  • the model file may be pre-stored in the electronic device. At this time, the electronic device can directly read the model file from the local to construct the target model.
  • the model file may be pre-stored in the cloud server.
  • the electronic device may download the model file from the cloud server by using a wired connection method or a wireless connection method, and store the model file to the local device, and then read the model file to construct the target model.
  • the above wireless connection manner may include but is not limited to 3G/4G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods that are now known or developed in the future. .
  • Step 202 Extract posture information of the imaging device collected by the inertial measurement device, and determine a scaling ratio and a rotation angle of the target model based on the posture information.
  • the electronic device may extract posture information of the imaging device collected by the inertial measurement device, and determine a scaling ratio and a rotation angle of the target model based on the posture information.
  • the posture information may be a rotation matrix R of the imaging device.
  • each column of the rotation matrix R can represent the direction vector of the three coordinate axes of the camera coordinate system in the world coordinate system.
  • the posture information collected by the inertial measurement device can be used as the posture information of the electronic device and the imaging device.
  • the camera coordinate system may be a Cartesian coordinate system in which the optical axis of the imaging device is the origin and the optical axis of the imaging device is one of the coordinate axes.
  • the electronic device may determine the scaling and rotation angle of the target model by the following steps:
  • the electronic device may analyze and calculate the posture information to determine an angle ⁇ between an optical axis of the imaging device and a vertical axis of the imaging device perpendicular to a ground plane.
  • the electronic device can determine the scaling ratio S of the target model according to the following formula:
  • the electronic device may determine a direction vector v of the optical axis of the imaging device in the world coordinate system, that is,
  • the electronic device can determine the rotation angle ⁇ of the target model according to the following formula:
  • x is the first component of the direction vector v and y is the second component of the direction vector v.
  • the electronic device in response to determining that the angle between the optical axis of the imaging device and the target plane is not greater than a preset angle (eg, 10 degrees), the electronic device may extract a preset zoom ratio (eg, The scaling ratio S) corresponding to the angle ⁇ is 80 degrees and the preset rotation angle (for example, the rotation angle ⁇ corresponding to the angle ⁇ of 80 degrees), and the preset scaling ratio is determined as the scaling ratio of the target model, and The preset rotation angle is determined as the rotation angle of the target model.
  • a preset zoom ratio eg, The scaling ratio S
  • the preset rotation angle for example, the rotation angle ⁇ corresponding to the angle ⁇ of 80 degrees
  • the preset scaling ratio is determined as the scaling ratio of the target model
  • the preset rotation angle is determined as the rotation angle of the target model.
  • the above target plane may be a ground plane or a ceiling plane.
  • Step 203 Determine an initial display size of the target model based on the scaling and size information.
  • the electronic device may determine an initial display size of the target model based on the scaling ratio and the size information, wherein the initial display size may include an initial display size of each component of the target model, and each component The initial display size may be the product of the above scaling ratio and the actual size of the component in the above size information.
  • the target model is a table composed of a table top and a bracket.
  • the initial display size of the desktop is the product of the actual size and the zoom ratio of the desktop
  • the initial display size of the bracket is the actual size of the bracket.
  • step 204 the target model is rotated according to the rotation angle, and the target model is presented in the display screen of the mobile device in the initial display size.
  • the electronic device may rotate the target model according to the rotation angle, and present the target model on the display screen of the mobile device by using the initial display size.
  • a default angle of the target model may be recorded, and the electronic device may use the default angle as an initial angle of the target model, and rotate the target model by the rotation angle, and then use the initial display size. Perform the presentation of the above target model.
  • FIG. 3 is a schematic diagram of an application scenario of the model display method according to the embodiment.
  • the user first opens a page in the shopping application installed by the mobile device (as indicated by reference numeral 301), and a set of tables and chairs is displayed in the interface.
  • the user clicks a button in the interface for indicating the presentation of the target model (ie, the three-dimensional model of the above-mentioned table and chair) to perform the loading operation of the target model.
  • the mobile device loads the model file for constructing the target model to construct the target model, and determines the scaling and rotation angle of the target model based on the posture information of the mounted camera device to determine the initial display size.
  • the mobile device rotates the target model according to the determined rotation angle and presents the target model in the display screen (as indicated by reference numeral 302) with the determined initial display size.
  • the foregoing embodiment of the present application provides a method for constructing a target model by detecting a loading operation of a target model after the user opens the camera to construct a target model, and then extracting the posture information to determine a scaling ratio of the target model.
  • Rotating angle then determining the initial display size of the target model based on the scaling and the size information in the model file, and finally rotating the target model according to the determined rotation angle and displaying it in the initial display size, so that it can be determined based on the current posture information Zooming the scale and displaying the size, and then displaying the model, achieves uniform size of the model when loading the model in different poses and converting to the same pose, improving the accuracy of the model display.
  • the model shows a method 400 of the method, including the following steps:
  • Step 401 In response to detecting a loading operation of the target model by the user, loading a model file for constructing the target model to construct the target model.
  • the electronic device on which the model display method runs (for example, the mobile device 101, 102, 103 shown in FIG. 1) can be loaded for constructing the above in response to detecting a loading operation of the target model performed by the user.
  • the model file may record the size information of the target model.
  • Step 402 extracting posture information of the imaging device collected by the inertial measurement device, and determining a scaling ratio and a rotation angle of the target model based on the posture information.
  • the electronic device may extract posture information of the imaging device collected by the inertial measurement device, and determine a scaling ratio and a rotation angle of the target model based on the posture information.
  • the posture information may be a rotation matrix of the imaging device.
  • Step 403 determining an initial display size of the target model based on the scaling and size information.
  • the electronic device may determine an initial display size of the target model based on the scaling ratio and the size information, wherein the initial display size may include an initial display size of each component of the target model, and each component The initial display size may be the product of the above scaling ratio and the actual size of the component in the above size information.
  • step 404 the target model is rotated according to the rotation angle, and the target model is presented in the display screen of the mobile device in the initial display size.
  • the electronic device may rotate the target model according to the rotation angle, and present the target model on the display screen of the mobile device by using the initial display size.
  • a default angle of the target model may be recorded, and the electronic device may use the default angle as an initial angle of the target model, and rotate the target model by the rotation angle, and then use the initial display size. Perform the presentation of the above target model.
  • the electronic device may further determine a position where the target model is presented in the display screen according to the following steps: First, the electronic device may determine that an intersection of an optical axis of the imaging device and a target plane is in a world coordinate system. The coordinates of the above target plane are the ground plane or the ceiling plane. Thereafter, the electronic device may determine, as the target model, the screen coordinates in the screen coordinate system corresponding to the original coordinates in the original coordinate of the world coordinate system as the target model, wherein the screen coordinate system is based on the above display A plane rectangular coordinate system pre-established on the screen (for example, a plane rectangular coordinate system pre-established with the vertices of one corner of the display screen as the origin and the two sides of the angle as the coordinate axes).
  • a plane rectangular coordinate system pre-established on the screen for example, a plane rectangular coordinate system pre-established with the vertices of one corner of the display screen as the origin and the two sides of the angle as the coordinate axes).
  • the electronic device may determine the screen coordinates in the screen coordinate system corresponding to the original coordinates by using a coordinate transformation manner. Finally, the electronic device may present the target model at a position indicated by the screen coordinates. It should be noted that the above method for determining the coordinates of the screen coordinate system based on the coordinates in the world coordinate system is a well-known technology that has been widely studied and applied at present, and will not be described herein.
  • the electronic device may perform steps 405-407, steps 408-410, or steps 411-413.
  • Step 405 in response to detecting the first sliding operation of the display screen by the user, determining the number of horizontal pixels involved in the first sliding operation.
  • the first sliding may be determined in response to detecting the first sliding operation of the display screen by the user.
  • the first sliding operation may be any sliding operation such as single-finger sliding or two-finger sliding.
  • Step 406 determining a target rotation angle of the model based on the number of horizontal pixels and the total number of horizontal pixels in the display screen.
  • the electronic device may determine the target rotation angle ⁇ ' of the model according to the following formula based on the number d of horizontal pixels and the total number W of horizontal pixels in the display screen:
  • Step 407 The target model is rotated in the vertical axis of the target model, and the target model is rotated in the target rotation angle and then presented in the display screen.
  • the electronic device may display the target model in the display screen after the target model is rotated by the target rotation angle ⁇ ′ with the vertical axis of the target model (ie, the axis of the target model is perpendicular to the axis of the ground plane).
  • Step 408 in response to detecting a second sliding operation of the user on the display screen, determining a starting position and an ending position of the second sliding operation, mapping the ending position to a near clipping plane of the camera coordinate system to determine that the mapped point is The coordinates in the camera coordinate system.
  • the second sliding operation may be determined in response to detecting the second sliding operation of the display screen by the user.
  • the starting position and the ending position mapping the ending position to the near clipping plane of the camera coordinate system to determine the coordinates of the mapped point in the camera coordinate system, wherein the camera coordinate system is based on the optical center of the camera device
  • the second sliding operation may be various sliding operation gestures, such as an operation gesture of clicking a model or a blank and then sliding, and the second sliding operation is different from the operation gesture of the first sliding operation.
  • Step 409 The coordinates of the mapped point in the camera coordinate system are used as mapping coordinates, and the first target coordinates of the target model in the world coordinate system are determined based on the mapping coordinates, the original coordinates, and the normal vector of the ground plane.
  • the electronic device may use the coordinates of the mapped point in the camera coordinate system as mapping coordinates, and then determine the target model in the world coordinate based on the mapping coordinates, the original coordinates, and the normal vector of the ground plane.
  • the first target coordinate in the system may be used as mapping coordinates, and then determine the target model in the world coordinate based on the mapping coordinates, the original coordinates, and the normal vector of the ground plane.
  • the electronic device may represent the mapping coordinates by (a, b, -1) T , and represent the original coordinates by (p x , p y , p z ) T to (n x , n y , n z T represents a normal vector of the above ground plane, wherein a and b respectively represent components of the x-axis and y-axis in the camera coordinate system indicated by the mapping coordinates, and p x , p y , and p z respectively represent the above
  • the position indicated by the original coordinates is the components of the x-axis, y-axis, and z-axis of the world coordinate system, and n x , n y , and n z respectively represent the normal vector of the above-mentioned ground plane in the x-axis, y-axis, and z of the world coordinate system.
  • the above first target coordinates are:
  • Step 410 The coordinates in the screen coordinate system corresponding to the first target coordinates are used as the first target screen coordinates, and the target model is moved from the position indicated by the screen coordinates to the position indicated by the first target screen coordinates.
  • the electronic device may use the coordinates in the screen coordinate system corresponding to the first target coordinate as the first target screen coordinates, and move the target model from the position indicated by the screen coordinates to the first target screen coordinate.
  • the indicated location It should be noted that the above method for determining the coordinates of the screen coordinate system based on the coordinates in the world coordinate system is a well-known technique widely studied and applied at present, and will not be described herein.
  • Step 411 Determine the acquired posture information as initial posture information, and extract current posture information of the imaging device collected by the inertial measurement device.
  • the acquired posture information may be determined as initial posture information, and the inertial measurement device is extracted.
  • the current posture information of the above-mentioned imaging device may be periodically extracted.
  • Step 412 in response to determining the posture change based on the comparison of the current posture information and the initial posture information, determining the second target coordinate of the target model in the world coordinate system based on the initial posture information, the current posture information, and the original coordinates.
  • the electronic device determines the posture change of the photographing device based on the comparison between the current posture information and the initial posture information, and determines the target model in the world coordinates based on the initial posture information, the current posture information, and the original coordinates.
  • the second target coordinate in the system may represent the original coordinates by (p x , p y , p z ) T , and the second target coordinates are:
  • R is the initial rotation matrix corresponding to the initial posture information
  • R' is the current rotation matrix corresponding to the current posture information
  • Step 413 The coordinates in the screen coordinate system corresponding to the second target coordinates are used as the second target screen coordinates, and the target model is moved from the position indicated by the screen coordinates to the position indicated by the second target screen coordinates.
  • the electronic device may use the coordinates in the screen coordinate system corresponding to the second target coordinate as the second target screen coordinates, and move the target model from the position indicated by the screen coordinates to the second target screen coordinate.
  • the indicated location It should be noted that the above method for determining the coordinates of the screen coordinate system based on the coordinates in the world coordinate system is a well-known technology that has been widely studied and applied at present, and will not be described herein.
  • the flow 400 of the model display method in the present embodiment highlights the steps of moving, rotating, and the like based on the user operation after the model is presented, as compared with the embodiment corresponding to Fig. 2. Therefore, the solution described in this embodiment can perform operations such as moving, rotating, and the like of the model based on calculating the mapping between the screen coordinate system and the world coordinate system, thereby not only improving the accuracy of the model display, but also further improving the flexibility of the model display.
  • the present application provides an embodiment of a model display device, which corresponds to the method embodiment shown in FIG. 2, and the device may specifically Applied to mobile devices.
  • the mobile device can be equipped with an imaging device and an inertial measurement device.
  • the model display apparatus 500 of the present embodiment includes: a loading unit 501 configured to load a model file for constructing the target model in response to detecting a loading operation of a target model performed by a user, Constructing the target model, wherein the model file records the size information of the target model; the first extracting unit 502 is configured to extract the posture information of the imaging device collected by the inertial measurement device, and determine the above based on the posture information.
  • the first determining unit 503 is configured to determine an initial display size of the target model based on the scaling and the size information;
  • the first rendering unit 504 is configured to use the target model according to the foregoing The rotation angle is rotated, and the above target model is presented in the display screen of the above mobile device in the above initial display size.
  • the loading unit 501 may load a model file for constructing the target model to construct a target model in response to detecting a loading operation of the target model performed by the user.
  • the model file may record the size information of the target model.
  • the first extraction unit 502 may extract posture information of the imaging device collected by the inertial measurement device, and determine a scaling ratio and a rotation angle of the target model based on the posture information.
  • the posture information may be a rotation matrix of the imaging device.
  • the first determining unit 503 may determine an initial display size of the target model based on the scaling ratio and the size information, wherein the initial display size may include an initial display size of each component of the target model.
  • the initial display size of a component may be the product of the above scaling ratio and the actual size of the component in the above size information.
  • the first rendering unit 504 may rotate the target model according to the rotation angle, and present the target model in the display screen of the mobile device in the initial display size.
  • the model display device 500 may further include a second determining unit, a third determining unit, and a second presenting unit (not shown).
  • the second determining unit may be configured to determine the number of horizontal pixels involved in the first sliding operation in response to detecting a first sliding operation of the display screen by the user.
  • the third determining unit may be configured to determine a target rotation angle of the model based on the number of the horizontal pixels and the total number of horizontal pixels in the display screen.
  • the second rendering unit may be configured to display the target model in the display screen after the target model is rotated by using the vertical axis of the target model as a rotation axis.
  • the first presentation unit may include a first determining module, a second determining module, and a rendering module (not shown).
  • the first determining module may be configured to determine coordinates of an intersection of an optical axis of the imaging device and a target plane in a world coordinate system, wherein the target plane is a ground plane or a ceiling plane.
  • the second determining module may be configured to use the determined coordinates as the original coordinates of the target model in the world coordinate system, and determine screen coordinates in a screen coordinate system corresponding to the original coordinates, wherein the screen coordinate system is A plane rectangular coordinate system that is pre-established based on the above display screen.
  • the rendering module may be configured to present the target model at a location indicated by the screen coordinates.
  • the model display device 500 may further include a fourth determining unit, a fifth determining unit, and a first moving unit (not shown).
  • the fourth determining unit may be configured to determine a start position and an end position of the second sliding operation in response to detecting a second sliding operation of the display screen by the user, and map the ending position to a camera coordinate system. Nearly clipping a surface to determine a coordinate of the mapped point in the camera coordinate system, wherein the camera coordinate system is a rectangular coordinate with the optical axis of the imaging device as an origin and the optical axis of the imaging device is one of the coordinate axes system.
  • the fifth determining unit may be configured to use the coordinates of the mapped point in the camera coordinate system as mapping coordinates, and determine the target model in the world coordinate based on the mapping coordinate, the original coordinate, and a normal vector of a ground plane.
  • the first moving unit may be configured to use the coordinates in the screen coordinate system corresponding to the first target coordinate as the first target screen coordinates, and move the target model from the position indicated by the screen coordinates to the first target The position indicated by the screen coordinates.
  • the model display device 500 may further include a second extraction unit, a sixth determination unit, and a second movement unit (not shown).
  • the second extraction unit is configured to determine the acquired posture information as initial posture information, and extract current posture information of the imaging device collected by the inertial measurement device.
  • the sixth determining unit may be configured to determine a posture change based on the comparison between the current posture information and the initial posture information, and determine the target model in the world coordinate based on the initial posture information, the current posture information, and the original coordinates.
  • the second moving unit may be configured to use the coordinates in the screen coordinate system corresponding to the second target coordinate as the second target screen coordinates, and move the target model from the position indicated by the screen coordinates to the second target. The position indicated by the screen coordinates.
  • the first extraction unit 502 may be further configured to extract a preset zoom ratio in response to determining that an angle between an optical axis of the imaging device and a target plane is not greater than a preset angle. And the preset rotation angle, determining the preset zoom ratio as the zoom ratio of the target model, and determining the preset rotation angle as the rotation angle of the target model.
  • the device provided by the foregoing embodiment of the present application loads the model file by the loading unit 501 after detecting the loading operation of the target model performed by the user after the camera device is turned on to construct the target model, and then the first extracting unit 502 extracts the posture information.
  • the first determining unit 503 determines the initial display size of the target model based on the scaling and the size information in the model file, and finally the first rendering unit 504 follows the determined rotation angle of the target model. Rotating and displaying in the initial display size, the scaling scale and the display size can be determined based on the current posture information, thereby displaying the model, and the size of the model is unified when the model is loaded in different postures and converted to the same posture. Improve the accuracy of the model display.
  • FIG. 6 a block diagram of a computer system 600 suitable for use in implementing the mobile device of an embodiment of the present application is shown.
  • the mobile device shown in FIG. 6 is merely an example, and should not impose any limitation on the function and scope of use of the embodiments of the present application.
  • computer system 600 includes a central processing unit (CPU) 601 that can be loaded into a program in random access memory (RAM) 603 according to a program stored in read only memory (ROM) 602 or from storage portion 608. And perform various appropriate actions and processes.
  • RAM random access memory
  • ROM read only memory
  • RAM random access memory
  • various programs and data required for the operation of the system 600 are also stored.
  • the CPU 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604.
  • An input/output (I/O) interface 605 is also coupled to bus 604.
  • the following components are connected to the I/O interface 605: an input portion 606 including a touch screen, a touch panel, etc.; an output portion 607 including a liquid crystal display (LCD) or the like, a speaker, etc.; a storage portion 608 including a hard disk or the like; and including, for example, a LAN card
  • the communication section 609 performs communication processing via a network such as the Internet.
  • Driver 610 is also coupled to I/O interface 605 as needed.
  • a removable medium 611 such as a semiconductor memory or the like, is mounted on the drive 610 as needed so that a computer program read therefrom is installed into the storage portion 608 as needed.
  • an embodiment of the present disclosure includes a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for executing the method illustrated in the flowchart.
  • the computer program can be downloaded and installed from the network via communication portion 609, and/or installed from removable media 611.
  • the central processing unit (CPU) 601 the above-described functions defined in the method of the present application are performed.
  • the computer readable medium described herein may be a computer readable signal medium or a computer readable storage medium or any combination of the two.
  • the computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of computer readable storage media may include, but are not limited to, electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable Programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the foregoing.
  • a computer readable storage medium may be any tangible medium that can contain or store a program, which can be used by or in connection with an instruction execution system, apparatus or device.
  • a computer readable signal medium may include a data signal that is propagated in the baseband or as part of a carrier, carrying computer readable program code. Such propagated data signals can take a variety of forms including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the foregoing.
  • the computer readable signal medium can also be any computer readable medium other than a computer readable storage medium, which can transmit, propagate, or transport a program for use by or in connection with the instruction execution system, apparatus, or device.
  • Program code embodied on a computer readable medium can be transmitted by any suitable medium, including but not limited to wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
  • each block of the flowchart or block diagram can represent a module, a program segment, or a portion of code that includes one or more of the logic functions for implementing the specified.
  • Executable instructions can also occur in a different order than that illustrated in the drawings. For example, two successively represented blocks may in fact be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending upon the functionality involved.
  • each block of the block diagrams and/or flowcharts, and combinations of blocks in the block diagrams and/or flowcharts can be implemented in a dedicated hardware-based system that performs the specified function or operation. Or it can be implemented by a combination of dedicated hardware and computer instructions.
  • the units involved in the embodiments of the present application may be implemented by software or by hardware.
  • the described unit may also be provided in the processor, for example, as a processor including a loading unit, a first extraction unit, a first determination unit, and a first presentation unit.
  • the names of these units do not constitute a limitation on the unit itself under certain circumstances.
  • the loading unit can also be described as "a unit of a model file for loading a target model".
  • the present application also provides a computer readable medium, which may be included in the apparatus described in the above embodiments, or may be separately present and not incorporated into the apparatus.
  • the computer readable medium carries one or more programs, when the one or more programs are executed by the device, causing the device to: in response to detecting a loading operation of a target model performed by the user after turning on the camera device, loading Constructing a model file of the target model to construct the target model, wherein the model file records size information of the target model; extracting posture information collected by the inertial measurement device, and determining a scaling of the target model based on the posture information a ratio and a rotation angle; determining an initial display size of the target model based on the scaling and the size information; rotating the target model according to the rotation angle, and presenting the target model on a display screen of the mobile device with the initial display size in.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • User Interface Of Digital Computer (AREA)
  • Controls And Circuits For Display Device (AREA)

Abstract

本申请公开了模型显示方法和装置。该方法的一具体实施方式包括：响应于检测到用户在开启摄像装置后进行的目标模型的加载操作，加载用于构建该目标模型的模型文件，以构建该目标模型，其中，该模型文件记录有该目标模型的尺寸信息；提取惯性测量装置采集到的姿态信息，并基于该姿态信息确定该目标模型的缩放比例和旋转角度；基于该缩放比例和该尺寸信息确定该目标模型的初始展示尺寸；将该目标模型按照该旋转角度旋转，并以该初始展示尺寸将该目标模型呈现在该移动设备的显示屏幕中。该实施方式提高了模型显示的准确性。

Description

模型显示方法和装置
本专利申请要求于2017年7月5日提交的、申请号为201710540596.8、申请人为北京京东尚科信息技术有限公司和北京京东世纪贸易有限公司、发明名称为“模型显示方法和装置”的中国专利申请的优先权，该申请的全文以引用的方式并入本申请中。
技术领域
本申请涉及计算机技术领域，具体涉及互联网技术领域，尤其涉及模型显示方法和装置。
背景技术
增强现实(Augmented Reality，AR)技术是一种实时计算摄像机位置和姿态，并在现实世界场景中叠加图像、视频和三维渲染模型的技术。随着电子产品计算能力的提升，增强现实的应用越来越广泛。移动设备(手机、平板电脑等)中利用增强现实技术进行模型显示成为一大热点。
现有的方法通常是将一个默认距离作为摄像装置与模型的距离，进行模型的尺寸缩放，进而显示初始模型。然而，当移动设备在不同的姿态下加载模型并转换到同一姿态时，会发生所显示的模型大小不一致的现象，从而，这种方式存在着模型显示的准确性较低的问题。
发明内容
本申请实施例的目的在于提出一种改进的模型显示方法和装置，来解决以上背景技术部分提到的技术问题。
第一方面，本申请实施例提供了一种用于移动设备的模型显示方法，移动设备安装有摄像装置和惯性测量装置，该方法包括：响应于检测到用户进行的目标模型的加载操作，加载用于构建目标模型的模 型文件，以构建目标模型，其中，模型文件记录有目标模型的尺寸信息；提取惯性测量装置采集到的摄像装置的姿态信息，并基于姿态信息确定目标模型的缩放比例和旋转角度；基于缩放比例和尺寸信息确定目标模型的初始展示尺寸；将目标模型按照旋转角度旋转，并以初始展示尺寸将目标模型呈现在移动设备的显示屏幕中。
在一些实施例中，在将目标模型按照旋转角度旋转，并以初始展示尺寸将目标模型呈现在移动设备的显示屏幕中之后，该方法还包括：响应于检测到用户对显示屏幕的第一滑动操作，确定第一滑动操作所涉及的水平像素的数量；基于水平像素的数量和显示屏幕中的水平像素的总数量，确定模型的目标旋转角度；以目标模型的竖直轴为旋转轴，将目标模型旋转目标旋转角度后呈现在显示屏幕中。
在一些实施例中，将目标模型按照旋转角度旋转，并以初始展示尺寸将目标模型呈现在移动设备的显示屏幕中，包括：确定摄像装置的光轴与目标平面的交点在世界坐标系中的坐标，其中，目标平面是地平面或天花板平面；将所确定的坐标作为目标模型在世界坐标系的原始坐标，确定与原始坐标相对应的屏幕坐标系中的屏幕坐标，其中，屏幕坐标系是基于显示屏幕而预先建立的平面直角坐标系；将目标模型呈现在屏幕坐标所指示的位置。
在一些实施例中，在将目标模型按照旋转角度旋转，并以初始展示尺寸将目标模型呈现在移动设备的显示屏幕中之后，该方法还包括：响应于检测到用户对显示屏幕的第二滑动操作，确定第二滑动操作的起始位置和结束位置，将结束位置映射到相机坐标系的近裁剪面，以确定映射后的点在相机坐标系中的坐标，其中，相机坐标系是以摄像装置的光心为原点、以摄像装置的光轴为其中一个坐标轴的直角坐标系；将映射后的点在相机坐标系中的坐标作为映射坐标，基于映射坐标、原始坐标和地平面的法向量，确定目标模型在世界坐标系中的第一目标坐标；将与第一目标坐标相对应的屏幕坐标系中的坐标作为第一目标屏幕坐标，将目标模型从屏幕坐标所指示的位置移动至第一目标屏幕坐标所指示的位置。
在一些实施例中，在将目标模型按照旋转角度旋转，并以初始展 示尺寸将目标模型呈现在移动设备的显示屏幕中之后，该方法还包括：将所获取到的姿态信息确定为初始姿态信息，提取惯性测量装置采集到的摄像装置的当前姿态信息；响应于基于当前姿态信息和初始姿态信息的比较确定出姿态变化，基于初始姿态信息、当前姿态信息和原始坐标确定目标模型在世界坐标系中的第二目标坐标；将与第二目标坐标相对应的屏幕坐标系中的坐标作为第二目标屏幕坐标，将目标模型从屏幕坐标所指示的位置移动至第二目标屏幕坐标所指示的位置。
在一些实施例中，基于姿态信息确定目标模型的缩放比例和旋转角度，包括：响应于确定摄像装置的光轴与目标平面的夹角不大于预设角度，提取预设缩放比例和预设旋转角度，将预设缩放比例确定为目标模型的缩放比例，将预设旋转角度确定为目标模型的旋转角度。
第二方面，本申请实施例提供了一种用于移动设备的模型显示装置，移动设备安装有摄像装置和惯性测量装置，该模型显示装置包括：加载单元，配置用于响应于检测到用户进行的目标模型的加载操作，加载用于构建目标模型的模型文件，以构建目标模型，其中，模型文件记录有目标模型的尺寸信息；第一提取单元，配置用于提取惯性测量装置采集到的摄像装置的姿态信息，并基于姿态信息确定目标模型的缩放比例和旋转角度；第一确定单元，配置用于基于缩放比例和尺寸信息确定目标模型的初始展示尺寸；第一呈现单元，配置用于将目标模型按照旋转角度旋转，并以初始展示尺寸将目标模型呈现在移动设备的显示屏幕中。
在一些实施例中，该模型显示装置还包括：第二确定单元，配置用于响应于检测到用户对显示屏幕的第一滑动操作，确定第一滑动操作所涉及的水平像素的数量；第三确定单元，配置用于基于水平像素的数量和显示屏幕中的水平像素的总数量，确定模型的目标旋转角度；第二呈现单元，配置用于以目标模型的竖直轴为旋转轴，将目标模型旋转目标旋转角度后呈现在显示屏幕中。
在一些实施例中，第一呈现单元包括：第一确定模块，配置用于确定摄像装置的光轴与目标平面的交点在世界坐标系中的坐标，其中，目标平面是地平面或天花板平面；第二确定模块，配置用于将所确定 的坐标作为目标模型在世界坐标系的原始坐标，确定与原始坐标相对应的屏幕坐标系中的屏幕坐标，其中，屏幕坐标系是基于显示屏幕而预先建立的平面直角坐标系；呈现模块，配置用于将目标模型呈现在屏幕坐标所指示的位置。
在一些实施例中，该模型显示装置还包括：第四确定单元，配置用于响应于检测到用户对显示屏幕的第二滑动操作，确定第二滑动操作的起始位置和结束位置，将结束位置映射到相机坐标系的近裁剪面，以确定映射后的点在相机坐标系中的坐标，其中，相机坐标系是以摄像装置的光心为原点、以摄像装置的光轴为其中一个坐标轴的直角坐标系；第五确定单元，配置用于将映射后的点在相机坐标系中的坐标作为映射坐标，基于映射坐标、原始坐标和地平面的法向量，确定目标模型在世界坐标系中的第一目标坐标；第一移动单元，配置用于将与第一目标坐标相对应的屏幕坐标系中的坐标作为第一目标屏幕坐标，将目标模型从屏幕坐标所指示的位置移动至第一目标屏幕坐标所指示的位置。
在一些实施例中，该模型显示装置还包括：第二提取单元，配置用于将所获取到的姿态信息确定为初始姿态信息，提取惯性测量装置采集到的摄像装置的当前姿态信息；第六确定单元，配置用于响应于基于当前姿态信息和初始姿态信息的比较确定出姿态变化，基于初始姿态信息、当前姿态信息和原始坐标确定目标模型在世界坐标系中的第二目标坐标；第二移动单元，配置用于将与第二目标坐标相对应的屏幕坐标系中的坐标作为第二目标屏幕坐标，将目标模型从屏幕坐标所指示的位置移动至第二目标屏幕坐标所指示的位置。
在一些实施例中，第一提取单元进一步配置用于：响应于确定摄像装置的光轴与目标平面的夹角不大于预设角度，提取预设缩放比例和预设旋转角度，将预设缩放比例确定为目标模型的缩放比例，将预设旋转角度确定为目标模型的旋转角度。
第三方面，本申请实施例提供了一种移动设备，包括：一个或多个处理器；存储装置，用于存储一个或多个程序；摄像装置，用于采集图像；惯性测量装置，用于采集姿态信息；当一个或多个程序被一 个或多个处理器执行，使得一个或多个处理器实现如模型显示方法中任一实施例的方法。
第四方面，本申请实施例提供了计算机可读存储介质，其上存储有计算机程序，该程序被处理器执行时实现如模型显示方法中任一实施例的方法。
本申请实施例提供的模型显示方法和装置，通过在检测到用户在开启摄像装置后进行的目标模型的加载操作后加载模型文件，以构建目标模型，而后提取姿态信息，以便确定目标模型的缩放比例和旋转角度，之后基于缩放比例和模型文件中的尺寸信息确定目标模型的初始展示尺寸，最后将目标模型按照所确定的旋转角度旋转，并以初始展示尺寸进行展示，从而可以基于当前的姿态信息确定缩放尺度和展示尺寸，进而显示模型，实现了在当在不同姿态下加载模型并转换到同一姿态时，模型的尺寸统一，提高了模型显示的准确性。
附图说明
通过阅读参照以下附图所作的对非限制性实施例所作的详细描述，本申请的其它特征、目的和优点将会变得更明显：
图1是本申请可以应用于其中的示例性系统架构图；
图2是根据本申请的模型显示方法的一个实施例的流程图；
图3是根据本申请的模型显示方法的一个应用场景的示意图；
图4是根据本申请的模型显示方法的又一个实施例的流程图；
图5是根据本申请的模型显示装置的一个实施例的结构示意图；
图6是适于用来实现本申请实施例的移动设备的计算机系统的结构示意图。
具体实施方式
下面结合附图和实施例对本申请作进一步的详细说明。可以理解的是，此处所描述的具体实施例仅仅用于解释相关发明，而非对该发明的限定。另外还需要说明的是，为了便于描述，附图中仅示出了与有关发明相关的部分。
需要说明的是，在不冲突的情况下，本申请中的实施例及实施例中的特征可以相互组合。下面将参考附图并结合实施例来详细说明本申请。
图1示出了可以应用本申请的模型显示方法或模型显示装置的示例性系统架构100。
如图1所示，系统架构100可以包括移动设备101、102、103，网络104和服务器105。网络104用以在移动设备101、102、103和服务器105之间提供通信链路的介质。网络104可以包括各种连接类型，例如有线、无线通信链路或者光纤电缆等等。
用户可以使用移动设备101、102、103通过网络104与服务器105交互，以接收或发送消息等。移动设备101、102、103上可以安装有各种装置、传感器等，例如摄像装置、惯性测量装置(Inertial measurement unit，IMU)等。其中，上述惯性测量装置也称为惯性测量单元，是测量物体三轴姿态角(或角速率)以及加速度的装置，可以用于采集上述摄像装置的姿态信息。该装置可以包括加速度计、磁力计和陀螺仪传感器等。另外，移动设备101、102、103上还可以安装有各种通讯客户端应用，例如购物类应用、网页浏览器应用、搜索类应用、即时通信工具、邮箱客户端、社交平台软件等。
移动设备101、102、103可以是具有显示屏并且支持网页浏览的各种电子设备，包括但不限于智能手机、平板电脑、膝上型便携计算机和台式计算机等等。
服务器105可以是提供各种服务的服务器，例如用于存储模型文件的云服务器。云服务器可以在移动设备101、102、103加载模型时，向移动设备101、102、103发送相应的模型文件。
需要指出的是，移动设备101、102、103也可以将模型文件存储在本地，此时，示例性系统架构100中可以不包含网络104和服务器105。
需要说明的是，本申请实施例所提供的模型显示方法一般由移动设备101、102、103执行，相应地，模型显示装置一般设置于移动设备101、102、103中。
应该理解，图1中的移动设备、网络和服务器的数目仅仅是示意性的。根据实现需要，可以具有任意数目的移动设备、网络和服务器。
继续参考图2，其示出了根据本申请的用于移动设备的模型显示方法的一个实施例的流程200。上述移动设备安装有摄像装置和惯性测量装置。所述的模型显示方法，包括以下步骤：
步骤201，响应于检测到用户进行的目标模型的加载操作，加载用于构建目标模型的模型文件，以构建目标模型。
在本实施例中，模型显示方法运行于其上的电子设备(例如图1所示的移动设备101、102、103)响应于检测到用户进行的目标模型的加载操作(例如点击上述电子设备的显示屏幕中所呈现的“加载模型”按键或点击用于指示对模型进行加载的图标等)，可以加载用于构建上述目标模型的模型文件，以构建目标模型。其中，上述模型文件可以记录有上述目标模型的尺寸信息，上述尺寸信息可以包括技术人员预先设定的该目标模型的各个组成部分的实际尺寸。作为示例，若目标模型为一张桌子，则尺寸信息可以包含该桌子各个组成部分的尺寸(例如长度、高度、厚度、直径等)。需要说明的是，上述模型文件还可以记录有上述目标模型的其他信息，例如图像、材质、灯光、几何网络、骨骼、场景等等。上述模型文件的格式可以是3ds格式、MD2格式、obj格式、bsp格式、x格式、dae格式或fbx格式等各种用于构建三维模型的文件格式。需要指出的是，上述目标模型可以是用户进行页面浏览时，该页面所涉及的物品的三维模型。
实践中，上述电子设备可以安装有购物类应用或搜索类应用。上述购物类应用或搜索类应用可以将物品在页面中进行展示。用户在浏览该页面时，可以点击该页面中的“加载模型”按键或点击用于指示对该物品进行模型加载的图标等，实现对模型的加载操作。
在本实施例的一些可选的实现方式中，上述模型文件可以预先存储于上述电子设备中。此时，上述电子设备可以从本地直接读取上述模型文件，以构建上述目标模型。
在本实施例的一些可选的实现方式中，上述模型文件可以预先存 储于云服务器。此时，上述电子设备可以通过有线连接方式或者无线连接方式从上述云服务器下载上述模型文件并存储至本地，进而读取上述模型文件，以构建上述目标模型。需要指出的是，上述无线连接方式可以包括但不限于3G/4G连接、WiFi连接、蓝牙连接、WiMAX连接、Zigbee连接、UWB(ultra wideband)连接、以及其他现在已知或将来开发的无线连接方式。
步骤202，提取惯性测量装置采集到的摄像装置的姿态信息，并基于姿态信息确定目标模型的缩放比例和旋转角度。
在本实施例中，上述电子设备可以提取上述惯性测量装置采集到的上述摄像装置的姿态信息，并基于上述姿态信息确定上述目标模型的缩放比例和旋转角度。其中，上述姿态信息可以是上述摄像装置的旋转矩阵R。实践中，旋转矩阵R的每一列可以表示相机坐标系的三个坐标轴在世界坐标系中的方向向量。需要说明的是，由于上述摄像装置和上述惯性测量装置均安装于上述电子设备中，因此，可以将上述惯性测量装置采集到的姿态信息作为上述电子设备和上述拍摄装置的姿态信息。需要说明的是，相机坐标系可以是以摄像装置的光心为原点、以摄像装置的光轴为其中一个坐标轴的直角坐标系。
在本实施例中，上述电子设备可以通过以下步骤确定上述目标模型的缩放比例和旋转角度：
首先，上述电子设备可以对上述姿态信息进行分析计算，确定上述摄像装置的光轴与上述摄像装置垂直于地平面的竖直轴之间的角度θ。
而后，上述电子设备可以按照以下公式确定上述目标模型的缩放比例S：
Figure PCTCN2018094524-appb-000001
之后，上述电子设备可以确定上述拍摄装置的光轴在世界坐标系的方向向量v，即:
Figure PCTCN2018094524-appb-000002
最后，上述电子设备可以按照以下公式确定上述目标模型的旋转角度ω:
Figure PCTCN2018094524-appb-000003
其中，x为方向向量v的第一个分量，y为方向向量v的第二个分量。
在本实施例的一些可选的实现方式中，响应于确定上述摄像装置的光轴与目标平面的夹角不大于预设角度(例如10度)，上述电子设备可以提取预设缩放比例(例如角度θ为80度时所对应的缩放比例S)和预设旋转角度(例如角度θ为80度时所对应的旋转角度ω)，将上述预设缩放比例确定为上述目标模型的缩放比例，将上述预设旋转角度确定为上述目标模型的旋转角度。需要说明的是，上述目标平面可以是地平面或天花板平面。
步骤203，基于缩放比例和尺寸信息确定目标模型的初始展示尺寸。
在本实施例中，上述电子设备可以基于上述缩放比例和上述尺寸信息确定目标模型的初始展示尺寸，其中，上述初始展示尺寸可以包括上述目标模型的各个组成部分的初始展示尺寸，每一个组成部分的初始展示尺寸可以是上述缩放比例与上述尺寸信息中的、该组成部分的实际尺寸的乘积。作为示例，上述目标模型是一张由桌面和支架组合而成的桌子，上述桌面的初始展示尺寸则为上述桌面的实际尺寸与缩放比例的乘积，上述支架的初始展示尺寸则为上述支架的实际尺寸与缩放比例的乘积。
步骤204，将目标模型按照旋转角度旋转，并以初始展示尺寸将目标模型呈现在移动设备的显示屏幕中。
在本实施例中，上述电子设备可以将上述目标模型按照上述旋转角度旋转，并以上述初始展示尺寸将上述目标模型呈现在上述移动设备的显示屏幕中。需要说明的是，上述模型文件中可以记录有上述目标模型的默认角度，上述电子设备可以将上述默认角度作为上述目标模型的初始角度，将上述目标模型旋转上述旋转角度，之后以上述初始展示尺寸进行上述目标模型的呈现。
继续参见图3，图3是根据本实施例的模型显示方法的应用场景的一个示意图。在图3的应用场景中，用户首先开启了移动设备所安装的购物类应用中的某一页面(如标号301所示)，该界面中展示有一组桌椅。之后，用户点击了该界面中用于指示呈现目标模型(即上述桌椅的三维模型)的按键，以进行目标模型的加载操作。而后，移动设备加载用于构建目标模型的模型文件，以构建目标模，并基于所安安装的摄像装置的姿态信息，确定目标模型的缩放比例和旋转角度，以确定初始展示尺寸。最后，移动设备将目标模型按照所确定旋转角度旋转，并以所确定的初始展示尺寸将上述目标模型呈现在显示屏幕中(如标号302所示)。
本申请的上述实施例提供的方法，通过在检测到用户在开启摄像装置后进行的目标模型的加载操作后加载模型文件，以构建目标模型，而后提取姿态信息，以便确定目标模型的缩放比例和旋转角度，之后基于缩放比例和模型文件中的尺寸信息确定目标模型的初始展示尺寸，最后将目标模型按照所确定的旋转角度旋转，并以初始展示尺寸进行展示，从而可以基于当前的姿态信息确定缩放尺度和展示尺寸，进而显示模型，实现了在当在不同姿态下加载模型并转换到同一姿态时，模型的尺寸统一，提高了模型显示的准确性。
进一步参考图4，其示出了模型显示方法的又一个实施例的流程400。该模型显示方法的流程400，包括以下步骤：
步骤401，响应于检测到用户进行的目标模型的加载操作，加载用于构建目标模型的模型文件，以构建目标模型。
在本实施例中，模型显示方法运行于其上的电子设备(例如图1所示的移动设备101、102、103)响应于检测到用户进行的目标模型的加载操作，可以加载用于构建上述目标模型的模型文件，以构建目标模型。其中，上述模型文件可以记录有上述目标模型的尺寸信息。
步骤402，提取惯性测量装置采集到的摄像装置的姿态信息，并基于姿态信息确定目标模型的缩放比例和旋转角度。
在本实施例中，上述电子设备可以提取上述惯性测量装置采集到 的上述摄像装置的姿态信息，并基于上述姿态信息确定上述目标模型的缩放比例和旋转角度。其中，上述姿态信息可以是上述摄像装置的旋转矩阵。
步骤403，基于缩放比例和尺寸信息确定目标模型的初始展示尺寸。
在本实施例中，上述电子设备可以基于上述缩放比例和上述尺寸信息确定目标模型的初始展示尺寸，其中，上述初始展示尺寸可以包括上述目标模型的各个组成部分的初始展示尺寸，每一个组成部分的初始展示尺寸可以是上述缩放比例与上述尺寸信息中的、该组成部分的实际尺寸的乘积。
需要说明的是，上述步骤401-403的操作与上述步骤201-203的操作基本相同，在此不再赘述。
步骤404，将目标模型按照旋转角度旋转，并以初始展示尺寸将目标模型呈现在移动设备的显示屏幕中。
在本实施例中，上述电子设备可以将上述目标模型按照上述旋转角度旋转，并以上述初始展示尺寸将上述目标模型呈现在上述移动设备的显示屏幕中。需要说明的是，上述模型文件中可以记录有上述目标模型的默认角度，上述电子设备可以将上述默认角度作为上述目标模型的初始角度，将上述目标模型旋转上述旋转角度，之后以上述初始展示尺寸进行上述目标模型的呈现。
在本实施例中，上述电子设备还可以按照以下步骤确定上述目标模型呈现在上述显示屏幕中的位置：首先，上述电子设备可以确定上述摄像装置的光轴与目标平面的交点在世界坐标系中的坐标，其中，上述目标平面是地平面或天花板平面。之后，上述电子设备可以将所确定的坐标作为上述目标模型在上述世界坐标系的原始坐标，确定与上述原始坐标相对应的屏幕坐标系中的屏幕坐标，其中，上述屏幕坐标系是基于上述显示屏幕而预先建立的平面直角坐标系(例如以上述显示屏幕的一个角的顶点作为原点，以该角的两条边作为坐标轴而预先建立的平面直角坐标系)。实践中，上述电子设备可以利用坐标变换方式确定与上述原始坐标相对应的屏幕坐标系中的屏幕坐标。最后， 上述电子设备可以将上述目标模型呈现在上述屏幕坐标所指示的位置。需要说明的是，上述基于世界坐标系中的坐标确定屏幕坐标系的坐标的方法是是目前广泛研究和应用的公知技术，在此不再赘述。
在将上述目标模型呈现在移动设备的显示屏幕中后，上述电子设备可以执行步骤405-407、步骤408-410或步骤411-413。
步骤405，响应于检测到用户对显示屏幕的第一滑动操作，确定第一滑动操作所涉及的水平像素的数量。
在本实施例中，上述电子设备在以上述初始展示尺寸将上述目标模型呈现在上述移动设备的显示屏幕中之后，响应于检测到用户对显示屏幕的第一滑动操作，可以确定上述第一滑动操作所涉及的水平像素的数量d。其中，上述第一滑动操作可以是单指滑动、双指滑动等任一滑动操作。
步骤406，基于水平像素的数量和显示屏幕中的水平像素的总数量，确定模型的目标旋转角度。
在本实施例中，上述电子设备可以基于水平像素的数量d和显示屏幕中的水平像素的总数量W，按照以下公式，确定模型的目标旋转角度θ'：
Figure PCTCN2018094524-appb-000004
步骤407，以目标模型的竖直轴为旋转轴，将目标模型旋转目标旋转角度后呈现在显示屏幕中。
在本实施例中，上述电子设备可以以目标模型的竖直轴(即目标模型的中心垂直于地平面的轴)为旋转轴，将目标模型旋转目标旋转角度θ'后呈现在显示屏幕中。
步骤408，响应于检测到用户对显示屏幕的第二滑动操作，确定第二滑动操作的起始位置和结束位置，将结束位置映射到相机坐标系的近裁剪面，以确定映射后的点在相机坐标系中的坐标。
在本实施例中，上述电子设备在以上述初始展示尺寸将上述目标模型呈现在上述移动设备的显示屏幕中之后，响应于检测到用户对显示屏幕的第二滑动操作，可以确定第二滑动操作的起始位置和结束位置，将上述结束位置映射到相机坐标系的近裁剪面，以确定映射后的 点在相机坐标系中的坐标，其中，相机坐标系是以摄像装置的光心为原点、以摄像装置的光轴为其中一个坐标轴(即z轴)的直角坐标系。此处，上述近裁剪面可以是上述显示屏幕的等比例缩放后的平面，在上述相机坐标系中，上述近裁剪面中的各个点的坐标与z轴的对应的值为-1(即上述近裁剪面在上述相机坐标系z＝-1的平面中)。需要说明的是，上述第二滑动操作可以是各种滑动操作手势，例如先点击模型或空白处再进行滑动的操作手势，且上述第二滑动操作与第一滑动操作的操作手势不同。
步骤409，将映射后的点在相机坐标系中的坐标作为映射坐标，基于映射坐标、原始坐标和地平面的法向量，确定目标模型在世界坐标系中的第一目标坐标。
在本实施例中，上述电子设备可以将映射后的点在上述相机坐标系中的坐标作为映射坐标，而后基于上述映射坐标、上述原始坐标和地平面的法向量，确定上述目标模型在世界坐标系中的第一目标坐标。具体地，上述电子设备可以以(a,b,-1) T表示将上述映射坐标，以(p x,p y,p z) T表示上述原始坐标，以(n x,n y,n z) T表示将上述地平面的法向量，其中，a、b分别表示上述映射坐标所指示的位置在相机坐标系中的x轴、y轴的分量，p x、p y、p z分别表示上述原始坐标所指示的位置在世界坐标系的x轴、y轴、z轴的分量，n x、n y、n z分别表示上述地平面的法向量在世界坐标系的x轴、y轴、z轴的分量。上述第一目标坐标为：
Figure PCTCN2018094524-appb-000005
步骤410，将与第一目标坐标相对应的屏幕坐标系中的坐标作为第一目标屏幕坐标，将目标模型从屏幕坐标所指示的位置移动至第一目标屏幕坐标所指示的位置。
在本实施例中，上述电子设备可以将与上述第一目标坐标相对应的屏幕坐标系中的坐标作为第一目标屏幕坐标，将目标模型从屏幕坐标所指示的位置移动至第一目标屏幕坐标所指示的位置。需要说明的 是，上述基于世界坐标系中的坐标确定屏幕坐标系的坐标的方法是是目前广泛研究和应用的公知技术，在此不再赘述。
步骤411，将所获取到的姿态信息确定为初始姿态信息，提取惯性测量装置采集到的摄像装置的当前姿态信息。
在本实施例中，上述电子设备在以上述初始展示尺寸将上述目标模型呈现在上述移动设备的显示屏幕中之后，可以将所获取到的姿态信息确定为初始姿态信息，提取惯性测量装置采集到的上述摄像装置的当前姿态信息。实践中，上述电子设备在以上述初始展示尺寸将上述目标模型呈现在上述移动设备的显示屏幕中之后，可以周期性地提取惯性测量装置采集到的姿态信息。
步骤412，响应于基于当前姿态信息和初始姿态信息的比较确定出姿态变化，基于初始姿态信息、当前姿态信息和原始坐标确定目标模型在世界坐标系中的第二目标坐标。
在本实施例中，上述电子设备响应于基于上述当前姿态信息和上述初始姿态信息的比较确定出上述拍摄装置的姿态变化，可以基于初始姿态信息、当前姿态信息和原始坐标确定目标模型在世界坐标系中的第二目标坐标。具体地，上述电子设备可以以(p x,p y,p z) T表示上述原始坐标，则第二目标坐标为：
RR'(p x,p y,p z) T
其中，R为上述初始姿态信息对应的初始旋转矩阵，R'为上述当前姿态信息对应的当前旋转矩阵。
步骤413，将与第二目标坐标相对应的屏幕坐标系中的坐标作为第二目标屏幕坐标，将目标模型从屏幕坐标所指示的位置移动至第二目标屏幕坐标所指示的位置。
在本实施例中，上述电子设备可以将与上述第二目标坐标相对应的屏幕坐标系中的坐标作为第二目标屏幕坐标，将目标模型从屏幕坐标所指示的位置移动至第二目标屏幕坐标所指示的位置。需要说明的是，上述基于世界坐标系中的坐标确定屏幕坐标系的坐标的方法是是目前广泛研究和应用的公知技术，在此不再赘述。
从图4中可以看出，与图2对应的实施例相比，本实施例中的模 型显示方法的流程400突出了在将模型呈现后，基于用户操作进行模型的移动、旋转等步骤。由此，本实施例描述的方案可以基于计算屏幕坐标系与世界坐标系的映射进行模型的移动、旋转等操作，从而不仅提高了模型显示的准确性，还进一步提高了模型显示的灵活性。
进一步参考图5，作为对上述各图所示方法的实现，本申请提供了一种模型显示装置的一个实施例，该装置实施例与图2所示的方法实施例相对应，该装置具体可以应用于移动设备中。其中，移动设备可以安装有摄像装置和惯性测量装置。
如图5所示，本实施例所述的模型显示装置500包括：加载单元501，配置用于响应于检测到用户进行的目标模型的加载操作，加载用于构建上述目标模型的模型文件，以构建上述目标模型，其中，上述模型文件记录有上述目标模型的尺寸信息；第一提取单元502，配置用于提取上述惯性测量装置采集到的上述摄像装置的姿态信息，并基于上述姿态信息确定上述目标模型的缩放比例和旋转角度；第一确定单元503，配置用于基于上述缩放比例和上述尺寸信息确定上述目标模型的初始展示尺寸；第一呈现单元504，配置用于将上述目标模型按照上述旋转角度旋转，并以上述初始展示尺寸将上述目标模型呈现在上述移动设备的显示屏幕中。
在本实施例中，上述加载单元501响应于检测到用户进行的目标模型的加载操作，可以加载用于构建上述目标模型的模型文件，以构建目标模型。其中，上述模型文件可以记录有上述目标模型的尺寸信息。
在本实施例中，上述第一提取单元502可以提取上述惯性测量装置采集到的上述摄像装置的姿态信息，并基于上述姿态信息确定上述目标模型的缩放比例和旋转角度。其中，上述姿态信息可以是上述摄像装置的旋转矩阵。
在本实施例中，上述第一确定单元503可以基于上述缩放比例和上述尺寸信息确定目标模型的初始展示尺寸，其中，上述初始展示尺寸可以包括上述目标模型的各个组成部分的初始展示尺寸，每一个组 成部分的初始展示尺寸可以是上述缩放比例与上述尺寸信息中的、该组成部分的实际尺寸的乘积。
在本实施例中，上述第一呈现单元504可以将上述目标模型按照上述旋转角度旋转，并以上述初始展示尺寸将上述目标模型呈现在上述移动设备的显示屏幕中。
在本实施例的一些可选的实现方式中，上述模型显示装置500还可以包括第二确定单元、第三确定单元和第二呈现单元(图中未示出)。其中，上述第二确定单元可以配置用于响应于检测到用户对上述显示屏幕的第一滑动操作，确定上述第一滑动操作所涉及的水平像素的数量。上述第三确定单元可以配置用于基于上述水平像素的数量和上述显示屏幕中的水平像素的总数量，确定上述模型的目标旋转角度。上述第二呈现单元可以配置用于以上述目标模型的竖直轴为旋转轴，将上述目标模型旋转上述目标旋转角度后呈现在上述显示屏幕中。
在本实施例的一些可选的实现方式中，上述第一呈现单元可以包括第一确定模块、第二确定模块和呈现模块(图中未示出)。其中，上述第一确定模块可以配置用于确定上述摄像装置的光轴与目标平面的交点在世界坐标系中的坐标，其中，上述目标平面是地平面或天花板平面。上述第二确定模块可以配置用于将所确定的坐标作为上述目标模型在上述世界坐标系的原始坐标，确定与上述原始坐标相对应的屏幕坐标系中的屏幕坐标，其中，上述屏幕坐标系是基于上述显示屏幕而预先建立的平面直角坐标系。上述呈现模块可以配置用于将上述目标模型呈现在上述屏幕坐标所指示的位置。
在本实施例的一些可选的实现方式中，上述模型显示装置500还可以包括第四确定单元、第五确定单元和第一移动单元(图中未示出)。其中，上述第四确定单元可以配置用于响应于检测到用户对上述显示屏幕的第二滑动操作，确定上述第二滑动操作的起始位置和结束位置，将上述结束位置映射到相机坐标系的近裁剪面，以确定映射后的点在相机坐标系中的坐标，其中，上述相机坐标系是以上述摄像装置的光心为原点、以上述摄像装置的光轴为其中一个坐标轴的直角坐标系。上述第五确定单元可以配置用于将上述映射后的点在上述相机坐标系 中的坐标作为映射坐标，基于上述映射坐标、上述原始坐标和地平面的法向量，确定上述目标模型在上述世界坐标系中的第一目标坐标。上述第一移动单元可以配置用于将与上述第一目标坐标相对应的屏幕坐标系中的坐标作为第一目标屏幕坐标，将上述目标模型从上述屏幕坐标所指示的位置移动至上述第一目标屏幕坐标所指示的位置。
在本实施例的一些可选的实现方式中，上述模型显示装置500还可以包括第二提取单元、第六确定单元和第二移动单元(图中未示出)。其中，上述第二提取单元，配置用于将所获取到的姿态信息确定为初始姿态信息，提取上述惯性测量装置采集到的上述摄像装置的当前姿态信息。上述第六确定单元可以配置用于响应于基于上述当前姿态信息和上述初始姿态信息的比较确定出姿态变化，基于上述初始姿态信息、上述当前姿态信息和上述原始坐标确定上述目标模型在上述世界坐标系中的第二目标坐标。上述第二移动单元可以配置用于将与上述第二目标坐标相对应的屏幕坐标系中的坐标作为第二目标屏幕坐标，将上述目标模型从上述屏幕坐标所指示的位置移动至上述第二目标屏幕坐标所指示的位置。
在本实施例的一些可选的实现方式中，上述第一提取单元502可以进一步配置用于响应于确定上述摄像装置的光轴与目标平面的夹角不大于预设角度，提取预设缩放比例和预设旋转角度，将上述预设缩放比例确定为上述目标模型的缩放比例，将上述预设旋转角度确定为上述目标模型的旋转角度。
本申请的上述实施例提供的装置，通过加载单元501在检测到用户在开启摄像装置后进行的目标模型的加载操作后加载模型文件，以构建目标模型，而后第一提取单元502提取姿态信息，以便确定目标模型的缩放比例和旋转角度，之后第一确定单元503基于缩放比例和模型文件中的尺寸信息确定目标模型的初始展示尺寸，最后第一呈现单元504将目标模型按照所确定的旋转角度旋转，并以初始展示尺寸进行展示，从而可以基于当前的姿态信息确定缩放尺度和展示尺寸，进而显示模型，实现了在当在不同姿态下加载模型并转换到同一姿态时，模型的尺寸统一，提高了模型显示的准确性。
下面参考图6，其示出了适于用来实现本申请实施例的移动设备的计算机系统600的结构示意图。图6示出的移动设备仅仅是一个示例，不应对本申请实施例的功能和使用范围带来任何限制。
如图6所示，计算机系统600包括中央处理单元(CPU)601，其可以根据存储在只读存储器(ROM)602中的程序或者从存储部分608加载到随机访问存储器(RAM)603中的程序而执行各种适当的动作和处理。在RAM 603中，还存储有系统600操作所需的各种程序和数据。CPU 601、ROM 602以及RAM 603通过总线604彼此相连。输入/输出(I/O)接口605也连接至总线604。
以下部件连接至I/O接口605：包括触摸屏、触摸板等的输入部分606；包括诸如液晶显示器(LCD)等以及扬声器等的输出部分607；包括硬盘等的存储部分608；以及包括诸如LAN卡、调制解调器等的网络接口卡的通信部分609。通信部分609经由诸如因特网的网络执行通信处理。驱动器610也根据需要连接至I/O接口605。可拆卸介质611，诸如半导体存储器等等，根据需要安装在驱动器610上，以便于从其上读出的计算机程序根据需要被安装入存储部分608。
特别地，根据本公开的实施例，上文参考流程图描述的过程可以被实现为计算机软件程序。例如，本公开的实施例包括一种计算机程序产品，其包括承载在计算机可读介质上的计算机程序，该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，该计算机程序可以通过通信部分609从网络上被下载和安装，和/或从可拆卸介质611被安装。在该计算机程序被中央处理单元(CPU)601执行时，执行本申请的方法中限定的上述功能。需要说明的是，本申请所述的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦 式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本申请中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本申请中，计算机可读的信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读的信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：无线、电线、光缆、RF等等，或者上述的任意合适的组合。
附图中的流程图和框图，图示了按照本申请各种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，该模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
描述于本申请实施例中所涉及到的单元可以通过软件的方式实现，也可以通过硬件的方式来实现。所描述的单元也可以设置在处理器中，例如，可以描述为：一种处理器包括加载单元、第一提取单元、第一确定单元和第一呈现单元。其中，这些单元的名称在某种情况下并不构成对该单元本身的限定，例如，加载单元还可以被描述为“加 载目标模型的模型文件的单元”。
作为另一方面，本申请还提供了一种计算机可读介质，该计算机可读介质可以是上述实施例中描述的装置中所包含的；也可以是单独存在，而未装配入该装置中。上述计算机可读介质承载有一个或者多个程序，当上述一个或者多个程序被该装置执行时，使得该装置：响应于检测到用户在开启摄像装置后进行的目标模型的加载操作，加载用于构建该目标模型的模型文件，以构建该目标模型，其中，该模型文件记录有该目标模型的尺寸信息；提取惯性测量装置采集到的姿态信息，并基于该姿态信息确定该目标模型的缩放比例和旋转角度；基于该缩放比例和该尺寸信息确定该目标模型的初始展示尺寸；将该目标模型按照该旋转角度旋转，并以该初始展示尺寸将该目标模型呈现在该移动设备的显示屏幕中。
以上描述仅为本申请的较佳实施例以及对所运用技术原理的说明。本领域技术人员应当理解，本申请中所涉及的发明范围，并不限于上述技术特征的特定组合而成的技术方案，同时也应涵盖在不脱离上述发明构思的情况下，由上述技术特征或其等同特征进行任意组合而形成的其它技术方案。例如上述特征与本申请中公开的(但不限于)具有类似功能的技术特征进行互相替换而形成的技术方案。

Claims (14)

  1. 一种用于移动设备的模型显示方法，其特征在于，所述移动设备安装有摄像装置和惯性测量装置，所述方法包括：
    响应于检测到用户进行的目标模型的加载操作，加载用于构建所述目标模型的模型文件，以构建所述目标模型，其中，所述模型文件记录有所述目标模型的尺寸信息；
    提取所述惯性测量装置采集到的所述摄像装置的姿态信息，并基于所述姿态信息确定所述目标模型的缩放比例和旋转角度；
    基于所述缩放比例和所述尺寸信息确定所述目标模型的初始展示尺寸；
    将所述目标模型按照所述旋转角度旋转，并以所述初始展示尺寸将所述目标模型呈现在所述移动设备的显示屏幕中。
  2. 根据权利要求1所述的模型显示方法，其特征在于，在所述将所述目标模型按照所述旋转角度旋转，并以所述初始展示尺寸将所述目标模型呈现在所述移动设备的显示屏幕中之后，所述方法还包括：
    响应于检测到用户对所述显示屏幕的第一滑动操作，确定所述第一滑动操作所涉及的水平像素的数量；
    基于所述水平像素的数量和所述显示屏幕中的水平像素的总数量，确定所述模型的目标旋转角度；
    以所述目标模型的竖直轴为旋转轴，将所述目标模型旋转所述目标旋转角度后呈现在所述显示屏幕中。
  3. 根据权利要求1或2所述的模型显示方法，其特征在于，所述将所述目标模型按照所述旋转角度旋转，并以所述初始展示尺寸将所述目标模型呈现在所述移动设备的显示屏幕中，包括：
    确定所述摄像装置的光轴与目标平面的交点在世界坐标系中的坐标，其中，所述目标平面是地平面或天花板平面；
    将所确定的坐标作为所述目标模型在所述世界坐标系的原始坐 标，确定与所述原始坐标相对应的屏幕坐标系中的屏幕坐标，其中，所述屏幕坐标系是基于所述显示屏幕而预先建立的平面直角坐标系；
    将所述目标模型呈现在所述屏幕坐标所指示的位置。
  4. 根据权利要求3所述的模型显示方法，其特征在于，在所述将所述目标模型按照所述旋转角度旋转，并以所述初始展示尺寸将所述目标模型呈现在所述移动设备的显示屏幕中之后，所述方法还包括：
    响应于检测到用户对所述显示屏幕的第二滑动操作，确定所述第二滑动操作的起始位置和结束位置，将所述结束位置映射到相机坐标系的近裁剪面，以确定映射后的点在相机坐标系中的坐标，其中，所述相机坐标系是以所述摄像装置的光心为原点、以所述摄像装置的光轴为其中一个坐标轴的直角坐标系；
    将所述映射后的点在所述相机坐标系中的坐标作为映射坐标，基于所述映射坐标、所述原始坐标和地平面的法向量，确定所述目标模型在所述世界坐标系中的第一目标坐标；
    将与所述第一目标坐标相对应的屏幕坐标系中的坐标作为第一目标屏幕坐标，将所述目标模型从所述屏幕坐标所指示的位置移动至所述第一目标屏幕坐标所指示的位置。
  5. 根据权利要求3所述的模型显示方法，其特征在于，在所述将所述目标模型按照所述旋转角度旋转，并以所述初始展示尺寸将所述目标模型呈现在所述移动设备的显示屏幕中之后，所述方法还包括：
    将所获取到的姿态信息确定为初始姿态信息，提取所述惯性测量装置采集到的所述摄像装置的当前姿态信息；
    响应于基于所述当前姿态信息和所述初始姿态信息的比较确定出姿态变化，基于所述初始姿态信息、所述当前姿态信息和所述原始坐标确定所述目标模型在所述世界坐标系中的第二目标坐标；
    将与所述第二目标坐标相对应的屏幕坐标系中的坐标作为第二目标屏幕坐标，将所述目标模型从所述屏幕坐标所指示的位置移动至所述第二目标屏幕坐标所指示的位置。
  6. 根据权利要求1所述的模型显示方法，其特征在于，所述基于所述姿态信息确定所述目标模型的缩放比例和旋转角度，包括：
    响应于确定所述摄像装置的光轴与目标平面的夹角不大于预设角度，提取预设缩放比例和预设旋转角度，将所述预设缩放比例确定为所述目标模型的缩放比例，将所述预设旋转角度确定为所述目标模型的旋转角度。
  7. 一种用于移动设备的模型显示装置，其特征在于，所述移动设备安装有摄像装置和惯性测量装置，所述模型显示装置包括：
    加载单元，配置用于响应于检测到用户进行的目标模型的加载操作，加载用于构建所述目标模型的模型文件，以构建所述目标模型，其中，所述模型文件记录有所述目标模型的尺寸信息；
    第一提取单元，配置用于提取所述惯性测量装置采集到的所述摄像装置的姿态信息，并基于所述姿态信息确定所述目标模型的缩放比例和旋转角度；
    第一确定单元，配置用于基于所述缩放比例和所述尺寸信息确定所述目标模型的初始展示尺寸；
    第一呈现单元，配置用于将所述目标模型按照所述旋转角度旋转，并以所述初始展示尺寸将所述目标模型呈现在所述移动设备的显示屏幕中。
  8. 根据权利要求7所述的模型显示装置，其特征在于，所述模型显示装置还包括：
    第二确定单元，配置用于响应于检测到用户对所述显示屏幕的第一滑动操作，确定所述第一滑动操作所涉及的水平像素的数量；
    第三确定单元，配置用于基于所述水平像素的数量和所述显示屏幕中的水平像素的总数量，确定所述模型的目标旋转角度；
    第二呈现单元，配置用于以所述目标模型的竖直轴为旋转轴，将所述目标模型旋转所述目标旋转角度后呈现在所述显示屏幕中。
  9. 根据权利要求7或8所述的模型显示装置，其特征在于，所述第一呈现单元包括：
    第一确定模块，配置用于确定所述摄像装置的光轴与目标平面的交点在世界坐标系中的坐标，其中，所述目标平面是地平面或天花板平面；
    第二确定模块，配置用于将所确定的坐标作为所述目标模型在所述世界坐标系的原始坐标，确定与所述原始坐标相对应的屏幕坐标系中的屏幕坐标，其中，所述屏幕坐标系是基于所述显示屏幕而预先建立的平面直角坐标系；
    呈现模块，配置用于将所述目标模型呈现在所述屏幕坐标所指示的位置。
  10. 根据权利要求9所述的模型显示装置，其特征在于，所述模型显示装置还包括：
    第四确定单元，配置用于响应于检测到用户对所述显示屏幕的第二滑动操作，确定所述第二滑动操作的起始位置和结束位置，将所述结束位置映射到相机坐标系的近裁剪面，以确定映射后的点在相机坐标系中的坐标，其中，所述相机坐标系是以所述摄像装置的光心为原点、以所述摄像装置的光轴为其中一个坐标轴的直角坐标系；
    第五确定单元，配置用于将所述映射后的点在所述相机坐标系中的坐标作为映射坐标，基于所述映射坐标、所述原始坐标和地平面的法向量，确定所述目标模型在所述世界坐标系中的第一目标坐标；
    第一移动单元，配置用于将与所述第一目标坐标相对应的屏幕坐标系中的坐标作为第一目标屏幕坐标，将所述目标模型从所述屏幕坐标所指示的位置移动至所述第一目标屏幕坐标所指示的位置。
  11. 根据权利要求9所述的模型显示装置，其特征在于，所述模型显示装置还包括：
    第二提取单元，配置用于将所获取到的姿态信息确定为初始姿态 信息，提取所述惯性测量装置采集到的所述摄像装置的当前姿态信息；
    第六确定单元，配置用于响应于基于所述当前姿态信息和所述初始姿态信息的比较确定出姿态变化，基于所述初始姿态信息、所述当前姿态信息和所述原始坐标确定所述目标模型在所述世界坐标系中的第二目标坐标；
    第二移动单元，配置用于将与所述第二目标坐标相对应的屏幕坐标系中的坐标作为第二目标屏幕坐标，将所述目标模型从所述屏幕坐标所指示的位置移动至所述第二目标屏幕坐标所指示的位置。
  12. 根据权利要求7所述的模型显示装置，其特征在于，所述第一提取单元进一步配置用于：
    响应于确定所述摄像装置的光轴与目标平面的夹角不大于预设角度，提取预设缩放比例和预设旋转角度，将所述预设缩放比例确定为所述目标模型的缩放比例，将所述预设旋转角度确定为所述目标模型的旋转角度。
  13. 一种移动设备，包括：
    一个或多个处理器；
    存储装置，用于存储一个或多个程序；
    摄像装置，用于采集图像；
    惯性测量装置，用于采集姿态信息；
    当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现如权利要求1-6中任一所述的方法。
  14. 一种计算机可读存储介质，其上存储有计算机程序，其特征在于，该程序被处理器执行时实现如权利要求1-6中任一所述的方法。
PCT/CN2018/094524 2017-07-05 2018-07-04 模型显示方法和装置 WO2019007372A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710540596.8A CN107329671B (zh) 2017-07-05 2017-07-05 模型显示方法和装置
CN201710540596.8 2017-07-05

Publications (1)

Publication Number Publication Date
WO2019007372A1 true WO2019007372A1 (zh) 2019-01-10

Family

ID=60196653

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/094524 WO2019007372A1 (zh) 2017-07-05 2018-07-04 模型显示方法和装置

Country Status (2)

Country Link
CN (1) CN107329671B (zh)
WO (1) WO2019007372A1 (zh)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107329671B (zh) * 2017-07-05 2020-06-30 北京京东尚科信息技术有限公司 模型显示方法和装置
CN109544702B (zh) * 2018-11-21 2023-05-02 北京像素软件科技股份有限公司 三维变换处理方法、装置、电子设备及存储介质
CN111161429B (zh) * 2019-12-19 2023-09-26 武汉中科医疗科技工业技术研究院有限公司 模型图像显示方法、装置、计算机设备和存储介质
CN111640206A (zh) * 2020-06-08 2020-09-08 上海商汤智能科技有限公司 一种动态控制方法及装置
CN114116081B (zh) * 2020-08-10 2023-10-27 抖音视界有限公司 交互式动态流体效果处理方法、装置及电子设备
CN114170363A (zh) * 2021-12-10 2022-03-11 北京字跳网络技术有限公司 3d目标点渲染方法、装置、设备及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103473692A (zh) * 2013-06-17 2013-12-25 展讯通信（上海）有限公司 一种三维购物平台显示系统
CN104134229A (zh) * 2014-08-08 2014-11-05 李成 实时交互的增强现实系统以及方法
CN106383655A (zh) * 2016-09-19 2017-02-08 北京小度互娱科技有限公司 在全景播放过程中控制视角转换的交互控制方法及装置
US20170153787A1 (en) * 2015-10-14 2017-06-01 Globalive Xmg Jv Inc. Injection of 3-d virtual objects of museum artifact in ar space and interaction with the same
CN107329671A (zh) * 2017-07-05 2017-11-07 北京京东尚科信息技术有限公司 模型显示方法和装置

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9129375B1 (en) * 2012-04-25 2015-09-08 Rawles Llc Pose detection
CN106896940B (zh) * 2017-02-28 2020-01-07 杭州乐见科技有限公司 虚拟物品呈现效果控制方法及装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103473692A (zh) * 2013-06-17 2013-12-25 展讯通信（上海）有限公司 一种三维购物平台显示系统
CN104134229A (zh) * 2014-08-08 2014-11-05 李成 实时交互的增强现实系统以及方法
US20170153787A1 (en) * 2015-10-14 2017-06-01 Globalive Xmg Jv Inc. Injection of 3-d virtual objects of museum artifact in ar space and interaction with the same
CN106383655A (zh) * 2016-09-19 2017-02-08 北京小度互娱科技有限公司 在全景播放过程中控制视角转换的交互控制方法及装置
CN107329671A (zh) * 2017-07-05 2017-11-07 北京京东尚科信息技术有限公司 模型显示方法和装置

Also Published As

Publication number Publication date
CN107329671B (zh) 2020-06-30
CN107329671A (zh) 2017-11-07

Similar Documents

Publication Publication Date Title
CN107329671B (zh) 模型显示方法和装置
US10643386B2 (en) Presenting image transition sequences between viewing locations
EP3095092B1 (en) Method and apparatus for visualization of geo-located media contents in 3d rendering applications
US10600150B2 (en) Utilizing an inertial measurement device to adjust orientation of panorama digital images
US9639988B2 (en) Information processing apparatus and computer program product for processing a virtual object
US9268410B2 (en) Image processing device, image processing method, and program
EP2583254B1 (en) Mobile device based content mapping for augmented reality environment
CN106846497B (zh) 应用于终端的呈现三维地图的方法和装置
US20160335796A1 (en) Method and apparatus for visualization of geo-located media contents in 3d rendering applications
US20130208005A1 (en) Image processing device, image processing method, and program
US10606360B2 (en) Three-dimensional tilt and pan navigation using a single gesture
JP6260241B2 (ja) ユーザの入力を受け付けるシステム、プログラムおよび方法
WO2023207522A1 (zh) 视频合成方法、装置、设备、介质及产品
CN110662015A (zh) 用于显示图像的方法及装置
CN111710047A (zh) 一种信息展示方法、装置和电子设备
CN114049403A (zh) 一种多角度三维人脸重建方法、装置及存储介质
CN106919260B (zh) 网页操作方法和装置
US11562538B2 (en) Method and system for providing a user interface for a 3D environment
CN115578432A (zh) 图像处理方法、装置、电子设备及存储介质
GB2555521A (en) Improved object painting through use of perspectives or transfers in a digital medium environment
WO2023035958A1 (zh) 界面显示方法、装置、电子设备和存储介质
US20230409121A1 (en) Display control method, apparatus, electronic device, medium, and program product
Ohta et al. Photo-based Desktop Virtual Reality System Implemented on a Web-browser
CN118051123A (zh) 基于全景3d场景的浏览方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18827984

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205 DATED 09/06/2020)

122 Ep: pct application non-entry in european phase

Ref document number: 18827984

Country of ref document: EP

Kind code of ref document: A1