WO2019128302A1 - 互动操作实现方法、装置及客户端设备 - Google Patents

互动操作实现方法、装置及客户端设备 Download PDF

Info

Publication number
WO2019128302A1
WO2019128302A1 PCT/CN2018/104437 CN2018104437W WO2019128302A1 WO 2019128302 A1 WO2019128302 A1 WO 2019128302A1 CN 2018104437 W CN2018104437 W CN 2018104437W WO 2019128302 A1 WO2019128302 A1 WO 2019128302A1
Authority
WO
WIPO (PCT)
Prior art keywords
street view
virtual
scene
information
client
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2018/104437
Other languages
English (en)
French (fr)
Inventor
曾志荣
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
UC MOBILE CO Ltd
UC Mobile Ltd
Original Assignee
UC MOBILE CO Ltd
UC Mobile Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by UC MOBILE CO Ltd, UC Mobile Ltd filed Critical UC MOBILE CO Ltd
Publication of WO2019128302A1 publication Critical patent/WO2019128302A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/20—Input arrangements for video game devices
    • A63F13/21—Input arrangements for video game devices characterised by their sensors, purposes or types
    • A63F13/216—Input arrangements for video game devices characterised by their sensors, purposes or types using geographical information, e.g. location of the game device or player using GPS
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/20—Input arrangements for video game devices
    • A63F13/21—Input arrangements for video game devices characterised by their sensors, purposes or types
    • A63F13/217—Input arrangements for video game devices characterised by their sensors, purposes or types using environment-related information, i.e. information generated otherwise than by the player, e.g. ambient temperature or humidity
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/50—Controlling the output signals based on the game progress
    • A63F13/52—Controlling the output signals based on the game progress involving aspects of the displayed game scene
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/80—Special adaptations for executing a specific game genre or game mode
    • A63F13/803—Driving vehicles or craft, e.g. cars, airplanes, ships, robots or tanks
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/80—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game specially adapted for executing a specific type of game
    • A63F2300/8017—Driving on land or water; Flying
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/80—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game specially adapted for executing a specific type of game
    • A63F2300/8058—Virtual breeding, e.g. tamagotchi
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/80—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game specially adapted for executing a specific type of game
    • A63F2300/8082—Virtual reality

Definitions

  • the present invention relates to the field of information technology, and in particular, to a method, an apparatus, and a client device for performing an interactive operation in a virtual scene based on a real scene.
  • FIGS. 1A and 1B show examples of screenshots of the Pokémon GO game.
  • FIGS. 1A and 1B Although the field and the real scene are involved, as shown in FIGS. 1A and 1B, the game only uses the LBS and AR technologies respectively (FIG. 1A is an example of a map mode, and FIG. 1B is an example of an AR mode), not in a specific scenario. Integrate related technologies to give a more realistic experience.
  • the present invention proposes a new interactive operation scheme, which generates a virtual street view map by acquiring existing street view information, and then superimposes the scene frame according to a specific application scenario, thereby being able to be in existing network conditions. Based on the calculations that the existing mobile device can withstand, the user is provided with a flexible and highly integrated interactive scene.
  • the solution is also able to further integrate AR technology to provide users with a more novel and interesting experience.
  • a method for implementing an interactive operation implemented on a client side includes: transmitting geographic location information to a server; and receiving related street view information related to street view information, wherein the street view information is the The street view information associated with the geographic location information queried by the server; the virtual street view map based on the street view information is merged with the scene frame to obtain a virtual scene; the virtual scene is displayed; and in response to user input, the virtual scene Interact in the scene.
  • the existing real-time image information can be utilized to obtain a specific application scenario combined with the depth of the real environment through reasonable transformation, thereby providing the user with an immersive application experience with achievable calculation amount and network transmission requirements.
  • the fusion of the virtual street view map and the scene frame may be to add street view information from the loaded scene frame, but the opposite is true.
  • the street view information may be added to a loaded scene frame to form a virtual street view map loaded with a scene frame, the virtual scene being a virtual street view map loaded with a scene frame; or constructing the based on the street view data a virtual street view map on which the scene frame is loaded to obtain the virtual scene. Therefore, the manner of constructing the virtual scene can be flexibly selected based on the specific application scenario.
  • the related street view information may further include map information associated with the street view information, and the configuration of the virtual street view map is further based on the map information.
  • map information By introducing map information, it is more convenient and efficient to implement the construction of virtual street view maps.
  • the scene framework can include many implementations.
  • the scene framework may include an operation panel that enables the user to interact with the displayed virtual scene or objects within it.
  • the environment or display style of the virtual street view map may be determined based at least in part on the scene frame.
  • the line of sight height in the virtual scene is determined based at least in part on the scene frame.
  • the virtual street view map may be generated by splicing from the received street view information picture according to a predetermined map algorithm. Therefore, compared with the simple modeling or map mode in the prior art, the user can provide a more realistic use scenario at a much lower cost.
  • the interactive form of the interaction may be determined at least in part by the scene framework.
  • the interactive operation implementation method of the present invention may further relate to AR technology.
  • an AR object to be loaded may be acquired based on the geographic location information, the street view information, and/or the scene frame; and the AR object is loaded in the virtual scene.
  • the loading of the above AR object may be conditional.
  • an AR object may be loaded in a virtual scene, but need not be displayed immediately, but may be triggered by a specific operation, angle, task completion, and the like.
  • the interaction described above may also include an interaction with an AR object loaded and displayed in the virtual street view map. Thereby, the application scenario and the applicable scope of the solution can be further enriched, and the participation and interest of the interaction can be improved.
  • the loading of the AR object can also be responsive to the shooting of the target object. Further, the relevant data for loading the AR object is used to determine current geographic location information and/or generation and update of the virtual scene.
  • the geographic location information submitted by the client may be the geographical location information selected by the user or the current geographic location information of the user.
  • the virtual scene is constructed according to the selected geographic information, so that the user can experience the virtual scenes established in various actual locations under various scene frameworks without leaving the home, thereby enriching the user experience. If the current geographical location information is based on, the virtual scene can be constructed according to the actual location of the user, further adding the user's sense of presence.
  • the virtual scene may be presented in a first perspective of the user.
  • the moving trajectory of the mobile end can be captured by using a gyroscope built in the mobile terminal, and reflected in real time on the display content of the virtual scene, thereby improving the immersive feeling of the application.
  • current geographic location information, speed information, and/or perspective information of, for example, a mobile terminal held by a user may be sent to the server in real time; and real-time according to the current geographic location, speed information, and/or perspective information. Change the perspective and content of the virtual Street View map displayed.
  • the interactive operation scheme of the present invention may also involve multi-user networking operations.
  • the user may obtain information (other user information) of one or more other clients connected to the network and display the avatars of the one or more other clients in the virtual street view map of the user.
  • the user can interact with the avatars of other clients via their avatars. This further enhances the participation of specific scenarios and socializes the application.
  • a method for implementing an interaction operation implemented on a server side includes: acquiring geographic location information sent by a client; querying street view information associated with the geographic location information or based on the street view The virtual street view map information generated by the information; the street view information or the virtual street view map information is sent to the client, so that the client displays a virtual scene in which the client user can perform an interaction operation, wherein the virtual scene is The virtual street view map is merged with the scene frame.
  • the method may further include: transmitting an AR object to be loaded in the virtual scene based on the geographical location information, the street view information, the scene frame, and/or a predetermined presentation condition, to For display in the virtual scene.
  • the method may further comprise: obtaining information of one or more other clients of the network; synchronizing the avatars of the one or more other clients in the virtual space platform; and The avatars of other clients are sent to the client for presentation in the virtual scene of the client.
  • the method may further include: receiving an operation of the avatar of the client and/or other client in real time and performing synchronization processing; and delivering the synchronization process to implement the avatar between the client and other clients. Interaction.
  • an apparatus for implementing an interactive operation implemented on a client side includes: a location information sending unit for transmitting geographic location information to a server; and a street view information receiving unit for receiving and viewing a street view Information related to the street view information, wherein the street view information is street view information associated with the geographic location information queried by the server; the virtual scene construction unit is configured to merge the virtual street view map based on the street view information with the scene frame Obtaining a virtual scene; a virtual scene display unit, configured to display the virtual scene; and an interaction unit, configured to perform an interaction operation in the virtual scene in response to user input.
  • the interaction operation implementation apparatus may further include: a target object acquisition unit, acquiring a target object to be loaded based on the geographical location information, the street view information, and/or the scene framework; and a target object loading unit, Loading the target object in the virtual scene.
  • a target object acquisition unit acquiring a target object to be loaded based on the geographical location information, the street view information, and/or the scene framework
  • a target object loading unit Loading the target object in the virtual scene.
  • the target object loading unit displays the loaded target object in the virtual scene if a predetermined display condition is met.
  • the interactive operation implementing device may further comprise an information synchronization unit, configured to synchronously display the avatars of the one or more other clients in the virtual street view map.
  • an apparatus for implementing an interactive operation implemented on a server side including:
  • a location information obtaining unit configured to query street view information associated with the geographic location information or virtual street view map information generated based on the street view information;
  • a street view information query unit configured to query street view information associated with the geographic location information or virtual street view map information generated based on the street view information
  • a street view information sending unit configured to send the street view information or the virtual street view map information to a client, so that the client displays a virtual scene in which the client user can perform an interaction operation, wherein the virtual scene is virtual
  • the street view map is merged with the scene frame.
  • the interaction operation implementation device further includes:
  • a building unit for constructing a virtual street view map based on street view information, and/or loading a scene frame on the virtual street view map to construct a virtual scene space.
  • the street view information sending unit is configured to send an AR object to be loaded in the virtual scene based on the geographical location information, the street view information, the scene frame, and/or a predetermined display condition, for Displayed in a virtual scene.
  • the interaction operation implementation device further includes:
  • a synchronization unit configured to synchronize operations and locations of the avatars of the plurality of clients in the virtual space platform.
  • a client device comprising: a processor; and a memory having stored thereon executable code that, when executed by the processor, causes the processor The interactive operation implementation method implemented on the client side as described above is performed.
  • a server comprising: a processor; and a memory having stored thereon executable code that, when executed by the processor, causes the processor to execute as above The method for implementing an interactive operation implemented on the server side.
  • a non-transitory machine readable storage medium having stored thereon executable code that, when executed by a processor of an electronic device, causes the processor to execute The interactive operation implementation method as described above.
  • the present invention can provide a virtual scene that is highly integrated with the real environment but is flexible and variable, thereby constructing a virtual scene based on the real-life map image for user interaction, thereby further improving the user experience.
  • Figures 1A and 1B show examples of screenshots of the "Pokémon GO" game.
  • FIG. 2 is a schematic diagram of an environment for implementing an embodiment of the present invention.
  • FIG. 3 shows a schematic diagram of an interactive operation scheme in accordance with one embodiment of the present invention.
  • 4A and 4B illustrate an example of a racing application scenario in accordance with one embodiment of the present invention.
  • FIG. 5 is a flow chart showing a method of implementing an interactive operation on the client side according to an embodiment of the present invention.
  • FIG. 6 is a flow chart showing a method of implementing an interactive operation on the server side according to an embodiment of the present invention.
  • FIG. 7 shows a schematic diagram of an interactive operation implementing apparatus implemented on a client side according to an embodiment of the present invention.
  • FIG. 8 shows a schematic diagram of an interactive operation implementing apparatus implemented on a server side according to an embodiment of the present invention.
  • Figure 9 is a client device in accordance with one embodiment of the present invention.
  • the present invention achieves a good combination of real scenes and virtual scenes with relatively low computational and network requirements, thereby providing users with a deep immersive experience.
  • FIG. 2 is a schematic diagram of an environment 200 for implementing an embodiment of the present invention.
  • mobile terminal 10 in environment 200 can effect messaging with server 20 via network 40.
  • the server 20 can acquire the content required by the mobile terminal 10 by accessing the database 30.
  • the database 30 may be a street view information database storing street view information, and the server 20 may obtain the street view information requested by the mobile terminal 10 by accessing the database.
  • the mobile terminals (e.g., between 10_1 and 10_2 or 10_N) may optionally communicate with one another via the network 40.
  • Network 40 may be a network for information transfer in a broad sense, and may include one or more communication networks, such as a wireless communication network, the Internet, a private area network, a local area network, a metropolitan area network, a wide area network, or a cellular data network. In one embodiment, network 40 may also include a satellite network, thereby transmitting GPS signals of mobile terminal 10 to server 20. It should be noted that if additional modules are added to or removed from the illustration 200, the underlying concepts of the example embodiments of the present invention are not altered. In addition, although a two-way arrow from the database 30 to the server 20 is shown in the figure for convenience of explanation, those skilled in the art can understand that the above-mentioned data can also be transmitted and received through the network 40.
  • Mobile terminal 10 is any suitable portable electronic device that can be used for network access, including but not limited to a smart phone, tablet or other portable client.
  • Server 20 is any server that is accessible over the network to provide the information needed for the interactive service.
  • a plurality of mobile terminals 10-1...N and a single server 20 and database 30 are shown in the figure, one or a part of the mobile terminals will be described in the following description (for example, the mobile terminal 10-1), However, it should be immediately apparent to those skilled in the art that the above 1...N mobile terminals are intended to represent a plurality of mobile terminals existing in a real network, and the illustrated single server 20 and database 30 are intended to indicate that the technical solution of the present invention relates to a server and The operation of the database.
  • the specific numbered mobile end and the individual servers and databases are detailed for at least convenience of explanation, and do not imply restrictions on the type or location of the mobile end and the server.
  • FIG. 3 shows a schematic diagram of an interactive operation scheme in accordance with one embodiment of the present invention.
  • the implementation environment of the interactive operation scheme at least one server S and at least one client A are included.
  • the implementation environment includes at least two clients A and B.
  • at least one server S may be the server 20 in the environment shown in FIG. 2, and the client A may be any mobile terminal 10 shown in FIG. 2.
  • the server S is connected to the Street View information server via a network or other connection to obtain the Street View information of the location of the client A.
  • client A sends a certain geographic location information to server S.
  • the geographical location information sent by client A to server S can theoretically be any geographic location information selected by the user of client A.
  • the user may select a certain coordinate in the map application, or even an image screenshot of the specific street view selected in the street view mode of the map application, and send the corresponding information to the server S.
  • client A may send its current geographic location information.
  • the client A can use the LBS (Location Based Service) to obtain the geographical location information of the location, and send it to the server S, for example, via a wireless communication network (for example, 4G network or WiFi).
  • LBS Location Based Service
  • the geographic location information required by client A is different based on the accuracy requirements of geographic location information in different application scenarios. Considering that the GPS positioning accuracy of the smartphone outdoors is usually less than 10 meters, it is possible to design an application scenario based on this accuracy requirement.
  • the application scene of the first angle of view following mode for example, the user of client A may be required to additionally have a handheld GPS device with an accuracy of less than one meter or even several centimeters, and the device may Extremely accurate geographic location information is sent to Client A in real time.
  • client A may first be required to target a particular direction, for example, or to target a particular target, as a baseline starting point for the interaction. Then, through the built-in compass, gyroscope, etc. of the client A, the LBS service is combined with the LBS service to correct the position and orientation of the client A in real time.
  • the server S receives the geographical location information transmitted by the client A, and queries the street view information database 30 shown in FIG. 2 for the street view information associated with the geographical location information.
  • street view information refers to a 360-degree panoramic real-life image capable of displaying a street, an indoor, a public building, or other environment.
  • "Street View Map” is a real-life map service, for example, an image that can be queried by a user in a street view mode of Google Maps or a panoramic mode of a Baidu map, which is aimed at an actual scene and has a variable viewing angle. Since the raw data of the street view map is usually collected by a rotatable lens mounted on a collecting vehicle traveling on a specific road surface, the real scene information provided by the street view map is usually the height of the line of the installation lens of the vehicle, the full view angle and the movable Query continuous street view information.
  • the server S can directly access the street view information database collected and maintained by the existing "Street View Map" service, for example, directly accessing the Street View information database of Baidu or Gaode maps.
  • the server S may also access a street view information database that it collects, builds, and maintains. This is especially useful when the range of virtual street view maps is limited. For example, real-time information data of the Forbidden City in Beijing (including panoramic views of indoor and outdoor buildings) can be collected to construct a street view (real view) database of the Forbidden City, which can be accessed by the server S.
  • a virtual street view map can be constructed.
  • the virtual street view map is built on the server side (ie, on server S) and sent to client A via the network.
  • Client A directly obtains the above virtual street view map and performs display or subsequent processing.
  • the server S can send street view information to the client A and complete the construction of the virtual street view map at the client A.
  • the server S can complete the partial construction of the virtual street view map and complete the construction of the remaining portion at the client A.
  • the construction of the virtual street view map may even be completed prior to receiving geographic location information from client A.
  • the server S may pre-store the virtual street view map of some specific geographic locations, or pre-deliver the virtual street view map (or a part thereof) to the client A.
  • the server S receives the geographical location information from the client A that matches the specific geographic location, the virtual street view map is directly transmitted to the client A or the virtual street view map that has been downloaded on the client A is enabled. This is especially true when a particular location is a location that is frequently requested or needs to be commercialized, such as a popular attraction or a business event promotion goal.
  • the "relevant street view information" transmitted by the server S to the client A refers to information related to the above-mentioned street view information (ie, street view information associated with the geographical location information queried by the server), which may be the foregoing
  • the street view information itself may be part or all of the virtual street view map built based on the street view information, or may be an instruction to enable the virtual street view map stored on the client A local machine.
  • “virtual street view map” refers to a map space constructed with real-life images and highly realistic and reductive.
  • the "virtual street view map” may optionally refer to a map space that is variable in perspective, similar to a three-dimensional real scene.
  • the virtual street view map in the present invention may be generated by splicing from the received street view information picture according to a predetermined map algorithm. Thereby, the authenticity of the use can be provided at an acceptable computational cost.
  • the "virtual street view map” may have a level of reconstruction that is appropriate for the particular application scenario.
  • reconstructions for example, need to include physical buildings, or even specific structures within a physical building, to operate with a particular store.
  • the "virtual street view map” may be built entirely based on street view data (eg, via street view picture stitching).
  • a "virtual street view map” can be constructed based on map data and street view data.
  • map data can be used to mark roads and buildings or even merchants to facilitate the partitioning of roads and/or buildings in street view data (pictures) to enable virtual street view maps of different reconstruction levels for different application scenarios.
  • the client A can also obtain the map data required to construct the virtual street view map from the server S.
  • the map data described above may be included, for example, in the related street view data or transmitted therewith.
  • the "virtual street view map” is a virtual map space, it is displayed to the user on the client A, which is similar to the "virtual" version of the live view image in the current map application street view mode.
  • the virtual street view images that continuously change as the user moves or operates can form a virtual street view map space.
  • Client A can obtain the street view information itself or construct a virtual street view map based on the received related street view information, and the virtual street view map based on the street view information can be merged with the scene frame to obtain the virtual scene.
  • the scene frame may be preloaded in the client A and loaded with street view information obtained from the server, thereby merging the loaded scene frame with the virtual street view map based on the street view information, thereby obtaining a virtual scene.
  • a virtual street view map may be constructed based on the street view information, and the scene frame is loaded on the virtual street view map, and the virtual street view map loaded with the scene frame is displayed to its user, thereby obtaining a virtual scene.
  • the virtual street view map loaded with the scene frame or the scene frame loaded with the virtual street view map may be referred to as a “virtual scene”.
  • the user can perform corresponding interaction operations in the above virtual scene to implement interaction with the specific scene or the object displayed therein.
  • Framework can refer to a reusable design of a whole or part of a system, represented as a set of abstract components and methods of interaction between component instances.
  • the "scene framework” is a framework associated with a specific application scenario, that is, a reusable design associated with a specific application scenario.
  • the loading of the above “scene frame” can be implemented by weex, Html5 or directly implanted.
  • the "scene framework” may include an operator panel that enables a user to interact with the displayed scene or objects within it.
  • the "scene frame" may include settings for an environment or display style that are adapted to the particular application scenario. That is, the display style or display environment of the virtual street view map can be determined at least in part by the scene frame.
  • the "scene frame" may include settings for line of sight height that are appropriate for the particular application scenario. That is, the line of sight height in the virtual scene is determined based at least in part on the scene frame.
  • the loading of the scene framework can be implemented on the client A side, but the acquisition time of the scene framework itself is not limited.
  • the scenario framework may be, for example, a scenario framework that is included in the application when the client A downloads the application (for example, the mobile APP), that is, the scenario frame is pre-downloaded and stored on the client A side.
  • the scene frame may be followed by the related street view information and transmitted from the server S to the client A.
  • the update of the scene frame may be issued together when the application updates or transmits related street view information.
  • multiple scene frames may be provided.
  • a "racing scene” framework as shown in Figure 4 below can be provided, which can provide a "city shooting scene” framework, a “business promotion scene” framework, an "ancient RPG scene” framework, and the like. These frameworks can be loaded on the same virtual street view map based on user selection or specifically installed applications, thereby enriching the scope of application of embodiments of the present invention.
  • the “virtual scene” refers to a map space obtained by combining a specific application scenario with a real-life map.
  • the road displayed in the "virtual scene” is at least consistent with the direction of the road in the physical scene (actual scene in reality) associated with the geographic location information uploaded by the client A, so that the user The interaction is completed smoothly. But the surrounding buildings and even the road forms can be different from reality.
  • FIG. 4A and 4B illustrate an example of a racing application scenario in accordance with one embodiment of the present invention.
  • Figure 4A shows a screenshot of a view and variable position street view at the height of the street view car's viewing angle.
  • 4B is a screenshot of a racing application scenario based on a virtual street view map of the location and loaded with a racing control room and its formal model framework. Since FIG. 4 relates to a racing application scenario, as shown in FIG. 4B, in the "scene frame", a panel and indicator display for racing operation is included, and the line of sight height setting is equivalent to the height of the car cab. Since it only involves road operations, the reconstruction in its virtual street view map mainly involves the road itself, ignoring the surrounding buildings. The loaded scene frame is filled with industrial buildings. The resulting virtual scene allows the user to immersively drive the car in line with the actual road direction while being somewhat away from the fact that he is in the living area, thus providing the user with a wonderful racing experience.
  • the operational form of the user's interaction with a particular scene or object within it is determined, at least in part, by the scene framework.
  • the user's interactive operation form is a driving operation determined by the racing scene.
  • the virtual scene constructed according to the real scene image and the scene frame is a virtual space at least partially conforming to the real scene.
  • the changed spatial content in the virtual scene can be displayed in a continuously changing perspective and position based on the user's screen or keyboard operation.
  • the virtual scene is constructed based on the current geographic location information of the user, the geographic location information reported by the client A in real time and the physical motion of the user that can be perceived by various sensors built in the client A may be used.
  • the spatial content of the changes in the virtual scene is displayed in a continuously changing perspective and position.
  • the live view image used to construct the virtual scene may be continuously acquired by client A from server S.
  • the client A can continuously update its current geographical location information and communicate with the server S in order to continuously obtain the real scene corresponding to the geographical location information from the server S.
  • Images, these real-life images are used for real-time updates of the spatial content displayed in the virtual scene. This is especially true when the user of client A has a large physical displacement during the interaction process, the network transmission performance is excellent (for example, high-speed WiFi is covered), and/or the real-time processing capability of client A is excellent.
  • the live view image used to construct the virtual scene may be obtained once by client A from server S. That is, the real scene range involved in a complete virtual scene interaction may be predetermined, and the real scene image involved may be transmitted to the client A at one time when the related street view information is transmitted.
  • “one-time” refers to continuously returning all of the required content in the response based on a single acquisition request of the client A, for example, 100 frames of real-time images required for continuous transmission.
  • the live image may also be obtained in batches, for example, each time a physical hotspot (eg, a WiFi point) is passed. The foregoing acquisition manners may be arbitrarily combined based on a specific application environment to obtain an optimal implementation scheme.
  • the perspective and content of the presentation virtual scene may be changed in real time according to the current geographic location, speed information, and/or perspective information.
  • the client A may send the current geographical location information, the speed information, and/or the perspective information to the server S in real time, and change the perspective and content of the displayed virtual scene when the updated content returned by the server S.
  • the display of the virtual scene may be the user's first perspective following display. That is, following the physical motion of the user, the display content in the fake scene is continuously changed.
  • the above display content is consistent with, or at least approximate to, the actual environment of the user. This can be achieved by measuring the user's motion trajectory in real time through the client A's high-precision positioning capability and the sensors such as gyroscopes or spirits equipped in it. In other embodiments, it may also be implemented based, at least in part, on a reference in the environment.
  • the interactive operation scheme of the present invention can be further extended to an interactive operation scheme of a plurality of networked users under the shared virtual scene platform provided by the server S.
  • the server S can be connected to a plurality of clients 10 as shown by the server 20 of FIG.
  • the server S can maintain a common platform for users under the same scene framework, and can interact with each other when the user's avatars are close in the virtual scene. Different users can also interact by, for example, seeing a game of the same scene in the same virtual scene, for example, a certain racing game passing through a real street in Beijing in a virtual scene.
  • the server S can synchronize the avatar information of one or more clients (users) in the virtual space.
  • Client A may obtain information for one or more other clients of the network and present the avatars of the one or more other clients in their virtual scenes. Further, client A can interact with other clients' avatars via avatars, which are based on a virtual street view map that loads the scene frame.
  • AR objects can be loaded in a virtual scene to further enhance the participation and interest of the interactive scheme of the present invention.
  • the AR object to be loaded may be acquired based on the geographical location information, the street view information, and/or the scene frame, and loaded in the virtual scene.
  • the loaded AR object but does not need to be displayed immediately, but can be triggered by a specific operation, angle, task completion, and the like.
  • the loaded AR object may be displayed in the virtual scene if a predetermined presentation condition is met.
  • a specific AR object may be displayed when the user reaches the target location or in the vicinity of the virtual scene; may be displayed when the user and the AR object are at a specific perspective, for example, the viewing angle is opposite to a certain store, and the store elf is displayed; After the specific task is reached, for example, the virtual trophy appears after the achievement of the racing result.
  • the trophy can be located, for example, in a car display store.
  • the map can generate a route indication to the store, and the trophy can be used as a discount in the car display store. Consumption certificate.
  • Client A can interact with AR objects loaded and displayed in the virtual scene.
  • the user may use the client
  • the camera function of the end A captures an actual target to trigger the AR object.
  • the above captured and triggered data may further be used for the positioning of the client A itself or the generation of the virtual scene.
  • target objects such as non-AR objects
  • may also be loaded in the virtual scene for example, objects in the virtual RPG scene that are integrated into the scene and opened via specific operations, and the objects may also be based on geographic location information, street view information, and/or The scene frame is acquired and triggered under predetermined conditions.
  • all the street view information required in, for example, the virtual scene is not completely acquired at the client A, but is obtained in batch or in real time from the server S.
  • the information interaction between the client A and the server S is also involved, for example, real-time uploading of current geographical location information/operation information (in the case of non-field) and obtaining corresponding street view information.
  • the server needs to synchronize the respective locations, actions, and interactions of the multi-user avatars and present them to users within the corresponding scope.
  • FIG. 5 is a flow chart showing a method of implementing an interactive operation on the client side according to an embodiment of the present invention.
  • the geographic location information is transmitted to the server.
  • related street view information related to the street view information is received, wherein the street view information is street view information associated with the geographical location information queried by the server.
  • the virtual street view map based on the street view information is merged with the scene frame to obtain a virtual scene.
  • the virtual scene is presented.
  • an interactive operation is performed in the virtual scene in response to the user input.
  • the related street view information transmitted by the server S to the client A may be the street view information itself, may be a virtual street view map generated based on the street view information, or may be related to the related street view information that has been downloaded to the client A. Or instructions for a virtual street view map.
  • FIG. 6 is a flow chart showing a method of implementing an interactive operation on the server side according to an embodiment of the present invention.
  • geographic location information sent by the client is obtained.
  • street view information associated with the geographic location information or virtual street view map information generated based on the street view information is queried.
  • the street view information or the virtual street view map information is sent to the client, so that the client displays a virtual scene in which the client user can perform an interaction operation, wherein the virtual scene is a virtual street view map. Blended with the scene frame.
  • FIG. 7 shows a schematic diagram of an interactive operation implementing apparatus implemented on a client side according to an embodiment of the present invention.
  • the interactive operation implementing apparatus 700 includes a location information transmitting unit 710, a street view information receiving unit 720, a virtual scene building unit 730, a virtual scene displaying unit 740, and an interaction unit 750.
  • the location information sending unit 710 can be configured to send the geographic location information to the server.
  • the street view information receiving unit 720 can be configured to receive related street view information related to the street view information, wherein the street view information is street view information that is queried by the server and associated with the geographic location information.
  • the virtual scene construction unit 730 can be configured to fuse the virtual street view map based on the street view information with the scene frame to obtain a virtual scene.
  • virtual scene construction unit 730 can include a virtual street view map construction sub-unit that builds a virtual street view map based on street view information.
  • the virtual scene display unit 740 can be used to display the virtual scene.
  • the interaction unit 750 can then perform an interaction in the virtual scene in response to user input.
  • the interaction operation implementing apparatus 700 may further include a target object acquisition unit and a target object loading unit.
  • the target object acquisition unit may acquire a target object to be loaded based on the geographic location information, the street view information, and/or the scene frame.
  • a target object loading unit can be used to load the target object in the virtual scene.
  • the target object loading unit displays the loaded target object in the virtual scene if a predetermined presentation condition is satisfied.
  • the target object can be an AR object.
  • the interactive operation implementing apparatus 700 may further include an information synchronization unit for synchronously displaying the avatars of the one or more other clients in the virtual street view map.
  • FIG. 8 shows a schematic diagram of an interactive operation implementing apparatus implemented on a server side according to an embodiment of the present invention.
  • the interactive operation implementing apparatus 800 includes a location information acquiring unit 810, a street view information inquiring unit 820, and a street view information transmitting unit 830.
  • the interactive operation implementation device 800 can be configured to obtain geographic location information sent by the client.
  • the street view information query unit 820 may be configured to query street view information associated with the geographic location information or virtual street view map information generated based on the street view information.
  • the street view information sending unit 830 may send the street view information or the virtual street view map information to the client, so that the client displays a virtual scene in which the client user can perform an interaction operation, wherein the virtual scene is a virtual street view.
  • the map is merged with the scene frame.
  • the virtual scene, virtual street view map, and/or at least a portion thereof may be built on the server side.
  • the interactive operation implementation apparatus 800 can optionally include a building unit 840 for constructing a virtual street view map based on the street view information, and/or loading the scene frame on the virtual street view map to construct the virtual scene space.
  • the street view information transmitting unit 830 may transmit content including street view information, for example, a virtual street view map that has been constructed, or even a virtual scene, or a part thereof.
  • the street view information transmitting unit 830 may transmit the AR object to be loaded in the virtual scene based on the geographical location information, the street view information, the scene frame, and/or a predetermined presentation condition, to For display in the virtual scene.
  • the location information obtaining unit 810 can continuously acquire the location information or the operation information of the client to send the content required by the client.
  • the location information obtaining unit 810 can acquire information of a plurality of clients connected to the network.
  • the interactive operation implementing apparatus 800 may include a synchronization unit for synchronizing operations and locations of the avatars of the plurality of clients in the virtual space platform.
  • the street view information sending unit 830 can send the synchronization information to the corresponding client, for example, sending the avatars of other clients involved in the virtual scene to a certain client to display in the virtual scene.
  • the synchronization unit may also receive the operation of the avatar of the client and/or other clients in real time and perform synchronization processing, and the synchronization processing is performed by the street view information sending unit 830 correspondingly to implement the client and other Interaction between the avatars of the client.
  • FIG. 9 is a client device 900 in accordance with one embodiment of the present invention.
  • the client device 900 includes an input output device 910, a memory 920, and a processor 930.
  • the input output device 910 can be a touch screen for receiving input information and displaying output information.
  • Memory 920 is used to store information.
  • Processor 930 is coupled to input and output device 910 and memory 920 and is used to present virtual scenes and interact with the user as described above.
  • a server 1000 is also provided.
  • the server 1000 includes a memory 1010 and a processor 1020.
  • the memory 1010 is for storing executable code.
  • the processor 1020 is coupled to the memory 1010, and when the executable code is executed by the processor 1020, causes the processor 1020 to perform the above-described interactive operation implementation method implemented on the server side.
  • the real-life streetscape is generated by the algorithm, which is closer to reality than the traditional map application (for example, map game). Taking Beijing as an example, the actual street map of Beijing can be quickly generated, and the user has high recognition, strong cognition and high reception. .
  • the generation of the real scene scene can be selected based on the splicing of the real scene image, so that the virtual street view map of at least part of the information to restore the actual scene can be realized with acceptable data processing and network transmission requirements. Different application scenarios can be superimposed for the same virtual street view map according to the loaded scene frame.
  • the constructed virtual scene can be associated with the geographical location selected by the user or the current geographical location, the real map can be directly generated by combining the map data, and the virtual map can be generated according to the key coordinate information, far more than any online line.
  • the map app is rich in content.
  • the user can launch the virtual Street View racing game function using the application installed on Client A.
  • the application may, for example, include a plurality of scene frames that are downloadable or loadable to the local, and the user may select them based on preferences.
  • client A can use the weex capability, H5 capability or directly into the existing racing first control room and formal model framework.
  • the server S can read the current geographic information of the client A (for example, can be provided by the client A's LBS capability), and read the street view data corresponding to the map data read from the map service or its database. Since the existing street view data is a pure picture, a map algorithm can be specially used on the server S or the client A to integrate the map and the street view (picture) information, so that the map and the picture information are combined to generate a virtual street view map.
  • the virtual street view map can be drawn not by the UI, but mainly by image stitching.
  • the virtual street view map can reconstruct the landmark buildings where the racing route passes, such as the Tiananmen Gate Tower and Plaza, which are presented as they pass through the actual streets in Beijing.
  • the moving speed and arrival location of the car can be calculated by LBS displacement and time; or after the map is loaded, the interaction can be performed according to the user's interface operation (ie, the display scene can be currently faced by the user). The scene is different).
  • a user networking system can be established, and each user can access using the LBS to enter a multi-person interaction scenario. For example, users near the same location can participate in the same car chase in the same virtual scene range.
  • the user can launch the city treasure hunt RPG game to achieve a deeper immersion game.
  • the user can launch the city treasure hunt RPG game to achieve a deeper immersion game.
  • Client A can first load the scene frame of the RPG game, such as an operation interface including a small map display, a jump, an AR shooting activation, and the like, and send its current geographic location to the server S in real time.
  • the server S can send the map data and the street view data to the client A, so as to load the virtual street view map with a higher level of the real scene restoration on the client A.
  • the building itself can be restored and the tasks required for customs clearance can be set at the location associated with the building.
  • the virtual street view displayed in real-time in client A can be highly consistent with the real scene that the user sees.
  • the user can scan the specific target through the camera function of client A to start the display of the AR object, and the user can interact with the AR object to complete the corresponding game task.
  • the scanning and interaction of the AR object can also be combined with the height of the real scene, for example, involving the introduction of a shop or building or a promotional event.
  • the method according to the invention may also be embodied as a computer program or computer program product comprising computer program code instructions for performing the various steps defined above in the above method of the invention.
  • the present invention may be embodied as a non-transitory machine readable storage medium (or computer readable storage medium, or machine readable storage medium) having stored thereon executable code (or computer program, or computer instruction code)
  • executable code or computer program, or computer instruction code
  • a processor of an electronic device or computing device, server, etc.
  • each block of the flowchart or block diagram can represent a module, a program segment, or a portion of code that includes one or more of the Executable instructions.
  • the functions noted in the blocks may also occur in a different order than the ones in the drawings. For example, two consecutive blocks may be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending upon the functionality involved.
  • each block of the block diagrams and/or flowcharts, and combinations of blocks in the block diagrams and/or flowcharts can be implemented in a dedicated hardware-based system that performs the specified function or operation. Or it can be implemented by a combination of dedicated hardware and computer instructions.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • Environmental & Geological Engineering (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biodiversity & Conservation Biology (AREA)
  • Ecology (AREA)
  • Environmental Sciences (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Processing Or Creating Images (AREA)
  • User Interface Of Digital Computer (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

本发明提供了一种基于实景地图图像构建虚拟场景以供用户交互的方法、装置和客户端设备。一种互动操作实现方法包括：将客户端地理位置信息发送给服务器，服务器查询与地理位置信息相关联的街景信息；客户端或服务器基于该街景信息构建虚拟街景地图；客户端将虚拟街景地图与场景框架相融合以得到虚拟场景；客户端向其用户展示所述虚拟场景以便用户在其中进行交互操作。由此，能够提供与现实环境高度融合但却灵活可变的虚拟场景，进一步提升用户的使用体验。 (图3)

Description

互动操作实现方法、装置及客户端设备 技术领域
本发明涉及信息技术领域，特别是涉及一种在基于实景的虚拟场景下进行互动操作的方法、装置和客户端设备。
背景技术
随着“精灵宝可梦GO”这一游戏风靡全球，结合了实景的沉浸式应用开始进入大众视野。地理位置信息的结合提升了人们的实地参与感，AR技术则通过将计算机生成的虚拟物体、场景或系统提示信息叠加到真实场景中，实现对现实的增强。图1A和1B示出了Pokémon GO游戏的截图示例。
虽然涉及了实地与实景，但如图1A和1B所示，该游戏只是分别使用了LBS和AR技术(图1A是地图模式的例子，图1B则是AR模式的例子)，并非在一个具体场景中融合相关技术以给出更为逼真的使用体验。
类似地，现有技术中缺乏对现实场景的深度融合，因此用户的使用沉浸度并不高。
发明内容
为了解决上述至少一个问题，本发明提出了一种全新的互动操作方案，其通过获取现有的街景信息生成虚拟街景地图，再根据具体的应用场景叠加场景框架，从而能够在现有网络条件的基础上，以现有移动端设备可承受的计算量为用户提供灵活的，且与环境高度融合的互动场景。该方案还能够进一步地融合AR技术，以便为用户提供更为新奇、有趣的使用体验。
根据本发明的一个方面，提供了一种在客户端侧实现的互动操作实现方法，包括：将地理位置信息发送给服务器；接收与街景信息有关的相 关街景信息，其中所述街景信息是所述服务器查询到的与地理位置信息相关联的街景信息；将基于所述街景信息的虚拟街景地图与场景框架相融合以得到虚拟场景；展示所述虚拟场景；以及响应于用户输入，在所述虚拟场景中进行交互操作。
由此，能够利用现有的实景图像信息，经过合理变换得出与现实环境深度结合的具体应用场景，从而以可实现的计算量和网络传输要求为用户提供沉浸式的应用体验。
虚拟街景地图与场景框架的融合可以是由装载的场景框架添加街景信息，而可以正好相反。例如，可以在向已装载的场景框架添加所述街景信息，以形成加载有场景框架的虚拟街景地图，所述虚拟场景是加载有场景框架的虚拟街景地图；或者基于所述街景数据构建所述虚拟街景地图，在所述虚拟街景地图上加载所述场景框架以得到所述虚拟场景。由此，可以基于具体应用场景，灵活选择虚拟场景的构建方式。
可选地，相关街景信息还可以包括与所述街景信息相关联的地图信息，并且所述虚拟街景地图的构造还基于所述地图信息。通过引入地图信息，可以更方便高效且准确地实现虚拟街景地图的构建。
场景框架可以包括诸多实现。可选地，场景框架可以包括使用户能够与所显示的虚拟场景或其内目标进行交互的操作面板。可选地，可以至少部分基于场景框架确定虚拟街景地图的环境或显示风格。可选地，虚拟场景中的视线高度至少部分基于所述场景框架确定。
可选地，虚拟街景地图可以是根据预定地图算法从接收到的街景信息图片中拼接生成的。从而相比于现有技术中单纯的建模或是地图模式，能够以小得多的代价为用户提供更为真实的使用场景。
在用户与场景的互动中，交互操作的交互形式可以至少部分由场景框架决定。由此，能够为相同的虚拟街景地图提供多种多样的交互场景，从而提升本方案的灵活性和适用广泛程度。
本发明的互动操作实现方法还可以进一步涉及AR技术。可选地可以基于所述地理位置信息、所述街景信息和/或所述场景框架获取要加载的AR对象；以及在所述虚拟场景中加载所述AR对象。上述AR对象的加载可 以是有条件的，例如虚拟场景中可以加载AR对象，但无需立即展示，而是可由特定操作、角度、任务完成等触发。上述的交互操作还可以包括与所述虚拟街景地图中加载并展示的AR对象进行的交互操作。由此，能够进一步丰富本方案的应用场景和适用范围，并提升交互的参与度和趣味性。AR对象的加载还可以响应于对目标对象的拍摄。进一步地，加载所述AR对象的相关数据被用于确定当前地理位置信息和/或所述虚拟场景的生成与更新。
客户端提交的地理位置信息可以是用户选定的地理位置信息或是用户的当前地理位置信息。根据选定地理信息构建虚拟场景，可以使得用户足不出户，尽情体验各种场景框架下建立在各类实际地点的虚拟场景，从而丰富用户的使用体验。倘若基于的是当前地理位置信息，则能够根据用户的实际所在地构建虚拟场景，进一步增添用户的临场感。
根据具体的应用场景，可以为用户提供各种交互水平。可选地，可以以用户的第一视角展示所述虚拟场景。例如，可以使用移动端内置的陀螺仪等捕捉移动端的运动轨迹，将其实时反映在虚拟场景的显示内容上，从而提升应用的身临其境感。例如，在具体实现中，可以将例如用户手持的移动端的当前地理位置信息、速度信息和/或视角信息实时发送给所述服务器；并根据所述当前地理位置、速度信息和/或视角信息实时改变展示的虚拟街景地图的视角和内容。
本发明的互动操作方案还可以涉及多用户的联网操作。可选地，用户可以获取联网的一个或多个其他客户端的信息(其他用户信息)，并在该用户的虚拟街景地图中展示所述一个或多个其他客户端的化身。该用户可以经由其化身与其他客户端的化身进行交互。由此进一步提升具体场景的参与度，并为应用赋予社交性。
根据本发明的另一个方面，提供了一种在服务器侧实现的互动操作实现方法，包括：获取客户端发送的地理位置信息；查询与所述地理位置信息相关联的街景信息或基于所述街景信息生成的虚拟街景地图信息；将所述街景信息或所述虚拟街景地图信息发送给客户端，以使得客户端展示客户端用户能在其中进行交互操作的虚拟场景，其中，所述虚拟场景是将 虚拟街景地图与场景框架相融合得到的。
在涉及AR技术的情况下，该方法还可以包括：基于所述地理位置信息、所述街景信息、所述场景框架和/或预定展示条件发送要在所述虚拟场景中加载的AR对象，以供在所述虚拟场景中展示。
在涉及多人联网的情况下，该方法还可以包括：获取联网的一个或多个其他客户端的信息；在所述虚拟空间平台中同步所述一个或多个其他客户端的化身；以及将所述其他客户端的化身发送给所述客户端以在所述客户端的虚拟场景中展示。可选地，该方法还可以包括：实时接收所述客户端和/或其他客户端的化身的操作并进行同步处理；以及下发所述同步处理以实现所述客户端与其他客户端的化身之间的交互操作。
根据本发明的再一个方面，提供了一种在客户端侧实现的互动操作实现装置，包括：位置信息发送单元，用于将地理位置信息发送给服务器；街景信息接收单元，用于接收与街景信息有关的相关街景信息，其中所述街景信息是所述服务器查询到的与地理位置信息相关联的街景信息；虚拟场景构建单元，用于将基于街景信息的虚拟街景地图与场景框架相融合以得到虚拟场景；虚拟场景展示单元，用于展示所述虚拟场景；以及交互单元，用于响应于用户输入，在所述虚拟场景中进行交互操作。
可选地，该互动操作实现装置还可以包括：目标对象获取单元，基于所述地理位置信息、所述街景信息和/或所述场景框架获取要加载的目标对象；以及目标对象加载单元，用于在所述虚拟场景中加载所述目标对象。
可选地，所述目标对象加载单元在满足预定展示条件的情况下，在所述虚拟场景中展示加载的所述目标对象。
相应的，所述互动操作实现装置还可以包括信息同步单元，用于在所述虚拟街景地图中同步展示所述一个或多个其他客户端的化身。
根据本发明的再一个方面，提供了一种在服务器侧实现的互动操作实现装置，包括：
位置信息获取单元，用于查询与所述地理位置信息相关联的街景信息或基于所述街景信息生成的虚拟街景地图信息；
街景信息查询单元，用于查询与所述地理位置信息相关联的街景信息或基于所述街景信息生成的虚拟街景地图信息；
街景信息发送单元，用于将所述街景信息或所述虚拟街景地图信息发送给客户端，以使得客户端展示客户端用户能在其中进行交互操作的虚拟场景，其中，所述虚拟场景是虚拟街景地图与场景框架融合得到的。
可选地，互动操作实现装置还包括：
构建单元，用于基于街景信息构建虚拟街景地图，和/或在虚拟街景地图上加载场景框架以构建虚拟场景空间。
可选地，街景信息发送单元用于基于所述地理位置信息、所述街景信息、所述场景框架和/或预定展示条件发送要在所述虚拟场景中加载的AR对象，以供在所述虚拟场景中展示。
可选地，互动操作实现装置还包括：
同步单元，用于在所述虚拟空间平台中同步所述多个客户端的化身的操作与位置。
根据本发明的一个方面，提供了一种客户端设备，包括：处理器；以及存储器，其上存储有可执行代码，当所述可执行代码被所述处理器执行时，使所述处理器执行如上所述的在客户端侧实现的互动操作实现方法。
根据本发明的一个方面，提供了一种服务器，包括：处理器；以及存储器，其上存储有可执行代码，当所述可执行代码被所述处理器执行时，使所述处理器执行如上所述的在服务器侧实现的互动操作实现方法。
根据本发明的再一个方面，提供了一种非暂时性机器可读存储介质，其上存储有可执行代码，当所述可执行代码被电子设备的处理器执行时，使所述处理器执行如上所述的互动操作实现方法。
由此，本发明通过基于实景地图图像构建虚拟场景以供用户交互，能够提供与现实环境高度融合但却灵活可变的虚拟场景，进一步提升用户的使用体验。
附图说明
通过结合附图对本公开示例性实施方式进行更详细的描述，本公开 的上述以及其它目的、特征和优势将变得更加明显，其中，在本公开示例性实施方式中，相同的参考标号通常代表相同部件。
图1A和1B示出了“精灵宝可梦GO”游戏的截图示例。
图2是用于实现本发明实施例的环境的示意图。
图3示出了根据本发明一个实施例的互动操作方案的示意图。
图4A和4B示出了根据本发明一个实施例的赛车应用场景的例子。
图5示出了根据本发明一个实施例的客户端侧的互动操作实现方法的流程图。
图6示出了根据本发明一个实施例的服务器侧的互动操作实现方法的流程图。
图7示出了根据本发明一个实施例的在客户端侧实现的互动操作实现装置的示意图。
图8示出了根据本发明一个实施例的在服务器侧实现的互动操作实现装置的示意图。
图9是根据本发明一个实施例的客户端设备。
具体实施方式
下面将参照附图更详细地描述本公开的可选实施方式。虽然附图中显示了本公开的可选实施方式，然而应该理解，可以以各种形式实现本公开而不应被这里阐述的实施方式所限制。相反，提供这些实施方式是为了使本公开更加透彻和完整，并且能够将本公开的范围完整地传达给本领域的技术人员。
随着移动端处理能力的增强和网络质量的全面提升，现有的结合当前地理位置信息的应用由于其使用场景单一、真实性差和/或交互性弱，已经越来越无法满足人们生活、学习和娱乐的丰富需求。有鉴于此，本发明以相对较低的计算和网络要求实现了真实场景和虚拟场景的良好结合，从而为用户提供深度的沉浸式体验。
本发明实施例所提供的互动场景构建和参与方案可应用于如图2所示的环境中。图2是用于实现本发明实施例的环境200的示意图。在一个 实施例中，环境200中的移动终端10可以经由网络40实现与服务器20的信息收发。服务器20可以通过访问数据库30来获取移动终端10所需的内容。在一个实施例中，该数据库30可以是存储有街景信息的街景信息数据库，服务器20则可以通过访问该数据库获取移动端10所请求的街景信息。移动终端之间(例如，10_1与10_2或10_N之间)可选地可以经由网络40彼此通信。网络40可以是广义上的用于信息传递的网络，可以包括一个或多个通信网络，诸如无线通信网络、因特网、私域网、局域网、城域网、广域网或是蜂窝数据网络等。在一个实施例中，网络40也可以包括卫星网络，由此将移动终端10的GPS信号传送给服务器20。应当注意，如果向图示200中添加或从图示200中去除附加模块，不会改变本发明的示例实施例的底层概念。另外，虽然为了方便说明而在图中示出了从数据库30到服务器20的双向箭头，但本领域技术人员可以理解的上，上述数据的收发也是可以通过网络40实现的。
移动终端10是可用来进行网络访问的任何合适的便携式电子设备，包括但不限于智能电话、平板电脑或是其他便携式客户端。服务器20则是能够通过网络访问的提供交互服务所需信息的任何服务器。图中虽然示出了多个移动终端10-1…N以及单个服务器20和数据库30，并且在随后的描述中会选择其中的一个或部分移动终端加以描述(例如，移动终端10-1)，但是本领域技术人员应该立即的是，上述1…N个移动终端旨在表示真实网络中存在的多个移动终端，示出的单个服务器20和数据库30旨在表示本发明的技术方案涉及服务器及数据库的操作。对特定编号的移动端以及单个服务器和数据库加以详述至少为了说明方便，而非暗示对移动端和服务器的类型或是位置等具有限制。
图3示出了根据本发明一个实施例的互动操作方案的示意图。在该互动操作方案的实现环境中，包括至少一台服务器S和至少一个客户端A。在互动方案包括多人联网交互的可选实施例中，该实现环境包括至少两个客户端A和B。在此，至少一台服务器S可以是图2所示环境中的服务器20，客户端A可以是图2所示的任一移动端10。该服务器S经由网络或其他连接与街景信息服务器相连，以便为客户端A获取其所在位置的街景信 息。
首先，客户端A将某一地理位置信息发送给服务器S。
客户端A发送给服务器S的地理位置信息理论上可以是客户端A的用户选定的任何地理位置信息。例如，用户可以在地图应用中选定某一坐标，甚至在地图应用的街景模式下选定的特定街景的图像截图，并将相应的信息发送给服务器S。
在另一个实施例中，客户端A发送的可以是其当前地理位置信息。在此，客户端A可以利用LBS(基于位置的服务)来获取其所在的地理位置信息，并例如经由无线通讯网络(例如，4G网络或是WiFi)将其发送给服务器S。
基于不同的应用场景下对地理位置信息精度要求的不同，客户端A所需发送的地理位置信息也不同。考虑到智能手机在室外的GPS定位精度通常小于10米，因此可以设计基于此精度要求的应用场景。在高精度场景要求下，例如，在真实世界中第一视角跟随模式的应用场景，例如可以要求客户端A的用户额外配备精度在一米以下，甚至几厘米的手持GPS设备，该设备可以将极为精确的地理位置信息实时发送给客户端A。在另一个例子中，针对地理位置信息的高精度要求，可以首先要求客户端A例如站在特定位置朝向特定方向或是对准特定目标，以此作为交互动作的基准起点。随后通过客户端A内置的指南针、陀螺仪等再结合LBS服务，实时修正客户端A的位置和朝向。
随后，服务器S接收客户端A发送的地理位置信息，并例如在图2所示的街景信息数据库30中查询与该地理位置信息相关联的街景信息。
在此，街景信息指的是能够显示街道、室内、公共建筑或其他环境的360度的全景实景图像。“街景地图”是一种实景地图服务，例如，用户可以在谷歌地图的街景模式下，或是百度地图的全景模式下查询到的、针对实际景物且视角可变的图像。由于街景地图的原始数据通常是由在特定路面上行驶的采集车辆上安装的可旋转镜头采集的，因此街景地图提供的实景信息通常是以采集车辆安装镜头高度为视线高度、全视角且可移动查询的连续的街景信息。但应该理解的是，虽然被称为“街景信息”，但 其实际指代的是视角可变的实景信息，实景可以并且通常是街景，但也可以指代真实“街景”之外的其他环境的实景，例如公共建筑或其内部的实景信息，自然景观的实景信息等。
在一个实施例中，服务器S可以直接访问由现有的“街景地图”服务采集并维护的街景信息数据库，例如，直接访问百度或高德地图的街景信息数据库。在其他实施例中，服务器S也可以访问其自行采集、构建并维护的街景信息数据库。这在实现虚拟街景地图的范围有限时尤为使用。例如，可以采集北京故宫的(包括室内外建筑全景的)实景信息数据，构造故宫的街景(实景)数据库，并可由服务器S访问。
基于上述街景信息，可以构建虚拟街景地图。在一个实施例中，虚拟街景地图在服务器端(即，服务器S上)构建，并经由网络发送给客户端A。客户端A直接获取上述虚拟街景地图，并进行显示或后续处理。在另一个实施例中，服务器S可以将街景信息发送给客户端A，并在客户端A完成对虚拟街景地图的构建。在又一个实施例中，服务器S可以完成虚拟街景地图的部分构建，并在客户端A处完成剩余部分的构建。
在再一个实施例中，虚拟街景地图(或其部分)的构建甚至可以在接收到来自客户端A的地理位置信息之前就已经完成。例如，服务器S可以预先存储某些特定地理位置的虚拟街景地图，或是将该虚拟街景地图(或其部分)预先下发至客户端A。当服务器S接收到来自该客户端A的与该特定地理位置相符的地理位置信息时，就直接向客户端A传输该虚拟街景地图或是启用已经下载在客户端A上的虚拟街景地图。这在特定地理位置是被频繁请求或是需要进行商业推广的地点，例如热门景点或是商业活动宣传目标时尤为有效。
由此，由服务器S传送给客户端A的“相关街景信息”指的是与上述街景信息(即，与服务器查询到的与地理位置信息相关联的街景信息)有关的信息，其可以是前述的街景信息本身，可以是基于街景信息构建的部分或全部虚拟街景地图，也可以是启用客户端A本机上存储的虚拟街景地图的指令。
在此，“虚拟街景地图”指的是基于实景图像构建的，具有高度真 实感和还原性的地图空间。在一个实施例中，“虚拟街景地图”可选可以指代视角可变的、类似于三维实景的地图空间。但不同于需要大量计算的三维建模方法，本发明中的虚拟街景地图可以是根据预定地图算法从接受到的街景信息图片中拼接生成的。由此，能够以可接受的计算代价来提供使用的真实性。在一个实施例中，“虚拟街景地图”可以根据具体应用场景，具有与其相适应的重建水平。例如，在游戏场景中，可以仅根据实景重建道路等主体内容，而忽略其中的实体建筑。在商业推广场景中，重建例如需要包括实体建筑，甚至实体建筑内的具体结构，以便与具体的店铺进行操作。
在一个实施例中，“虚拟街景地图”可以完全基于街景数据(例如，经由街景图片拼接)构建。在另一个实施例中，“虚拟街景地图”可以基于地图数据和街景数据构建。例如，地图数据可以用于标记道路和建筑甚至商户，以便于对街景数据(图片)中道路和/或建筑物的划分，从而实现针对不同应用场景的不同重建级别的虚拟街景地图。在此情况下，客户端A还可以从服务器S获取构建虚拟街景地图所需的地图数据。上述地图数据例如可以包括在相关街景数据中，或与其一并发送。虽然“虚拟街景地图”是一个虚拟地图空间，但其在客户端A显示给用户的，是类似于当前地图应用街景模式中的实景图像的“虚拟”版本。而这些随用户移动或操作而连续变化的虚拟街景图像可以构成虚拟街景地图空间。
客户端A可以基于接收到的相关街景信息得到街景信息本身或是构建虚拟街景地图，基于街景信息的虚拟街景地图可以与场景框架相融合，以得到虚拟场景。在一个实施例中，可以在客户端A中预先加载场景框架，并装入从服务器获得的街景信息，由此使得加载的场景框架与基于街景信息的虚拟街景地图相融合，从而得到虚拟场景。在另一个实施例中，可以基于街景信息构建虚拟街景地图，并在该虚拟街景地图上加载场景框架，并向其用户展示加载了所述场景框架的虚拟街景地图，从而得到虚拟场景。在此，可以将加载了所述场景框架的虚拟街景地图或是加载了虚拟街景地图的场景框架称为“虚拟场景”。用户可以在上述虚拟场景中进行相应的交互操作，以实现与该具体场景或其中显示的对象的互动。
“框架(Framework)”可以指的是整个或部分系统的可复用设计，表现为一组抽象构件及构件实例间交互的方法。在此，“场景框架”是与具体应用场景相关联的框架，即与具体应用场景相关联的可复用的设计。在具体实现中，可以通过weex、Html5或是直接植入的方式来实现上述“场景框架”的加载。
在一个实施例中，“场景框架”可以包括使用户能够与所显示的场景或其内目标进行交互的操作面板。
在一个实施例中，“场景框架”可以包括与具体应用场景相适应的环境或显示风格的设定。即，虚拟街景地图的显示风格或是显示环境，可以至少部分由场景框架所确定。
在一个实施例中，“场景框架”可以包括与具体应用场景相适应的视线高度的设定。即，虚拟场景中的视线高度至少部分基于所述场景框架确定。
另外，如图3所示，场景框架的加载可以在客户端A侧实现，但对该场景框架本身的获取时间不做限制。在一个实施例中，该场景框架可以是例如客户端A在下载应用(例如，移动端APP)时，应用内自带的场景框架，即，场景框架是预先下载并存储在客户端A侧的。在另一个实施例中，该场景框架可以是跟随如上相关街景信息一并从服务器S传送至客户端A的。在又一个实施例中，可以在应用更新或是传输相关街景信息时，一并下发对场景框架的更新。
在本发明的实施例中，可以提供有多个场景框架。例如，可以提供如下图4所示的“赛车场景”框架，可以提供“城市射击场景”框架，“商务推广场景”框架，“古代RPG场景”框架等等。这些框架可以基于用户选择或是具体安装的应用而被加载在相同的虚拟街景地图上，由此丰富本发明实施例的应用范围。
在此，“虚拟场景”是指具体应用场景与实景地图结合后得到的地图空间。在一个更为常见的实施例中，“虚拟场景”中显示的道路至少与客户端A上传的地理位置信息相关联的物理场景(现实中的实际场景)中的道路的走向相一致，以便用户交互的顺利完成。但周围建筑甚至是道路 形式都可以与现实不同。
图4A和4B示出了根据本发明一个实施例的赛车应用场景的例子。图4A示出了在街景采集车视角高度下的、视角和位置可变的街景视图的一张截图。图4B是基于该位置的虚拟街景地图并加载了赛车控制室及其形式模型框架的赛车应用场景的截图。由于图4涉及的是赛车应用场景，因此如图4B所示，在该“场景框架”中，包括用于赛车操作的面板和指标显示，且视线高度设定与汽车驾驶室高度相当。由于仅涉及路面操作，因此其虚拟街景地图中的重建主要涉及道路本身，忽略了周边建筑。加载的场景框架则用工业建筑加以填充。由此得到的虚拟场景使得用户能够身临其境地进行符合实际道路走向的赛车驾驶的同时，又在一定程度上远离自己身处生活区的事实，从而为用户提供奇妙的赛车体验。
在本发明的一个实施例中，用户与具体场景或其内对象的交互操作的操作形式至少部分由所述场景框架决定。如图4B所示，当加载了赛车场景框架时，用户的交互操作形式是由该赛车场景决定的驾驶操作。
由上可知，根据实景图像和场景框架构建的虚拟场景是至少部分与现实场景相符的虚拟空间。在将构建好的虚拟场景经由客户端A展现给用户之后，可以基于用户的屏幕或键盘操作，以连续变化的视角和位置来显示虚拟场景中变化的空间内容。而在虚拟场景的构建是基于用户的当前地理位置信息的情况下，可以基于客户端A实时上报的自身的地理位置信息，以及能由客户端A内置的各类传感器感知的用户的物理运动，以连续变化的视角和位置来显示虚拟场景中变化的空间内容。
在一个实施例中，用于构建虚拟场景的实景图像可以是客户端A从服务器S持续获取的。例如，尤其在客户端A发送的位置是当前地理位置信息的情况下，客户端A可以持续更新其当前地理位置信息并与服务器S通信，以便持续从服务器S获取与其地理位置信息相对应的实景图像，这些实景图像用于虚拟场景中显示的空间内容的实时更新。这尤其适用于客户端A的用户在交互过程中物理位移较大的情况，网络传输性能优良的情况(例如，覆盖了高速WiFi)，和/或客户端A实时处理能力优秀的情况。
在另一个实施例中，用于构建虚拟场景的实景图像可以是客户端A 从服务器S一次性获取的。即，在一次完整的虚拟场景交互中涉及到的实景范围可以是预先确定的，并可以在相关街景信息传输时将所涉及的实景图像一次性地传输给客户端A。在此，“一次性”指的是基于客户端A的单次获取请求而将全部所需的内容在应答中持续全部返回，例如，连续逐张的传输所需的例如100帧实景图像。在其他实施例中，实景图像还可以分批获取，例如，在每次经过物理热点(例如，WiFi点)时获取。上述获取方式可以基于具体应用环境而任意组合，以获得最优的实现方案。
在一个实施例中，在交互过程中，可以根据当前地理位置、速度信息和/或视角信息实时改变展示虚拟场景的视角和内容。在虚拟场景预先在客户端A中加载的情况下，上述改变可以仅在客户端A侧实现。而在其他实施例中，客户端A可以将当前地理位置信息、速度信息和/或视角信息实时发送给服务器S，并经由服务器S返回的更新内容来时改变展示的虚拟场景的视角和内容。
在一个实施例中，虚拟场景的显示可以是用户的第一视角跟随显示。即，跟随用户的物理运动，连续改变虚假场景中的显示内容。上述显示内容与用户实际所处环境相一致，或者至少近似。这可以通过客户端A高精度定位能力，以及其内配备的诸如陀螺仪或水平仪的传感器来实时测量用户的运动轨迹来实现。在其他实施例中，还可以至少部分基于环境中的参照物来实现。
本发明的互动操作方案可以进一步拓展为在由服务器S提供的共享虚拟场景平台下多个联网用户的互动操作方案。服务器S可以如图2的服务器20所示，连入多个客户端10。服务器S可以为同一场景框架下的用户维护一个共同的平台，当用户的化身在虚拟场景中接近的时候，可以彼此进行互动。不同的用户也可以通过例如参见同一虚拟场景中的某一相同场次的游戏来进行互动，例如，虚拟场景中途经某北京实际街道的某一次赛车游戏。服务器S可以在虚拟空间中同步一个或多个客户端(的用户)的化身信息。客户端A可以获取联网的一个或多个其他客户端的信息，并在其虚拟场景中展示所述一个或多个其他客户端的化身。进一步地，客户端A可以经由化身与其他客户端的化身进行交互，其中所述交互是基于加 载了所述场景框架的虚拟街景地图进行的。
在一个实施例中，虚拟场景中可以加载AR对象，以进一步提升本发明的互动方案的参与性和趣味性。可以基于所述地理位置信息、所述街景信息和/或所述场景框架获取要加载的AR对象，并在虚拟场景中加载该AR对象。加载的AR对象，但无需立即展示，而是可由特定操作、角度、任务完成等触发。可以在满足预定展示条件的情况下，在所述虚拟场景中展示加载的所述AR对象。例如，特定AR对象可以在用户在虚拟场景中到达目标位置或其附近时展现；可以在用户与AR对象处于特定视角时展现，比如，视角正对某店铺后展示该店铺小精灵等；也可以在特定任务达成后展现，比如赛车成绩达标后出现虚拟奖杯，该奖杯例如可以位于某汽车展示店内，地图中可以生成去往该店内的路线指示，并且该奖杯可以作为在该汽车展示店内的优惠消费凭证。客户端A可以与虚拟场景中载入并展示的AR对象进行的交互操作。
在一个实施例中，尤其是在基于客户端A提交的当前地理位置信息生成虚拟场景的情况下(即，用户实际身处虚拟场景所基于的实际场景内的情况下)，用户可以通过使用客户端A的摄像功能拍摄某实际目标来实现AR对象的触发。可选地，上述拍摄和触发的数据还可以进一步用于客户端A本身的定位或是虚拟场景的生成。
应该理解的是，虚拟场景中也可以加载其他目标对象，例如非AR对象，例如，虚拟RPG场景中融入场景并经由特定操作开启的对象，上述对象同样可以基于地理位置信息、街景信息和/或场景框架获取并在预定条件下触发。
由此，如图3中的虚线箭头所显示的，在例如虚拟场景所需的全部街景信息并非在客户端A构建该虚拟场景是完全获取，而是分批或是实时从服务器S获取的情况下，在用户与该虚拟场景的交互过程中，还涉及客户端A与服务器S的信息交互，例如，实时上传当前地理位置信息/操作信息(非实地的情况下)并获取相应的街景信息。另外，在涉及多人联网的情况下，服务器需要同步多用户化身各自的位置、动作及其间的交互，并将其展示给相应范围内的用户。
如上已经结合图3描述了本发明的互动操作方案的大致流程。如下将结合附图5-8示出客户端A和服务器S各自的互动操作实现方法和装置。
图5示出了根据本发明一个实施例的客户端侧的互动操作实现方法的流程图。在步骤S510，将地理位置信息发送给服务器。在步骤S520，接收与街景信息有关的相关街景信息，其中所述街景信息是所述服务器查询到的与地理位置信息相关联的街景信息。在步骤S530，将基于街景信息的虚拟街景地图与场景框架相融合以得到虚拟场景。在步骤S540，展示所述虚拟场景。在步骤S550，响应于用户输入，在虚拟场景中进行交互操作。
根据具体的应用场景，由服务器S传递给客户端A的相关街景信息可以是街景信息本身，可以是基于街景信息生成的虚拟街景地图，也可以是启用已经下载到客户端A上的相关街景信息或虚拟街景地图的指令。上述内容已在上文结合图3描述，在此不再赘述。
图6示出了根据本发明一个实施例的服务器侧的互动操作实现方法的流程图。在步骤S610，获取客户端发送的地理位置信息。在步骤S620，查询与所述地理位置信息相关联的街景信息或基于所述街景信息生成的虚拟街景地图信息。在步骤S630，将所述街景信息或所述虚拟街景地图信息发送给客户端，以使得客户端展示客户端用户能在其中进行交互操作的虚拟场景，其中，所述虚拟场景是将虚拟街景地图与场景框架融合得到的。
图7示出了根据本发明一个实施例的在客户端侧实现的互动操作实现装置的示意图。如图7所示，互动操作实现装置700包括位置信息发送单元710、街景信息接收单元720、虚拟场景构建单元730、虚拟场景展示单元740和交互单元750。
位置信息发送单元710可以用于将地理位置信息发送给服务器。街景信息接收单元720可以用于接收与街景信息有关的相关街景信息，其中所述街景信息是所述服务器查询到的与地理位置信息相关联的街景信息。虚拟场景构建单元730可以用于将基于街景信息的虚拟街景地图与场景框架相融合以得到虚拟场景。在一个实施例中，虚拟场景构建单元730可以包括基于街景信息构建虚拟街景地图的虚拟街景地图构建子单元。虚拟场景展示单元740可以用于展示所述虚拟场景。交互单元750则可响应于用 户输入，在所述虚拟场景中进行交互操作。
在一个实施例中，互动操作实现装置700还可以包括目标对象获取单元和目标对象加载单元。目标对象获取单元可以基于所述地理位置信息、所述街景信息和/或所述场景框架获取要加载的目标对象。目标对象加载单元可以用于在所述虚拟场景中加载所述目标对象。目标对象加载单元在满足预定展示条件的情况下，在所述虚拟场景中展示加载的所述目标对象。该目标对象可以是AR对象。另外，在涉及多人联网等的情况下，互动操作实现装置700还可以包括信息同步单元，用于在所述虚拟街景地图中同步展示所述一个或多个其他客户端的化身。
图8示出了根据本发明一个实施例的在服务器侧实现的互动操作实现装置的示意图。如图8所示，互动操作实现装置800包括位置信息获取单元810、街景信息查询单元820和街景信息发送单元830。
互动操作实现装置800可以用于获取客户端发送的地理位置信息。街景信息查询单元820可以用于查询与所述地理位置信息相关联的街景信息或基于所述街景信息生成的虚拟街景地图信息。街景信息发送单元830可以将所述街景信息或所述虚拟街景地图信息发送给客户端，以使得客户端展示客户端用户能在其中进行交互操作的虚拟场景，其中，所述虚拟场景是虚拟街景地图与场景框架融合得到的。
在一个实施例中，虚拟场景、虚拟街景地图和/或其至少一部分可以在服务器侧构建。于是，互动操作实现装置800可以可选地包括构建单元840，用于基于街景信息构建虚拟街景地图，和/或在虚拟街景地图上加载场景框架以构建虚拟场景空间。在此情况下，街景信息发送单元830发送的可以是包含街景信息的内容，例如，已经构建好的虚拟街景地图，甚至是虚拟场景，或其一部分。
在支持AR功能的情况下，街景信息发送单元830可以基于所述地理位置信息、所述街景信息、所述场景框架和/或预定展示条件发送要在所述虚拟场景中加载的AR对象，以供在所述虚拟场景中展示。
另外，位置信息获取单元810可以持续获取客户端的位置信息或操作信息，以便发送客户端所需的内容。在涉及多人联网的情况下，位置信 息获取单元810可以获取联网多个客户端的信息。互动操作实现装置800可以包括同步单元，用于在所述虚拟空间平台中同步所述多个客户端的化身的操作与位置。街景信息发送单元830可以将这些同步信息发送给相应的客户端，例如将虚拟场景中涉及的其他客户端的化身发送给某一客户端以在其虚拟场景中展示。可选地，同步单元还可以实时接收所述客户端和/或其他客户端的化身的操作并进行同步处理，并由街景信息发送单元830相应地下发所述同步处理以实现所述客户端与其他客户端的化身之间的交互操作。
图9是根据本发明一个实施例的客户端设备900。客户端设备900包括输入输出装置910、存储器920和处理器930。输入输出装置910可以是触摸屏，用于接收输入信息并显示输出信息。存储器920用于存储信息。处理器930连接至输入输出装置910和存储器920，并用于如上所述展示虚拟场景并与用户交互。
本发明的一个实施例中，还提供服务器1000。服务器1000包括存储器1010和处理器1020。存储器1010用于存储可执行代码。处理器1020连接至存储器1010，当所述可执行代码被所述处理器1020执行时，使所述处理器1020执行上述在服务器侧实现的互动操作实现方法。
应该理解，如上基于图3描述的可选实施例同样适用于图5-9所示方法中的客户端和服务器及其上实现的相应装置，在此不再赘述。
如上已经结合图3-9描述了本发明的互动操作方案的基本原理。通过算法生成实景街景，比传统的地图应用(例如，地图游戏)更为接近现实，以北京为例，可以很快的生成北京实际街道地图，用户认识度高，认知性强，接收度高。实景街景的生成可选基于实景图像的拼接实现，因此能够以可接受的数据处理和网络传输要求来实现至少部分信息高度还原实际场景的虚拟街景地图。可以根据加载的场景框架，来为相同的虚拟街景地图叠加不同的应用场景。例如，用于赛车游戏的工业建筑场景和操作面板，用于射击类游戏的丛林场景和操作面板等等，从而拓展了本发明的应用领域。另外，由于构建的虚拟场景可以与用户选定的地理位置或是当前所在的地理位置相关联，因此可以直接结合地图数据生成实景地图，亦 可以根据关键坐标信息生成虚拟地图，远比线上任何地图应用内容丰富。
[应用例1]
用户可以使用客户端A上安装的应用，启动虚拟街景赛车游戏功能。该应用例如可以包括多个可下载或可加载至本地的场景框架，用户可以基于喜好自行选取。在赛车游戏的场景下，客户端A可以通过weex能力、H5能力或直接植入现有的赛车第一控制室及形式模型框架。
随后，服务器S可以读取客户端A的当前地理信息(例如，可由客户端A的LBS能力提供)，从地图服务或其数据库中读取地图数据相对应的读取街景数据。由于现有的街景数据是纯图片，因此可以在服务器S或是客户端A上利用专门制定地图算法，整合地图与街景(图片)信息，使得地图与图片信息结合，生成虚拟街景地图。在此，该虚拟街景地图可以不通过UI绘制，而是主要利用图片拼接得到。
在街景赛车的应用场景下，可以利用地图数据和如图4A所示的街景数据(图片)，重建以实景道路为骨架的虚拟街景地图，并为虚拟街景地图添加与赛车场景相适应的建筑物和道路风格(如图4B所示)。为了增添实际参与感，虚拟街景地图中可以重建赛车路线经过的地标建筑，例如途经北京实际街道时原样呈现的天安门城楼和广场等。
在赛车游戏框架运行时，可以通过LBS位移及时间计算汽车的移动速度及到达地点等；也可以在载入地图之后，按照用户的界面操作进行交互(即，显示场景可以与用户当前面对的场景不同)。
可选地，可以建立用户联网体系，各个用户可以使用LBS接入，从而进入多人互动场景。例如，在同一地点附近的用户可以参加在同一虚拟场景范围下的同一场汽车追逐赛。
[应用例2]
相比于前例中非严格的当前场景虚拟化实现(因为正参与赛车游戏的用户无法在游戏中自身真正达到游戏中赛车的移动速度)，用户可以启动城市寻宝RPG游戏来实现沉浸程度更深的游戏体验。
客户端A可以首先装载该RPG游戏的场景框架，例如包括小地图显示、跳跃、AR拍摄激活等菜单的操作界面，并实时向服务器S发送其当前 地理位置。服务器S可以将地图数据和街景数据发送给客户端A，以便在客户端A上加载实景还原级别更高的虚拟街景地图。例如，除了还原道路之外，还可以还原建筑本身，并在与建筑关联的位置上设置通关所需完成的任务。通过客户端A的LBS能力及其内置的各类传感器，客户端A中实时显示的虚拟街景可以与用户真实眼见的实景保持高度一致。
在游戏进行中，用户可以通过客户端A的相机功能扫描特定目标来启动AR对象的显示，用户可以与AR对象进行交互，以完成相应的游戏任务。另外，该AR对象的扫描和互动还可以与实景高度结合，例如，涉及扫描店铺或建筑的介绍或是优惠活动等等。
上文中已经参考附图详细描述了根据本发明的互动操作实现方法和装置。
此外，根据本发明的方法还可以实现为一种计算机程序或计算机程序产品，该计算机程序或计算机程序产品包括用于执行本发明的上述方法中限定的上述各步骤的计算机程序代码指令。
或者，本发明还可以实施为一种非暂时性机器可读存储介质(或计算机可读存储介质、或机器可读存储介质)，其上存储有可执行代码(或计算机程序、或计算机指令代码)，当所述可执行代码(或计算机程序、或计算机指令代码)被电子设备(或计算设备、服务器等)的处理器执行时，使所述处理器执行根据本发明的上述方法的各个步骤。
本领域技术人员还将明白的是，结合这里的公开所描述的各种示例性逻辑块、模块、电路和算法步骤可以被实现为电子硬件、计算机软件或两者的组合。
附图中的流程图和框图显示了根据本发明的多个实施例的系统和方法的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段或代码的一部分，所述模块、程序段或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标记的功能也可以以不同于附图中所标记的顺序发生。例如，两个连续的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也 要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
以上已经描述了本发明的各实施例，上述说明是示例性的，并非穷尽性的，并且也不限于所披露的各实施例。在不偏离所说明的各实施例的范围和精神的情况下，对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。本文中所用术语的选择，旨在最好地解释各实施例的原理、实际应用或对市场中的技术的改进，或者使本技术领域的其它普通技术人员能理解本文披露的各实施例。

Claims (27)

  1. 一种互动操作实现方法，包括：
    将地理位置信息发送给服务器；
    接收与街景信息有关的相关街景信息，其中所述街景信息是所述服务器查询到的与地理位置信息相关联的街景信息；
    将基于所述街景信息的虚拟街景地图与场景框架相融合以得到虚拟场景；
    展示所述虚拟场景；以及
    响应于用户输入，在所述虚拟场景中进行交互操作。
  2. 如权利要求1所述的方法，其中，所述将基于所述街景信息的虚拟街景地图与场景框架相融合以得到虚拟场景包括：
    在向已装载的场景框架添加所述街景信息，以形成加载有场景框架的虚拟街景地图，所述虚拟场景是加载有场景框架的虚拟街景地图；或者，基于所述街景数据构建所述虚拟街景地图，在所述虚拟街景地图上加载所述场景框架以得到所述虚拟场景。
  3. 如权利要求1或2所述的方法，其中，所述相关街景信息还包括与所述街景信息相关联的地图信息，并且，
    所述虚拟街景地图的构造还基于所述地图信息。
  4. 如权利要求1-3任意一项所述的方法，其中，所述场景框架包括使用户能够与所显示的虚拟场景或其内目标进行交互的操作面板。
  5. 如权利要求1-4任意一项所述的方法，其中，至少部分基于场景框架确定虚拟街景地图的环境或显示风格。
  6. 如权利要求1-5任意一项所述的方法，其中，所述虚拟场景中的视线高度至少部分基于所述场景框架确定。
  7. 如权利要求1-6任意一项所述的方法，还包括：
    基于所述地理位置信息、所述街景信息和/或所述场景框架获取要加载的AR对象；以及
    在所述虚拟场景中加载所述AR对象。
  8. 如权利要求1-7任意一项所述的方法，还包括：
    在满足预定展示条件的情况下，在所述虚拟场景中展示加载的所述AR对象。
  9. 如权利要求1-8任意一项所述的方法，还包括：
    响应于对目标对象的拍摄，加载AR对象。
  10. 如权利要求1-9任意一项所述的方法，其中，加载所述AR对象的相关数据被用于确定当前地理位置信息和/或所述虚拟场景的生成与更新。
  11. 如权利要求1-10任意一项所述的方法，其中，
    客户端提交的地理位置信息可以是用户选定的地理位置信息或是用户的当前地理位置信息。
  12. 如权利要求1-11任意一项所述的方法，其中，
    以用户的第一视角展示所述虚拟场景。
  13. 如权利要求1-12任意一项所述的方法，其中，
    获取联网的一个或多个其他客户端的信息，并在该用户的虚拟街景地图中展示所述一个或多个其他客户端的化身。
  14. 一种互动操作实现方法，包括：
    获取客户端发送的地理位置信息；
    查询与所述地理位置信息相关联的街景信息或基于所述街景信息生成的虚拟街景地图信息；
    将所述街景信息或所述虚拟街景地图信息发送给客户端，以使得客户端展示客户端用户能在其中进行交互操作的虚拟场景，
    其中，所述虚拟场景是将虚拟街景地图与场景框架相融合得到的。
  15. 如权利要求14所述的方法，其中，还包括：
    基于所述地理位置信息、所述街景信息、所述场景框架和/或预定展示条件发送要在所述虚拟场景中加载的AR对象，以供在所述虚拟场景中展示。
  16. 如权利要求14或15所述的方法，其中，还包括：
    获取联网的一个或多个其他客户端的信息；
    在所述虚拟空间平台中同步所述一个或多个其他客户端的化身；
    以及将所述其他客户端的化身发送给所述客户端以在所述客户端的虚拟场景中展示。
  17. 如权利要求14-16任意一项所述的方法，其中，还包括：
    实时接收所述客户端和/或其他客户端的化身的操作并进行同步处理；
    以及下发所述同步处理以实现所述客户端与其他客户端的化身之间的交互操作。
  18. 一种互动操作实现装置，包括：
    位置信息发送单元，用于将地理位置信息发送给服务器；
    街景信息接收单元，用于接收与街景信息有关的相关街景信息，其中所述街景信息是所述服务器查询到的与地理位置信息相关联的街景信 息；
    虚拟场景构建单元，用于将基于街景信息的虚拟街景地图与场景框架相融合以得到虚拟场景；
    虚拟场景展示单元，用于展示所述虚拟场景；以及交互单元，用于响应于用户输入，在所述虚拟场景中进行交互操作。
  19. 如权利要求18所述的互动操作实现装置，还包括：
    目标对象获取单元，基于所述地理位置信息、所述街景信息和/或所述场景框架获取要加载的目标对象；
    以及目标对象加载单元，用于在所述虚拟场景中加载所述目标对象。
  20. 如权利要求18或19所述的互动操作实现装置，其中，
    所述目标对象加载单元在满足预定展示条件的情况下，在所述虚拟场景中展示加载的所述目标对象。
  21. 如权利要求18-20任意一项所述的互动操作实现装置，还包括：
    信息同步单元，用于在所述虚拟街景地图中同步展示所述一个或多个其他客户端的化身。
  22. 一种互动操作实现装置，包括：
    位置信息获取单元，用于查询与所述地理位置信息相关联的街景信息或基于所述街景信息生成的虚拟街景地图信息；
    街景信息查询单元，用于查询与所述地理位置信息相关联的街景信息或基于所述街景信息生成的虚拟街景地图信息；
    街景信息发送单元，用于将所述街景信息或所述虚拟街景地图信息发送给客户端，以使得客户端展示客户端用户能在其中进行交互操作的虚拟场景，其中，所述虚拟场景是虚拟街景地图与场景框架融合得到的。
  23. 如权利要求22所述的互动操作实现装置，其中，还包括：
    构建单元，用于基于街景信息构建虚拟街景地图，和/或在虚拟街景地图上加载场景框架以构建虚拟场景空间。
  24. 如权利要求22或23所述的互动操作实现装置，其中，
    街景信息发送单元用于基于所述地理位置信息、所述街景信息、所述场景框架和/或预定展示条件发送要在所述虚拟场景中加载的AR对象，以供在所述虚拟场景中展示。
  25. 如权利要求22-24任意一项所述的互动操作实现装置，其中，还包括：
    同步单元，用于在所述虚拟空间平台中同步所述多个客户端的化身的操作与位置。
  26. 一种客户端设备，包括：
    处理器；以及存储器，其上存储有可执行代码，当所述可执行代码被所述处理器执行时，使所述处理器执行如权利要求1-13任意一项所述的互动操作实现方法。
  27. 一种服务器，包括：
    处理器；以及存储器，其上存储有可执行代码，当所述可执行代码被所述处理器执行时，使所述处理器执行如权利要求14-17任意一项所述的互动操作实现方法。
PCT/CN2018/104437 2017-12-26 2018-09-06 互动操作实现方法、装置及客户端设备 Ceased WO2019128302A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201711433566.3A CN108144294B (zh) 2017-12-26 2017-12-26 互动操作实现方法、装置及客户端设备
CN201711433566.3 2017-12-26

Publications (1)

Publication Number Publication Date
WO2019128302A1 true WO2019128302A1 (zh) 2019-07-04

Family

ID=62462961

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/104437 Ceased WO2019128302A1 (zh) 2017-12-26 2018-09-06 互动操作实现方法、装置及客户端设备

Country Status (2)

Country Link
CN (1) CN108144294B (zh)
WO (1) WO2019128302A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111930082A (zh) * 2020-07-22 2020-11-13 青岛海信智慧家居系统股份有限公司 一种智能家居设备的更换方法及装置
CN112863229A (zh) * 2020-12-30 2021-05-28 中兴智能交通股份有限公司 一种基于停车设备和技术实现无人值守的系统及其方法

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108144294B (zh) * 2017-12-26 2021-06-04 阿里巴巴（中国）有限公司 互动操作实现方法、装置及客户端设备
CN109099902A (zh) * 2018-06-29 2018-12-28 中国航空规划设计研究总院有限公司 一种基于Unity 3D的虚拟现实全景导航系统
CN110448912A (zh) * 2019-07-31 2019-11-15 维沃移动通信有限公司 终端控制方法及终端设备
CN110704557B (zh) * 2019-09-10 2023-03-10 广东华远国土工程有限公司 一种电子地图的实景浏览方法
CN111815759B (zh) * 2020-06-18 2021-04-02 广州建通测绘地理信息技术股份有限公司 一种可量测实景图片的生成方法、装置、计算机设备
CN112764629B (zh) * 2021-01-28 2022-02-18 北京城市网邻信息技术有限公司 增强现实界面展示方法、装置、设备和计算机可读介质
CN112882569B (zh) * 2021-01-28 2024-02-23 咪咕文化科技有限公司 Ar交互方法、终端设备及云端地图管理系统
CN113313840A (zh) * 2021-06-15 2021-08-27 周永奇 实时虚拟系统及实时虚拟交互方法
CN115437731A (zh) * 2022-08-02 2022-12-06 广东虚拟现实科技有限公司 环境信息的交互显示方法、装置、设备和可读存储介质
CN117152324B (zh) * 2023-09-04 2024-07-12 艾迪普科技股份有限公司 基于三维播放器的数据驱动方法和装置

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104199944A (zh) * 2014-09-10 2014-12-10 重庆邮电大学 一种实现街景视图展示的方法及装置
WO2016077493A1 (en) * 2014-11-11 2016-05-19 Bent Image Lab, Llc Real-time shared augmented reality experience
CN106372260A (zh) * 2016-10-25 2017-02-01 广州卓能信息科技有限公司 一种信息交互方法、装置及系统
CN106371603A (zh) * 2016-09-18 2017-02-01 成都动鱼数码科技有限公司 基于位置服务和增强现实技术的角色定位捕捉方法
CN106648322A (zh) * 2016-12-21 2017-05-10 广州市动景计算机科技有限公司 与虚拟对象进行互动操作的触发方法、装置和系统
CN107493228A (zh) * 2017-08-29 2017-12-19 北京易讯理想科技有限公司 一种基于增强现实的社交互动方法和系统
CN108144294A (zh) * 2017-12-26 2018-06-12 优视科技有限公司 互动操作实现方法、装置及客户端设备

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103150759B (zh) * 2013-03-05 2015-11-25 腾讯科技（深圳）有限公司 一种对街景图像进行动态增强的方法和装置
JP6522882B2 (ja) * 2014-03-20 2019-05-29 任天堂株式会社 情報処理装置、情報処理システム、情報処理プログラム、および情報処理方法
CN111899003B (zh) * 2016-12-13 2024-11-22 创新先进技术有限公司 基于增强现实的虚拟对象分配方法及装置

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104199944A (zh) * 2014-09-10 2014-12-10 重庆邮电大学 一种实现街景视图展示的方法及装置
WO2016077493A1 (en) * 2014-11-11 2016-05-19 Bent Image Lab, Llc Real-time shared augmented reality experience
CN106371603A (zh) * 2016-09-18 2017-02-01 成都动鱼数码科技有限公司 基于位置服务和增强现实技术的角色定位捕捉方法
CN106372260A (zh) * 2016-10-25 2017-02-01 广州卓能信息科技有限公司 一种信息交互方法、装置及系统
CN106648322A (zh) * 2016-12-21 2017-05-10 广州市动景计算机科技有限公司 与虚拟对象进行互动操作的触发方法、装置和系统
CN107493228A (zh) * 2017-08-29 2017-12-19 北京易讯理想科技有限公司 一种基于增强现实的社交互动方法和系统
CN108144294A (zh) * 2017-12-26 2018-06-12 优视科技有限公司 互动操作实现方法、装置及客户端设备

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111930082A (zh) * 2020-07-22 2020-11-13 青岛海信智慧家居系统股份有限公司 一种智能家居设备的更换方法及装置
CN112863229A (zh) * 2020-12-30 2021-05-28 中兴智能交通股份有限公司 一种基于停车设备和技术实现无人值守的系统及其方法

Also Published As

Publication number Publication date
CN108144294B (zh) 2021-06-04
CN108144294A (zh) 2018-06-12

Similar Documents

Publication Publication Date Title
CN108144294B (zh) 互动操作实现方法、装置及客户端设备
CN108446310B (zh) 虚拟街景地图生成方法、装置和客户端设备
CN109865289B (zh) 一种基于增强现实技术的实景环境娱乐系统及其方法
US9324298B2 (en) Image processing system, image processing apparatus, storage medium having stored therein image processing program, and image processing method
US10022626B2 (en) Information processing system, information processing apparatus, storage medium having stored therein information processing program, and information processing method, for performing augmented reality
CN102054289B (zh) 一种基于全景实景和地理信息的3d虚拟社区构建方法
KR101736477B1 (ko) 저장된 콘텐츠 및 ar 통신의 로컬 센서 증강
US20140267234A1 (en) Generation and Sharing Coordinate System Between Users on Mobile
US20180174369A1 (en) Method, apparatus and system for triggering interactive operation with virtual object
CN106803966B (zh) 一种多人网络直播方法、装置及其电子设备
US9947139B2 (en) Method and apparatus for providing hybrid reality environment
CN105843396B (zh) 在共享的稳定虚拟空间上维持多视图的方法
CN102054121B (zh) 3d全景实景网络游戏平台构建方法
US20140152764A1 (en) Information processing system, information processing apparatus, storage medium having stored therein information processing program, and information transmission/reception method
CN112312111A (zh) 虚拟图像的显示方法、装置、电子设备及存储介质
US12361632B2 (en) Information processing system, information processing method, and information processing program
JP2019535090A (ja) 仮想現実アトラクション制御方法及びシステム
CN104484327A (zh) 一种项目工程的环境展示方法
CN107638690B (zh) 增强现实的实现方法、装置、服务器及介质
CN114401414A (zh) 沉浸式直播的信息显示方法及系统、信息推送方法
JP2014217566A (ja) 宝探しゲーム配信システム
JP2016122277A (ja) コンテンツ提供サーバ、コンテンツ表示端末、コンテンツ提供システム、コンテンツ提供方法、および、コンテンツ表示プログラム
KR20250055536A (ko) 3d 지도 세그먼트에서 객체 정렬 유지
WO2018149321A1 (zh) 提供交互对象信息的方法及装置
CN118488221A (zh) 展示方法、数据处理方法、装置、电子设备及计算机介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18897755

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18897755

Country of ref document: EP

Kind code of ref document: A1