CN114942716A

CN114942716A - VR scene establishing method and device

Info

Publication number: CN114942716A
Application number: CN202210515673.5A
Authority: CN
Inventors: 陈苑锋; 杜鹏; 林航东; 卢祺; 奉飞飞
Original assignee: Midea Group Co Ltd; Midea Group Shanghai Co Ltd
Current assignee: Midea Group Co Ltd; Midea Group Shanghai Co Ltd
Priority date: 2022-05-11
Filing date: 2022-05-11
Publication date: 2022-08-26

Abstract

The application relates to the technical field of VR (virtual reality), and provides a VR scene establishing method and device. The method comprises the following steps: receiving a plurality of collected images of a first scene acquired by an image collecting terminal; determining the displacement of the target object according to the position of the target object in the first target image in the first scene and the position of the target object in the second target image; and establishing a VR scene according to the displacement of the target object, the collected image and the pre-stored image. According to the VR scene establishing method and device, the moving state and the static state of the image acquisition terminal for shooting the first scene are determined according to the multiple acquired images of the first scene acquired by the image acquisition terminal, and then the acquired images and the pre-stored images are combined according to different states to establish different VR game scenes, so that the interestingness of the combination of the robot and the VR technology can be improved, the practicability of the combination of the robot and the VR technology is effectively expanded, and the VR scene establishing method and device have a very wide application prospect.

Description

VR scene establishing method and device

Technical Field

The application relates to the technical field of VR (virtual reality), in particular to a VR scene establishing method and device.

Background

At present, the robot is provided with various software and hardware functions, and a great deal of convenience is provided in life and work. On the other hand, VR (Virtual Reality) technology is increasingly used in life, and typical applications include VR games and the like.

However, the prior art is rarely applicable to combining robotics with VR technology. Most, if any, the method is used for specific work scenes, such as dangerous and complex work scenes of deep well mining, earthquake rescue, submarine exploration and the like by combining the robot technology and VR technology.

Therefore, how to expand the application of combining the robot and the VR technology is of great significance to improving the practicability of the robot and the VR technology.

Disclosure of Invention

The present application is directed to solving at least one of the technical problems occurring in the related art. Therefore, the method for establishing the VR scene can improve the interestingness of the combination of the robot and the VR technology and effectively expands the practicability of the combination of the robot and the VR technology.

The application also provides a VR scene establishing system.

The application also provides a VR scene establishing device.

The VR scene establishing method according to the embodiment of the first aspect of the application comprises the following steps:

receiving a plurality of collected images of a first scene acquired by an image collecting terminal; the plurality of captured images includes a first target image and a second target image;

determining the displacement of the object according to the position of the object in the first object image in the first scene and the position of the object in the second object image; wherein the first target image is a captured image acquired prior to the second target image; the object is a stationary object in the first scene;

and establishing a VR scene according to the displacement of the target object, the acquired image and a pre-stored image.

According to the VR scene establishing method, the movement and the static state of the image acquisition terminal for shooting the first scene are determined according to the multiple acquired images of the first scene acquired by the image acquisition terminal, and then the acquired images and the pre-stored images are combined according to different states to establish different VR game scenes, so that the interestingness of the combination of the robot and the VR technology can be improved, the practicability of the combination of the robot and the VR technology is effectively expanded, and the VR scene establishing method has a very wide application prospect.

According to an embodiment of the present application, the establishing a VR scene according to the displacement, the captured image and a pre-stored image includes:

determining that the size of the displacement of the target object is larger than 0, and determining a first matching image matched with the direction of the displacement of the target object from a plurality of pre-stored images according to the direction of the displacement of the target object;

and overlapping the acquired image and the first matching image to establish the VR scene.

According to the VR scene establishing method, the VR game scene is established by determining the first matching image according to the displacement direction of the target object under the condition that the displacement of the target object of the first scene is larger than 0, so that the established VR game scene is matched with the motion state of the image acquisition terminal, the fidelity of the VR game scene is effectively improved, and the user experience is guaranteed.

According to an embodiment of the application, said superimposing said acquired image and said first matching image comprises:

superimposing the first matching image over the acquired image.

According to an embodiment of the application, the first matching image is an animated image, the method further comprising:

controlling display of the animated image according to at least one of a magnitude of the displacement of the target object or a direction of the displacement of the target object.

According to an embodiment of the present application, the creating a VR scene according to the displacement of the target object, the captured image and the pre-stored image includes:

determining that the displacement of the target object is 0, and determining a second matching image matched with the target object with the displacement of 0 from a plurality of pre-stored images;

and overlapping the acquired image and the second matching image to establish the VR scene.

According to the VR scene establishing method provided by the embodiment of the application, under the condition that the displacement of the target object of the first scene is 0, the second matching image is determined from the pre-stored images, and the VR game scene is established according to the collected image and the second matching image, so that the established VR game scene is matched with the motion state of the image collecting terminal, the fidelity of the VR game scene is effectively improved, and the user experience is guaranteed.

According to an embodiment of the application, the second matching image comprises a target object, the method further comprising:

determining the position of the target object in the superposed image according to the content of the acquired image; the superimposed image is an image formed by superimposing the second matching image on the captured image.

According to an embodiment of the application, after the superimposing the captured image and the second matching image to establish the VR scene, the method further includes:

receiving input of a user;

responding to the input, generating a control instruction, and sending the control instruction to the image acquisition terminal; the control instruction is used for controlling the movement of the image acquisition terminal.

According to the VR game scene establishing method, the control instruction is generated in response to the input of the user, and is sent to the image acquisition terminal, interaction between the user and the established VR game scene can be achieved, so that the participation degree of the user on the VR game scene is effectively improved, and the user experience is further improved.

According to one embodiment of the application, the image acquisition terminal comprises a mechanical arm and a shooting device; the shooting device is arranged at one end of the mechanical arm;

the acquired image is acquired by any one of the following methods:

under the condition that the mechanical arm moves, the shooting device moves along with the mechanical arm and acquires the acquired image in the moving process;

in a case where the robot arm is stationary, the photographing device acquires the captured image from a stationary position of the photographing device.

According to one embodiment of the application, the displacement of the target object corresponds to the displacement generated by the mechanical arm driving the shooting device to move in a target time interval;

wherein the target time interval is a time interval between acquiring the first target image and acquiring the second target image.

The VR scene creation system according to the second aspect of the present application includes:

the system comprises an image acquisition terminal, a first scene acquisition terminal and a second scene acquisition terminal, wherein the image acquisition terminal is used for acquiring a plurality of acquired images of a first scene;

the VR display terminal is in communication connection with the image acquisition terminal; the VR display terminal is used for:

receiving the plurality of collected images acquired by the image collecting terminal; the plurality of captured images includes a first target image and a second target image;

and establishing a VR scene according to the displacement of the target object, the collected image and a pre-stored image.

The VR scene establishment system provided by the embodiment of the application determines the motion and the static state of the image acquisition terminal for shooting the first scene through a plurality of acquired images of the first scene acquired by the image acquisition terminal, combines the acquired images and the pre-stored images according to different states to establish different VR game scenes, can improve the interestingness of the combination of the robot and the VR technology, effectively expands the practicability of the combination of the robot and the VR technology, and has very wide application prospect

the image acquisition terminal is specifically configured to:

According to an embodiment of the application, under the condition that the mechanical arm drives the shooting device to move within a target time interval to generate a displacement larger than 0, the VR display terminal is specifically configured to determine a first matching image matched with the displacement direction of the target object from a plurality of pre-stored images according to the displacement direction of the target object, and superimpose the acquired image and the first matching image to establish the VR scene;

and under the condition that the mechanical arm drives the shooting device to move within a target time interval to generate 0 displacement, the VR display terminal is specifically used for determining a second matching image matched with the target object in the displacement of 0 from a plurality of pre-stored images, and overlapping the acquired image and the second matching image to establish the VR scene.

According to an embodiment of the present application, the image capturing terminal is further configured to:

receiving a control instruction sent by the VR display terminal;

and controlling the movement of the mechanical arm based on the control instruction.

The VR scene creation apparatus according to the third aspect of the present application includes:

the receiving module is used for receiving a target image of a first scene acquired by the image acquisition terminal;

the determining module is used for determining the displacement of the target object according to the position of the target object in the first target image in the first scene and the position of the target object in the second target image; wherein the first target image is a target image acquired prior to the second target image; the object is a stationary object in the first scene;

and the establishing module is used for establishing a VR scene according to the displacement, the target image and a pre-stored image.

The VR scene establishing device provided by the embodiment of the application confirms the movement and the static state of the image acquisition terminal for shooting the first scene through a plurality of acquired images of the first scene acquired according to the image acquisition terminal, and establishes different VR game scenes according to the target image and the pre-stored image in different states, so that the interestingness of the combination of the robot and the VR technology can be improved, the practicability of the combination of the robot and the VR technology is effectively expanded, and the VR scene establishing device has a very wide application prospect.

An electronic device according to an embodiment of the fourth aspect of the present application includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor implements the VR scene establishment method according to the first aspect when executing the program.

A non-transitory computer-readable storage medium according to an embodiment of the fifth aspect of the present application, having stored thereon a computer program which, when executed by a processor, implements the VR scene creation method as described in the first aspect.

A computer program product according to an embodiment of the sixth aspect of the present application comprises a computer program which, when executed by a processor, implements the VR scene creation method as described in the first aspect.

One or more technical solutions in the embodiments of the present application have at least one of the following technical effects:

the VR scene is established through the displacement of the target object in the first scene, the acquired image acquired by the image acquisition terminal and the pre-stored image, so that the established VR scene is vivid and lively, and the game experience of the user is improved.

Because the collection image that image acquisition terminal shot can be arbitrary scene, for example house scene etc. consequently still effectively improved the interest in VR scene.

Additional aspects and advantages of the present application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the present application.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments or related technologies of the present application, the drawings needed to be used in the description of the embodiments or related technologies are briefly introduced below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.

Fig. 1 is a schematic flowchart of a VR scene establishment method provided in an embodiment of the present application;

fig. 2 is a schematic structural diagram of a VR scene setup system provided in an embodiment of the present application;

fig. 3 is a schematic structural diagram of a VR scene creation apparatus according to an embodiment of the present application;

fig. 4 is a schematic physical structure diagram of an electronic device according to an embodiment of the present application.

Detailed Description

Embodiments of the present application will be described in further detail with reference to the drawings and examples. The following examples are intended to illustrate the present application but are not intended to limit the scope of the present application.

In the description of the embodiments of the present application, it should be noted that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", and the like indicate orientations or positional relationships based on orientations or positional relationships shown in the drawings, and are only for convenience of description of the embodiments of the present application and to simplify the description, but do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus should not be construed as limiting the embodiments of the present application. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.

In the description of the embodiments of the present application, it should be noted that the terms "connected" and "connected" are to be interpreted broadly, and may be, for example, fixedly connected, detachably connected, or integrally connected, unless explicitly stated or limited otherwise; can be mechanically or electrically connected; may be directly connected or indirectly connected through an intermediate. Specific meanings of the above terms in the embodiments of the present application can be understood in specific cases by those of ordinary skill in the art.

In the embodiments of the present application, unless otherwise explicitly specified or limited, a first feature "on" or "under" a second feature may be directly contacting the first and second features or indirectly contacting the first and second features through an intermediate. Also, a first feature "on," "over," and "above" a second feature may be directly or diagonally above the second feature, or may simply indicate that the first feature is at a higher level than the second feature. A first feature being "under," "below," and "beneath" a second feature may be directly under or obliquely under the first feature, or may simply mean that the first feature is at a lesser elevation than the second feature.

In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of an embodiment of the application. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Moreover, various embodiments or examples and features of various embodiments or examples described in this specification can be combined and combined by one skilled in the art without being mutually inconsistent.

Fig. 1 is a schematic flowchart of a VR scene establishment method provided in an embodiment of the present application. Referring to fig. 1, a VR scene establishment method provided in an embodiment of the present application may include:

step 110, receiving a plurality of collected images of a first scene obtained by an image collecting terminal; the plurality of acquired images comprise a first target image and a second target image;

step 120, determining the displacement of the target object according to the position of the target object in the first target image in the first scene and the position of the target object in the second target image; wherein the first target image is a captured image acquired prior to the second target image; the object is a stationary object in a first scene;

and step 130, establishing a VR scene according to the displacement of the target object, the collected image and the pre-stored image.

It should be noted that an execution subject of the VR scene establishment method provided in the embodiment of the present application may be a VR display terminal, such as VR glasses. The image acquisition terminal may be a combination of a robot arm and a photographing device of the robot. Alternatively, the photographing device may be provided at one end of the robot arm, for example, fixedly or detachably provided at one end of the robot arm, and the image photographed by the photographing device may be changed according to the movement of the robot arm.

In step 110, the image capturing terminal may acquire a captured image of a first scene (i.e., a current scene that can be captured by the image capturing terminal) in real time, and send the captured image to the VR display terminal.

Wherein, image acquisition terminal and VR display terminal can communication connection. Furthermore, the data interaction between the image acquisition terminal and the VR display terminal can be realized by adopting a high-speed communication protocol, so that the data transmission delay is reduced, and the real-time performance of the data interaction between the image acquisition terminal and the VR display terminal is ensured. For example, the image capturing terminal may transmit the captured image to the VR display terminal through an UWB (Ultra Wide Band) technology.

It should be noted that, when the image capturing terminal includes the mechanical arm and the shooting device, and the shooting device is disposed at one end of the mechanical arm, the captured image may be acquired in any one of the following manners:

mode 1: under the condition that the mechanical arm moves, the shooting device moves along with the mechanical arm and acquires an acquired image in the moving process;

it can be understood that, when the mechanical arm moves, the mechanical arm can drive the shooting device to move along with the movement of the mechanical arm, and at the moment, the shooting device can acquire the collected images of the shooting device in the movement process in real time or at certain time intervals.

For example, when the mechanical arm dives from top to bottom, the shooting device may send the captured image of the first scene shot in the top-to-bottom movement process to the VR display terminal in real time. When the mechanical arm rotates from left to right, the shooting device can send the collected image of the first scene shot in the process of moving from left to right to the VR display terminal in real time.

Mode 2: in the case where the robot arm is stationary, the photographing device acquires a captured image from a stationary position of the photographing device.

It can be understood that, when the mechanical arm is stationary, the shooting device may send the captured image of the first scene shot to the VR display terminal in real time.

For example, when the mechanical arm makes the photographing device still photograph food (a first scene) in a refrigerator, the photographing device may transmit a captured image including the food in the refrigerator to the VR display terminal in real time; when the mechanical arm makes the shooting device still shoot the obstacle (first scene) on the desktop, the shooting device can send the collected image of the obstacle including the desktop to the VR display terminal in real time.

Adopt multiple mode to acquire the collection image through the combination of arm and shooting device, can realize gathering the variety of image to for follow-up different VR recreation scenes of establishing provide the basis, and then guarantee the interest that robot and VR technique combine.

In step 120, the VR display terminal may determine a first target image and a second target image according to a plurality of captured images of a first scene transmitted by the image capturing terminal in real time, and then determine a displacement of the target object according to a position of the target object in the first scene in the first target image and a position of the target object in the second target image.

Wherein the object in the first scene may be any stationary object in the first scene. For example, when the first scene is a room, the target object may be furniture such as a table, a chair, and a cabinet in the room, or may be an ornament such as a wall painting and a wall clock on a wall in the room. The displacement of the object refers to the displacement of the object in the second target image relative to its displacement in the first target image.

The VR display terminal can determine the displacement of the target object according to the two continuous collected images. For example, the VR display terminal may determine that the table moves upward by 0.01m in a unit time by using a first acquired image of two consecutive acquired images as a first target image and using a second acquired image of the two consecutive acquired images as a second target image; the VR display terminal can also randomly sample two images from a plurality of continuous acquired images acquired by the image acquisition terminal to respectively serve as a first target image and a second target image, and further determine that the table moves downwards by 0.02m and the like in unit time.

It should be noted that the displacement of the target corresponds to the displacement generated by the mechanical arm driving the shooting device to move within the target time interval;

wherein the target time interval is the time interval between acquiring the first target image and acquiring the second target image.

It can be understood that, under the condition that the mechanical arm moves, the shooting device moves along with the mechanical arm and acquires the target image in the moving process, so that the displacement of the mechanical arm driving the shooting device to move in the target time interval is reflected on the content included in the first target image and the second target image. That is, in the case that the first target image and the second target image both include a stationary target object, the displacement of the target object may correspond to the displacement generated when the mechanical arm drives the shooting device to move within the target time interval.

The displacement of the target object is achieved through the combination of the mechanical arm and the shooting device, the mechanical arm drives the shooting device to move in the target time interval, and the displacement of the mechanical arm can be accurately reflected by the collected images, so that the established VR game scene has a vivid effect, and the interestingness of the combination of the robot and the VR technology is guaranteed.

In step 130, the VR display terminal may establish a VR scene according to the displacement of the target object, the collected image, and the pre-stored image.

For example, when the VR display terminal determines that the displacement of the target object is 0, it may be determined that the mechanical arm is in a stationary state at the time, that is, the mechanical arm does not drive the shooting device to move, and the VR display terminal may establish a game scene, corresponding to the mechanical arm being in the stationary state, for playing the game in the first scene according to the collected image and the pre-stored image. Wherein, the pre-stored image may be an image pre-stored by the VR display terminal.

When the VR display terminal determines that the displacement of the target object is not 0, the mechanical arm can be determined to be in a motion state at the moment, namely the mechanical arm drives the shooting device to move, and the VR display terminal can establish a game scene corresponding to the mechanical arm in the motion state and used for playing in a first scene according to the displacement, the collected image and the pre-stored image of the target object.

According to the VR scene establishing method, the moving state and the static state of the image acquisition terminal for shooting the first scene are determined according to the multiple acquired images of the first scene acquired by the image acquisition terminal, and then the acquired images and the pre-stored images are combined according to different states to establish different VR game scenes, so that the interestingness of the combination of the robot and the VR technology can be improved, the practicability of the combination of the robot and the VR technology is effectively expanded, and the VR scene establishing method has a very wide application prospect.

In one embodiment, step 130 may include:

determining that the displacement of the target object is larger than 0, and determining a first matching image matched with the displacement direction of the target object from a plurality of pre-stored images according to the displacement direction of the target object;

and overlapping the collected image and the first matching image to establish a VR scene.

The VR display terminal can judge whether the displacement of the target object in unit time is larger than 0 or not, if so, the mechanical arm is in a motion state, namely, the mechanical arm drives the shooting device to move. At this time, the VR display terminal may further determine a first matching image matching the direction of the displacement of the target object from among the pre-stored plurality of images according to the direction of the displacement.

For example, when the mechanical arm of the image acquisition terminal dives from top to bottom, the shooting device of the image acquisition terminal sends the acquired image of the first scene shot from top to bottom to the VR display terminal in real time, and the VR display terminal determines whether the displacement of the target object is greater than 0 according to the position of the target object in the second target image and the position of the target object in the first target image. If the displacement direction is from top to bottom, the VR display terminal determines an image including a coaster element, which is matched with the displacement direction from top to bottom, from a plurality of pre-stored images as a first matching image, and overlaps the collected image and the first matching image to establish a VR game scene for the coaster game in the first scene.

The image including the roller coaster element may include a roller coaster track and other objects reflecting the roller coaster element, such as a roller coaster seat, a roller coaster head, a roller coaster front seat, and the like.

For example, when the mechanical arm of the image acquisition terminal rotates from left to right, the shooting device of the image acquisition terminal sends the acquired image of the first scene shot from left to right to the VR display terminal in real time, and the VR display terminal determines whether the displacement of the target object is greater than 0 according to the position of the target object in the second target image and the position of the target object in the first target image. If the displacement direction of the target object is from left to right, the VR display terminal determines an image which is matched with the displacement direction from left to right and comprises racing car elements from a plurality of pre-stored images to serve as a first matching image, and the collected image and the first matching image are overlapped to establish a VR game scene for racing car games in a first scene.

The image including the elements of the race may include, among other things, the track and other objects reflecting the elements of the race, such as the cockpit of the race, the rear view mirrors of the race, other races in front, etc.

In one embodiment, superimposing the target image and the first matching image may include:

the first matching image is superimposed over the acquired image.

After the target image and the first matching image are determined, the VR display terminal can superpose the collected image and the first matching image to obtain a superposed image, and establishes a VR scene based on the superposed image.

In this embodiment, the VR display terminal may preferentially superimpose the first matching image on the captured image, that is, the VR display terminal takes the captured image as a bottom layer image, takes the first matching image as an upper layer image, and then superimposes the captured image and the first matching image.

It should be noted that, the first matching image is superimposed on the collected image, so that the first matching image in the superimposed image can be displayed more clearly and obviously, and the interest of the game can be more prominent in the finally established VR scene.

In an embodiment, the first matching image may be an animated image, and the VR scene establishment method provided in the embodiment of the present application may further include:

For example, when the VR display terminal establishes a VR game scene for a roller coaster game in a first scene according to the captured image and the first matching image, and the first matching image is an animation image of a roller coaster track, the VR display terminal may control display of the roller coaster track according to a direction of displacement of the target object. For example, when the displacement of the target object is upward, the VR display terminal may control the roller coaster track to lengthen in the vertical direction to simulate a scene when sitting over the roller coaster and climbing upward.

The VR display terminal can also control the display of the roller coaster track according to the displacement of the target object. For example, when the displacement of the target object in the unit time is large, the VR display terminal can control the roller coaster track to move at a fast speed to simulate the scene when the roller coaster accelerates.

The VR display terminal can also control the display of the roller coaster track according to the magnitude and direction of displacement of the target object. For example, when the displacement of the target object is upward and the displacement per unit time is large, the VR display terminal may control the roller coaster track to be elongated in the vertical direction and to move at a fast speed to simulate a scene when the roller coaster climbs fast. When the displacement of the target object is upward and the displacement in unit time is small, the VR display terminal can control the roller coaster track to extend in the vertical direction and move at a slow speed so as to simulate a scene when the roller coaster climbs at a low speed.

Of course, when the VR display terminal establishes the VR game scene of the roller coaster according to the collected image and the first matching image, the VR display terminal may also control the display of the seat of the roller coaster, the head of the roller coaster, the front seat of the roller coaster, and the like according to the size and/or the direction of the displacement of the target object, which is not given as an example in the embodiments of the present application.

When the VR display terminal establishes a VR game scene for racing in a first scene according to the collected image and the first matching image, and the first matching image is an animation image of the track, the VR display terminal can control the display of the track according to the displacement direction of the target object. For example, when the displacement of the target object is to the left, the VR display terminal may control the track to lengthen to the left to simulate a scene when the car is turning left.

The VR display terminal may control the display of the track according to the magnitude of the displacement of the target object. For example, when the displacement of the target object in a unit time is large, the VR display terminal may control the track to move at a fast speed to simulate a scene when the racing car accelerates.

The VR display terminal may also control the display of the track according to the magnitude and direction of the displacement of the target object. For example, when the displacement of the target object is to the left and the displacement per unit time is large, the VR display terminal may control the track to lengthen in the left and move at a faster speed to simulate a scene when a racing car makes a fast left turn. When the displacement of the target object is leftward and the displacement in the unit time is small, the VR display terminal can control the track to be lengthened leftward and move at a slow speed so as to simulate the scene when the racing car turns left at a low speed.

Of course, when the VR display terminal establishes the VR game scene of the racing car according to the collected image and the first matching image, the VR display terminal may also control the display of the driving cab of the racing car, the rear view mirror of the racing car, other racing cars ahead, and the like according to the size and/or the direction of the displacement of the target object, which is not an example in the embodiment of the present application.

According to the VR scene establishing method provided by the embodiment of the application, the first matching image is controlled to change along with at least one of the displacement size and the displacement direction of the target object, so that the fidelity and the interestingness of the VR game scene can be further improved, and the user experience is further improved.

In one embodiment, step 130 may include:

determining that the displacement is 0, and determining a second matching image matched with the target object in the displacement of 0 from a plurality of pre-stored images;

and overlapping the acquired image and the second matching image to establish a VR scene.

VR display terminal can judge earlier whether the displacement of target object in the unit interval is 0, if, indicates that the arm is quiescent condition, and the arm does not drive the shooting device motion promptly. At this time, the VR display terminal may determine a second matching image matching the size of the displacement of 0 from among the pre-stored plurality of images.

When the mechanical arm of the image acquisition terminal is static, the shooting device of the image acquisition terminal can send the shot acquisition image of the first scene to the VR display terminal in real time, the VR display terminal firstly determines whether the displacement of the target object is 0 according to the position of the target object in the second target image and the position of the target object in the first target image, if so, the VR display terminal can determine the image which is matched with the displacement in a size of 0 and comprises the cartoon image as a second matching image from a plurality of pre-stored images, and the acquisition image and the second matching image are overlapped to establish the VR game scene.

For example, when the mechanical arm of the image capturing terminal makes the shooting device still shoot food (a first scene) in the refrigerator, the VR display terminal may determine an image including a fire spray from a plurality of pre-stored images as a second matching image, and superimpose the captured image and the second matching image to establish a VR game scene in which the fire spray explores in the refrigerator.

For example, when the mechanical arm of the image capturing terminal causes the shooting device to still shoot an obstacle (a first scene) on the desktop, the VR display terminal may determine an image including the cartoon character from a plurality of pre-stored images as a second matching image, and superimpose the captured image and the second matching image to create a VR game scene in which the cartoon character is expeditioned on the desktop.

According to the VR scene establishing method, under the condition that the displacement of the target object is 0, the second matching image is determined from the pre-stored images, and the VR game scene is established according to the collected image and the second matching image, so that the established VR game scene can be matched with the motion state of the image collecting terminal, the fidelity of the VR game scene is effectively improved, and the user experience is guaranteed.

In an embodiment, the second matching image may include a target object, and the VR scene setup method provided in the embodiment of the present application may further include:

determining the position of the target object in the superposed image according to the content of the acquired image; the superimposed image is an image formed by superimposing the second matching image on the captured image. The content of the acquired image can be target content having a matching relationship with the target object; the matching relationship may be a custom relationship, for example the matching relationship may include: the target object may be placed on top of the target content, the target content is food for the target object, the target content may obstruct the target object from advancing, etc.

For example, when the target object is a flamingo, a cake with candles inserted in the captured image is included in the captured image, the VR display terminal may place the flamingo next to the candles in the superimposed image to simulate a VR game scene where the flamingo ignites the candles. When the target object is a cartoon character and the collected image includes a cup located on the dining table, the VR display terminal can place the cartoon character on the cup in the superposed image to simulate a VR game scene in which the cartoon character jumps down from the cup to explore.

It should be noted that, by superimposing the second matching image on the acquired image, it can be ensured that the second matching image in the superimposed image can be displayed more clearly and obviously, so that the interest of the game can be more prominent in the finally established VR scene.

According to the VR scene establishing method, the position of the target object in the superposed image is determined according to the content of the collected image, the established VR game scene can be ensured to be matched with the content of the superposed image, the fidelity of the VR game scene is effectively improved, and the user experience is ensured.

In an embodiment, after the acquired image and the second matching image are superimposed to establish the VR scene, the VR scene establishment method provided in the embodiment of the present application may further include:

receiving input of a user;

After the VR display terminal establishes the VR scene according to the collected image and the second matching image, the input of a user can be received, and the movement of the image collection terminal is controlled according to the input, so that expedition in the VR game scene is simulated.

Taking the VR display terminal to establish the VR game scene of the cartoon character expedition on the desktop according to the target image and the second matching image as an example: when the cartoon character is positioned above the teacup, the VR display terminal can receive input indicating 'jump' by the user, and the VR display terminal responds to the input to generate a control instruction and sends the control instruction to the image acquisition terminal. The image acquisition terminal can enable the mechanical arm to move obliquely upwards (the acquired image shot by the shooting device can simulate the view angle when the cartoon character jumps up at the moment) and then enable the mechanical arm to move obliquely downwards according to a certain parabola (the acquired image shot by the shooting device can simulate the view angle when the cartoon character jumps down from a high place at the moment). Through the operations of the VR display terminal and the image acquisition terminal, the scene that the cartoon character jumps down from the teacup can be completely simulated.

In summary, according to the VR scene establishment method provided by the embodiment of the application, on one hand, the VR scene is established through the displacement of the target object in the first scene, the acquired image acquired by the image acquisition terminal and the pre-stored image, so that the established VR scene is vivid and lively, and the game experience of the user is improved; on the other hand, as the acquired image shot by the image acquisition terminal can be any scene, such as a home scene, the interestingness of the VR scene is effectively improved.

The embodiment of the present application further provides a VR scene setup system, which may be used to execute the VR scene setup method described above.

Fig. 2 is a schematic structural diagram of a VR scene creation system according to an embodiment of the present application. Referring to fig. 2, the VR scene establishment system provided in the embodiment of the present application may include:

an image capture terminal 210 for obtaining a plurality of captured images of a first scene;

the VR display terminal 220 is in communication connection with the image acquisition terminal 210; VR display terminal 220 is configured to:

receiving a plurality of captured images acquired by the image capture terminal 210; the plurality of acquired images comprise a first target image and a second target image;

determining the displacement of the target object according to the position of the target object in the first target image in the first scene and the position of the target object in the second target image; wherein the first target image is a captured image acquired prior to the second target image; the object is a stationary object in a first scene;

and establishing a VR scene according to the displacement of the target object, the collected image and the pre-stored image.

It should be noted that the image capturing terminal 210 may be disposed on the robot, and the data interaction between the image capturing terminal 210 and the VR display terminal 220 may be implemented by using a high-speed communication protocol, so as to reduce data transmission delay and ensure real-time performance of the data interaction between the image capturing terminal 210 and the VR display terminal 220. For example, the image capturing terminal 210 may transmit the captured image to the VR display terminal 220 by an UWB (Ultra Wide Band) technology.

It should be noted that, in the VR scene establishment system provided in the embodiment of the present application, the working principle of the image acquisition terminal 210 is the same as that of the image acquisition terminal in the method embodiment, and the working principle of the VR display terminal 220 is the same as that of the VR display terminal in the method embodiment, which is not described herein again.

The VR scene establishment system determines the movement and the static state of the image acquisition terminal for shooting the first scene through a plurality of acquired images of the first scene acquired according to the image acquisition terminal, combines the acquired images and pre-stored images according to different states to establish different VR game scenes, can improve the interestingness of the combination of a robot and a VR technology, effectively expands the practicability of the combination of the robot and the VR technology, and has a very wide application prospect.

In one embodiment, the image capture terminal 210 includes a robotic arm 211 and a camera 212; the shooting device 212 is arranged at one end of the mechanical arm 211;

image capture terminal 210 is specifically configured to:

under the condition that the mechanical arm 211 moves, the photographing device 212 moves along with the mechanical arm 211 and acquires a collected image in the moving process;

in the case where the robot arm 211 is stationary, the camera 212 acquires a captured image from a stationary position of the camera 212.

The photographing device 212 may be, for example, fixedly or detachably provided at one end of the robot arm 211 so that an image photographed by the photographing device may be changed according to the movement of the robot arm 211.

It can be understood that, when the mechanical arm 211 moves, the mechanical arm 211 drives the camera 212 to move along with the movement of the mechanical arm 211, and at this time, the camera 212 can acquire the captured image of the camera 212 in real time or at certain time intervals during the movement.

For example, when the robot arm 211 is pitched down from top to bottom, the camera 212 may transmit the captured image of the first scene captured during the top-to-bottom motion to the VR display terminal 220 in real time. When the mechanical arm 211 rotates from left to right, the camera 212 may transmit the captured image of the first scene captured during the left-to-right movement to the VR display terminal 220 in real time.

It is understood that when the robot 211 is stationary, the camera 212 may transmit the captured image of the first scene to the VR display terminal 220 in real time.

For example, when the robot arm 211 makes the photographing device 212 still photograph food (first scene) in a refrigerator, the photographing device 212 may transmit a captured image including the food in the refrigerator to the VR display terminal 220 in real time; when the mechanical arm 211 makes the photographing device 212 still photograph an obstacle (first scene) of the table top, the photographing device 212 may transmit a captured image including the obstacle of the table top to the VR display terminal 220 in real time.

Adopt multiple mode to acquire the collection image through the combination of arm 212 and shooting device 222, can realize the variety of gathering the image to for follow-up different VR recreation scenes of establishing provide the basis, and then guarantee the interest that robot and VR technique combine.

In one embodiment, the displacement of the target corresponds to the displacement of the robotic arm 211 to move the camera 212 during the target time interval;

It can be understood that, in the case that the mechanical arm 211 moves, the camera 212 moves along with the mechanical arm 211 and acquires the target image in the moving process, so that the displacement of the mechanical arm 211 driving the camera 212 to move is reflected on the content included in the first target image and the second target image in the target time interval. That is, in the case where the first target image and the second target image each include a stationary target object, the displacement of the target object may correspond to the displacement resulting from the robot arm 211 moving the camera 212 within the target time interval.

The displacement of the target object is realized through the combination of the mechanical arm 211 and the shooting device 212, and the displacement generated when the mechanical arm 211 drives the shooting device 212 to move in a target time interval corresponds to the displacement, so that the collected image can accurately reflect the displacement of the mechanical arm 211, the established VR game scene has a vivid effect, and the interestingness of the combination of the robot and the VR technology is further ensured.

In one embodiment, in a case that the mechanical arm 211 drives the shooting device 212 to move within a target time interval, so that the displacement size is greater than 0, the VR display terminal 220 is specifically configured to determine, according to the displacement direction of the target object, a first matching image that matches the displacement direction of the target object from a plurality of pre-stored images, and superimpose the acquired image and the first matching image to establish a VR scene;

for example, when the robot 211 dives from top to bottom within the target time interval, the camera 212 may send the captured image of the first scene captured from top to bottom to the VR display terminal 220 in real time, and the VR display terminal 220 first determines whether the displacement of the target object is greater than 0 according to the position of the target object in the second target image and the position of the target object in the first target image. If so, the VR display terminal 220 further determines the direction of the displacement, and if the direction of the displacement is from top to bottom, the VR display terminal 220 determines an image including a coaster element, which is matched with the direction of the displacement from top to bottom, from a plurality of pre-stored images as a first matching image, and superimposes the acquired image and the first matching image to establish a VR game scene in which the coaster game is performed in the first scene.

The image including the roller coaster element may include a roller coaster track and other objects reflecting the roller coaster element, such as a seat of the roller coaster, a head of the roller coaster, a front seat of the roller coaster, and the like.

For example, when the mechanical arm 211 rotates from left to right within the target time interval, the camera 212 may send the captured image of the first scene captured from left to right to the VR display terminal 220 in real time, and the VR display terminal 220 first determines whether the displacement of the target object is greater than 0 according to the position of the target object in the second target image and the position of the target object in the first target image. If so, the VR display terminal 220 further determines the direction of the displacement of the target object, and if the direction of the displacement of the target object is from left to right, the VR display terminal 220 determines an image including a racing car element, which is matched with the direction of the displacement from left to right, from a plurality of pre-stored images as a first matching image, and superimposes the acquired image and the first matching image to establish a VR game scene for the racing car game in the first scene.

The image including the elements of the race car may include, among other things, the track and other objects reflecting the elements of the race car, such as the cockpit of the race car, the rear view mirror of the race car, other race cars in front, and so on.

In an embodiment, in a case that the mechanical arm 211 drives the camera 212 to move within the target time interval, so that the displacement size is 0, the VR display terminal 220 is specifically configured to determine a second matching image matching the displacement size of the target object as 0 from the pre-stored multiple images, and superimpose the acquired image and the second matching image to establish a VR scene.

When the mechanical arm 211 is stationary within the target time interval, the shooting device 212 sends the shot collected image of the first scene to the VR display terminal 220 in real time, the VR display terminal 220 determines whether the displacement of the target object is 0 according to the position of the target object in the second target image and the position of the target object in the first target image, if so, the VR display terminal 220 determines an image including a cartoon image matched with the displacement of 0 from a plurality of pre-stored images as a second matching image, and superimposes the collected image and the second matching image to establish a VR game scene.

For example, when the mechanical arm 211 makes the photographing device 212 still photograph food (a first scene) in a refrigerator within a target time interval, the VR display terminal 220 may determine an image including a fire dragon from a plurality of pre-stored images as a second matching image, and superimpose the captured image and the second matching image to create a VR game scene in which the fire dragon explores in the refrigerator.

For example, when the mechanical arm 211 makes the camera 212 still shoot an obstacle (a first scene) on the table within the target time interval, the VR display terminal 220 may determine an image including a cartoon character from a plurality of pre-stored images as a second matching image, and superimpose the captured image and the second matching image to create a VR game scene in which the cartoon character is expeditioned on the table.

According to the VR scene establishing method provided by the embodiment of the application, the second matching image is determined from the pre-stored images under the condition that the displacement of the target object is 0, and the VR game scene is established according to the collected image and the second matching image, so that the established VR game scene is matched with the motion state of the image collecting terminal, the fidelity of the VR game scene is effectively improved, and the user experience is guaranteed.

In one embodiment, the image capture terminal 210 is further configured to:

receiving a control instruction sent by the VR display terminal 220;

the movement of the robot arm 211 is controlled based on the control command.

It should be noted that, after the VR display terminal 220 establishes the VR scene according to the captured image and the second matching image, the VR display terminal may further receive an input from a user, generate a control instruction in response to the input, and then send the control instruction to the image capturing terminal 210.

After receiving the control command sent by the VR display terminal 220, the image capturing terminal 210 may control the movement of the mechanical arm 211 based on the control command, so as to simulate a quest in a VR game scene.

Taking the VR display terminal 220 to establish a VR game scene of the cartoon character expedition on the desktop according to the target image and the second matching image as an example:

when the cartoon character is positioned over the cup, the VR display terminal 220 can receive an input from the user indicating "jump," in response to which the VR display terminal 220 generates control instructions and sends the control instructions to the image capture terminal 210. According to the control instruction, the image capturing terminal 210 first moves the mechanical arm 211 obliquely upward (at this time, the captured image captured by the capturing device 212 can simulate the view angle of the cartoon character jumping up), and then moves the mechanical arm 211 obliquely downward according to a certain parabola (at this time, the captured image captured by the capturing device 212 can simulate the view angle of the cartoon character jumping down from a high place). Through the operations of the VR display terminal 220 and the image acquisition terminal 210, a scene that the cartoon character jumps down from the teacup can be completely simulated.

According to the VR scene establishing method provided by the embodiment of the application, the image acquisition terminal 210 receives the control instruction sent by the VR display terminal 220, and the movement of the mechanical arm 211 is controlled based on the control instruction, so that interaction between a user and an established VR game scene can be realized, the participation degree of the user to the VR game scene is effectively improved, and the user experience is further improved.

The embodiment of the application also provides a VR scene establishing device, and the device and the VR scene establishing method can be referred to correspondingly.

Fig. 3 is a schematic structural diagram of a VR scene creation apparatus according to an embodiment of the present application. Referring to fig. 3, a VR scene creation apparatus provided in an embodiment of the present application may include:

a receiving module 310, configured to receive multiple captured images of a first scene acquired by an image capturing terminal; the plurality of captured images includes a first target image and a second target image;

a determining module 320, configured to determine a displacement of the object according to a position of the object in the first object image in the first scene and a position of the object in the second object image; wherein the first target image is a captured image acquired prior to the second target image; the object is a stationary object in the first scene;

and the establishing module 330 is configured to establish a VR scene according to the displacement, the acquired image, and a pre-stored image.

The VR scene establishing device provided by the embodiment of the application confirms the motion and the static state of the image acquisition terminal for shooting the first scene through a plurality of acquired images of the first scene acquired according to the image acquisition terminal, combines the acquired images and the pre-stored images according to different states to establish different VR game scenes, can improve the interestingness of the combination of the robot and the VR technology, effectively expands the practicability of the combination of the robot and the VR technology, and has very wide application prospect.

In one embodiment, the establishing module 330 is specifically configured to:

superimposing the first matching image over the acquired image.

In one embodiment, the establishing module 330 is further configured to:

In an embodiment, the establishing module 330 is specifically configured to:

determining that the displacement of the target object is 0, and determining a second matching image matched with the displacement of the target object with the displacement of 0 from a plurality of pre-stored images;

In one embodiment, the establishing module 330 is further configured to:

In an embodiment, the VR scene creation apparatus provided in the embodiment of the present application further includes a generation module, configured to:

receiving input of a user;

In one embodiment, the image acquisition terminal comprises a mechanical arm and a shooting device; the shooting device is arranged at one end of the mechanical arm;

the acquired image is acquired by any one of the following methods:

In one embodiment, the displacement of the target corresponds to the displacement of the mechanical arm which drives the shooting device to move in a target time interval;

Fig. 4 illustrates a physical structure diagram of an electronic device, which may include, as shown in fig. 4: a processor (processor)410, a communication Interface (Communications Interface)420, a memory (memory)430 and a communication bus 440, wherein the processor 410, the communication Interface 420 and the memory 430 are in communication with each other via the communication bus 440. The processor 410 may call logic instructions in the memory 430 to perform the following method:

receiving a plurality of acquisitions of a first scene acquired by an image acquisition terminal; the plurality of captured images includes a first target image and a second target image;

In addition, the logic instructions in the memory 430 may be implemented in the form of software functional units and stored in a computer readable storage medium when the software functional units are sold or used as independent products. Based on such understanding, the technical solution of the present application or portions thereof that contribute to the related art in essence may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk, and various media capable of storing program codes.

In another aspect, embodiments of the present application disclose a computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, which when executed by a computer, enable the computer to perform the methods provided by the above-mentioned method embodiments, for example, including:

In yet another aspect, the present application also provides a non-transitory computer-readable storage medium, on which a computer program is stored, where the computer program is implemented by a processor to execute the method provided by the foregoing method embodiments, for example, including:

determining the displacement of the target object according to the position of the target object in the first target image in the first scene and the position of the target object in the second target image; wherein the first target image is a captured image acquired prior to the second target image; the object is a stationary object in the first scene;

The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.

Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. Based on such understanding, the above technical solutions substantially or contributing to the related art may be embodied in the form of a software product, which may be stored in a computer-readable storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the embodiments or some parts of the embodiments.

Finally, it should be noted that the above embodiments are only for illustrating the present application, and do not limit the present application. Although the present application has been described in detail with reference to the embodiments, it should be understood by those skilled in the art that various combinations, modifications or equivalents may be made to the technical solutions of the present application without departing from the spirit and scope of the technical solutions of the present application, and the technical solutions of the present application should be covered by the claims of the present application.

Claims

1. A VR scene establishment method is characterized by comprising the following steps:

receiving a plurality of acquired images of a first scene acquired by an image acquisition terminal; the plurality of captured images includes a first target image and a second target image;

2. The VR scene creation method of claim 1, wherein creating the VR scene from the displacement of the target, the captured image, and a pre-stored image comprises:

3. The VR scene creation method of claim 2, wherein the overlaying the captured image and the first matching image comprises:

superimposing the first matching image over the acquired image.

4. The VR scene creation method of claim 2, wherein the first matching image is an animated image, the method further comprising:

5. The VR scene creation method of claim 1, wherein creating the VR scene from the displacement of the target, the captured image, and a pre-stored image comprises:

6. The VR scene creation method of claim 5, wherein the second matching image includes a target object, the method further comprising:

7. The VR scene creation method of claim 5, wherein after the superimposing the captured image and the second matching image to create the VR scene, further comprising:

receiving input of a user;

8. The VR scene setup method of any of claims 1 to 7, wherein the image capture terminal includes a robotic arm and a camera; the shooting device is arranged at one end of the mechanical arm;

the acquired image is acquired by any one of the following methods:

9. The VR scene creation method of claim 8, wherein the displacement of the target corresponds to a displacement of the robotic arm that results from moving the camera within a target time interval;

10. A VR scene creation system comprising:

11. The VR scene creation system of claim 10, wherein the image capture terminal includes a robotic arm and a camera; the shooting device is arranged at one end of the mechanical arm;

the image acquisition terminal is specifically configured to:

12. The VR scene creation system of claim 11, wherein the displacement of the target corresponds to a displacement of the robotic arm that results from moving the camera over a target time interval;

13. The VR scene creation system of claim 12,

under the condition that the mechanical arm drives the shooting device to move within a target time interval to generate displacement larger than 0, the VR display terminal is specifically used for determining a first matching image matched with the displacement direction of the target object from a plurality of pre-stored images according to the displacement direction of the target object, and superposing the acquired image and the first matching image to establish the VR scene;

14. The VR scene creation system of any of claims 11 to 13, wherein the image capture terminal is further configured to:

receiving a control instruction sent by the VR display terminal;

controlling the movement of the robotic arm based on the control command.

15. A VR scene creation apparatus, comprising:

the receiving module is used for receiving a plurality of acquired images of a first scene acquired by the image acquisition terminal; the plurality of captured images includes a first target image and a second target image;

a determining module, configured to determine a displacement of the object according to a position of the object in the first object image in the first scene and a position of the object in the second object image; wherein the first target image is a captured image acquired prior to the second target image; the object is a stationary object in the first scene;

and the establishing module is used for establishing a VR scene according to the displacement of the target object, the acquired image and a pre-stored image.

16. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the VR scene creation method of any of claims 1 to 9 when executing the program.

17. A non-transitory computer-readable storage medium having stored thereon a computer program, which when executed by a processor implements the VR scene creation method of any of claims 1 to 9.

18. A computer program product comprising a computer program, wherein the computer program, when executed by a processor, implements the VR scene creation method of any of claims 1 to 9.