WO2018227757A1

WO2018227757A1 - Prompting method and vr device

Info

Publication number: WO2018227757A1
Application number: PCT/CN2017/098593
Authority: WO
Inventors: 康凤霞; 陈浩; 周胜丰
Original assignee: 华为技术有限公司
Priority date: 2017-06-16
Filing date: 2017-08-23
Publication date: 2018-12-20

Abstract

Disclosed in the present application are a prompting method and a VR device. The method in the present application comprises: a VR device collects at least two images by means of a camera, the at least two images having a same photographed object; and the VR device gives a prompt according to a change of a position relationship between the photographed object and a preset position, the change of the position relationship between the photographed object and the preset position being determined by the VR device according to the at least two images. When a user uses the VR device, the VR device can take at least two images; accordingly, the existence of an abnormal situation can be indicated according to the at least two images; and if the abnormal situation exists, a prompt can be given to make the user boost alertness, so as to improve the safety.

Description

A prompting method and VR device

This application claims the priority of the Chinese Patent Application, filed on Jun. 16, 2017, the application Serial No.

Technical field

The present application relates to the field of terminals, and in particular, to a prompting method and a VR device.

Background technique

When a user uses a virtual reality (VR) device in public, since the VR device is worn by both eyes, there is no danger of incidents such as theft. Therefore, a measure is needed for the surrounding. Early warning of dangerous events.

Summary of the invention

The embodiment of the invention provides a prompting method and a VR device, which are used for acquiring an image and issuing an early warning prompt according to the image.

A first aspect of the embodiments of the present invention provides a prompting method, where the method is applied to a VR device, the method includes: the VR device collects at least two images by using a camera, and any two images of the at least two images are from different At the time point, the at least two images have the same subject, and in some possible embodiments, the subject may be a hand or a dice. Then, the VR device may determine a change in the positional relationship between the subject and the preset position according to the at least two images, and issue a prompt according to the positional relationship change between the subject and the preset position. In some possible embodiments, the prompt includes an alarm sound, a closed VR video, a text prompt on the screen of the VR device, or an image prompt, which is not limited herein.

When the user uses the VR device, the VR device can capture at least two images, and the abnormality can be indicated according to the at least two images. If there is an abnormal situation, a prompt can be issued to make the user alert to improve security.

With reference to the first aspect of the embodiments of the present invention, a first implementation manner of the first aspect of the embodiment of the present invention includes: determining, by the VR device, the location of the object and the preset location by using the at least two images a relationship change, the positional relationship change is used to indicate a distance between the subject and the preset position, and a motion trajectory of the subject, if the distance between the subject and the preset position is shortened and the moving direction of the subject For the preset position, it can be considered that an abnormal situation has occurred, and the VR device can issue a prompt.

The positional relationship change between the two can be determined by the distance between the subject and the preset position and the moving direction of the subject, and the distance between the two is shortened according to the positional relationship change, and the moving direction of the subject Whether or not the preset position is determined is a method for judging whether or not an abnormal condition exists.

With reference to the first aspect of the embodiments of the present invention, a second implementation manner of the first aspect of the embodiments of the present invention includes: including, in the at least two images, a first image and a second image, the first image and the The photographing time of the second image is different. Assuming that the first image is before the second image, the VR device respectively determines a first position of the subject according to the first image, and determines the photographing according to the second image. The second position of the object. Since the photographing time of the first image and the second image are different, after determining the first position and the second position of the subject, the first position and the first The second position is the motion trajectory of the subject. When the motion trajectory of the subject is determined, the VR device may determine a change in the positional relationship with the preset position according to the motion trajectory of the subject, and in some feasible embodiments, the preset position is relative to the The VR device is in a fixed position, such as a pocket, or a backpack. In some possible embodiments, the preset position is within the shooting range of the camera of the VR device, such as a backpack or other VR device that can be photographed at the chest.

Since the first position and the second position are determined by the first image and the second image, and the first position and the second position are the motion trajectory of the subject, the manner of determining the motion trajectory of the subject is clarified .

With reference to the first aspect of the embodiments of the present invention, a third implementation manner of the first aspect of the embodiment of the present invention includes: the VR device can acquire a motion track of the VR device, because the preset location is relative to the VR device For a fixed position, the motion track of the preset position can be determined by the motion track, and the VR device can determine the subject and the preset position according to the motion track of the subject and the motion track of the preset position. The positional relationship changes.

Since the preset position is a fixed position relative to the VR device, the motion track of the preset position may be determined according to the motion track of the VR device, and then the positional relationship and the positional relationship between the subject and the preset position are determined. Variety.

With reference to the first aspect of the embodiments of the present invention, a fourth implementation manner of the first aspect of the embodiments of the present invention includes: the VR device can collect motion information in real time through at least one sensor, obtain motion data, and generate motion data according to the motion data. Determine the motion trajectory of the VR device.

Since the motion information can be acquired by the at least one sensor, and thus the motion data is obtained, and the motion trajectory of the VR device is determined based on the motion data, the determination manner of the motion trajectory of the VR device is determined.

With reference to the first aspect of the embodiments of the present invention, a fifth implementation manner of the first aspect of the embodiments of the present invention includes: determining, by the VR device, a first maximum grayscale block of the first image and the second image, respectively a second maximum gray scale block, and then determining a first centroid of the first maximum gray scale block and a second centroid of the second maximum gray scale block, and then determining a first position of the first centroid and a second centroid of the second centroid Two locations.

Since the centroid is determined by the maximum gray scale block, the position is determined by the centroid, so the position of the VR device can be accurately determined.

A second aspect of the embodiments of the present invention provides a prompting method, where the method is applied to a VR device, including: the VR device collects at least two images by using a camera, and the two images are taken at different time points, such as a video. Two adjacent frames in the middle, the at least two images have the same subject, and the subject may be a hand or a dice, or other items showing danger. Then the VR device can respectively determine a first maximum gray block of the first image and a second maximum gray block of the second image, and determine a first centroid and the second maximum gray of the first maximum gray block a second centroid of the block, and determining a first position of the first centroid and a second position of the second centroid, wherein the first position and the second position are positions of the subject at different points in time, Then the first position and the second position are motion trajectories of the subject. When the motion trajectory of the subject is determined, if the VR device is moving, it is also necessary to determine its own motion trajectory, and the VR device can collect its own motion information in real time through at least one sensor, obtain motion data, and pass the The motion data determines the motion trajectory of the VR device.

In some feasible embodiments, the preset position is a fixed position relative to the VR device, and the motion trajectory of the preset position may be determined according to the motion trajectory of the VR device. In some feasible embodiments, in order to ensure that the situation can be accurately determined, the preset position may be within the shooting range of the camera of the VR device. After determining the motion trajectory of the subject and the motion trajectory of the preset position, the VR device can determine the positional relationship change of the subject with the preset position. If the positional relationship change indicates that the distance between the subject and the preset position is shortened, and the movement of the subject If the direction is the preset position, the VR device can issue a prompt, including an alarm sound, turning off the VR video, a text prompt on the screen of the VR device, or an image prompt.

A third aspect of the embodiments of the present invention provides a VR device, including an acquisition module and a prompt module. The acquiring module collects at least two images having the same subject through the camera, and the prompting module issues a prompt according to the positional relationship change between the subject and the preset position, and the positional relationship between the subject and the preset position changes. The VR device is determined based on the at least two images.

With reference to the third aspect of the embodiments of the present invention, the first implementation manner of the third aspect of the embodiment of the present invention includes: the prompting module includes a determining submodule and a prompting submodule, wherein the determining determining submodule is based on the at least two images Determining a change in the positional relationship between the subject and the preset position, and if the distance between the subject and the preset position is shortened according to the change in the positional relationship, and the moving direction of the subject is the preset position, The prompt submodule issues a prompt.

Since the positional relationship change between the subject and the preset position and the moving direction of the subject can be determined, whether or not the VR device issues a specific basis is provided.

With reference to the third aspect of the embodiments of the present invention, the second implementation manner of the first aspect of the embodiment of the present invention includes a first determining unit, a second determining unit, and a third determining unit, wherein the first determining unit respectively according to the at least The first image and the second image of the two images determine a first position and a second position of the subject, and then the second determining unit determines a motion trajectory of the subject according to the first position and the second position, Then the third determining unit is based on the motion track of the subject

With reference to the third aspect of the embodiments of the present invention, in a third implementation manner of the first aspect of the embodiment, the acquiring unit acquires a motion track of the VR device, and then the fourth determining unit determines, according to the motion track of the VR device The motion trajectory of the preset position, and then the third determining unit determines a change in the positional relationship between the subject and the preset position according to the motion trajectory of the subject and the motion trajectory of the preset position.

With reference to the third aspect of the embodiments of the present invention, in a fourth implementation manner of the first aspect of the embodiment, the collecting subunit collects motion information in real time by using at least one sensor to obtain motion data, and then determines that the subunit is configured according to the The motion data determines the motion trajectory of the VR device.

With reference to the third aspect of the embodiments of the present invention, in a fifth implementation manner of the first aspect of the embodiments, the first determining subunit respectively determines the first maximum grayscale block and the second image of the first image a second maximum gray scale block, and then the second determining subunit determines a first centroid of the first maximum gray scale block and a second centroid of the second maximum gray scale block, and then a third The determining subunit determines a first position of the first centroid and a second position of the second centroid.

A fourth aspect of the embodiments of the present invention provides a VR device. In the fourth aspect, the acquiring module collects at least two images by using a camera, where the at least two images have the same subject, in some feasible embodiments. In the middle, the subject includes a hand or a dice. The at least two images include a first image and a second image, and then the first determining unit determines the first position and the second position of the subject according to the first image and the second image, respectively, and then the first determining subunit Determining a first maximum grayscale block of the first image and a second maximum grayscale block of the second image, respectively, the second determining subunit may determine a first centroid and the second maximum of the first largest grayscale block The second centroid of the gray scale block, such that the third determining subunit can determine the first position of the first centroid and the second position of the second centroid. Then, the collecting sub-unit can collect the motion information in real time through the at least one sensor, and obtain the motion data, so that the determining subunit determines the motion trajectory of the VR device according to the motion data. In some feasible embodiments, the preset position is relative to the The VR device is a fixed position, and the fourth determining unit may determine a motion trajectory of the preset position according to the motion trajectory of the VR device, and the third determining unit is configured according to the motion track of the object and the preset position. The motion trajectory determines a change in the positional relationship of the subject with the preset position. In some possible embodiments, the preset position is within the shooting range of the camera of the VR device. Finally, if the distance between the subject and the preset position is shortened according to the change of the positional relationship, and the moving direction of the subject is the preset position, the prompting sub-module may issue a prompt, in some feasible embodiments. The prompt includes an alarm sound, a VR video off, a text prompt or an image prompt on the screen of the VR device.

A fifth aspect of the embodiments of the present invention provides a VR device, including:

a processor, a memory, one or more applications, wherein the one or more applications are stored in the memory, the one or more applications including instructions that, when executed by the VR device, cause the The VR device performs the method of the above aspects.

A sixth aspect of the embodiments of the present invention provides a computer program product, characterized in that, when the computer program product is run on a VR device, the VR device is caused to perform the method of the above aspects.

A seventh aspect of an embodiment of the present invention provides a computer readable storage medium having stored therein instructions that, when executed on a computer, cause the computer to perform the method of the above aspects.

DRAWINGS

FIG. 1 is a schematic flowchart diagram of a prompting method in an embodiment of the present application;

FIG. 2 is another schematic flowchart of a prompting method in an embodiment of the present application;

3-1 is a schematic diagram of a VR device according to an embodiment of the present application;

FIG. 3-2 is another schematic diagram of a VR device according to an embodiment of the present application;

FIG. 3-3 is another schematic diagram of a VR device according to an embodiment of the present application;

3-4 is another schematic diagram of a VR device according to an embodiment of the present application;

3-5, another schematic diagram of a VR device according to an embodiment of the present application;

FIG. 3-6 is another schematic diagram of a VR device according to an embodiment of the present application;

4-1 is a schematic diagram of a collection of terminals and VR components in the embodiment of the present application;

4-2 is a schematic diagram of a screen effect of a terminal in an embodiment of the present application;

4-3 is a schematic diagram of a main device of a terminal in an embodiment of the present application;

5-1 is a flowchart of operation of a VR device in the present application in the embodiment of the present application;

5-2 is a schematic diagram of a motion trajectory of a VR device from point A to point B in the embodiment of the present application;

5-3 is a schematic diagram of a fixed position coordinate system in the embodiment of the present application;

FIG. 5-4 is a schematic diagram of a VR coordinate system according to an embodiment of the present application; FIG.

5-5 are schematic diagrams of coordinates of point C in the embodiment of the present application;

5-6 are schematic diagrams of photographing effects in the embodiment of the present application;

5-7 are schematic diagrams of images collected by a DVS camera in the embodiment of the present application;

5-8 are schematic diagrams of processing effects of a maximum connected domain in the embodiment of the present application;

5-9 are schematic diagrams showing a centroid of a gray scale block in an embodiment of the present application;

5-10, the position of point D in two DVS cameras in the embodiment of the present application;

5-11 are schematic diagrams of coordinates of a point D in a VR device coordinate system according to an embodiment of the present application;

5-12 are schematic diagrams showing coordinates of point D in a fixed position coordinate system in the embodiment of the present application;

FIG. 6 is a schematic diagram of the early warning system E in the embodiment of the present application;

7-1 is a schematic flowchart of an embodiment of an embodiment of the present application;

7-2 is a schematic diagram of a user using a VR device in the embodiment of the present application;

7-3 is a schematic diagram of a user walking in the embodiment of the present application;

7-4, in the embodiment of the present application, the VR device can determine a fixed point A and a fixed point coordinate system with the point A as the origin in the embodiment of the present application;

7-5 are schematic diagrams of characters appearing in front of a user in the embodiment of the present application;

7-6 is a schematic diagram of a moving hand intercepted by a DVS camera in the embodiment of the present application;

7-7 are schematic diagrams of grayscale blocks and centroids of an image of a moving hand in an embodiment of the present application;

7-8, the positions of the centroids of the moving hands in the two DVS cameras in the embodiment of the present application;

7-9 are coordinates of a VR device coordinate system of point F in the embodiment of the present application;

7-10 are coordinates of a fixed point device coordinate system of point F in the embodiment of the present application;

7-11 are schematic diagrams of valuable items of a user in an embodiment of the present application;

7-12, the positional relationship between the G point and the F point of the fixed position coordinate system in the embodiment of the present application;

FIG. 7-13 is a schematic diagram showing a moving direction of point F and a positional relationship with G in the embodiment of the present application;

7-14 are schematic diagrams showing warnings on a screen of a VR device in the embodiment of the present application;

7-15 are schematic diagrams showing an early warning of displaying text on a screen of a VR device according to an embodiment of the present application;

7-16 are schematic diagrams of an early warning of a VR device emitting a strong light or an alarm sound according to an embodiment of the present application.

detailed description

The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is obvious that the described embodiments are only a part of the embodiments of the present invention, but not all embodiments.

The terms "first", "second", "third", "fourth", etc. (if present) in the specification and claims of the present invention and the above figures are used to distinguish similar objects without having to use To describe a specific order or order. It is to be understood that the data so used may be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than what is illustrated or described herein. In addition, the terms "comprises" and "comprises" and "the" and "the" are intended to cover a non-exclusive inclusion, for example, a process, method, system, product, or device that comprises a series of steps or units is not necessarily limited to Those steps or units may include other steps or units not explicitly listed or inherent to such processes, methods, products or devices.

In the embodiment of the present application, when the VR device is used by the user, the VR device can detect at least two images according to the at least two images, and if there is an abnormal situation, the VR device can issue a prompt to make the user Be alert to improve safety.

Based on the above, the embodiment of the present application provides a prompting method. Referring to FIG. 1 , the prompting method in the embodiment of the present application includes

steps

101 and 102 .

Wherein, the VR device collects at least two images through the camera, and the at least two images have the same subject. 102. The VR device sends a prompt according to the change of the positional relationship between the subject and the preset position, and the positional relationship between the subject and the preset position is changed by the VR device according to the at least two images.

For the sake of understanding, the specific process in the embodiment of the present application is described below. Referring to FIG. 2, it is a schematic flowchart of a prompting method. The method includes steps 201-210.

201. The VR device collects at least two images through a camera.

In the embodiment of the present application, the VR device may have one or more cameras built in or externally used for taking an image. In the embodiment of the present application, the VR device can capture at least two images at different time points by using any built-in camera, such as at least two frames in a video segment obtained by the screen shooting, or at least in the pre-made time. Two images are not limited here. The at least two images are used by the VR device to determine whether there is an abnormal situation, such as a person or an object approaching, and the target is a valuable item on the body, and if there is an abnormal situation, a prompt may be issued. In some feasible embodiments, the at least two images may be photographed to the same subject, and the subject may be a human hand or a dice, or other object that may cause a threat, if it is determined that the photograph is abnormal. In order to pose a threat to the user, a prompt can be issued to alert the user to improve the security of the user.

Specifically, the camera may be a DVS camera, and the DVS camera is used to capture a moving image. If no object in the moving image has motion, no subject may be captured, and if there is a moving object, the camera The DVS camera can determine the relevant subject. In some possible embodiments, the VR device may set at least two DVS cameras at different positions for determining the position of the human eye through at least two DVS cameras when capturing a moving subject. The principle determines the position of the subject relative to itself.

In some possible embodiments, if a black and white image is captured by the DVS camera, the position of the subject may be determined by the following method. Specifically, refer to step 202-step 204.

202. The VR device determines a first maximum grayscale block of the first image and a second largest grayscale block of the second image, respectively.

In the embodiment of the present application, since different parts of the image captured by the DVS camera have the same gray level, To determine the range of the subject, the image of the subject in the image can be determined by determining the maximum grayscale block of the image. When at least two images are captured, if the at least two images include the first image and the second image, the largest grayscale block in the first image and the second image may be separately determined. It should be noted that the at least two images may be only the first image and the second image, and may further include other multiple images. For the sake of brief description, the following steps 203 to 204 in the embodiment of the present application, the first image and the second image The image is taken as an example for explanation.

203. The VR device determines a first centroid of the first maximum grayscale block and a second centroid of the second largest grayscale block.

In the embodiment of the present application, after determining the maximum gray scale block of an image, the centroid of the maximum gray scale block may be determined, and the centroid is used as the centroid of the subject. Then the VR device can determine the first centroid and the second centroid according to the first maximum gray scale block and the second maximum gray scale block.

204. The VR device determines a first location of the first centroid and a second location of the second centroid.

In some feasible embodiments, if a frame is captured by the VR device, if two images are captured by two DVS cameras, each of which determines two centroids, the The two centroids determine the position of the subject relative to itself in this frame. Then the VR device can determine the first position of the first image and the second position of the second image of the subject in the at least two images according to the first centroid and the second centroid.

205. The VR device determines a motion trajectory of the subject according to the first location and the second location.

In the embodiment of the present application, if the first position and the second position of the object are determined, two different positions of the object at two different time points may be determined, and the subject may be determined. Motion trajectories at two different points in time. When more than two images are taken, such as a video, the motion trajectory of the subject in the video can be determined.

In some feasible embodiments, the VR device may be stationary or mobile, and if the subject is determined to gradually approach the VR device with respect to the motion track of the VR device, The VR device is based on whether or not to issue a prompt. When the VR device is in motion and the subject is stationary, the motion trajectory of the subject relative to the VR device may gradually approach the VR device. In this case, it should not be judged. It is an abnormal situation. Therefore, in the embodiment of the present application, the motion trajectory of the VR device can be added as a consideration factor to determine whether a prompt should be issued. Specifically, please refer to step 206-step 208.

206. The VR device collects motion information of the VR device in real time through at least one sensor to obtain motion data.

In the embodiment of the present application, the VR device can collect its own motion information in real time through at least one built-in sensor to obtain motion data. Specifically, the sensor may be a motion sensor or an inertial sensor, and the motion information of the VR device is obtained by the sensor to obtain motion data, such as speed and direction, which are not limited herein.

207. The VR device determines a motion trajectory of the VR device by using the motion data.

In some possible embodiments, when determining the speed and direction of the VR device at different points in time, the motion trajectory of the VR device can be determined according to the speed and direction. Specifically, the VR device may use the starting point as a fixed point, and obtain the speed and direction of each time point after starting from the fixed point by using the fixed point, thereby determining the displacement and direction of each time period, thereby obtaining The position of the VR device relative to the fixed point is determined relative to the position of the fixed point at each point in time.

208. Determine a motion track of the preset position according to a motion track of the VR device.

In some possible embodiments, the preset position may be a position of an item that the user needs to protect, such as a critical part of the body or an important part, such as a placement position of a valuable object, which is not limited herein. In some possible embodiments, the preset position A fixed position that can be fixed relative to the VR device, that is, when determining the position of the VR device relative to a certain fixed point, that is, determining the position of the preset position relative to the fixed point, when determining that the VR device is relative to the certain The motion trajectory of a fixed point, that is, the trajectory of the preset position relative to the fixed point is determined. In some feasible embodiments, the preset position may be within the shooting range of the camera of the VR device. It should be noted that when the camera of the VR device is adjustable in angle, the camera may be set. Automatically aligning the preset position, or the user adjusts the camera to the preset position so that the preset position is placed within the range monitored by the camera; if the camera of the VR device is not adjustable, Place valuables and the like on the position that the camera of the VR device can capture, and use this position as a preset position to enhance the monitoring of the preset position.

209. The VR device determines, according to the motion track of the object and the motion track of the preset position, a change in the positional relationship between the object and the preset position.

In the embodiment of the present application, when the motion trajectory of the subject relative to the VR device and the motion trajectory of the VR device are determined, the motion trajectory of the subject relative to the fixed position may be determined thereby, thereby determining the Whether the subject is moving to avoid a situation in which the erroneous judgment is made that the subject is abnormal due to the VR device gradually approaching the subject due to the preset position.

In the embodiment of the present application, the VR device may determine the position of the subject and the preset position according to the motion track of the subject relative to the fixed position and the motion track of the preset position relative to the fixed position. Relationship changes. In some feasible embodiments, according to the motion trajectories of the two, it can be seen that the other side of the motion is stationary, or both are stationary, or both have motion, so the positional relationship between the two can be gradually separated. One party gradually approaches the other, both are in motion but the distance is constant, etc., and is not limited here. In some feasible embodiments, if the position change relationship is displayed, the subject is moving, and the moving direction is the preset position, the subject is considered to be gradually approaching the preset position. If it is determined that the subject continues to be close to the preset position, and the distance is already less than the preset distance, then an abnormal situation may be considered, and a prompt may be issued.

210. The VR device can issue a prompt.

In some embodiments, the VR device may issue a prompt, which may include displaying text on the user's VR video, such as "There is an abnormal situation, please pay attention to personal safety", or pop up an animation or other image information to indicate There is an abnormal situation, or an alarm sounds to indicate that there is an abnormal situation. In some feasible embodiments, a strong light can also be emitted. If the other party is a theft or other abnormal situation, the other party can indicate that he or she has detected that the VR video can be turned off. , open the rear camera to see the situation in front of you, not limited here.

The above description is made in the perspective of a method for prompting the following. The following describes the perspective of the function device for performing the prompting method. Referring to FIG. 3-1, an embodiment of the present application further provides a VR device 300, including:

The acquisition module 301 and the prompt module 302. The acquiring module 301 can collect images having the same subject by using a camera. If at least two images are acquired at different time points, the prompting module 302 can determine a positional relationship between the subject and the preset position, and A prompt is issued according to the change in the positional relationship.

When the VR device 300 is used by the user, the VR device 300 can capture at least two images, and the abnormality can be indicated according to the at least two images. If an abnormal situation exists, a prompt can be issued to improve the alert for the user to improve security. Sex.

In some possible embodiments, referring to FIG. 3-2, the prompting module 302 may include a determining sub-module 3022 and a prompting sub-module 3021, wherein the determining sub-module 3022 determines the subject according to the at least two images and the Preset position In the positional relationship of each image, the change of the positional relationship may be determined. If the abnormality is indicated according to the change of the positional relationship, if the distance between the subject and the preset position is shortened, and the moving direction of the subject is the pre- With the location set, the prompt sub-module 3022 can issue a prompt.

Since the positional relationship change between the subject and the preset position and the moving direction of the subject can be determined, whether the prompting sub-module 3022 issues the proposed specific basis is provided.

In some possible embodiments, referring to FIG. 3-3, the determining submodule 3021 may include a first determining unit 30211, a second determining unit 30212, and a third determining unit 30213. a first image and a second image of the at least two images, the first determining unit 30211 may determine a first position of the subject according to the first image, and determine a second of the subject according to the second image position. If there is a third image, the VR device may determine the third position of the subject according to the third determination, so at least two positions of the subject may be determined according to the at least two images, and then the second determining unit 30212 A motion trajectory of the subject is determined according to the at least two positions, that is, at least two positions at different time points. Then, the third determining unit 30213 determines a change in the positional relationship with the preset position according to the motion trajectory of the subject, and the positional relationship change may be that the two gradually move away, one of the parties gradually approaches the other, and both are moving but the distance is not Change, etc., not limited here.

In some possible embodiments, referring to FIG. 3-4, the determining submodule 3021 further includes an obtaining unit 30214 and a fourth determining unit 30215. The obtaining unit 30214 can obtain the motion track of the VR device 300. Since the preset position is a fixed position with respect to the VR device 300, the fourth determining unit 30215 can determine the motion track according to the motion track of the VR device 300. The motion track of the preset position, and then the third determining unit 30213 determines a change in the positional relationship between the subject and the preset position according to the motion track of the subject and the motion track of the preset position.

Since the preset position is a fixed position relative to the VR device 300, the motion trajectory of the preset position may be determined according to the motion trajectory of the VR device 300, and then the positional relationship between the subject and the preset position may be determined. The positional relationship changes.

In some possible embodiments, specifically, referring to FIG. 3-5, the obtaining unit 30214 further includes a collecting subunit 302141 and a determining subunit 302142, wherein the collecting subunit 302141 can collect motion information in real time through at least one sensor. To obtain motion data, the motion data includes speed and motion direction of the VR device at different time points, so that the determining subunit 302142 determines a motion trajectory of the VR device 300 according to the motion data.

Since the motion information can be acquired by the at least one sensor, and thus the motion data is obtained, and the motion trajectory of the VR device 300 is determined based on the motion data, the determination manner of the motion trajectory of the VR device 300 is determined.

In some possible embodiments, referring to FIG. 3-6, the second determining unit 3012 may further include a first determining subunit 302121 and a second determining subunit 302122 and a third determining subunit 302123. In the embodiment of the present application, the first determining subunit 302121 is configured to respectively determine a first maximum grayscale block of the first image and a second largest grayscale block of the second image, so that the second determining subunit 302122 determines a first centroid of the first maximum gray scale block and a second centroid of the second maximum gray scale block, and then a third determining subunit 302123 determines a first position of the first centroid and a second position of the second centroid, Then the second determining unit may determine at least two locations of the subject.

Since the centroid is determined by the maximum gray scale block, the position is determined by the centroid, so the position of the VR device 300 can be accurately determined.

The embodiment of the present invention further provides a VR device 300. The acquisition module 301 can collect images having the same subject through a camera. If at least two images are acquired at different time points, the first of the at least two images An image and a second image, the first determining unit 30211 may determine a first position of the subject according to the first image, and determine a second position of the subject according to the second image. If there is a third image, the VR device may determine the third position of the subject according to the third determination, so at least two positions of the subject may be determined according to the at least two images.

The first determining subunit 302121 is then configured to respectively determine a first maximum grayscale block of the first image and a second largest grayscale block of the second image, such that the second determining subunit 302122 determines the first maximum grayscale a first centroid of the block and a second centroid of the second largest grayscale block, and then the third determining subunit 302123 determines a first location of the first centroid and a second location of the second centroid, then the second determining unit At least two positions of the subject can be determined.

The collection subunit 302141 may collect motion information in real time by using at least one sensor to obtain motion data, where the motion data includes speed and motion direction of the VR device at different time points, so that the determining subunit 302142 is based on the motion data. The motion trajectory of the VR device 300 is determined. Therefore, the fourth determining unit 30215 can determine the motion trajectory of the preset position according to the motion trajectory of the VR device 300, and then the third determining unit 30213 determines the motion trajectory of the subject and the motion trajectory of the preset position. The positional relationship between the subject and the preset position changes.

Then, the third determining unit 30213 determines a change in the positional relationship with the preset position according to the motion trajectory of the subject, and the positional relationship change may be that the two gradually move away, one of the parties gradually approaches the other, and both are moving but the distance is not It is not limited here. In some feasible embodiments, if the position change relationship is displayed, the subject is moving, and the moving direction is the preset position, the subject is considered to be gradually If the object continues to be close to the preset position and the distance is already less than the preset distance, the abnormality may be considered, and the prompting sub-module 3022 may issue a prompt.

The following describes the function device of the VR device as an example. The following describes the VR device as a terminal and the VR device as an example. The VR device can be a terminal. It can also be a collection of terminals and VR accessories. As long as the related functions can be performed, it is not limited here. Please refer to Figure 4-1, which is a collection of terminals and VR accessories.

In the embodiment of the present application, the VR accessory may be a VR glasses. When the terminal and the VR glasses are used together, the user can view the 3-dimensional effect through the lens of the VR glasses. In some possible embodiments, the VR accessory can also be connected to the terminal to play related audio to the user through the earphone, which is not limited herein. In the embodiment of the present application, when the VR device is running, the terminal screen can achieve the effect shown in FIG. 4-2 (FIG. 4-2, which is a schematic diagram of the screen effect of the terminal), and the image on the left side is viewed by the left eye. The image on the right is viewed by the right eye, and the images on the two sides are slightly different on the screen to simulate the viewing mode of the two eyes. When the user uses the screen of the terminal with the VR glasses, a three-dimensional viewing effect can be obtained.

Specifically, please refer to FIG. 4-3, which is a schematic diagram of a main device of the terminal A, which may include a mobile phone, a tablet computer, a personal digital assistant (PDA), and a point of sales (Point of Sales, POS), on-board computer and other terminal devices, taking the terminal as a mobile phone as an example:

4-3 is a block diagram showing a part of the structure of a mobile phone related to the terminal A provided by the embodiment of the present invention. Referring to FIG. 4-3, the mobile phone includes: radio frequency (RF) circuit C-10, memory C-20, input unit C-30, display unit C-40, sensor C-50, audio circuit C-60, wireless Components such as the wireless fidelity (Wi-Fi) module C-70, processor C-80, and power supply C-90. Those skilled in the art will appreciate that the handset structure illustrated in Figures 4-3 does not constitute a limitation to a handset, and may include more or fewer components than those illustrated, or some components may be combined, or different components may be arranged.

The following describes the components of the mobile phone in conjunction with Figure 4-3:

The RF circuit C-10 can be used for receiving and transmitting signals during the transmission or reception of information or during a call. Specifically, after receiving the downlink information of the base station, it is processed by the processor C-80. In addition, the uplink data is designed to be sent to the base station. Generally, RF circuit C-10 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a Low Noise Amplifier (LNA), a duplexer, and the like. In addition, the RF circuit C-10 can also communicate with the network and other devices through wireless communication. The above wireless communication may use any communication standard or protocol, including but not limited to Global System of Mobile communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (Code Division). Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), E-mail, Short Messaging Service (SMS), and the like.

The memory C-20 can be used to store software programs and modules, and the processor C-80 executes various functional applications and data processing of the mobile phone by running software programs and modules stored in the memory C-20. The memory C-20 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application required for at least one function (such as a sound playing function, an image playing function, etc.), and the storage data area may be Stores data created based on the use of the phone (such as audio data, phone book, etc.). Further, the memory C-20 may include a high speed random access memory, and may also include a nonvolatile memory such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.

Input unit C-30 can be used to receive input numeric or character information and to generate key signal inputs related to user settings and function controls of the handset. Specifically, the input unit C-30 may include a touch panel C-31 and other VR devices C-32. The touch panel C-31, also called a touch screen, can collect touch operations on or near the user (such as a user using a finger, a stylus, or the like, on any suitable object or accessory on the touch panel C-31 or in touch The operation near the panel C-31), and the corresponding connecting device is driven according to a preset program. Optionally, the touch panel C-31 may include two parts: a touch detection device and a touch controller. Wherein, the touch detection device detects the touch orientation of the user, and detects a signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts the touch information into contact coordinates, and sends the touch information. The processor C-80 is provided and can receive commands from the processor C-80 and execute them. In addition, the touch panel C-31 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch panel C-31, the input unit C-30 may also include other VR devices C-32. Specifically, other VR devices C-32 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control buttons, switch buttons, etc.), trackballs, mice, joysticks, and the like.

The display unit C-40 can be used to display information input by the user or information provided to the user as well as various menus of the mobile phone. The display unit C-40 may include a display panel C-41. Alternatively, the display panel C-41 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like. . Further, the touch panel C-31 can cover the display panel C-41, and when the touch panel C-31 detects a touch operation on or near it, it is transmitted to the processor C-80 to determine the type of the touch event. Processor C-80 then provides a corresponding visual output on display panel C-41 depending on the type of touch event. Although in FIG. 4-3, the touch panel C-31 and the display panel C-41 are used as two independent components to implement the input and input functions of the mobile phone, in some embodiments, the touch panel C may be The -31 integrates with the display panel C-41 to realize the input and output functions of the mobile phone.

The handset may also include at least one sensor C-50, such as a light sensor, motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel C-41 according to the brightness of the ambient light, and the proximity sensor may close the display panel C when the mobile phone moves to the ear. -41 and / or backlight. As a kind of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (usually three axes). When it is stationary, it can detect the magnitude and direction of gravity. It can be used to identify the gesture of the mobile phone (such as horizontal and vertical screen switching, related Game, magnetometer attitude calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for the mobile phone can also be configured with gyroscopes, barometers, hygrometers, thermometers, infrared sensors and other sensors, no longer Narration.

Audio circuit C-60, speaker C-61, and microphone C-62 provide an audio interface between the user and the handset. The audio circuit C-60 can transmit the converted electrical signal of the received audio data to the speaker C-61, and convert it into a sound signal output by the speaker C-61; on the other hand, the microphone C-62 converts the collected sound signal. The electrical signal is received by the audio circuit C-60 and converted into audio data, and then processed by the audio data output processor C-80, sent to the other mobile phone via the RF circuit C-10, or output the audio data to Memory C-20 for further processing.

Wi-Fi is a short-range wireless transmission technology. The mobile phone can help users to send and receive e-mail, browse web pages and access streaming media through the Wi-Fi module C-70. It provides users with wireless broadband Internet access. Although FIG. 4-3 shows the Wi-Fi module C-70, it can be understood that it does not belong to the essential configuration of the mobile phone, and can be omitted as needed within the scope of not changing the essence of the invention.

The processor C-80 is the control center of the mobile phone, and connects various parts of the entire mobile phone by using various interfaces and lines, by running or executing software programs and/or modules stored in the memory C-20, and calling the memory in the memory C- The data in 20, the various functions of the mobile phone and the processing of data, so as to monitor the mobile phone as a whole. Optionally, the processor C-80 may include one or more processing units; preferably, the processor C-80 may integrate an application processor and a modem processor, where the application processor mainly processes an operating system and a user interface. And applications, etc., the modem processor primarily handles wireless communications. It can be understood that the above modem processor may not be integrated into the processor C-80.

The mobile phone also includes a power source C-90 (such as a battery) that supplies power to various components. Preferably, the power source can be logically connected to the processor C-80 through a power management system to manage charging, discharging, and power management through the power management system. And other functions. Although not shown, the mobile phone may further include a camera, a Bluetooth module, and the like, and details are not described herein again.

When a user uses a VR device in a public place, since the VR device is worn by both eyes, there is no danger of incidents such as theft, and therefore, a measure is needed to warn dangerous events that may be encountered around. At present, it is possible to perform an early warning by ultrasonically testing the distance between the surrounding object and the device, but this can only obtain the distance between the object and itself, and thus cannot conclude that this is dangerous.

The present invention performs data collection on the acquired image by performing motion pattern recognition on the VR device, and then performs image processing, thereby determining whether the distance between the object and itself and whether the object gradually approaches itself, thereby determining whether an early warning is required. If necessary, the user can be notified, so it can effectively help the user to eliminate dangerous situations when using the VR device.

Specifically, please refer to FIG. 5-1, which is a schematic flowchart of performing the embodiment of the present application by the VR device in the present application:

D1, open the early warning mode.

The user can enable the early warning mode on the VR application of the terminal, or enable the early warning mode on the VR accessory (for example, pressing a shortcut key on the VR accessory), which is not limited herein. Further, the VR device can automatically start the early warning mode. For example, after the user wears the VR device, the current geographic location is collected in real time, and the early warning mode is automatically started according to the geographical location information (for example, when the current geographic location is monitored as a relatively large traffic volume) , the warning mode is automatically turned on; or, after the user wears the VR device, the current acceleration is collected in real time, and the early warning mode is automatically turned on when the acceleration reaches a certain threshold (for example, when the current user is in a mobile environment, such as a bus) In the car, the warning mode is automatically turned on).

D2, motion pattern recognition.

When the warning mode is turned on, the VR device recognizes its own motion mode through inertial sensors or other sensors to obtain its own motion information, such as motion speed (including real-time speed and average speed), motion direction, displacement distance, and motion. Trajectories, etc., and can represent the position of VR devices, motion trajectories, and other items by calculating points in the coordinate system.

In the embodiment of the present application, the VR device can obtain its own motion data, such as speed, direction, and displacement through the inertial sensor in the motion recognition module, and then process the above data, so that the VR device can obtain its own motion track and real-time. The speed of movement. Figure 5-2 is a schematic diagram of the motion trajectory from point A to point B using a VR device. In this diagram, the motion trajectory of the VR device is the dotted line from point A to point B, and the VR device is in the process from point A to point B. The speed, direction and displacement are obtained by the inertial sensor, and the motion trajectory and real-time speed are obtained through waveform recognition.

As shown in Figure 5-3, it is a schematic diagram of a fixed position coordinate system. In this diagram, the VR device can set the A point as the starting point and set the point to the origin (0, 0, 0) of the fixed position coordinate system, and determine The directions of the x-axis, the y-axis, and the z-axis are used to obtain a fixed-position coordinate system, wherein the x-axis, the y-axis, and the z-axis can be selected at will, as long as the amount is vertical, and generally, the x-axis and the y-axis are set. The axis of the ground plane, z is perpendicular to the ground plane, and its positive direction points to the sky. Since the origin A of the fixed position coordinate system is a fixed point with reference to the earth, the fixed position coordinate system can also be called the earth coordinate system.

In the embodiment of the present application, when the VR device moves from point A to point B, the coordinates of any point in the path from point A to point B and the point of point B can be determined by inertial sensor and waveform recognition, and point B is set. The coordinates of the fixed position coordinate system are (b1', b2', b3'), that is, the value of the vector AB.

However, for a VR device, a coordinate system, also referred to as a VR coordinate system, as shown in FIG. 5-4, is a schematic diagram of the VR coordinate system. In the schematic diagram, the VR coordinate system takes the B point as the origin (0, 0). , 0), the x-axis, y-axis, and z-axis directions coincide with the x-axis, y-axis, and z-axis directions of the fixed position coordinate system, so the coordinates of the VR coordinate system at point B are (0, 0, 0), A The coordinates of the VR coordinate system of the point are (a1, a2, a3), that is, the value of the vector BA, which is equivalent to (a1, a2, a3) = - (b1', b2', b3').

In some possible embodiments, for any point C, the coordinates of point C for the VR device coordinate system are (c1, c2, c3), and the coordinates for the fixed position coordinate system are (c1', c2', c3'), Since (c1, c2, c3) is actually the value of vector BC, (c1', c2', c3') is the value of vector AC, so vector AC = vector AB + vector BC, ie for any point C, its fixed position coordinates The relationship between the coordinates of the system and the coordinates of the VR device coordinate system is: (c1', c2', c3') = (b1', b2', b3') + (c1, c2, c3), as shown in Figure 5-5. A schematic diagram of the coordinates of point C. For the fixed point B, (b1', b2', b3') is fixed, so it can be assumed that the coordinate of the coordinate of the VR coordinate system of point B is converted to the coordinate of the fixed position coordinate system of point A. _AB . When the conversion coefficient Δ _AB is determined, the VR device can convert the coordinates of the VR coordinate system at any point to the coordinates of the fixed position coordinate system, and can be used to determine whether one point is gradually approaching another point.

D3, image acquisition.

In the embodiment of the present application, the VR device can acquire an object around the VR device through image acquisition, for example, through a camera of a mobile phone or a camera of the VR glasses. In some possible embodiments, data acquisition can be performed by a motion recognition camera (DVS) on the VR device. Specifically, two or more DVSs are built in the VR device, and the DVS is used to capture moving objects, that is, non-moving objects cannot be photographed. The principle of judging the distance between two human eyes is simulated, and the positional relationship between the object and itself is judged by the images taken by the two DVSs.

In the embodiment of the present application, the image acquisition module is used to collect images. Alternatively, images can also be acquired by a color camera. Optionally, to reduce the power consumption of the color camera, images can also be acquired by a Dynamic Vision Sensor (DVS) camera. The DVS camera is used to capture moving objects, and can only capture moving objects. The captured images can be black and white. Figure 5-6 shows the shooting effect. Some palms are moving, such as waving and intercepting. One of the frames is a black and white rendering of the DVS camera.

D4, image processing.

After the motion pattern recognition and image acquisition are performed, the position and movement state of the object can be known by the motion pattern recognition, and the position of the object relative to itself can be obtained through image acquisition, and the position of the object can be obtained. By recording the object and its position multiple times, it is judged whether the object is gradually approaching itself, and whether it is dangerous or not, whether an early warning is required.

In the embodiment of the present application, when the image acquisition module collects an image, as shown in FIG. 5-7, a schematic diagram of an image acquired by the DVS camera, the maximum connected domain may be processed for the image, and a grayscale block is obtained. The gray scale block can be considered as an object. Figure 5-8 shows the processing effect of the maximum connected domain.

Then, the centroid of the gray block is determined according to the sliding window method, as shown in FIG. 5-9, which is a schematic diagram of the centroid of the gray block, and the centroid is set to point D. When the D point in the image is determined, the position of the D point in the three-dimensional space can be determined.

In the embodiment of the present application, since the shooting is performed by two DVS cameras, the position of the D point is different for different DVS cameras, as shown in FIG. 5-10, and the D points are respectively in the two DVS cameras. The location in .

Similar to the principle of the human eye, through the difference of D points for two DVS cameras, the coordinates of D relative to the VR device are obtained as shown in (d1, d2, d3), as shown in Figure 5-11, which is D point. Schematic diagram of coordinates in the VR device coordinate system.

Please refer to Figure 5-12, which is the coordinate diagram of the D point in the fixed position coordinate system. When the VR device coordinate system of the D point is determined, the coordinates of the D point relative to the fixed position coordinate system can be calculated. The specific calculation method is Let the coordinates of the fixed position coordinate system of D be (d1', d2', d3'), then:

(d1', d2', d3') = (d1, d2, d3) + (b1', b2', b3') = (d1, d2, d3) + △ _AB .

In the embodiment of the present application, when the fixed position coordinates of the D point and the B point are determined, the distance between the two points can be determined. When the D point moves relative to the fixed position coordinate, it can be determined whether the moving direction of the D point is For point B, it can be determined whether point D is getting closer to the VR device. In some feasible embodiments, the other two points may also be determined, and the fixed position coordinates of the two points are periodically acquired to determine whether the two points are gradually approaching, which is not limited herein.

D5, for early warning.

If an early warning is required if it is judged to be dangerous by image processing, an early warning is issued. Specifically, the warning method can include Displaying text or images on the VR video that the user is watching may also emit a strong light or a sound, etc., which is not limited herein.

It should be noted that, in order to meet the above functions, the VR device can be built with an early warning system E. Please refer to FIG. 6 , which is a schematic diagram of the early warning system E. The early warning system E includes a system setting module E1, a motion pattern recognition module E2, and image acquisition. The module E3, the image processing detection module E4, and the warning module E5 are respectively used to run the functions of the above D1-D5.

The functions and implementation principles of the image acquisition module, the motion recognition module, and the image processing module are described above. The following is described in conjunction with the embodiments. Referring to FIG. 7-1, a flow chart of the embodiment is shown.

S1, open the warning function.

Please refer to Figure 7-2 for a schematic diagram of the VR device for the user. In the embodiment of the present application, when the user uses the VR device, if the VR device is fully enclosed and worn by the user, the user may not be able to observe surrounding objects and things happening through the glasses. When the user uses the VR device outdoors, if there is someone stealing or other dangerous situations, an early warning function is required. In the embodiment of the present application, the user walks in the outdoor park as an example for description.

In some possible embodiments, the VR device automatically starts the early warning function, and may also be opened by the button, or may be opened by a voice command, or may be opened by other means, which is not limited herein. It should be noted that the VR device may be a mobile phone and a VR glasses, or may be an integrated VR device, which is not limited herein. In the embodiment of the present application, the image acquisition module built in the VR device may include two DVS cameras for capturing an object and determining the position of the observed object in a three-dimensional space.

S2, collecting motion data.

In the embodiment of the present application, the user walks in the park using the VR device, as shown in FIG. 7-3, which is a schematic diagram of walking for the user. At this time, please refer to Figure 7-4. The VR device can determine a fixed point A and a fixed position coordinate system with the point A as the origin. When the user moves to point B, the VR device with the point B as the origin can be determined. A coordinate system in which the value of the vector AB is a coordinate system conversion coefficient Δ _AB for transforming an arbitrary point between the coordinates of the VR device coordinate system and the coordinates of the fixed position coordinate system.

S3. Acquire an image.

In the embodiment of the present application, when a person or thing that appears to be moving in front of or around the user (refer to FIG. 7-5, a schematic diagram of the character appears in front of the user), the image of the motion can be obtained by the DVS camera, such as the motion. Human hand (please refer to Figure 7-6 for a schematic diagram of the moving hand taken by the DVS camera). Since the VR device has two DVS cameras built in, two DVS cameras respectively capture the image of the hand, and the principle of the human eye determines the position of the hand for the VR device.

S4, discriminate dangerous behavior.

In the embodiment of the present application, after the image of the moving hand is acquired by the DVS camera, the image processing module may perform processing. Specifically, the grayscale block in the image is determined by the maximum connected domain method, and then the centroid F of the grayscale block is determined, (as shown in FIG. 7-7, the grayscale block of the image of the moving hand and the schematic diagram of the centroid) Two different images are presented in the two DVS cameras (as shown in Figure 7-8, where the center of mass of the moving hand is in the two DVS cameras), then the coordinates of the F point for the VR device can be determined ( F1, f2, f3) (Figure 7-9, the coordinates of the VR device coordinate system at point F).

In the embodiment of the present application, when the coordinates of the VR device coordinate system of the F point are determined, the coordinates (f1', f2', f3') of the fixed position coordinate system of the F point can be calculated by the coordinate system conversion coefficient Δ _AB ( As shown in Figure 7-10, the coordinates of the coordinate system of the fixed point device at point F), specifically:

(f1', f2', f3') = (f1, f2, f3) + △ _AB

In some feasible embodiments, as shown in FIG. 7-11, a schematic diagram of a user's valuables, assuming that there is a valuable item in the bag of the user's shoulder, the valuable object is a mobile phone, and the coordinates of the mobile phone relative to the VR device, ie, VR The coordinates of the device coordinate system can be preset before the warning mode is turned on. Here, it is assumed that the position of the mobile phone is G point, and the coordinate of the VR device coordinate system is (g1, g2, g3), which is converted into the coordinate of the fixed position coordinate system. Is (g1', g2', g3'). It should be noted that the valuable item is a fixed position for the VR device, that is, the valuable item is stationary with respect to the VR device. In addition, in general, when the user turns on the early warning function of the VR device, the valuable item is placed as far as possible within the shooting range of the camera of the VR device. Specifically, if the DVS camera of the VR device is adjustable in angle, the automatic item can be automatically aligned to make the valuable item within the shooting range of the DVS camera. If the DVS camera is not adjustable, The user can try to place valuables in the position that the DVS camera can capture to enhance its security.

In summary, the coordinates of the fixed position coordinate system of the position G of the valuable item and the position F of the moving hand are determined. The positional relationship between the G point and the F point is shown in Fig. 7-12 with respect to the fixed position coordinate system. Is the positional relationship between the G point and the F point of the fixed position coordinate system.

In the embodiment of the present application, the basis for determining whether the moving hand is dangerous may be: whether the F point gradually approaches the G point, and if the F point gradually approaches the G point, and the distance from the F point and the G point is within a preset range, Then point F is determined to be a dangerous event. Specifically, please refer to FIG. 7-13, which is a schematic diagram of the moving direction of the F point and the positional relationship with the G. When the distance between point F and point G is less than or equal to the first threshold (such as 2 meters), the VR device can start the counter M, the initial value of M can be 0, and record G point and F periodically (such as once every 0.1 seconds). The positional relationship of the point and the moving direction of the point F; if the distance from the second point to the point G is reduced, and the moving direction of the point F is the point G, then M will be incremented by 1, ie, M=M+1=1 If the distance from the F point to the G point is reduced in the third time, and the movement direction of the point F is G point, then M will accumulate 1 , that is, M=M+1=2; and so on, when the point F continues to approach At point G, M continues to accumulate until M reaches the threshold M0 (eg, M0=10). When M>=M0, and the distance between point F and point G is less than or equal to the second threshold (eg, 1 meter), then F can be determined. The hand of the movement represented by the point is dangerous, that is, there is a dangerous situation! In some possible embodiments, the value of M may be reset to zero if the terminal F point is away from the G point or the direction of the transfer.

S5, for early warning.

In the embodiment of the present application, when a dangerous situation is determined by the processing of the image processing module, the early warning module may perform an early warning. The specific warning mode can display an alarm on the screen of the VR device, as shown in Figure 7-14. The alarm is displayed on the screen of the VR device. The warning can be a text warning. For example, please note that there is a suspected theft. !) You can also display the image captured by the DVS camera (as shown in Figure 7-15, which is a schematic diagram of the warning of the text displayed on the screen of the VR device). It can also emit a strong light or an alarm sound (Figure 7-16, VR). A schematic diagram of the device emitting a strong light or warning of an alarm sound, which is not limited here.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, it may be implemented in whole or in part in the form of a computer program product.

The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in accordance with embodiments of the present invention are generated in whole or in part. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be from a website site, computer, server or data center By wired (eg coaxial cable, Fiber, Digital Subscriber Line (DSL) or wireless (eg infrared, wireless, microwave, etc.) is transmitted to another website, computer, server or data center. The computer readable storage medium can be any available media that can be stored by a computer or a data storage device such as a server, data center, or the like that includes one or more available media. The usable medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (such as a solid state disk (SSD)).

Claims

A prompting method, the method being applied to a VR device, comprising:

The VR device collects at least two images through a camera, the at least two images having the same subject;

The VR device issues a prompt according to a change in the positional relationship between the subject and the preset position, and the positional relationship between the subject and the preset position is changed by the VR device according to the at least two images.
The method according to claim 1, wherein the prompting by the VR device according to the change of the positional relationship between the subject and the preset position comprises:

Determining, by the VR device, a change in a positional relationship between the subject and the preset position according to the at least two images;

The VR device issues a prompt if the distance between the subject and the preset position is shortened according to the positional relationship change, and the moving direction of the subject is the preset position.
The method according to claim 2, wherein the determining, by the VR device, the change in the positional relationship between the subject and the preset position according to the at least two images comprises:

Determining, by the VR device, a first location and a second location of the subject according to the first image and the second image of the at least two images, respectively;

Determining, by the VR device, a motion trajectory of the subject according to the first location and the second location;

The VR device determines a change in the positional relationship with the preset position according to a motion trajectory of the subject.
The method of any of claims 1-3, wherein the predetermined position is a fixed position relative to the VR device.
The method according to any one of claims 1 to 4, wherein the preset position is within a shooting range of a camera of the VR device.
The method according to claim 3 or 4, wherein before the VR device determines the change in the positional relationship with the preset position according to the motion trajectory of the subject, the method further includes:

The VR device acquires a motion track of the VR device;

Determining, by the VR device, a motion trajectory of the preset position according to a motion trajectory of the VR device;

Determining, by the VR device, a change in the positional relationship with the preset position according to the motion trajectory of the subject includes:

The VR device determines a change in the positional relationship between the subject and the preset position according to the motion trajectory of the subject and the motion trajectory of the preset position.
The method of claim 6, wherein the acquiring, by the VR device, the motion track of the VR device comprises:

The VR device collects motion information in real time through at least one sensor to obtain motion data;

The VR device determines a motion trajectory of the VR device according to the motion data.
The method according to any one of claims 2-7, wherein the VR device determines a first position of the subject and a first image based on the first image and the second image of the at least two images, respectively The second location includes:

Determining, by the VR device, a first maximum grayscale block of the first image and a second largest grayscale block of the second image, respectively;

Determining, by the VR device, a first centroid of the first maximum gray scale block and a second centroid of the second maximum gray scale block;

The VR device determines a first location of the first centroid and a second location of the second centroid.
A method according to any one of claims 1-8, wherein the subject comprises a hand or a dice.
Method according to any of the claims 1-9, characterized in that the prompt comprises an alarm sound and the VR is turned off Video, text prompts or image prompts on the screen of the VR device.
A VR device, comprising:

An acquisition module, configured to acquire at least two images by using a camera, the at least two images having the same subject;

And a prompting module, configured to issue a prompt according to the change of the positional relationship between the subject and the preset position, where the positional relationship between the subject and the preset position is changed by the VR device according to the at least two images.
The VR device according to claim 11, wherein the prompting module comprises:

Determining a sub-module, configured to determine, according to the at least two images, a change in a positional relationship between the subject and the preset position;

The prompting sub-module is configured to issue a prompt if the distance between the subject and the preset position is shortened according to the positional relationship change, and the moving direction of the subject is the preset position.
The VR device according to claim 12, wherein the determining submodule comprises:

a first determining unit, configured to determine a first location and a second location of the subject according to the first image and the second image of the at least two images, respectively;

a second determining unit, configured to determine a motion trajectory of the subject according to the first location and the second location;

And a third determining unit, configured to determine a change in the positional relationship with the preset position according to the motion trajectory of the subject.
The VR device according to claim 12 or 13, wherein the determining submodule further comprises:

An acquiring unit, configured to acquire a motion track of the VR device;

a fourth determining unit, configured to determine a motion trajectory of the preset position according to a motion trajectory of the VR device;

The third determining unit is specifically configured to determine a positional relationship change between the subject and the preset position according to the motion trajectory of the subject and the motion trajectory of the preset position.
The VR device according to claim 14, wherein the obtaining unit comprises:

The collecting subunit is configured to collect motion information in real time by using at least one sensor to obtain motion data;

Determining a subunit for determining a motion trajectory of the VR device based on the motion data.
The VR device according to any one of claims 12-15, wherein the second determining unit comprises:

a first determining subunit, configured to respectively determine a first maximum grayscale block of the first image and a second largest grayscale block of the second image;

a second determining subunit, configured to determine a first centroid of the first maximum grayscale block and a second centroid of the second largest grayscale block;

And a third determining subunit, configured to determine a first position of the first centroid and a second position of the second centroid.
A VR device, comprising: a processor, a memory, one or more applications; wherein

The one or more applications are stored in the memory, the one or more applications including instructions that, when executed by the VR device, cause the VR device to perform the following steps:

Acquiring at least two images by a camera, the at least two images having the same subject;

And prompting according to the change of the positional relationship between the subject and the preset position, the positional relationship of the subject and the preset position is changed by the VR device according to the at least two images.
The VR device according to claim 17, wherein in said step of issuing a prompt according to a change in positional relationship between said subject and a preset position, said at least one processor executes said plurality of instructions to cause said VR device to at least Perform the following steps:

Determining a change in a positional relationship between the subject and the preset position according to the at least two images;

And indicating that the distance between the subject and the preset position is shortened according to the positional relationship change, and the moving direction of the subject is the preset position.
The VR device according to claim 18, wherein in the step of determining a positional relationship between the subject and the preset position based on the at least two images, the at least one processor Executing the instructions causes the VR device to perform at least the following steps:

Determining a first location and a second location of the subject according to the first image and the second image of the at least two images, respectively;

Determining a motion trajectory of the subject according to the first position and the second position;

A change in the positional relationship with the preset position is determined according to a motion trajectory of the subject.
The VR device according to claim 18 or 19, wherein said at least one processor executes said plurality of instructions to cause said VR device to further perform the following steps:

Obtaining a motion track of the VR device;

Determining a motion trajectory of the preset position according to a motion trajectory of the VR device;

Determining a change in the positional relationship with the preset position according to the motion trajectory of the subject includes:

Determining a change in the positional relationship of the subject with the preset position according to the motion trajectory of the subject and the motion trajectory of the preset position.
The VR device according to claim 20, wherein in the step of acquiring a motion track of the VR device, the at least one processor executing the instructions causes the VR device to perform at least the following steps:

Collecting motion information in real time through at least one sensor to obtain motion data;

A motion trajectory of the VR device is determined based on the motion data.
The VR device according to any one of claims 17 to 21, wherein the first position and the second position of the subject are determined based on the first image and the second image of the at least two images, respectively In the location step, the at least one processor executes the instructions to cause the VR device to perform at least the following steps:

Determining a first maximum grayscale block of the first image and a second largest grayscale block of the second image, respectively;

Determining a first centroid of the first maximum gray scale block and a second centroid of the second maximum gray scale block;

A first position of the first centroid and a second position of the second centroid are determined.
A computer program product, characterized in that, when the computer program product is run on a VR device, the VR device is caused to perform the method of any of claims 1-10.
A computer readable storage medium comprising instructions, wherein the instructions, when executed on a VR device, cause the VR device to perform the method of any of claims 1-10.