WO2019033510A1

WO2019033510A1 - Method for recognizing vr application program, and electronic device

Info

Publication number: WO2019033510A1
Application number: PCT/CN2017/103193
Authority: WO
Inventors: 孟亚州
Original assignee: 歌尔科技有限公司
Priority date: 2017-08-16
Filing date: 2017-09-25
Publication date: 2019-02-21
Also published as: CN107506031B; CN107506031A

Abstract

Provided are a method for recognizing a VR application program, and an electronic device. The method comprises: acquiring, from four corners of an interface image of an application program to be recognized, at least four reference image blocks of appointed sizes; determining, according to grayscale values of pixel points included in the at least four reference image blocks, whether the at least four reference image blocks are all black image blocks; if the at least four image blocks are all black image blocks, recognizing, according to grayscale values of pixel points included in the interface image, a black image area in the interface image; determining, according to grayscale values of pixel points included in the black image area, whether left and right parts of the black image area correspond to each other; and if the left and right parts of the black image area correspond to each other, determining that the application program to be recognized is a VR application program. By means of the technical solution provided in the present invention, a VR application program can be efficiently recognized from a large number of application programs.

Description

Method for identifying VR application and electronic device

cross reference

The present application is hereby incorporated by reference in its entirety in its entirety in its entirety in its entirety in its entirety in the the the the the the the the the the the

Technical field

The present invention relates to the field of virtual reality technologies, and in particular, to a method and an electronic device for identifying a VR application.

Background technique

VR (Virtual Reality) technology is a computer simulation system that can create and experience virtual worlds. It uses a computer to generate a simulation environment, and through interactive multi-source information, interactive 3D dynamic vision and physical behavior. The system simulation immerses the user in the simulation environment.

With the maturity of VR technology, applications that are applied in VR mode continue to emerge. How to efficiently identify VR applications from a large number of applications becomes a technical problem that needs to be solved urgently.

Summary of the invention

The invention provides a method for identifying a VR application and an electronic device for efficiently identifying a VR application from a large number of applications.

The invention provides a method for identifying a VR application, comprising:

Obtaining at least four reference image blocks of a specified size from four corners of the interface image of the application to be identified;

Determining the at least four according to a gray value of a pixel point included in the at least four reference image blocks Whether the reference image blocks are black image blocks;

If the at least four image blocks are black image blocks, identifying a black image region in the interface image according to a gray value of a pixel point included in the interface image;

Determining, according to the gray value of the pixel point included in the black image area, whether the left and right portions of the black image area correspond to each other;

If the left and right portions of the black image area correspond, it is determined that the to-be-identified application is a VR application.

Further, determining whether the at least four reference image blocks are all black image blocks according to the gray value of the pixel points included in the at least four reference image blocks comprises: acquiring the at least four reference image blocks The number of pixels whose gray value is smaller than the specified gray threshold; the ratio of the number of pixels whose gray value is smaller than the specified gray threshold to the total number of pixels included in the at least four reference image blocks; The ratio is greater than or equal to the set ratio threshold, and then determining that the at least four image blocks are black image blocks.

Further optionally, the method further includes: if the at least four reference image blocks are not all black image blocks, determining that the to-be-identified application is a non-VR application.

Further optionally, acquiring, from the four corners of the interface image of the application to be recognized, at least four reference image blocks of a specified size, including: starting points selected by the four vertices of the interface image as images, An image area of a specified size is selected on the interface image to obtain the at least four as the reference image block.

Further, optionally, identifying the black image area in the interface image according to the gray value of the pixel point included in the interface image, comprising: acquiring, in the interface image, that the gray value is less than the specified gray threshold An area in which the pixel point is located; acquiring a geometric envelope map of the suspect black image area according to the coordinates of the pixel point included in the suspect black image area; if the geometric envelope image exists and the top angle of the interface image The coincident apex angle or the longitudinal center axis of the geometric envelope map coincides with the longitudinal center axis of the interface image, and it is determined that the suspect black image region belongs to the black image region of the interface image.

Further optionally, judging the gray value according to the pixel points included in the black image area Whether the left and right portions of the black image area correspond to each other includes: determining whether the black image area is symmetrical along a longitudinal central axis of the interface image according to a gray value of a pixel point included in the black image area.

Further optionally, determining whether the black image area is symmetrical along a longitudinal central axis of the interface image according to a gray value of a pixel point included in the black image area comprises: along a longitudinal central axis of the interface image The black image area is divided into two sub-image areas; the similarity rate of the two sub-image areas is calculated according to the total number of pixels included in the two sub-image areas and the number of symmetric pixel points having the same gray value; If the similarity ratio is greater than the set similarity threshold, it is determined that the black image area is symmetrical along a longitudinal center axis of the interface image.

Further optionally, calculating a similarity ratio of the two sub-image regions according to a total number of pixels included in the two sub-image regions and a number of symmetric pixel points having the same gray value in the two sub-image regions, including : establishing a coordinate system with the horizontal central axis of the interface image as an abscissa axis and the longitudinal central axis of the interface image as an ordinate axis; in the two sub-image regions, obtaining the same ordinate and opposite the abscissa a pixel as a symmetric pixel; counting the number of symmetric pixel points having the same gray value, and obtaining an average of the total number of pixels included in the two sub-image regions; symmetric pixels having the same gray value The ratio of the number of points to the average value determines the similarity rate of the two sub-image areas.

The present invention also provides a VR application identification electronic device, including: a memory and a processor;

The memory is configured to: store one or more instructions; the processor is configured to execute the one or more computer instructions for: obtaining at least four specified sizes from four corners of an interface image of an application to be identified Reference image block;

Determining, according to the gray value of the pixel point included in the at least four reference image blocks, whether the at least four reference image blocks are black image blocks;

If the left and right parts of the black image area correspond, determining that the application to be identified is a VR application.

Further, the processor is specifically configured to: acquire, in the at least four reference image blocks, a number of pixels whose gray value is smaller than a specified gray threshold; and calculate that the gray value is smaller than a specified gray threshold. a ratio of the number of pixels to the total number of pixels included in the at least four reference image blocks; if the ratio is greater than or equal to the set ratio threshold, determining that the at least four image blocks are black image blocks.

Further, the processor is specifically configured to: obtain, in the interface image, an area where a pixel point whose gray value is smaller than the specified gray level threshold is used as a suspicious black image area; according to the suspect black image area a coordinate of the included pixel, acquiring a geometric envelope of the suspect black image region; if the geometric envelope has an apex angle coincident with a vertex of the interface image or a longitudinal direction of the geometric envelope The axis coincides with the longitudinal center axis of the interface image, and it is determined that the suspect black image area belongs to the black image area of the interface image.

The VR application identification method and the electronic device provided by the present invention determine whether the left and right parts of the interface image correspond to each other by judging whether the black image block is a black image block at the four corners of the interface image of the application to be identified. . Further, when it is determined that the left and right portions of the interface image correspond, the application to be identified is determined to be a VR application. This method is not limited by the package name or name of the application, and the recognition efficiency of the VR application is high.

DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, a brief description of the drawings used in the embodiments or the prior art description will be briefly described below. Obviously, the drawings in the following description It is a certain embodiment of the present invention, and other drawings can be obtained from those skilled in the art without any creative work.

1a is a schematic flowchart of a method for identifying a VR application according to an embodiment of the present invention;

Figure 1b is a schematic diagram of an interface image of a VR application;

FIG. 2 is a schematic flowchart of another method for identifying a VR application according to an embodiment of the present invention; Figure

2b is a schematic diagram of selecting a reference image block at four corners of an interface image according to an embodiment of the present invention;

2c is a schematic diagram of a geometric envelope diagram for acquiring a suspicious black image region in an interface image according to an embodiment of the present invention;

2d is another schematic diagram of acquiring a geometric envelope diagram of a suspicious black image area in an interface image according to an embodiment of the present invention;

2e is a schematic diagram of establishing a coordinate system on an interface image according to an embodiment of the present invention;

3 is a schematic structural diagram of an electronic device according to an embodiment of the present invention;

FIG. 4 is a schematic diagram showing the internal configuration of the head mounted display device 400 in some embodiments provided by the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. It is a partial embodiment of the invention, and not all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

FIG. 1 is a schematic flowchart of a method for identifying a VR application according to an embodiment of the present invention. Referring to FIG. 1a, the method includes:

Step 101: Acquire at least four reference image blocks of a specified size from four corners of the interface image of the application to be identified.

Step 102: Determine, according to the gray value of the pixel points included in the at least four reference image blocks, whether the at least four reference image blocks are black image blocks; if yes, perform step 103; if not, execute Step 106.

Step 103: Identify a black image area in the interface image according to a gray value of a pixel point included in the interface image.

Step 104: Determine whether the left and right portions of the black image region correspond according to the gray value of the pixel included in the black image region; if yes, execute step 105; if no, execute step 106.

Step 105: Determine that the to-be-identified application is a VR application.

Step 106: Determine that the to-be-identified application is a non-VR application.

For step 101, the application to be identified may be any application installed on a VR device or a terminal device such as a mobile phone. The interface image is a corresponding image of any interface of the application to be recognized, and includes an interface style and an interface element of the application to be recognized. For the VR device, the application store or the application scenario can be accessed to find the application to be identified, the application to be recognized is opened, and the interface image of any interface of the application to be recognized is obtained. For a terminal device such as a mobile phone, the application to be identified can be found on the desktop or in the application list, and the application to be recognized is opened and an interface image of any interface of the application to be recognized is obtained. Generally, the shape of the interface image is a rectangle, and the four corners of the interface image are the areas near the four vertices of the interface image. In this step, at least one reference image block may be acquired at each corner of the interface image. Wherein, the size of the reference image block is specified, and the specified size is associated with the size of the interface image.

For step 102, typically, the display interface of the VR application includes a content image area and a black image area. The content image area is used to display interface information of the VR content or the VR application, and the black image area is a non-content image area. In order to allow the user to have a stereoscopic sense of depth when viewed, the display interface of the VR application includes two content image areas that are parallel to the direction in which the user's left and right eyes are connected. It can be considered that in the rectangular interface image of the VR application, the two content image areas are removed, and the remaining part is the black image area. 2b is a schematic diagram of an interface image of the VR application. In FIG. 1b, the mesh portion is a content image area, and the portion outside the mesh is a black image area.

As shown in FIG. 1b, in order to make the user feel a real immersion when viewing, both content image areas have a certain degree of barrel distortion. Furthermore, this barrel distortion causes the four corners of the interface image of the VR application to be black. In the interface image of a non-VR application, the four corners of the image are not necessarily black. Therefore, the embodiment can determine that the reference image block on the four corners of the interface image is No black, to initially screen out non-VR applications.

The black image contains pixels with a gray value that is usually small, usually between 0-10. Therefore, optionally, after acquiring at least four reference image blocks, determining, according to the gray value of the pixel points included in each reference image block, whether the reference image block is a black image block to implement preliminary screening of non-VR applications .

For step 103, in one possible scenario, the interface image of the non-VR application may also have a case where all four corners are black. Therefore, if at least four reference image blocks acquired in step 101 are not all black image blocks, the interface image may be considered as an interface image of a non-VR application. If at least four reference image blocks acquired in step 101 are black image blocks, the interface image may be considered to be an interface image of the VR application. However, it is necessary to further determine that the interface image is indeed an interface image of the VR application.

Optionally, in this embodiment, the interface image is determined to be an interface image of the VR application according to the corresponding features of the left and right portions of the black image area in the interface image. Before performing the correspondence determination on the left and right portions of the black image area, the black image area in the interface image may be identified according to the gray value of the pixel points included in the interface image.

For step 1024, for the VR application, the interface displayed by the VR application is a dual screen interface, and the left and right screens respectively correspond to the positions of the left eye and the right eye of the user. Further, at the same time, the display contents seen by the left and right eyes of the user are different to produce a strong stereoscopic depth. Furthermore, it can be considered that if an interface image is an interface image of a VR application, the left and right portions of the black image area in the interface image must correspond.

If the left and right parts of an image area correspond, the gray values of the corresponding pixel points in the left and right parts of the image area must be the same. Therefore, optionally, this step can determine whether the left and right portions of the interface image correspond to each other by the gray value of the pixel point included in the black image area of the interface image. If so, it can be determined that the application to be identified is a VR application, otherwise it is a non-VR application.

In this embodiment, by determining whether the four corners of the interface image of the application to be identified are all black image blocks to initially screen the non-VR application program, it is further determined whether the left and right portions of the interface image correspond. Further, when determining the corresponding left and right portions of the interface image, the application to be identified is determined. The order is a VR application. This method is not limited by the package name or name of the application, and the recognition efficiency of the VR application is high.

It should be noted that, in the foregoing or the following embodiments of the present invention, the interface image may be a screen capture image obtained by taking a screen shot for any interface of the application to be identified, or may be performed on any interface of the application to be identified. Take the captured image. The technical solution of the embodiment of the present invention is further described in the following section by taking a screen shot image of the object to be identified as an example.

FIG. 2 is a schematic flowchart of another method for identifying a VR application according to an embodiment of the present invention. Referring to FIG. 2a, the method includes:

Step 201: Obtain at least four reference image blocks of a specified size from four corners of the screen capture image of the application to be identified;

Step 202: Acquire, in the at least four reference image blocks, a number of pixels whose gray value is smaller than a specified gray threshold;

Step 203: Calculate a ratio of a number of pixel points whose gray value is smaller than a specified gray level threshold to a total number of pixel points included in the at least four reference image blocks.

Step 204: Determine whether the ratio is less than a set proportional threshold. If yes, go to step 210; if no, go to step 205.

Step 205: Identify a black image area in the screen capture image according to a gray value of a pixel point included in the screen capture image.

Step 206, dividing the black image area into two left and right sub-image areas along a longitudinal central axis of the screen capture image;

Step 207: Calculate a similarity ratio of the two sub-image regions according to a total number of pixels included in the two sub-image regions and a number of symmetric pixel points with the same gray value.

Step 208: Determine, according to the similarity ratio of the two sub-image regions, whether the black image region is symmetric along a longitudinal central axis of the screen capture image; if yes, perform step 209; if no, perform step 210.

Step 209: Determine that the to-be-identified application is a VR application.

Step 210: Determine that the to-be-identified application is a non-VR application.

For step 201, when at least four reference image blocks of a specified size are acquired from the four corners of the screen capture image of the application to be identified, the four vertices of the screen capture image may be used as a starting point for image selection, and the specified size is selected on the screen capture image. The image area obtains the at least four reference image blocks.

Optionally, in this embodiment, in order to improve the recognition efficiency and reduce the calculation amount in the recognition process, only one square reference image block may be taken at each corner of the screen capture image. As shown in FIG. 2b, four reference image blocks rect1, rect2, rect3, and rect4 are selected on the screen image, and the vertices of each reference image block coincide with the vertices of the screen image.

In this step, the advantage of selecting the starting point of the vertices of the screen image as the image is that, for the screen image with different degrees of distortion of the content image area, the possibility of including the content image in the reference image block can be reduced, and the black image is further improved. The accuracy of the block identification.

For example, the content image area on a VR application interface is highly distorted, resulting in a smaller black image area at the four corners of the VR application interface. At this time, the starting point selected by the vertex of the screen image as the image can avoid the pixel points selected to the content image area.

Optionally, the specified size is associated with the size of the screenshot image. After trial and error, when the specified size is one tenth of the size of the screen image, the selected reference image block is more effective, but the embodiment of the present invention does not limit the size of the specified size.

For step 202, after acquiring at least four reference image blocks, the number C of pixels of the at least four reference image blocks whose gray value is smaller than the specified gray threshold may be acquired. Among them, the specified gray value threshold is small, usually a single digit.

When the screen image is an image in the RGB color format, if it is determined whether the gray value of the pixel in the reference image block is less than the specified gray threshold, the gray value of the pixel may be acquired first.

Optionally, obtaining the gray value of the pixel included in the reference image block may adopt the following optional methods:

1, floating point algorithm: Gray = R * 0.3 + G * 0.59 + B * 0.11;

2. Integer method: Gray=(R*30+G*59+B*11)/100;

3. Shift method: Gray=(R*28+G*151+B*77)>>8;

4. Average method: Gray=(R+G+B)/3;

5. Take only the green method: Gray=G.

Where R, G, and B are the values of any pixel on the reference image block on the three color components of red, green, and blue, respectively, and Gray is the calculated gray value of the pixel.

For step 203, the ratio P1 of the number of pixel points whose gray value is smaller than the specified gray level threshold to the total number of pixel points included in the reference image block is calculated, and the following formula can be used:

Where a is the number of pixels included in the longitudinal direction of the reference image block, b is the number of pixels included in the height direction of the reference image block, and N is the total number of reference image blocks, N ≥ 4.

For step 204, in the present embodiment, preferably, when the set proportional threshold is 99%, the recognition efficiency and accuracy of the black image block are high. That is, if P1 is greater than 99%, it may be determined that at least four reference image blocks acquired in step 201 are black image blocks.

For the step 205, the black image area in the screen shot image refers to a portion of the screen shot image other than the content image area. To identify the portion, the suspicious black image region in which the grayscale value is smaller than the specified grayscale threshold is identified from the screen capture image, and the black image region is distinguished from the suspect black image region.

In one possible case, the image content displayed in the content image area of the screen shot image has black portions, and the black portion contains pixel points whose gray value is also smaller than the specified gray scale threshold. Thus, the suspect black image area may contain black portions in the content image area. Optionally, in order to improve the accuracy of the VR application identification, in the embodiment, the black image area of the screen capture image may be identified from the suspect black image area by using the following method:

First, for each suspicious black image area, a regular geometric figure can be used to draw the maximum contour of the suspect black image area as a geometric envelope of the suspect black image area. Optionally, when drawing the geometric envelope diagram of the suspect black image area, the maximum value of the abscissa of the pixel, the minimum value of the abscissa, the maximum value of the ordinate, and the ordinate of the pixel in the suspect black image area may be calculated first. The minimum value, and the geometric envelope diagram of the suspect black image area is determined by these four maximum values.

Secondly, after determining the geometric envelope map of the suspect black image area, determining that the geometric envelope map is There is an apex angle that coincides with the apex angle of the screen shot image, or whether the longitudinal center axis of the geometric envelope map coincides with the longitudinal center axis of the screen shot image. Determining that if the geometric envelope map has an apex angle that coincides with a vertex angle of the screen shot image, or determines that a longitudinal central axis of the geometric envelope map coincides with a longitudinal central axis of the screen capture image The suspicious black image area belongs to the black image area of the screen shot image. The following part will further explain the method for identifying the black image area provided by the embodiment in conjunction with FIG. 2c and FIG. 2d.

2c is a schematic diagram of a geometric envelope diagram for acquiring a suspicious black image in a screen capture image according to an embodiment of the present invention. The five shaded areas in Fig. 2c are suspect black image areas, and the geometric envelope Fig. 1 - geometric envelope Fig. 5 are the geometric envelope diagrams corresponding to the suspect black image areas in Fig. 2c, respectively. In Figure 2c, the four apex angles of the geometric envelope 1 coincide with the apex angle of the screen image, and the longitudinal center axis of the geometric envelope 1 coincides with the longitudinal center axis of the screen image. Therefore, it can be considered that the suspicious black image area corresponding to the geometric envelope diagram 1 is the black image area of the screen shot image. Geometric Envelope Figure 2 - Geometry Envelope Figure 5 does not coincide with the apex angle of the screen image, and the geometric envelope diagram 2 - the longitudinal center axis of the geometric envelope diagram 5 does not correspond to the longitudinal center axis of the screen image coincide. Therefore, it can be considered that the suspicious black image area corresponding to the geometric envelope diagram 1 belongs to the content image area of the screen capture image.

FIG. 2d is another schematic diagram of acquiring a geometric envelope diagram of a suspicious black image area in a screen capture image according to an embodiment of the present invention. In the case shown in Fig. 2d, the degree of distortion of the content image area is large, and the boundary of the content image area is tangent to the boundary of the screen image. In Fig. 2d, there are a total of 10 suspect black image areas of A1-A10. In FIG. 2d, the geometric envelope diagram corresponding to A1-A4 has a apex angle coincident with the apex angle of the screen image, and the suspect black image area A1-A4 may be considered to belong to the black image area of the screen capture image. In Fig. 2d, the longitudinal central axes of A5 and A6 coincide with the longitudinal central axis of the screen image, and the suspect black image areas A5 and A6 may be considered to belong to the black image area of the screen image. The apex angle of the A7-A10 does not coincide with the apex angle of the screen image, and any longitudinal central axis does not coincide with the longitudinal center axis of the screen image. Therefore, the suspicious black image area A7-A10 can be considered to belong to the content image area of the screen shot image.

For step 206, after the black image area is identified, the black image area is divided into two left and right sub-image areas along the longitudinal center axis.

The application to be tested is installed on the VR device or installed on the mobile phone embedded in the VR device for the user to view with both eyes. Therefore, the longer side of the captured image of the application to be tested is the side parallel to the direction in which the left and right eyes of the user are connected. The longitudinal center axis is a central axis perpendicular to the longer side of the screen image, and the longitudinal center axis is capable of dividing the screen image into two equal parts. That is to say, if the lengths of the screen images are w and h, respectively, the longitudinal center axis is along the line direction of (w/2, 0) and (w/2, h).

After step 207, the black image area is divided into two parts, the horizontal central axis of the screen image is the horizontal axis x, and the longitudinal central axis of the screen image is the vertical axis y. The coordinate system x0y shown in 2e.

Based on the coordinate system, in the two sub-image regions, pixels having the same ordinate and opposite abscissa are acquired as symmetric pixel points, such as pixel points A (-x1, y1) and pixel points B in FIG. 2e ( X1, y1).

The number of symmetric pixel points having the same gray value is counted and the average of the total number of pixel points included in the two sub-image areas is obtained.

The similarity ratio P2 of the two sub-image regions may be determined according to a ratio of the number of symmetric pixel points having the same gray value to the average of the total number of pixel points. Among them, the calculation formula of the similarity rate can be as follows:

Where M is the number of symmetric pixel points with the same gray value, and Ci is the number of pixel points of the i-th black region.

For the total number of pixels included in the two sub-image regions, n is the number of black image regions contained in the two sub-image regions.

If the similarity rate is greater than a set similarity threshold, it is determined that the black image area is symmetrical along a longitudinal center axis of the screen shot image. Wherein, the set similarity threshold may be 99%, that is, when P2 is greater than 99%, it is determined that the black image area is symmetrical along the longitudinal central axis of the screen image.

In this embodiment, it is first determined whether the four corners of the screen image of the application to be identified are black image blocks, and then the symmetry of the screen image is further determined, thereby determining whether the screen image includes a double-screen barrel distortion. Judging based on the barrel distortion of a double screen on a screen capture image As a result, it can be determined whether the application to be identified is a VR application. This method is not limited by the package name or name of the application, and has high recognition accuracy and high efficiency for the VR application. Furthermore, when the user uses a non-VR application on the VR device, the non-VR application can be quickly identified and the user is promptly reminded to enhance the user experience.

3 is a schematic structural diagram of an electronic device according to an embodiment of the present invention. Referring to FIG. 3, the electronic device includes a memory 301 and a processor 302.

The memory 301 is configured to: store one or more instructions.

The processor 302 is configured to invoke to execute the one or more instructions for: acquiring, from the four corners of the interface image of the application to be identified, at least four reference image blocks of a specified size; according to the at least four references a gray value of a pixel included in the image block, determining whether the at least four reference image blocks are black image blocks; if the at least four image blocks are black image blocks, according to the pixels included in the interface image a gray value of the point, identifying a black image area in the interface image; determining, according to a gray value of the pixel point included in the black image area, whether the left and right portions of the black image area correspond; if the black image The two parts of the area correspond to each other, and then the application to be identified is determined to be a VR application.

Further, the processor 302 is specifically configured to: acquire, in the at least four reference image blocks, a number of pixels whose gray value is smaller than a specified gray threshold; and calculate the gray value to be smaller than a specified gray threshold. The ratio of the number of pixels to the total number of pixels included in the at least four reference image blocks; if the ratio is greater than or equal to the set ratio threshold, determining that the at least four image blocks are all black image blocks.

Further, the processor 302 is specifically configured to: select, by using four vertices of the interface image, a starting point of an image, and select an image area of a specified size on the interface image to obtain the at least four As the reference image block.

Further, the processor 302 is specifically configured to: in the interface image, obtain an area where a pixel point whose gray value is smaller than the specified gray level threshold is used as a suspicious black image area; according to the suspect black image The coordinates of the pixel points included in the region, and the geometry of the suspect black image region is obtained. An envelope map; if the geometric envelope map has an apex angle coincident with a vertex angle of the interface image or a longitudinal central axis of the geometric envelope map coincides with a longitudinal central axis of the interface image, determining the The suspicious black image area belongs to the black image area of the interface image.

Further, determining whether the left and right portions of the black image area correspond to each other according to the gray value of the pixel point included in the black image area comprises: according to the gray value of the pixel point included in the black image area, It is determined whether the black image area is symmetrical along a longitudinal center axis of the interface image.

Further optionally, the processor 302 is specifically configured to: divide the black image area into two left and right sub-image areas along a longitudinal central axis of the interface image; and total pixels according to the two sub-image areas Calculating a similarity rate of the two sub-image regions by the number of symmetric pixel points having the same number and gray value; and determining the black image region along the longitudinal direction of the interface image if the similarity ratio is greater than a set similarity threshold The central axis is symmetrical.

Further, the processor 302 is specifically configured to: establish a coordinate system by using a horizontal central axis of the interface image as an axis of abscissa and a longitudinal axis of the interface image as an ordinate axis; In the sub-image area, obtain pixel points with the same ordinate and opposite abscissa as symmetric pixel points; count the number of symmetric pixel points with the same gray value and obtain the average of the total number of pixels included in the two sub-image areas a value; determining a similarity ratio of the two sub-image regions according to a ratio of the number of symmetric pixel points having the same gray value to the average of the total number of pixel points.

In this embodiment, by determining whether the four corners of the interface image of the application to be identified are all black image blocks to initially screen the non-VR application program, it is further determined whether the left and right portions of the interface image correspond. Further, when it is determined that the left and right portions of the interface image correspond, the application to be identified is determined to be a VR application. This method is not limited by the package name or name of the application, and the recognition efficiency of the VR application is high.

The electronic device provided by some embodiments of the present invention may be an external head mounted display device or an integrated head mounted display device, wherein the external head mounted display device needs to be used in conjunction with an external processing system (eg, a computer processing system).

FIG. 4 shows a schematic diagram of the internal configuration of the head mounted display device 400 in some embodiments.

The display unit 401 may include a display panel that is disposed on the head mounted display device 400 The side surface facing the user's face may be a whole panel or a left panel and a right panel corresponding to the left and right eyes of the user, respectively. The display panel may be an electroluminescence (EL) element, a liquid crystal display or a microdisplay having a similar structure, or a laser-scanned display in which the retina may be directly displayed or similar.

The virtual image optical unit 402 photographs the image displayed by the display unit 401 in an enlarged manner and allows the user to observe the displayed image in the enlarged virtual image. As the display image outputted to the display unit 401, it may be an image of a virtual scene supplied from a content reproduction device (a Blu-ray disc or a DVD player) or a streaming server, or an image of a real scene photographed using an external camera 410. In some embodiments, virtual image optical unit 402 can include a lens unit, such as a spherical lens, an aspheric lens, a Fresnel lens, and the like.

The input operation unit 403 includes at least one operation member for performing an input operation, such as a button, a button, a switch, or other similarly functioned component, receives a user instruction through the operation member, and outputs an instruction to the control unit 407.

The status information acquisition unit 404 is configured to acquire status information of the user wearing the head mounted display device 400. The status information acquisition unit 404 may include various types of sensors for detecting status information by itself, and may acquire status information from an external device such as a smartphone, a wristwatch, and other multi-function terminals worn by the user through the communication unit 405. The status information acquisition unit 404 can acquire location information and/or posture information of the user's head. The status information acquisition unit 404 may include one or more of a gyro sensor, an acceleration sensor, a global positioning system (GPS) sensor, a geomagnetic sensor, a Doppler effect sensor, an infrared sensor, and a radio frequency field intensity sensor. Further, the state information acquisition unit 404 acquires state information of the user wearing the head-mounted display device 400, for example, acquires, for example, an operation state of the user (whether the user wears the head-mounted display device 400), an action state of the user (such as standing, walking, running) And the state of movement such as the state of the hand or fingertip, the open or closed state of the eye, the direction of the line of sight, the size of the pupil, the mental state (whether the user is immersed in observing the displayed image, and the like), or even the physiological state.

The communication unit 405 performs communication processing, modulation and demodulation processing with an external device, and encoding and decoding processing of the communication signal. In addition, the control unit 407 can be outward from the communication unit 405 The device transmits transmission data. The communication method may be wired or wireless, such as mobile high-definition link (MHL) or universal serial bus (USB), high-definition multimedia interface (HDMI), wireless fidelity (Wi-Fi), Bluetooth communication, or low-power Bluetooth communication. And the mesh network of the IEEE802.11s standard. Additionally, communication unit 405 can be a cellular wireless transceiver that operates in accordance with Wideband Code Division Multiple Access (W-CDMA), Long Term Evolution (LTE), and the like.

In some embodiments, the head mounted display device 400 can also include a storage unit, which is a mass storage device configured to have a solid state drive (SSD) or the like. In some embodiments, storage unit 406 can store applications or various types of data. For example, content viewed by the user using the head mounted display device 400 may be stored in the storage unit 406.

In some embodiments, the head mounted display device 400 can also include a control unit, and the control unit 407 can include a computer processing unit (CPU) or other device having similar functionality. In some embodiments, control unit 407 can be used to execute an application stored by storage unit 406, or control unit 407 can also be used to perform the methods, functions, and operations disclosed in some embodiments of the present application.

The image processing unit 408 is for performing signal processing such as image quality correction related to the image signal output from the control unit 407, and converting its resolution into a resolution according to the screen of the display unit 401. Then, the display driving unit 409 sequentially selects each row of pixels of the display unit 401, and sequentially scans each row of pixels of the display unit 401 line by line, thereby providing pixel signals based on the signal-processed image signals.

In some embodiments, the head mounted display device 400 can also include an external camera. The external camera 410 may be disposed on the front surface of the body of the head mounted display device 400, and the external camera 410 may be one or more. The external camera 410 can acquire three-dimensional information and can also be used as a distance sensor. Additionally, a position sensitive detector (PSD) or other type of distance sensor that detects reflected signals from the object can be used with the external camera 410. The external camera 410 and the distance sensor can be used to detect the body position, posture, and shape of the user wearing the head mounted display device 400. In addition, under certain conditions, the user can directly view or preview the real scene through the external camera 410.

In some embodiments, the head mounted display device 400 may further include a sound processing unit, the sound portion The processing unit 411 can perform sound quality correction or sound amplification of the sound signal output from the control unit 407, signal processing of the input sound signal, and the like. Then, the sound input/output unit 412 outputs the sound to the outside and the sound from the microphone after the sound processing.

It should be noted that the structure or component illustrated by the dashed line in FIG. 4 may be independent of the head mounted display device 400, for example, may be disposed in an external processing system (eg, a computer system) for use with the head mounted display device 400; or The structure or components shown by the dashed box may be disposed inside or on the surface of the head mounted display device 400.

The embodiments of the electronic device described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, ie Located in one place, or distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the embodiment. Those of ordinary skill in the art can understand and implement without deliberate labor.

Through the description of the above embodiments, those skilled in the art can clearly understand that the various embodiments can be implemented by means of software plus a necessary general hardware platform, and of course, by hardware. Based on such understanding, the above-described technical solutions may be embodied in the form of software products in essence or in the form of software products, which may be stored in a computer readable storage medium such as ROM/RAM, magnetic Discs, optical discs, etc., include instructions for causing a computer device (which may be a personal computer, server, or network device, etc.) to perform the methods described in various embodiments or portions of the embodiments.

It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not limited thereto; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that The technical solutions described in the foregoing embodiments are modified, or the equivalents of the technical features are replaced. The modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

A method for identifying a VR application, comprising:

Obtaining at least four reference image blocks of a specified size from four corners of the interface image of the application to be identified;

Determining, according to the gray value of the pixel point included in the at least four reference image blocks, whether the at least four reference image blocks are black image blocks;

If the at least four image blocks are black image blocks, identifying a black image region in the interface image according to a gray value of a pixel point included in the interface image;

Determining, according to the gray value of the pixel point included in the black image area, whether the left and right portions of the black image area correspond to each other;

If the left and right portions of the black image area correspond, it is determined that the to-be-identified application is a VR application.
The method according to claim 1, wherein determining whether the at least four reference image blocks are black image blocks according to the gray value of the pixel points included in the at least four reference image blocks comprises:

Obtaining, in the at least four reference image blocks, a number of pixel points whose gray value is smaller than a specified gray threshold;

Calculating a ratio of a number of pixel points whose gray value is less than a specified gray level threshold to a total number of pixel points included in the at least four reference image blocks;

If the ratio is greater than or equal to the set ratio threshold, it is determined that the at least four image blocks are black image blocks.
The method according to claim 2, wherein acquiring at least four reference image blocks of a specified size from the four corners of the interface image of the application to be identified comprises:

The four vertices of the interface image are respectively selected starting points of the image, and the image regions of the specified size are selected on the interface image to obtain the at least four reference image blocks.
The method of claim 3 wherein said specified size is said interface One tenth of the size of the image.
The method according to claim 1, wherein the identifying the black image area in the interface image according to the gray value of the pixel point included in the interface image comprises:

In the interface image, an area where a pixel point whose gradation value is smaller than the specified gradation threshold is obtained as a suspicious black image area;

Obtaining a geometric envelope map of the suspect black image region according to coordinates of the pixel points included in the suspect black image region;

Determining, according to the geometric envelope map, whether the suspect black image area belongs to a black image area of the interface image.
The method according to claim 5, wherein determining whether the suspicious black image area belongs to the black image area of the interface image according to the geometric envelope map comprises:

Determining the suspect black image area if the geometric envelope map has an apex angle coincident with a vertex angle of the interface image or a longitudinal central axis of the geometric envelope image coincides with a longitudinal central axis of the interface image A black image area belonging to the interface image.
The method according to any one of claims 1 to 6, wherein determining whether the left and right portions of the black image area correspond to each other according to a gray value of a pixel point included in the black image area comprises:

And determining whether the black image area is symmetrical along a longitudinal central axis of the interface image according to a gray value of a pixel point included in the black image area.
The method according to claim 7, wherein determining whether the black image area is symmetrical along a longitudinal central axis of the interface image according to a gray value of a pixel point included in the black image area comprises:

Dividing the black image area into two left and right sub-image areas along a longitudinal central axis of the interface image;

Calculating a similarity ratio of the two sub-image regions according to a total number of pixels included in the two sub-image regions and a number of symmetric pixel points having the same gray value in the two sub-image regions;

Determining the black image area along the interface if the similarity rate is greater than a set similarity threshold The longitudinal center axis of the image is symmetrical.
The method according to claim 8, wherein the two are calculated according to the total number of pixels included in the two sub-image regions and the number of symmetric pixel points having the same gray value in the two sub-image regions. The similarity rate of sub-image areas, including:

Taking a horizontal central axis of the interface image as an abscissa axis, and a longitudinal central axis of the interface image as a ordinate axis, establishing a coordinate system;

Obtaining, in the two sub-image regions, pixel points having the same ordinate and opposite abscissa as symmetric pixel points;

Counting the number of symmetric pixel points having the same gray value, and obtaining an average of the total number of pixel points included in the two sub-image areas;

And determining a similarity ratio of the two sub-image regions according to a ratio of the number of symmetric pixel points having the same gray value to the average value.
An electronic device, comprising: a memory and a processor;

The memory is configured to: store one or more computer instructions;

The processor is configured to execute the one or more computer instructions for:

Obtaining at least four reference image blocks of a specified size from four corners of the interface image of the application to be identified;

Determining, according to the gray value of the pixel point included in the at least four reference image blocks, whether the at least four reference image blocks are black image blocks;

If the at least four image blocks are black image blocks, identifying a black image region in the interface image according to a gray value of a pixel point included in the interface image;

Determining, according to the gray value of the pixel point included in the black image area, whether the left and right portions of the black image area correspond to each other;

If the left and right portions of the black image area correspond, it is determined that the to-be-identified application is a VR application.
The electronic device according to claim 10, wherein the processor is specifically configured to:

Obtaining, in the at least four reference image blocks, a number of pixel points whose gray value is smaller than a specified gray threshold;

Calculating a ratio of a number of pixel points whose gray value is less than a specified gray level threshold to a total number of pixel points included in the at least four reference image blocks;

If the ratio is greater than or equal to the set ratio threshold, it is determined that the at least four image blocks are black image blocks.
The electronic device according to claim 10, wherein the processor is specifically configured to:

In the interface image, an area where a pixel point whose gradation value is smaller than the specified gradation threshold is obtained as a suspicious black image area;

Obtaining a geometric envelope map of the suspect black image region according to coordinates of the pixel points included in the suspect black image region;

Determining the suspect black image area if the geometric envelope map has an apex angle coincident with a vertex angle of the interface image or a longitudinal central axis of the geometric envelope image coincides with a longitudinal central axis of the interface image A black image area belonging to the interface image.