CN116721376A

CN116721376A - Scene display guiding method, device and storage medium

Info

Publication number: CN116721376A
Application number: CN202310273993.9A
Authority: CN
Inventors: 李春艳
Original assignee: Shenzhen Sensetime Technology Co Ltd
Current assignee: Shenzhen Sensetime Technology Co Ltd
Priority date: 2023-03-13
Filing date: 2023-03-13
Publication date: 2023-09-08

Abstract

The application discloses a scene display guiding method, equipment and a storage medium. The scene display guiding method comprises the steps of acquiring scene images in a scene space by using an image sensor; identifying image feature points in the scene image to obtain the distribution condition of the image feature points in the scene image; based on the distribution of the image feature points in the scene image, guiding information that changes the position of the image sensor in the scene space is determined. By the above embodiment, the region of interest in the scene is automatically analyzed from the distribution of the image feature points in the recognized scene image, and the position of the image sensor in the scene space is changed by using the guide information, and the region of interest in the scene is tracked.

Description

Scene display guiding method, device and storage medium

Technical Field

The present application relates to the field of image processing technologies, and in particular, to a scene display guidance method, apparatus, and storage medium.

Background

Augmented reality (Augmented Reality) technology is a technology that skillfully merges virtual information with the real world. The method widely uses various technical means such as multimedia, three-dimensional modeling, real-time tracking, registration, intelligent interaction, sensing and the like. The augmented reality is implemented by applying virtual information such as characters, images, three-dimensional models, music, video and the like generated by a computer to the real world after simulation so that the two kinds of information are mutually complemented, thereby realizing the enhancement of the real world.

The augmented reality technology has been developed for many years, and it is expected to be able to use the technology in daily life and work in one day, provide convenience for life, improve working efficiency, etc., and many working scenes have appeared at present, and along with the progress of the technology, there is also a global positioning method based on multi-sensor fusion such as visual high-precision map, signal positioning, etc., and various science fiction concepts such as digital twinning, parallel world, etc. are gradually possible.

In the prior art, situations that feature points are too few to be positioned successfully and the like may occur in an image acquired by a camera at present, so that the final presented effect is poor.

Disclosure of Invention

In order to solve the above problems in the prior art, the present application provides a scene display guidance method, apparatus, and storage medium.

In order to solve the technical problems in the prior art, the application provides a scene display guiding method, which comprises the following steps: acquiring a scene image in a scene space using an image sensor; identifying image feature points in the scene image to obtain the distribution condition of the image feature points in the scene image; and determining guide information for changing the position of the image sensor in the scene space based on the distribution condition of the image characteristic points in the scene image.

In order to solve the technical problems existing in the prior art, the application provides a scene display guiding device, which comprises: a processor and a memory, the memory storing a computer program, the processor being configured to execute the computer program to implement the method described above.

To solve the technical problems existing in the prior art, the present application provides a computer readable storage medium storing program instructions that when executed by a processor implement the above-mentioned method.

Compared with the prior art, the scene display guiding method comprises the steps of acquiring scene images in a scene space by using an image sensor; identifying image feature points in the scene image to obtain the distribution condition of the image feature points in the scene image; based on the distribution of the image feature points in the scene image, guiding information that changes the position of the image sensor in the scene space is determined. By the above embodiment, from the distribution situation of the image feature points in the identified scene image, the region of interest in the scene is automatically analyzed, and the position of the image sensor in the scene space is changed by using the guiding information to track the region of interest in the scene.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application as claimed.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flow chart of a scene display guidance method according to an embodiment of the present application;

FIG. 2 is a flowchart of step S101 in FIG. 1;

FIG. 3 is a flowchart illustrating the step S102 in FIG. 1;

FIG. 4 is a flowchart of step S103 in FIG. 1;

FIG. 5 is a flowchart of step S403 in FIG. 4;

FIG. 6 is a schematic structural diagram of an embodiment of a scene display guiding device according to the present application;

FIG. 7 is a schematic diagram of an embodiment of a scene display guidance device according to the present application;

fig. 8 is a schematic structural diagram of an embodiment of a computer storage medium provided by the present application.

Detailed Description

The application is described in further detail below with reference to the drawings and examples. It is specifically noted that the following examples are only for illustrating the present application, but do not limit the scope of the present application. Likewise, the following examples are only some, but not all, of the examples of the present application, and all other examples, which a person of ordinary skill in the art would obtain without making any inventive effort, are within the scope of the present application.

Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments.

In the description of the present application, it should be noted that, unless explicitly stated and limited otherwise, the terms "mounted," "disposed," "connected," and "connected" are to be construed broadly, and may be, for example, fixedly connected, detachably connected, or integrally connected; the connection can be mechanical connection or electric connection; may be directly connected or may be connected via an intermediate medium. It will be apparent to those skilled in the art that the foregoing is in the specific sense of the present application.

The present disclosure relates to the field of augmented reality, and more particularly, to the field of augmented reality, in which, by acquiring image information of a target object in a real environment, detection or identification processing of relevant features, states and attributes of the target object is further implemented by means of various visual correlation algorithms, so as to obtain an AR effect combining virtual and reality matching with a specific application. By way of example, the target object may relate to a face, limb, gesture, action, etc. associated with a human body, or a marker, a marker associated with an object, or a sand table, display area, or display item associated with a venue or location, etc. Vision related algorithms may involve vision localization, SLAM, three-dimensional reconstruction, image registration, background segmentation, key point extraction and tracking of objects, pose or depth detection of objects, and so forth. The specific application not only can relate to interactive scenes such as navigation, explanation, reconstruction, virtual effect superposition display and the like related to real scenes or articles, but also can relate to interactive scenes such as makeup beautification, limb beautification, special effect display, virtual model display and the like related to people.

The detection or identification processing of the relevant characteristics, states and attributes of the target object can be realized through a convolutional neural network. The convolutional neural network is a network model obtained by performing model training based on a deep learning framework.

Based on the above technical basis, the present application provides a scene display guiding method, referring to fig. 1, fig. 1 is a schematic flow chart of an embodiment of the scene display guiding method provided by the present application. Specifically, the following steps S101 to S103 may be included.

Step S101: a scene image is acquired in a scene space using an image sensor.

The image sensor, which may be a camera, is located in the display device, with which the display device may acquire images of the scene. Wherein the display device may be any electronic device capable of supporting AR functionality including, but not limited to, AR glasses, tablet computers, smartphones, etc. The AR effect may be displayed in the display device, which may be understood as displaying a virtual object fused to a real scene in the display device, or may be directly rendering the display content of the virtual object and fusing with the real scene, for example, displaying a group of virtual buildings, where the display effect is a real building placed in the real scene, or may be displaying a fused display screen after fusing the display content of the virtual object with the real scene screen. In this embodiment, the display device may be a display device such as a mobile terminal.

More environmental information may be included in the scene space, which may include indoor scene space or outdoor scene space, for example. By way of example, when an outdoor scene space is included, the outdoor scene space may include a highway, a building on both sides of the highway, a plant between the highway and the building, and the like. When included, the indoor scene space may be an elevator, corridor, light fixture, furniture, store, or the like.

A scene image is a picture taken in scene space with a camera having part of the scene content in scene space.

Step S102: and identifying image feature points in the scene image to obtain the distribution condition of the image feature points in the scene image.

The image feature points may be image corner points in the scene image, and may be identified from the scene image based on a preset identification algorithm, for example, when the image feature points include image corner points, the image corner points may be identified from the scene image by a corner detection algorithm, for example, the corner detection algorithm includes corner detection based on a gray scale image, corner detection based on a binary image, and corner detection based on a contour curve. The detection of the corner points based on the gray level image can be divided into 3 types of methods based on gradient, based on a template and based on the template gradient combination, wherein the method based on the template mainly considers the gray level change of the points in the pixel field, namely the change of the brightness of the image, and the points with enough contrast with the brightness of the adjacent points are defined as the corner points. Common corner detection algorithms based on templates are a Kitchen-Rosenfeld corner detection algorithm, a Harris corner detection algorithm, a KLT corner detection algorithm and a SUSAN corner detection algorithm. Compared with other corner detection algorithms, the SUSAN corner detection algorithm has the characteristics of simple algorithm, accurate position, strong noise resistance and the like.

After identifying the image feature points from the scene image, the distribution of the image feature points in the scene image may be obtained based on the locations of all the identified image feature points in the scene image, e.g. the image feature points are mostly distributed in the middle of the scene image or in other areas of the scene image.

Step S103: based on the distribution of the image feature points in the scene image, guiding information that changes the position of the image sensor in the scene space is determined.

In some scenes, for example, in an AR interactive game or an AR navigation application, positioning operation needs to be performed in the use process of the application, but when the number of image feature points in the scene is small, positioning is difficult to succeed, and at this time, a user is required to move the position of the image sensor to re-identify the scene image with more image feature points, so that positioning operation is finally realized. However, in the immersive experience process of the user, if the user is not prompted to move the guiding information of the position of the image sensor, it is difficult to quickly align the image sensor to the region with more image feature points in the scene space, which can seriously affect the use experience of the user.

The guiding information may be embodied in the form of text information or voice broadcast, and may be sent to prompt the user to move the image sensor in the direction of 12 o ' clock or prompt the user to adjust the angle of the image sensor so that the image sensor faces in the direction of 12 o ' clock when the image of the scene is positioned with more image feature points in the direction of 12 o ' clock, for example. The method comprises the steps of predicting the region with rich feature points in the scene space according to the distribution condition of the feature points in the scene image, and prompting a user to move the image sensor to the region with rich feature points by using guide information so as to display the scene region with more image feature points, thereby avoiding the possibility that the image feature points in the current scene image are out of compliance with conditions and cannot be subjected to subsequent positioning and other operations.

In one embodiment, a scene display guidance method includes: determining a virtual display space based on a position of the image sensor when the scene image is acquired; image feature points are displayed in the virtual display space.

The virtual display space may be established by acquiring a high-precision map, and, illustratively, the virtual display space may be established based on the high-precision map by acquiring a high-precision map of the scene space. The high-precision map data may be based on an image captured of the actual environment in which the scene space is located, which may be captured by a smart vision sensor or other type of camera. The number of the images may be multiple, for example, 10 images, 20 images, 50 images, etc., and the more the number of the images, the more accurate the virtual display space is constructed, but the more the time for constructing the virtual space is relatively, the images may be three-dimensional map data, and the three-dimensional map data may be three-dimensional point cloud.

The virtual display space may be determined by the position of the image sensor in the scene space, wherein the position of the virtual display space is determined to be the same as the position at which the scene image was acquired with the sensor. For example, when a user is located in a scene space while using the mobile terminal as a display device and holds the mobile terminal in his hand, a virtual display space may be generated within the angle of view of the image sensor and a scene image may be acquired at the point to analyze image feature points in the scene image using the acquired scene image. After the image characteristic points in the scene image are obtained, the characteristic points can be displayed at the positions corresponding to the virtual display space, so that a user can conveniently display the image characteristic points through the virtual display space, and can know what is the image characteristic points more easily, the learning cost of the user for using the virtual display space is reduced, the application experience of the user for the virtual display space is enhanced to a certain extent, and the interest in entertainment by using the virtual display space is effectively improved.

In one embodiment, the step of displaying the image feature points in the virtual display space includes: comparing the distance between the image feature points in the scene image with a preset distance threshold; merging a plurality of image feature points with the distance smaller than a preset distance threshold value into a fusion feature point; and commonly displaying the fusion characteristic points and the image characteristic points in a virtual display space.

When the number of the image feature points is large, if the image feature points are directly displayed in the scene image irregularly, the display effect of the final presentation in the scene image is poor. Therefore, after the image feature points are identified in the scene image, the distance between the image feature points can be acquired, then a plurality of image feature points with the distance smaller than a preset distance threshold value are combined to form a fusion feature point, and the image feature points and the fusion feature points are displayed at the same time. Wherein, the expression form of the fusion characteristic points can be different from the expression form of the image characteristic points, and the color of the fusion characteristic points can be different from the color of the image characteristic points, and can be determined based on the number of the fusion characteristic points, and the more the number of the fusion characteristic points is, the darker the color of the fusion characteristic points is; the shape of the fused feature points may be different from the shape of the image feature points, for example, the image feature points are represented by a circle shape, and the shape of the fused feature points are represented by a triangle or the like.

In one embodiment, a scene display guidance method includes: generating a prompt animation according to the guide information; the cue animation is displayed in the virtual display space.

When the guiding information may include guiding the user to move the image sensor in an up, down, left, right or other direction, a prompt animation may be generated and displayed, and the prompt animation may prompt the user to move the image sensor in a direction determined by the guiding information in a dynamic image form, for example, in a form of moving an arrow in a certain direction, and prompt the user to move the image sensor in a direction determined by the guiding information. When the guiding information can include guiding the user to incline the image sensor in the up, down, left, right or other directions so as to change the orientation of the image sensor, a prompting animation can be generated and displayed, and the prompting animation can be in the form of dynamic images to prompt the user to rotate the image sensor in the direction determined by the guiding information. Therefore, the position of the image sensor is prompted to move by the user in an animation mode, so that the interestingness of the user in the using process is improved.

Referring to fig. 2, fig. 2 is a flowchart of step S101 in fig. 1 according to an embodiment. Specifically, the method includes the following steps S201 to S203.

Step S201: multiple frames of selected images are acquired in a scene space using an image sensor.

The selected images of the multiple frames can be obtained by acquiring different points of the image sensor in the scene space. When the mobile terminal is used as the display device, the user holds the mobile terminal in the scene space, and obtains the first frame selection image when the mobile terminal is located at a certain point in the scene space, but the first frame selection image may not meet the expectations, that is, it cannot be ensured that one frame selection image obtained each time is a scene image meeting the conditions, and at this time, the image sensor can be used to obtain other selection images at different points so as to select the scene image most meeting the conditions from the selection images of multiple frames.

Step S202: and identifying the image characteristic points of the multi-frame selected image to obtain the quantity of the characteristic points in each frame of selected image.

The characteristic point recognition algorithm can be utilized to respectively recognize the image characteristic points of each frame of selected image from multiple frames of selected images, and the characteristic point quantity of the image characteristic points of each frame of selected image is calculated. For example, when the image feature points comprise image corners, the image corners may be identified from the scene image by a corner detection algorithm. For example, the corner detection algorithm comprises corner detection based on gray level images, corner detection based on binary images and corner detection based on contour curves.

Step S203: selecting the selected images with the number of the characteristic points being greater than or equal to a preset number threshold as scene images.

After the number of the characteristic points of each frame of the selected image is identified, the number of the characteristic points in each frame of the selected image can be compared with a preset number threshold value, and then a comparison result is obtained. And then selecting at least one frame of selected images with the number of the characteristic points larger than or equal to a preset number threshold value from the multi-frame selected images as scene images, so that the selected scene images can be used for accurately and conveniently obtaining the guide information.

Referring to fig. 3, fig. 3 is a flowchart of an embodiment of step S102 in fig. 1, specifically, the following steps S301 to S304 are included.

Step S301: image feature points in the scene image are identified.

Under the condition that the number of the image characteristic points in the identified scene image is larger than a first preset threshold value, the identification accuracy of the identified image characteristic points can be improved, so that the number of the identified image characteristic points is reduced; and under the condition that the number of the image characteristic points in the scene image is smaller than a second preset threshold value, the identification accuracy of the image characteristic points can be adjusted downwards, so that the number of the identified image characteristic points is increased, wherein the second preset threshold value is smaller than the first preset threshold value.

Step S302: dividing the scene image into a plurality of image areas according to a preset dividing rule.

The preset dividing rule includes dividing the scene image into a plurality of image areas in a form of a nine-grid, or dividing the scene image into a plurality of image areas in a form of a time clock, and in other embodiments, the preset dividing rule may also include other forms, which are not limited herein.

The step S302 and the step S301 may be performed in no order, that is, the scene image may be divided into a plurality of image areas according to a preset dividing rule, and then the image feature points in the scene image are identified.

Step S303: the number of image feature points in each image area is calculated separately.

After identifying the image feature points in the scene image and dividing the scene image into a plurality of image regions, the number of image feature points in each image region may also be determined, the number of image feature points in each image region being the sum of all image feature points in the region.

Step S304: based on the number of image feature points within each image region, the distribution of the image feature points in the scene image is determined.

After the number of the image feature points in each image area is obtained, a conclusion can be obtained that the number of the identified image feature points is the largest in which image area and the number of the identified image feature points is the smallest in which image area, so that the obtained distribution situation of the image feature points in the scene image is utilized to execute subsequent steps, and the accuracy of the distribution situation of the image feature points in the scene image is improved.

In the present embodiment, the step of determining the guide information for changing the position of the image sensor in the scene space based on the distribution situation of the image feature points in the scene image (step S103) includes: comparing the number of image feature points in the plurality of image areas, and determining the image area with the largest number of image feature points in the scene image; determining a moving direction of a center position of the scene image to an image area with the largest number of image feature points based on the position information; guide information for changing a position of the image sensor in the scene space is determined based on the moving direction.

After the number of image feature points in each image area is calculated, the number of image feature points in each image area can be compared to determine the image area with the largest number of at least one image feature point in the scene image, and after the position information of the image area with the largest number of image feature points is determined, the guiding information for changing the movement of the image sensor towards the position information can be sent to the user. Illustratively, taking an example of dividing the scene image into nine image areas arranged in a nine-grid form, and the image area with the largest number of image feature points in the image area in the upper left corner among the nine image areas is determined to be located in the upper left corner of the scene image, at this time, the image area can be moved in the position direction of the upper left corner according to the center position of the scene image, so as to obtain a moving direction, and further obtain guiding information for changing the position of the image sensor in the scene space, wherein the guiding information at this time includes moving the image sensor in the upper left corner direction, or rotating the image sensor in the direction of the upper left corner.

The above embodiment is to divide a scene image into a plurality of image areas after acquiring one scene image, thereby determining the distribution of image feature points in the scene graph, and in other embodiments, it is also possible to determine the guidance information by using a plurality of images. Referring to fig. 4, fig. 4 is a flowchart of step S103 in fig. 1 according to an embodiment. Specifically, the following steps S401 to S403 may be included.

Step S401: a predicted image is acquired with an image sensor at a location where the image of the scene is acquired, wherein the angle of view of the image sensor when acquiring the predicted image is different from the angle of view when acquiring the image of the scene.

The image sensor itself has a field angle, the size of the field angle determines the field of view of the image sensor, the larger the field angle is, the larger the field of view is, and the image sensor can adjust the field angle within the field of view to acquire images. Illustratively, the field angle range of the image sensor is between 0 and 60 degrees, at which time the adjustable image sensor acquires a first frame image at a field angle of 60 degrees, the first frame image being available as a scene image as described above, and then the readjustment image sensor acquires a second frame image at a field angle of 50 degrees, the second frame image being available as a predicted image. Wherein the scene image and the predictive image may be acquired at the same location in the scene space.

It should be noted that the same position here is not the same point in an absolute sense, but a range value of one position, that is, the position of the image sensor when acquiring the predicted image as well as the scene image may float in a small range.

Step S402: and identifying image characteristic points in the predicted image to obtain the distribution condition of the image characteristic points in the predicted image.

The image feature points in the predicted image may be image corner points, and the image feature points may be identified from the predicted image based on a preset identification algorithm, for example, when the image feature points include image corner points, the image corner points may be identified from the predicted image by a corner detection algorithm, for example, the corner detection algorithm includes corner detection based on a gray image, corner detection based on a binary image, and corner detection based on a contour curve.

After the image feature points are identified from the predicted image, the distribution situation of the image feature points in the predicted image can be obtained based on the positions of all the identified image feature points in the predicted image, for example, the image feature points are distributed in the middle of the predicted image or distributed in other areas of the predicted image.

Step S403: the guiding information that changes the position of the image sensor in the scene space is determined based on the distribution of the image feature points in the scene image and the distribution of the image feature points in the predicted image.

The guiding information may be embodied in the form of text information or voice broadcasting, and in this embodiment, the distribution condition of the image feature points in the scene image and the distribution condition of the image feature points in the prediction image are considered at the same time, so as to more accurately predict the region with rich feature points in the scene space, and the guiding information is used to prompt the user to move the image sensor to the region with rich feature points so as to display the scene region with more image feature points, thereby avoiding the possibility that the image feature points in the current scene image do not meet the conditions, and thus the subsequent positioning and other operations cannot be performed.

Referring to fig. 5, fig. 5 is a flowchart of step S403 in fig. 4. Specifically, the following steps S501 to S506 may be included.

Step S501: and determining a first position in the predicted image based on the distribution condition of the image feature points in the predicted image, wherein the first position is a point position in an image area with the largest number of the image feature points in the predicted image.

In this embodiment, the manner of determining the first position in the predicted image may be similar to the case of determining the distribution of the image feature points in the scene image in the embodiment shown in fig. 3, that is, the image feature points in the predicted image may be identified first; then dividing the predicted image into a plurality of image areas according to a preset dividing rule; respectively calculating the number of image characteristic points in each image area; comparing the number of the image characteristic points in the plurality of image areas to determine the image area with the largest number of the image characteristic points; then selecting a point from the image area with the largest number of image feature points as a first position, wherein the first position can be any point selected from the image area by way of example; or an image feature point selected from the image area; or the center of an image area selected from the image areas.

Step S502: and determining a second position in the scene image based on the distribution condition of the image feature points in the scene image, wherein the second position is a point position in an image area with the largest number of the image feature points in the scene image.

In this embodiment, the manner of determining the second position in the scene image may be similar to the case of determining the distribution of the image feature points in the scene image in the embodiment shown in fig. 3, that is, the image feature points in the scene image may be identified first; dividing the scene image into a plurality of image areas according to a preset dividing rule; respectively calculating the number of image characteristic points in each image area; comparing the number of the image feature points in the plurality of image areas to determine the image area with the largest number of the image feature points, and then selecting one point from the image area with the largest number of the image feature points as a second position, wherein the second position can be any point selected from the image areas by way of example; or an image feature point selected from the image area; or the center of an image area selected from the image areas, etc.

Step S503: a first location in the predicted image is mapped into the scene image.

In this embodiment, the angle of view of the image sensor when acquiring the predicted image may be smaller than the angle of view when acquiring the scene image; and the image sensor acquires the predicted image at the position where the scene image is acquired, namely, the image content in the predicted image can be embodied in the scene image. After determining the first location, the first location may be mapped in the scene image, resulting in the first location in the scene image.

Step S504: a third location is determined in the scene image based on the mapped first location and the second location.

The first position and the second position in the mapped scene image are at different positions of the scene image, that is, the first position and the second position are spaced from each other, and the third position can be further determined based on the first position and the second position. The third position may be a position of a midpoint of a line connecting the first position and the second position. Or a third location may be determined based on the weights of the scene image and the predicted image, e.g., the weight of the scene image is greater than the weight of the predicted image, whereby the determined third location is on the line of the first location and the second location and is closer to the second location.

Step S505: and determining a prediction direction from the central position of the scene image to the third position by using the third position and the central position of the scene image.

After the third position is determined, the predicted direction may be based on a predicted direction from the center position to the third position of the scene image, e.g., the direction of 12 o 'clock of the center position, where the predicted direction is the direction moving from the center position to the direction of 12 o' clock.

Step S506: guide information that changes the position of the image sensor in the scene space is determined based on the predicted direction.

After the prediction direction is determined, the guide information can be determined based on the prediction direction, so that a user can change the position of the image sensor based on the guide information, the first position and the second position can be used for determining the third position, the obtained prediction direction is more accurate, the user can change the position of the image sensor based on the guide information, the image sensor faces the region with rich feature points in the scene space, and the possibility that the image feature points in the current scene image are out of condition, and subsequent operations (such as positioning based on the feature points) cannot be performed can be avoided.

The scene display guiding method in the embodiment can be applied to a scene display guiding device, and the scene display guiding device can be a server, mobile equipment or a system formed by mutually matching the server and the mobile equipment. Accordingly, each part included in the mobile device, for example, each unit, sub-unit, module, and sub-module, may be all disposed in the server, may be all disposed in the mobile device, and may also be disposed in the server and the mobile device, respectively.

Further, the server may be hardware or software. When the server is hardware, the server may be implemented as a distributed server cluster formed by a plurality of servers, or may be implemented as a single server. When the server is software, it may be implemented as a plurality of software or software modules, for example, software or software modules for providing a distributed server, or may be implemented as a single software or software module, which is not specifically limited herein.

In order to realize the scene display guiding method of the embodiment, the application provides a scene display guiding device. Referring to fig. 6, fig. 6 is a schematic structural diagram of an embodiment of a scene display guiding device 60 provided by the present application.

Specifically, the scene display guidance device 60 may include: an acquisition module 61, an identification module 62 and a determination module 63.

The acquisition module 61 is for acquiring a scene image in a scene space using an image sensor.

The identifying module 62 is configured to identify image feature points in the scene image, and obtain a distribution of the image feature points in the scene image.

The determining module 63 is configured to determine, based on the distribution of the image feature points in the scene image, guiding information that changes the position of the image sensor in the scene space.

The above scheme determines the guiding information from the distribution situation of the image feature points in the identified scene image, so that a user can change the position of the image sensor in the scene space through the guiding information, thereby avoiding the possibility that the image feature points in the current scene image are out of condition, and subsequent operation cannot be performed.

In one embodiment of the present application, each module in the scene display guidance device 60 shown in fig. 6 may be separately or completely combined into one or several units, or some unit(s) thereof may be further split into a plurality of sub-units with smaller functions, so that the same operation may be implemented without affecting the implementation of the technical effects of the embodiment of the present application. The above modules are divided based on logic functions, and in practical applications, the functions of one module may be implemented by a plurality of units, or the functions of a plurality of modules may be implemented by one unit. In other embodiments of the present application, the scene display guidance device 60 may also include other units, and in practical applications, these functions may also be implemented with assistance from other units, and may be implemented by cooperation of a plurality of units.

The method is applied to scene display guiding equipment. Referring specifically to fig. 7, fig. 7 is a schematic structural diagram of an embodiment of a scene display guidance device provided by the present application, where the scene display guidance device 70 includes a processor 71 and a memory 72. Wherein the memory 72 stores a computer program, and the processor 71 is configured to execute the computer program to implement the above-described scene display guidance method.

The processor 71 may be an integrated circuit chip, and has signal processing capability. Processor 71 may also be a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

For the scenario display guidance method of the embodiment shown in fig. 1-5, which may be presented in the form of a computer program, the present application proposes a computer storage medium carrying the computer program, please refer to fig. 8, fig. 8 is a schematic structural diagram of an embodiment of the computer storage medium provided by the present application, and the computer storage medium 80 of the present embodiment includes a computer program 81, which may be executed to implement the scenario display guidance method described above.

The computer storage medium 80 of this embodiment may be a medium that may store program instructions, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disc, or may be a server that stores the program instructions, and the server may send the stored program instructions to other devices for execution, or may also self-execute the stored program instructions.

In addition, the above-described functions, if implemented in the form of software functions and sold or used as a separate product, may be stored in a mobile terminal-readable storage medium, i.e., the present application also provides a storage device storing program data that can be executed to implement the method of the above-described embodiments, the storage device may be, for example, a U-disk, an optical disk, a server, or the like. That is, the present application may be embodied in the form of a software product comprising instructions for causing a smart terminal to perform all or part of the steps of the method described in the various embodiments.

In the description of the present application, a description of the terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, the different embodiments or examples described in this specification and the features of the different embodiments or examples may be combined and combined by those skilled in the art without contradiction.

Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In the description of the present application, the meaning of "plurality" means at least two, for example, two, three, etc., unless specifically defined otherwise.

Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and further implementations are included within the scope of the preferred embodiment of the present application in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present application.

Logic and/or steps represented in the flowcharts or otherwise described herein, e.g., may be considered as a ordered listing of executable instructions for implementing logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device (which can be a personal computer, server, network device, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions). For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program is printed, as the program may be electronically captured, via, for instance, optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.

The foregoing description is only of embodiments of the present application, and is not intended to limit the scope of the application, and all equivalent structures or equivalent processes using the descriptions and the drawings of the present application or directly or indirectly applied to other related technical fields are included in the scope of the present application.

Claims

1. A scene display guidance method, characterized by comprising:

acquiring a scene image in a scene space using an image sensor;

identifying image feature points in the scene image to obtain the distribution condition of the image feature points in the scene image;

and determining guide information for changing the position of the image sensor in the scene space based on the distribution condition of the image characteristic points in the scene image.

2. The method according to claim 1, wherein the identifying the image feature points in the scene image to obtain the distribution of the image feature points in the scene image comprises:

identifying image feature points in the scene image;

dividing the scene image into a plurality of image areas according to a preset dividing rule;

respectively calculating the number of the image characteristic points in each image area;

And determining the distribution condition of the image characteristic points in the scene image based on the number of the image characteristic points in each image area.

3. The method of claim 2, wherein the determining, based on the distribution of the image feature points in the scene image, the guidance information that changes the position of the image sensor in the scene space comprises:

determining an image area with the largest number of image feature points in the scene image;

acquiring the position information of the image area with the largest number of the image feature points in the scene image;

determining the moving direction of the central position of the scene image to the image area with the largest number of image feature points based on the position information;

guide information that changes the position of the image sensor in the scene space is determined based on the movement direction.

4. The method of claim 1, wherein acquiring the scene image in the scene space using the image sensor comprises:

acquiring a plurality of frames of selected images in the scene space by utilizing the image sensor;

identifying the image characteristic points of the selected images of a plurality of frames to obtain the quantity of the characteristic points in the selected images of each frame;

And selecting the selected images with the number of the characteristic points being greater than or equal to a preset number threshold as the scene images.

5. The method of claim 1, wherein the determining, based on the distribution of the image feature points in the scene image, the guidance information that changes the position of the image sensor in the scene space comprises:

acquiring a predicted image with the image sensor at a position where the scene image is acquired, wherein a field angle of view of the image sensor when acquiring the predicted image is different from a field angle of view when acquiring the scene image;

identifying image feature points in the predicted image to obtain the distribution condition of the image feature points in the predicted image;

and determining guide information for changing the position of the image sensor in the scene space based on the distribution condition of the image feature points in the scene image and the distribution condition of the image feature points in the prediction image.

6. The method of claim 5, wherein the determining, based on the distribution of image feature points in the scene image and the distribution of image feature points in the predicted image, the guidance information that changes the position of the image sensor in the scene space comprises:

Determining a first position in the predicted image based on the distribution condition of image feature points in the predicted image, wherein the first position is a point position in an image area with the largest number of image feature points in the predicted image;

determining a second position in the scene image based on the distribution condition of the image feature points in the scene image, wherein the second position is a point position in an image area with the largest number of the image feature points in the scene image;

mapping a first location in the predicted image into the scene image;

determining a third location in the scene image based on the mapped first location and the second location;

determining a predicted direction from a center position of the scene image to the third position using the third position and the center position of the scene image;

guide information that changes the position of the image sensor in the scene space is determined based on the predicted direction.

7. The method according to any one of claims 1-6, characterized in that the method comprises:

determining a virtual display space based on a position of the image sensor when the scene image is acquired;

And displaying the image characteristic points in the virtual display space.

8. The method of claim 7, wherein the displaying the image feature points in the virtual display space comprises:

comparing the distance between the image feature points in the scene image with a preset distance threshold;

merging the image feature points with the distance smaller than the preset distance threshold value into a fusion feature point;

and displaying the fusion characteristic points and the image characteristic points in the virtual display space together.

9. The method according to claim 7, characterized in that the method comprises:

generating a prompt animation according to the guide information;

and displaying the prompt animation in the virtual display space.

10. A scene display guidance apparatus, characterized by comprising: a processor and a memory, the memory having stored therein a computer program for executing the computer program to implement the method of any of claims 1 to 9.

11. A computer readable storage medium having stored thereon program instructions, which when executed by a processor, implement the method of any of claims 1 to 9.