WO2018210055A1 - 增强现实处理方法及装置、显示终端及计算机存储介质 - Google Patents

增强现实处理方法及装置、显示终端及计算机存储介质 Download PDF

Info

Publication number
WO2018210055A1
WO2018210055A1 PCT/CN2018/080094 CN2018080094W WO2018210055A1 WO 2018210055 A1 WO2018210055 A1 WO 2018210055A1 CN 2018080094 W CN2018080094 W CN 2018080094W WO 2018210055 A1 WO2018210055 A1 WO 2018210055A1
Authority
WO
WIPO (PCT)
Prior art keywords
target object
information
image frame
display
point
Prior art date
Application number
PCT/CN2018/080094
Other languages
English (en)
French (fr)
Inventor
庞英明
魏扼
Original Assignee
腾讯科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技（深圳）有限公司 filed Critical 腾讯科技（深圳）有限公司
Publication of WO2018210055A1 publication Critical patent/WO2018210055A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/94—Hardware or software architectures specially adapted for image or video understanding
    • G06V10/95—Hardware or software architectures specially adapted for image or video understanding structured as a network, e.g. client-server architectures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/48—Matching video sequences
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Definitions

  • the present application relates to the field of information technology, and in particular, to an Augmented Reality (AR) processing method and apparatus, a display terminal, and a computer storage medium.
  • AR Augmented Reality
  • the AR is a display technology that performs various superimpositions on an image acquired based on the real world to expand the display content.
  • the information introduced in the AR in the prior art needs to be displayed around the corresponding graphic object.
  • the problem that the AR information deviates from the corresponding graphic object usually occurs, and there may also be a problem that the current video needs to be kept still, and the AR information is superimposed and displayed after the AR information is acquired.
  • the embodiments of the present application are expected to provide an AR processing method and a method for cooking a pointing finger, a display terminal, and a computer storage medium to reduce the problem that the AR information deviates from the corresponding graphic object, or need to suspend the video.
  • a first aspect of the embodiments of the present application provides an augmented reality AR processing method, which is applied to a display terminal, and includes:
  • the AR information is added to the current image frame according to the display position.
  • the second aspect of the embodiment of the present application provides an augmented reality AR processing device, which is applied to a display terminal, and includes:
  • a display unit configured to display a video based on a video stream
  • An obtaining unit configured to acquire AR information of a target object in the video
  • a tracking unit configured to track a display position of the target object in a current image frame of the currently displayed video
  • the display unit is further configured to add the AR information to the current image frame according to the display position.
  • a third aspect of the embodiment of the present application provides a display terminal, including:
  • Display configured as an information display
  • a memory configured to store a computer program
  • a processor coupled to the display and the memory, configured to control the display terminal to perform any of the AR processing methods described above by executing the computer program.
  • a fourth aspect of the embodiments of the present application provides a computer storage medium, where the computer program stores a computer program, and the computer program is configured to be executed by the processor to implement any of the foregoing AR processing methods.
  • a fifth aspect of the embodiments of the present application provides an information display method for displaying augmented reality information of a target object in real time on a display device, where the method includes:
  • the embodiment of the present application provides an AR processing method and apparatus, a display terminal, and a computer storage medium.
  • the display of the video is not suspended; however, when the video is displayed, the display position of the target object in the image frame is tracked.
  • the AR information is obtained, the AR information is superimposed and displayed on the current image frame according to the obtained display position, so that the AR information can be superimposed on the target object or the target object attachment, thereby reducing the target object in the video.
  • the movement of the target object in different image frames causes the AR information to deviate from the target object, thereby solving the problem that the AR information deviates from the corresponding target object, and does not need to suspend the display of the video in the acquisition of the AR information, waiting for the display of the AR information.
  • Improved user experience improves the user experience.
  • FIG. 1 is a schematic flowchart diagram of a first AR processing method according to an embodiment of the present disclosure
  • FIG. 2 is a schematic diagram showing display of a reference point, a feature point, an offset vector, and a mean offset vector according to an embodiment of the present application;
  • FIG. 3 is a schematic flowchart of a second AR processing method according to an embodiment of the present application.
  • FIG. 4 is a schematic diagram of a video display effect according to an embodiment of the present application.
  • FIG. 5 is a schematic diagram of another video display effect according to an embodiment of the present application.
  • FIG. 6 is a schematic structural diagram of an AR processing apparatus according to an embodiment of the present disclosure.
  • FIG. 7 is a schematic structural diagram of another AR processing apparatus according to an embodiment of the present disclosure.
  • FIG. 8 is a schematic diagram of another AR processing system and processing flow according to an embodiment of the present application.
  • FIG. 9 is a schematic flowchart diagram of another information display method according to an embodiment of the present application.
  • the embodiment provides an AR processing method, which is applied to a display terminal, and includes:
  • Step S110 Display a video based on the video stream
  • Step S120 Acquire AR information of the target object in the video.
  • Step S130 Tracking the display position of the target object in the current image frame of the currently displayed video
  • Step S140 Add the AR information to the current image frame according to the display position.
  • the display terminal here can be various types of display terminals having a display screen, for example, a human-mounted display terminal such as a mobile phone, a tablet computer or a wearable device, or various in-vehicle devices having a display screen.
  • the display here can be a variety of displays such as liquid crystal display, electronic ink display or projection display.
  • the display terminal displays the video based on the video stream in step S110.
  • the video stream includes a data stream having a multi-frame image of display timing.
  • the video stream may be formed by the current collection of the display terminal, or may be received from another device, or may be pre-stored in the display terminal, and may be used by the terminal to display a data stream forming a video.
  • the AR information of the target object in the video is acquired in step S120.
  • the AR information herein may include one or more of identification information of the target object, category information, various attribute information, and location information of a location of the target object.
  • Target object 1 and target object 2 are marked in FIG. 4; apparently target object 1 is a human figure; target object 2 is a figure of a vehicle.
  • An image in which the AR information is superimposed is displayed in FIG. 5; the words “Broadcast and Star A" are included in the AR information shown in FIG.
  • the AR information displayed by the overlay is displayed adjacent to the corresponding target object. For example, the word "transit" is adjacent to the vicinity of the target object 1 identified as a bus in the current image, thereby reducing the phenomenon that the AR information is far from the target object.
  • the identification information may be information such as a name or a recognition serial number of the collected object corresponding to the target object.
  • the identification information that the AR information can include in the present embodiment may be text information indicating that the vehicle is: a Mercedes-Benz, and the text “Mercedes-Benz” is the identifier. A kind of information.
  • the category information may be information indicating a category to which the target object belongs, for example, text information indicating that the vehicle is a public transportation, and the text “transit” may be used as one of the category information.
  • the attribute information may represent information of various attributes of the target object. For example, if the target object is identified as a bus and the bus number of the bus is identified, the attribute information may include: route information of the bus. The bus number may be another example of the identification information.
  • the location information may be used to indicate an approximate location where the target object is currently located, for example, if the target object is a vehicle, by image recognition and/or positioning, for example, global satellite system (GPS) positioning, etc.
  • the current location information of the vehicle is taken out. For example, it can be displayed that the current vehicle is located at Haidian South Road, Haidian District, Beijing.
  • the AR information may be text information or image information.
  • the AR information herein may be information superimposed on the current image.
  • the identification of the image to be identified may be locally identified by the display terminal based on the correspondence between the image stored in the local image recognition library and the predetermined information, thereby extracting part of the predetermined information as a The AR information is displayed.
  • the display position of the target object may be different in different image frames of the video.
  • the position of the target object in each image frame of the video is tracked, thereby determining the display position of the target object in the current image frame.
  • the tracking algorithm is used to predict the location of the target object in the current image frame based on the position of the target object in the previous image frame, and then the display position of the target object in the current image frame is determined by feature matching in the region.
  • the tracking algorithm may include: calculating a motion speed of the target object (which may include: a moving direction and a motion rate) based on an imaging size of the target objects of the two adjacent image frames, according to a time difference between the two image frames before and after, Therefore, it is possible to predict the target object in the area where the current image frame is located.
  • the feature of the target object is matched with the feature of each graphic in the current image frame, and the location of the graphic that satisfies the matching condition is selected as the display position of the target object in the current image frame.
  • step S140 according to the determined display position, the AR information is conveniently displayed on the target object or around the target object, and the information offset problem caused by the incorrect display position of the AR information is avoided, thereby improving the user experience.
  • step S140 the display superposition of the AR information is performed according to the display position of the current image frame of the target object, instead of the random superposition.
  • the current image frame here is the image frame displayed at the current time.
  • the display position here may include display position indication information such as coordinates of the target object in the current image frame.
  • step S140 the display position of each image frame in each video object in the video stream is tracked, so that the AR information is superimposed and displayed on the target object or around the target object, thereby avoiding information offset caused by the misalignment of the AR information superimposed display position. Problems that can enhance the user experience.
  • the AR information may be added to the multi-frame image of the video stream until the target object disappears from the image frame of the video stream. However, each time adding, it is necessary to re-determine the positional parameter of the target object in the current image frame, thereby realizing that the AR information follows the positional switching of the target object between the image frames and switches; avoiding the movement of the target object itself in the image frame, The phenomenon of separation from AR information to enhance the user experience again.
  • the step S120 may include:
  • One or more frames of the image to be recognized satisfying the preset clear condition are intercepted.
  • determining whether an image frame satisfies the preset clear condition may include: extracting contour information of the corresponding image frame, and if the contour information is successfully extracted, the image frame may be regarded as an image satisfying the preset clear condition. .
  • the method may further include: calculating a gray level difference between each pixel point in the image frame and a pixel point around the pixel, or having a gray level difference between the preset pixel point and a pixel point around the pixel is greater than a preset threshold, which may be considered as
  • the image frame may be an image frame that satisfies the preset clear condition.
  • the video stream is received from another device, or is pre-stored in the display terminal, and the image frame of the video stream is divided into a key frame and a non-key frame, which may be in this embodiment.
  • One or more key frames are selected as the image to be recognized that satisfies the preset clear condition.
  • the non-key frame may be a display of image information dependent on the key frame. The display of the key frames can be displayed independently.
  • any one of the graphic objects to be recognized may be identified as the target object, or only a partial image object may be used as the target object.
  • step S120 there is no certain sequence between the step S120 and the step 130.
  • the step S120 may be performed after the step S130 or may be performed synchronously with the step S130.
  • the step S120 may include:
  • Step S122 Send the image to be identified to the service platform, where the image to be identified is used for image recognition on the service platform to obtain a recognition result;
  • Step S123 Receive AR information returned by the service platform based on the recognition result.
  • the service platform may be a server that provides image recognition formed by one or more servers.
  • the display terminal sends the image to be identified to the service platform, that is, the client intercepts one or more frames of the image from the video stream, and sends the image to the service platform for image recognition by the service platform.
  • the identification of the service platform in this embodiment may be generalization identification.
  • the generalization identification here is that the service platform recognizes any identifiable graphic object in the image to be recognized, thereby realizing comprehensive recognition of each graphic object in the image to be recognized, thereby providing AR information as much as possible.
  • Image recognition is performed by the service platform, which can reduce the load of the display terminal itself, and reduce resource consumption and power consumption of the display terminal. If the display terminal is a mobile display terminal, the standby time of the mobile display terminal to be tested may be extended.
  • the step S120 may include:
  • the image feature may include: a contour feature, a texture feature, and/or a grayscale feature of each graphic object in the image to be recognized. These image features can all be features for image recognition.
  • the contour feature may include an outer contour of an image of an object, an inner contour in the outer contour, and the like, and the shape, size, and the like of the graphic object described by the contour are convenient for image recognition with the reference image when performing image recognition. Matching results in the AR information.
  • the texture feature can be used to describe the grayscale variation gradient between adjacent contours, and can also be used for image recognition, and can be used to reflect information such as the material of the target object.
  • the grayscale feature may directly include a grayscale value and a grayscale gradient, and the grayscale value and the grayscale gradient may be used to extract the contour feature, the texture feature, and the like.
  • the image feature further comprises: a color feature; the color feature can include: a hue of the color, a saturation, and the like.
  • the image features are various, and are not limited to any one of the above.
  • the image feature may further include: indicating the color of the overall or partial color of each target object. feature.
  • the extracting image features of at least a portion of the images in the video stream comprises:
  • the first gray point value of the first pixel point is A1
  • the second gray value of the second pixel point is B1
  • satisfying the preset difference condition may include: the absolute difference between the A1 and the B1 The value is not less than the difference threshold. If the gradation of the first gradation value and the second gradation value is sufficiently large, the pixel point may be a pixel point on the contour of the image object, or a highlight point of the highlighted portion, and may be used as an important pixel for identifying the corresponding image object. .
  • the second pixel point can be a pixel within a neighborhood of the first pixel point.
  • the neighborhood is an area formed by extending N pixels in the first direction and the second direction centering on the first pixel, and the pixel located in the area may be the Area.
  • the N can be a positive integer.
  • the first direction may be perpendicular to the second direction.
  • the neighborhood may be a rectangular area, and the neighborhood may also be a circular area centered on the first pixel, and the pixel located in the circular area is the second pixel. point.
  • the method further includes: calculating a distribution density of feature points of each sub-region based on the number of the feature points and the distribution of the feature points, where the distribution density of the M sub-regions in one image is greater than the density At the threshold, the image frame can be considered to be an image frame that satisfies the preset clear condition.
  • the step S140 may include:
  • a tracking manner is adopted, based on the gradation of the movement of the target object in two adjacent image frames in the video stream, combined with the first position of the previous image frame.
  • the parameter which locates the second position parameter in the current image frame, can reduce the positioning of the target object every time on the entire picture in the current image frame, reducing the amount of calculation.
  • the search range of the search target object is determined based on the first position parameter without searching the target object over the entire current image frame, thereby reducing the amount of calculation in the search process.
  • the first position parameter is used to extend a preset pixel as the search area at an edge position corresponding to the current image frame; and then the image is located by matching the search area with the image of the target object in the previous image frame.
  • the second positional parameter of the target object in the current image frame is used to extend a preset pixel as the search area at an edge position corresponding to the current image frame; and then the image is located by matching the search area with the image of the target object in the previous image frame.
  • the second positional parameter of the target object in the current image frame In this way, if the target object is not searched in the current search area, the search area is expanded again or the search area is changed until the entire current image frame is searched. In this case, it is obvious that some targets move slowly.
  • the object can be quickly determined as the second positional parameter in the current image frame.
  • the step S142 may include:
  • the mean shift vector comprises: a mean shift direction and a mean offset
  • mapping according to the reference point and the mean shift vector, a target point corresponding to the target object; wherein the target point is a reference point of a next image frame, and corresponds to the second position parameter.
  • the reference point may be a central position of the target object in the previous image frame, but is not limited to the center or the like.
  • each feature point in the current image frame is first extracted, and the feature point here is also a pixel point that satisfies a preset condition for the gray level difference of the pixel points around it.
  • Constructing an offset vector pointing to each feature point with the reference point as a vector starting position; acquiring an offset of each offset vector, and then obtaining an average of the offset to obtain an average of the mean offset vector The offset is further combined with each offset vector to perform a vector operation of the direction to determine the mean shift direction corresponding to the mean offset vector.
  • the offset direction of the mean shift vector points to a position where the feature point density is high.
  • the position of the target point may be the end point of the mean value offset vector starting from the reference point.
  • the target point may be used as a component of the second position parameter; when the target image is tracked for the next image frame, the target point of the current image frame may be used as a reference point of the next image frame; Iterative tracking, the way in which the positioning target object is displayed at the current image frame has the characteristics of small calculation amount and simple implementation.
  • each black solid point in Fig. 2 represents a feature point
  • the hollow origin represents the reference point
  • the one-line arrow represents the offset vector
  • the open arrow represents the mean offset vector
  • the mean mean offset vector is Starting from the current datum point, it points to the region where the feature point distribution density is high
  • the mean shift of the mean shift vector is equal to the mean of the offsets of all the offset vectors.
  • the target object is a graphical object displayed within a focus area of the image to be identified.
  • each frame of image can be performed by the camera based on a specific focus position, each image frame has its corresponding focus area; the image object usually located in the focus area is the clearest graphic object, and is also the user's focus.
  • the image object in the embodiment, is for reducing the recognition workload, the target object being a graphic object at least partially located within the focus area.
  • the method further includes:
  • the obtaining prompt is displayed on the screen of the video, wherein the obtaining prompt is used to prompt that the AR information is currently being acquired.
  • the user is prompted to be currently in the process of acquiring the AR information by displaying the prompt.
  • the obtaining prompt may be text information or image information.
  • it may be a translucent mask or the like displayed on the current image frame to further enhance the user experience.
  • the embodiment of the present invention further provides another information display method for displaying augmented reality information of a target object in real time on a display device, the method comprising:
  • Step S210 Acquire a video stream of the target object; the video stream includes a plurality of consecutive acquisition unit video frames;
  • Step S220 Receive augmented reality information of the target object in the video stream; the video frame displays a graphic of the target object, and the augmented reality information may be various information related to the target object, for example, attribute information indicating an attribute of the target object. And status information indicating the status of the target object;
  • Step S230 Acquire, according to a specific algorithm, latest location information of the target object in a current image frame of the video stream;
  • Step S240 Add the augmented reality information to the current image frame according to the latest location information.
  • the aforementioned specific algorithm may be a tracking algorithm for each tracking target.
  • the location information of the target object in the current image frame ie, the latest location information
  • the augmented reality information is displayed in the current image based on the latest location information in step S240.
  • the S230 may include:
  • the step S240 may include: locating a first location parameter of the target object of a previous image frame in the video stream; acquiring, according to the first location parameter, a number of the target object in the current image frame. Two position parameters.
  • the acquiring the second location parameter of the target object in the current image frame based on the first location parameter includes: determining the target based on tracking the target object in a previous image frame a reference point of the object in the current image frame, wherein the reference point is a pixel point characterizing a display position of the target object at a previous image frame; determining respective feature points of the current image frame relative to the reference point An offset vector, wherein the feature point is a first pixel point that satisfies a first gray value, and a difference between the first gray value and a second gray value of a second pixel adjacent to the first pixel point Satisfying a preset difference condition; determining, based on the offset vector, a mean shift vector of each of the feature points relative to the reference point, wherein the mean shift vector includes at least: a mean shift direction and a mean shift Locating a target point corresponding to the target object based on the reference point and the mean value offset vector; wherein the target point is a reference point of a next image
  • the present example provides an AR processing method, which can be applied to various smart terminals such as mobile phones and smart glasses.
  • the smart terminal collects video, it needs to add AR information (for example, superimposing) to improve the display effect.
  • AR information for example, superimposing
  • Step S110 Display a video based on the video stream
  • Step S121 Extract one or more frames of the image to be recognized that meet the preset clear condition in the video stream; wherein the image to be identified includes the target object;
  • Step S122 Send the image to be recognized to the service platform
  • Step S123 Receive AR information returned by the service platform based on the recognition result of the image to be identified;
  • Step S131 Tracking the display position of the target object in each image frame, thereby obtaining a position parameter of the target object in the current image frame;
  • Step S141 superimpose and display the AR information in the current image frame according to the position parameter.
  • the embodiment provides an augmented reality AR processing device, which is applied to a display terminal, and includes:
  • the display unit 110 is configured to display a video based on the video stream
  • the obtaining unit 120 is configured to acquire AR information of the target object in the video
  • the tracking unit 130 is configured to track a display position of the target object in a current image frame of the currently displayed video
  • the display unit 110 is further configured to add the AR information to the current image frame according to the display position.
  • the display terminal provided in this embodiment may be various terminals including a display screen, and the display screen here may be various types of display screens such as a liquid crystal display, an electronic ink display or a projection display.
  • the obtaining unit 120 and the tracking unit 130 correspond to a processor or a processing circuit in the terminal.
  • the processor can be a central processing unit (CPU), a microprocessor (MCU), a digital signal processor (DSP), an application processor (AP), or a programmable array (PLC).
  • the processor circuit can be an application specific integrated circuit (ASIC).
  • the processor or processing circuit can perform the above operations by execution of executable code.
  • the apparatus tracks the display position of the target object in each frame image in the video when the display terminal performs AR display, thereby ensuring that the AR information is superimposed and displayed on the corresponding target object attachment, thereby reducing the target.
  • the phenomenon that the AR information of the object A is superimposed around the target object B reduces the phenomenon that the AR information deviates from the target object and improves the user experience.
  • the acquiring unit 120 is configured to extract one or more frames of the to-be-identified image that meet the preset clear condition in the video stream, and acquire the target object in the to-be-identified image based on one or more frames.
  • the identification result corresponds to the AR information.
  • the obtaining unit 120 is configured to send the to-be-identified image to a service platform, where the image to be identified is used for image recognition on the service platform to obtain a recognition result; receiving the service platform is based on the Identify the AR information returned by the result.
  • the AR information is from a service platform, and the service platform can provide as much information as possible to the client through information search, thereby reducing AR information caused by insufficient information storage of the terminal itself. Not rich enough or small amount of information.
  • the acquiring unit 120 is configured to extract an image feature of at least part of the image in the video stream, and determine, according to the image feature, whether the image to be identified meets the preset clear condition.
  • the acquiring unit 120 is mainly used to select one or more frames of images that are sufficiently clear by image feature extraction, and send the image to the service platform or identify itself to improve the recognition accuracy and the probability of successful recognition.
  • the acquiring unit 120 is configured to extract feature points of at least part of the image in the video stream, where the feature point is a first pixel point of the first gray value; the first gray value The difference of the second gray value of the second pixel adjacent to the first pixel point satisfies a preset difference condition; and the image to be recognized that satisfies the preset clear condition is determined according to the number of the feature points.
  • the image to be recognized that satisfies the preset clear condition is determined by the extraction of the feature points, for example, the detection using the FAST feature points.
  • the FAST may be an abbreviation of Features from Accelerated Segment Test.
  • the tracking unit 130 is configured to locate a first location parameter of the target object of a previous image frame in the video stream; searching for the target object based on the first location parameter A second positional parameter in the current image frame.
  • the second position parameter of the target object in the current image frame is acquired, and the calculation amount of the positioning second position parameter is reduced.
  • the tracking unit 130 is configured to determine a reference point of the target object in a current frame based on tracking the target object in a previous image frame, wherein the reference point is to represent the target a pixel point of the object at the display position of the previous image frame; determining an offset vector of each feature point of the current image frame with respect to the reference point, wherein the feature point is the first pixel point of the first gray value Determining, by the first offset value, a difference of a second gray value of the second pixel point adjacent to the first pixel point, a preset difference condition; determining, based on the offset vector, each of the feature points relative to the a mean value offset vector of the reference point, wherein the mean value offset vector includes: a mean value offset direction and a mean value offset amount; and the target point corresponding to the target object is located based on the reference point and the mean value offset vector Wherein the target point is a reference point of a next image frame and corresponds to the second position parameter.
  • the first position parameter may include coordinates of a target point of a previous image frame; and the second position parameter may be a coordinate of a target point of a current image frame.
  • the target object is a graphic object displayed in a focus area of the image to be recognized. This can reduce the recognition of unnecessary graphic objects and the return of AR information, reduce the display of unnecessary graphic information, and reduce the interference of information to users.
  • the display unit 110 is further configured to display an acquisition prompt on the screen of the video in the process of acquiring the AR information, where the obtaining prompt is used to prompt that the AR is currently being acquired. information.
  • the user may be prompted to obtain the AR information, which reduces the anxiety state of the user during the waiting process, and improves the user experience again.
  • the embodiment provides a display terminal, including:
  • the display 210 is configured to display information
  • a memory 220 configured to store a computer program
  • the processor 230 coupled to the display and the memory, is configured to control the display terminal to perform the AR processing method provided by any one of the foregoing embodiments by executing the computer program, for example, the AR provided in FIG. Processing methods, etc.
  • the display 210 in this embodiment may be various types of displays, liquid crystal displays, projection displays, or electronic ink displays, and the like.
  • the memory 220 can be various types of storage media, such as a random storage medium, a read-only storage medium, a flash memory, or a compact disc.
  • the memory 220 includes at least a portion of a non-transitory storage medium, where the non-transitory storage medium can be used to store the computer program.
  • the processor 230 can be a CPU, an MCU, a DSP, an AP, or a PLC or an ASIC.
  • the processor or the ASIC can perform superimposed display of the AR information on the current image frame of the video by the execution of the computer program.
  • the terminal display 210, the memory 220, and the processor 230 are all connected by a bus 250, which may include an integrated circuit bus (IIC) bus or a Peripheral Component Interconnect Standard (PCI) bus. Wait.
  • IIC integrated circuit bus
  • PCI Peripheral Component Interconnect Standard
  • the client may also include a network interface 240 that is operable to connect to the network side and connect to the service platform.
  • the embodiment further provides a computer storage medium, wherein the computer storage medium stores a computer program; the computer program is configured to be executed by the processor to implement the AR processing method provided by any one of the foregoing embodiments, for example, as shown in the figure The AR processing method shown in 1 or the like.
  • the computer storage medium can be any type of storage medium, optionally a non-transitory storage medium.
  • the computer storage medium may be selected from the group consisting of a mobile storage device, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like. .
  • the example provides an AR processing system, including:
  • the client may be the terminal that displays AR information
  • the server side may provide a network side service platform supported by the AR processing for the client.
  • the client includes:
  • the kernel module corresponding to the kernel of the operating system, can be used for background information processing, for example, by acquiring interaction with the service platform to obtain AR information;
  • An AR engine is configured to acquire location information of the target object, and send the corresponding location parameter to the display screen based on the location information;
  • the display screen corresponding to the display user interface, can be used for video display. Based on the position parameters provided by the ASDDK, the AR information forwarded from the kernel module is correctly superimposed to the corresponding target object.
  • the server side may include: a proxy server, an identification server, and a search server;
  • the proxy server is configured to perform information interaction with the client, for example, receiving an image to be identified sent by the client;
  • the identification server is connected to the proxy server, and is configured to receive an image to be identified forwarded by the proxy server, and then send the recognition result to the search server;
  • the search server is connected to the identification server, and is configured to query the AR information based on the search result, and send the AR information to the client through the proxy server.
  • the following specifically provides an application method of the AR information in the foregoing system, including:
  • the real-time scanning and tracking of the object is a process in which the terminal displays the real-time picture cloud and then displays the terminal.
  • the background of the terminal is connected to the cloud identification server, the search server, and the proxy server for information integration.
  • the terminal includes an ASDDK, a data transmission unit for data transmission between the terminal and the network platform, and a UI unit for performing display interaction with the user.
  • the terminal opens the camera through the application.
  • the video stream is imported into the terminal network transmission unit by the UI unit, and the network transmission unit detects through the FAST feature point.
  • the number of FAST feature points can represent whether the object in the picture has sufficient recognition conditions to satisfy the feature point.
  • the requested image is an image with enough preset conditions, then we pass this frame image to the background proxy server.
  • the background proxy server receives the uploaded image from the terminal and sends the image to the cloud identification server.
  • the cloud recognition server performs image generalization recognition, and displays the object category, location, and number information in the image, and then transmits the image to the background proxy server.
  • the background proxy server After the background proxy server obtains the image category information, it sends the information to the consulting search center server to retrieve the relevant information about the object category, and returns directly if there is no relevant information. At this time, the proxy server transmits the image information and the consultation information to the terminal.
  • the related information information here is one of the aforementioned AR information.
  • the terminal has module receiving. If the number of times passes the information directly to the UI drawing, the current object motion may change due to network transmission and recognition time. If the position of the uploaded frame is also drawn, it is likely to be drawn. shift. Therefore, at this time, the data transmission module does not plug the information to the UI, but sends it to the ASDDK to obtain the location update.
  • the ARSDK transmits the image frame data of the transmission module to the local following module, and the local following module uses the mean shift algorithm to first calculate the offset mean value of the feature points of the first frame picture, and then use this as a new starting point to investigate the next frame again.
  • the image corresponds to the movement position of the feature point, and can always follow the object in the image, and can acquire the position information of the object in real time. At this time, after receiving the object transmitted from the kernel, the latest position of the object can be transmitted to the kernel module.
  • the kernel updates to the latest location information of the object, and transmits the consultation information, the category information, and the location information to the UI unit. After receiving the relevant information, the UI unit can draw the identifier on the screen. And the location will be accurate.
  • the AR information processing method may include:
  • Step 1 The display provides a video stream to the kernel module
  • Step 2 The kernel module provides the image to be identified to the proxy server
  • Step 3 The proxy server provides the identification server with the image to be identified
  • Step 4 The identification server sends the recognition result to the proxy server.
  • the identification server may directly feed back the identification result to the search server.
  • Step 5 The proxy server sends the recognition result to the search server
  • Step 6 The search server returns the AR information searched based on the recognition result to the proxy server;
  • Step 7 The proxy server forwards the AR information to the kernel module of the client;
  • Step 8 The kernel module acquires a new location parameter from the ASDSK; the location parameter here is a location parameter of the target object in the current image frame;
  • Step 9 ARSDK sends the updated location parameters to the kernel module
  • Step 10 The kernel module returns the AR information and the updated position parameter to the display screen, and the display screen superimposes the AR information in the vicinity of the target object while displaying the video.
  • the disclosed apparatus and method may be implemented in other manners.
  • the device embodiments described above are merely illustrative.
  • the division of the unit is only a logical function division.
  • there may be another division manner such as: multiple units or components may be combined, or Can be integrated into another system, or some features can be ignored or not executed.
  • the coupling, or direct coupling, or communication connection of the components shown or discussed may be indirect coupling or communication connection through some interfaces, devices or units, and may be electrical, mechanical or other forms. of.
  • the units described above as separate components may or may not be physically separated, and the components displayed as the unit may or may not be physical units, that is, may be located in one place or distributed to multiple network units; Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing module, or each unit may be separately used as one unit, or two or more units may be integrated into one unit;
  • the unit can be implemented in the form of hardware or in the form of hardware plus software functional units.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Software Systems (AREA)
  • User Interface Of Digital Computer (AREA)
  • Image Analysis (AREA)
  • Processing Or Creating Images (AREA)

Abstract

本申请实施例公开了一种AR处理方法及装置、显示终端及计算机存储介质。所述AR处理方法包括：获取所述视频中目标对象的AR信息；跟踪所述目标对象在当前显示的所述视频的当前图像帧中的显示位置；根据所述显示位置，将所述AR信息添加到所述当前图像帧中。本申请实施例还提供了一种信息显示方法。

Description

增强现实处理方法及装置、显示终端及计算机存储介质
本申请基于申请号为201710340898.0、申请日为2017年05月15日的中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此引入本申请作为参考。
技术领域
本申请涉及信息技术领域，尤其涉及一种增强现实(Augmented Reality，AR)处理方法及装置、显示终端及计算机存储介质。
背景技术
AR是在基于真实世界采集的图像上，进行各种叠加从而进行显示内容扩展的一种显示技术。在现有技术中AR中引入的信息需要显示在对应的图形对象周围。在现有技术中，通常出现AR信息偏离对应的图形对象的问题，同时还可能存在着需要保持当前视频静止，等到AR信息获取之后再进行AR信息的叠加显示的问题。
发明内容
有鉴于此，本申请实施例期望提供一种AR处理方法及煮给你指、显示终端及计算机存储介质，以减少AR信息偏离对应图形对象的问题，或需要中止视频的问题。
本申请的技术方案是这样实现的：
本申请实施例第一方面提供一种增强现实AR处理方法，应用于显示终端中，包括：
基于视频流，显示视频；
获取所述视频中目标对象的AR信息；
跟踪所述目标对象在当前显示的所述视频的当前图像帧中的显示位置；
根据所述显示位置，将所述AR信息添加到所述当前图像帧中。
本申请实施例第二方面提供一种增强现实AR处理装置，应用于显示终端中，包括：
显示单元，配置为基于视频流，显示视频；
获取单元，配置为获取所述视频中目标对象的AR信息；
跟踪单元，配置为跟踪所述目标对象在当前显示的所述视频的当前图像帧中的显示位置；
所述显示单元，还配置为根据所述显示位置，将所述AR信息添加到所述当前图像帧中。
本申请实施例第三方面提供一种显示终端，包括：
显示器，配置为信息显示；
存储器，配置为存储计算机程序；
处理器，与所述显示器及所述存储器连接，配置为通过执行所述计算机程序，控制所述显示终端执行任一项上述的AR处理方法。
本申请实施例第四方面提供一种计算机存储介质，所述计算机存储介质中存储计算机程序；所述计算机程序，用于被处理器执行后能够实现任一项上述的AR处理方法。
本申请实施例第五方面提供一种信息显示方法，用于在显示设备上实时显示目标对象的增强现实信息，该方法包括：
获取目标对象的视频流；
接收视频流中目标对象的增强现实信息；
根据特定算法获取所述目标对象在所述视频流的当前图像帧中的最新 位置信息；以及
根据所述最新位置信息，将所述增强现实信息添加到所述当前图像帧中。
本申请实施例提供AR处理方法及装置、显示终端及计算机存储介质，在进行视频的显示，不会中止视频的显示；但是会在进行视频显示时，会跟踪图像帧中的目标对象的显示位置，这样在获得AR信息之后，会根据获取的显示位置，将AR信息叠加显示到当前图像帧中，这样就可以将AR信息叠加目标对象上或目标对象附件，从而可以减少因为目标对象在视频的不同图像帧中目标对象的移动，导致AR信息偏离目标对象的现象，从而解决了AR信息偏离对应的目标对象的问题，且在获取AR信息中不用中止视频的显示，以等待AR信息的显示，提升了用户体验。
附图说明
图1为本申请实施例提供的第一种AR处理方法的流程示意图；
图2为本申请实施例提供的一种基准点、特征点、偏移向量及均值偏移向量的显示示意图；
图3为本申请实施例提供的第二种AR处理方法的流程示意图；
图4为本申请实施例提供的一种视频显示效果示意图；
图5为本申请实施例提供的另一种视频显示效果示意图；
图6为本申请实施例提供的一种AR处理装置的结构示意图；
图7为本申请实施例提供的另一种AR处理装置的结构示意图；
图8为本申请实施例提供的另一种AR处理系统及处理流程示意图；
图9为本申请实施例提供的另一种信息显示方法的流程示意图。
具体实施方式
以下结合说明书附图及具体实施例对本申请的技术方案做进一步的详 细阐述。
如图1所示，本实施例提供一种AR处理方法，应用于显示终端中，包括：
步骤S110：基于视频流，显示视频；
步骤S120：获取所述视频中目标对象的AR信息；
步骤S130：跟踪所述目标对象在当前显示的所述视频的当前图像帧中的显示位置；
步骤S140：根据所述显示位置，将所述AR信息添加到所述当前图像帧中。
本实施例提供一种AR处理方法，为应用于显示终端中的方法。这里的显示终端可为各种类型具有显示屏的显示终端，例如，手机、平板电脑或可穿戴式设备等人载显示终端，还可以是各种具有显示屏的车载设备。这里的显示屏可为液晶显示屏、电子墨水显示屏或投影显示屏等各种显示屏。
在步骤S110中显示终端会基于视频流显示视频。所述视频流包括：具有显示时序的多帧图像的数据流。在步骤S110中所述视频流可为所述显示终端当前采集形成的，也可以是从其他设备接收的，也可以是预先存储在所述显示终端内，可用于终端显示形成视频的数据流。
在步骤S120中会获取所述视频中目标对象的AR信息。这里的AR信息可包括：所述目标对象的标识信息、类别信息、各种属性信息及所述目标对象所在位置的位置信息的一种或多种。在图4中标注有目标对象1和目标对象2；显然目标对象1为人的图形；目标对象2为车辆的图形。在图5中显示叠加显示AR信息的图像；在图5所示的AR信息中包括字样：公交及明星A。叠加显示的AR信息都是邻近对应的目标对象显示的。例如，字样“公交”邻近当前图像中的识别为公交的目标对象1的附近，从而减少了AR信息远远偏离目标对象的现象。
所述标识信息，可为目标对象对应的被采集对象的名称或识别序列号等信息。例如，在一个图像中有一辆车的图形对象，在本实施中所述AR信息可包括的标识信息，可为指示该车辆为：奔驰车的文本信息，则文本“奔驰”即为所述标识信息的一种。
所述类别信息可为指代所述目标对象所归属的类别的信息，例如，指示该车辆为公交的文本信息，则文本“公交”可作为所述类别信息的一种。
所述属性信息可表征该目标对象各种属性的信息。例如，识别出目标对象为一个公交车，并识别出该公交车的公交号，则所述属性信息可包括：该公交车的路线信息。所述公交号可为所述标识信息的另一种示例。
所述位置信息，可用于指示所述目标对象当前所处的大致位置，例如，若所述目标对象为车辆，则可通过图像识别和/或定位，例如，全球卫星系统(GPS)定位等给出所述车辆当前位置信息。例如，可以显示出当前车辆位于北京市海淀区海淀南路等位置信息。
在本实施例中所述AR信息可为文本信息、也可以为图像信息。总之，这里的AR信息可为叠加显示在当前图像中的信息。
在本实施例中对所述待识别图像的识别，可以由所述显示终端基于本地的图像识别库中存储的图像与预定信息的对应关系，进行本地识别，从而提取部分所述预定信息作为所述AR信息进行显示。
目标对象可能在视频的不同图像帧中的显示位置是不同的，在本实施例中会跟踪目标对象在视频各个图像帧的位置，从而确定出目标对象在当前图像帧的显示位置。
例如，利用跟踪算法基于前一个图像帧中的目标对象的位置，预测出当前图像帧中目标对象的所在区域，然后在该区域内通过特征匹配确定出目标对象在当前图像帧的显示位置。所述跟踪算法可包括：基于相邻两个图像帧目标对象的成像尺寸，可以计算出目标对象的运动速度(可包括： 运动方向及运动速率)，根据前后两个图像帧之间的时间差，故可以预测出目标对象在当前图像帧的所在区域。
又例如，将目标对象的特征与当前图像帧中各个图形的特征进行匹配，选择出满足匹配条件的图形所在位置视为所述目标对象在当前图像帧的显示位置。
在步骤S140中会根据确定的显示位置，以方便将所述AR信息添加显示在目标对象上或目标对象的周围，避免AR信息添加显示位置不对导致的信息偏移问题，从而可以提升用户体验。
在本实施例中步骤S140中会根据目标对象在当前图像帧的显示位置，进行AR信息的显示叠加，而非随意的叠加。这里的当前图像帧为当前时刻显示的图像帧。
这里的显示位置可包括：目标对象在当前图像帧中的坐标等显示位置指示信息。
在步骤S140中会跟踪各个目标对象在视频流中各个图像帧的显示位置，方便将所述AR信息叠加显示在目标对象上或目标对象的周围，避免AR信息叠加显示位置不对导致的信息偏移问题，从而可以提升用户体验。
所述AR信息可以添加到视频流的多帧图像中，直至所述目标对象从所述视频流的图像帧中消失。但是每次添加时都需要重新确定目标对象在当前图像帧中的位置参数，从而实现AR信息跟随所述目标对象在图像帧之间的位置切换而切换；避免目标对象自身在图像帧的移动，与AR信息的脱离的现象，以再次提升用户体验。
在一些实施例中，所述步骤S120可包括：
会截取一帧或多帧满足预设清晰条件的待识别图像。
获取基于一帧或多帧所述待识别图像中所述目标对象的识别结果对应的AR信息。
在本实施例中判断一个图像帧是否满足所述预设清晰条件，可包括：提取对应图像帧的轮廓信息，若轮廓信息提取成功，可认为该图像帧为满足所述预设清晰条件的图像。
在一些实施例中还可包括：计算图像帧中各个像素点与其周围的像素点的灰度差，或存在预设个像素点与其周围的像素点的灰度差大于预设阈值，可认为该图像帧可为满足所述预设清晰条件的图像帧。
在一些情况下，所述视频流为从其他设备接收的，或者，预先存储在所述显示终端中的，所述视频流的图像帧分为关键帧和非关键帧，在本实施例中可以选择一个或多个关键帧作为满足所述预设清晰条件的待识别图像。在本实施例中，所述非关键帧可为依赖关键帧的图像信息的显示。所述关键帧的显示，可以独立显示。
总之，确定视频流中满足预设清晰条件的方式有很多种，不局限于上述任意一种。
在一些实施例中还包括：
将所述待识别图像发送给网络侧的服务平台，供服务平台进行识别。
在本实施例中进行图像识别时，所述待识别图像中任意一个图形对象都可视为所述目标对象进行识别，也可以仅是部分图像对象作为所述目标对象。
值得注意的在一些实施例中，所述步骤S120与步骤130之间没有一定的先后顺序，所述步骤S120可以在所述步骤S130后执行，也可以与所述步骤S130同步执行。
可选地，如图3所示，所述步骤S120可包括：
步骤S122：将所述待识别图像发送给服务平台，其中，所述待识别图像用于在所述服务平台进行图像识别，以获得识别结果；
步骤S123：接收所述服务平台基于所述识别结果返回的AR信息。
在本实施例中所述服务平台可为有一台或多台服务器形成的提供图像识别的服务端。
在本实施例中所述显示终端将待识别图像发送给服务平台，即所述客户端会从视频流中截取一帧或多帧图像，发送给服务平台，由服务平台进行图像识别。在本实施例中所述服务平台的识别可为泛化识别。这里的泛化识别为所述服务平台会对所述待识别图像中任意一个可识别的图形对象进行识别，从而实现待识别图像中各个图形对象的全面识别，从而尽可能多提供AR信息。
由所述服务平台进行图像识别，这样可以减少显示终端自身的负载量，减少对显示终端的资源消耗和功耗。若所述显示终端为移动显示终端时，可以延长所述移动显示终端待测待机时长。
在一些实施例中，所述步骤S120可包括：
提取所述视频流中至少部分图像的图像特征；
依据所述图像特征，确定所述待识别图像是否满足所述预设清晰条件。
在本实施例中所述图像特征可包括：待识别图像中各个图形对象的轮廓特征、纹理特征和/或灰度特征等。这些图像特征都可为用于进行图像识别的特征。
所述轮廓特征可包括某一个物体的图像的外轮廓，和外轮廓内的内轮廓等，这些轮廓描述的图形对象的形状、尺寸等信息，方便在进行图像识别时，通过与基准图像的图像匹配得到所述AR信息。
所述纹理特征可用于描述相邻轮廓之间的灰度变化梯度，同样可以用于图像识别，可以用于反映目标对象的材质等信息。
所述灰度特征，可直接包括灰度值及灰度梯度，所述灰度值和灰度梯度，可用于提取所述轮廓特征及所述纹理特征等。
在一些实施例中，所述图像特征还包括：色彩特征；所述色彩特征可 包括：颜色的色相、饱和度等参数。
总之，所述图像特征有多种，不局限于上述任意一种，例如，当所述待识别图像为彩色图像时，所述图像特征还可包括：指示各目标对象整体或局部颜色的色彩的特征。
在一些实施例中，所述提取所述视频流中至少部分图像的图像特征，包括：
提取所述视频流中至少部分图像的特征点，其中，所述特征点为第一灰度值的第一像素点；所述第一灰度值与第一像素点邻近的第二像素点的第二灰度值的差异满足预设差异条件；
根据所述特征点的数目，判断满足所述预设清晰条件的所述待识别图像。
例如，第一像素点的第一灰度值为A1，所述第二像素点的第二灰度值为B1；满足所述预设差异条件可包括：所述A1与所述B1差的绝对值不小于差异阈值。若第一灰度值和第二灰度值的灰度足够大，则该像素点可为图像对象的轮廓上的像素点，或高亮部分的高亮点，可作为识别对应图像对象的重要像素。
在一些实施例中所述第二像素点可为所述第一像素点的邻域内的像素点。
在一些实施例中，邻域为以所述第一像素点为中心，向第一方向和第二方向分别延伸N个像素点形成的区域，位于该区域内的像素点，均可为所述邻域。所述N可为正整数。所述第一方向可为垂直所述第二方向。
在一些实施例所述邻域可为矩形区域，所述邻域还可为所述第一像素点为中心的圆形区域，则位于所述圆形区域的像素点即为所述第二像素点。
若一个图像帧出现很明显的模糊现象，则在模糊区域内的各个像素点的灰度差会比较小，则出现的特征点就会很少。
在本实施例中可以基于所述特征点的数量，确定一个图像帧是否满足所述预设清晰条件。例如，当所述特征点的数量大于数量阈值时，可认为对应的图像帧满足所述预设清晰条件。
在还有一些实施例中，所述方法还包括：基于所述特征点的数量及特征点的分布，计算各个子区域特征点的分布密度，当一个图像中存在M个子区域的分布密度大于密度阈值时，则可认为该图像帧为满足所述预设清晰条件的图像帧。
在一些实施例中，所述步骤S140可包括：
定位所述视频流中的前一图像帧所述目标对象的第一位置参数；
基于所述第一位置参数，搜索所述目标对象在所述当前图像帧中的第二位置参数。
在本实施例中为了减少定位各个目标对象的显示位置，采用跟踪的方式，基于目标对象在视频流中相邻两个图像帧中的移动的渐变性，结合在前一个图像帧的第一位置参数，定位在当前图像帧中的第二位置参数，这样可以减少每一次对目标对象的定位都是在当前图像帧中的整个画面上，减少计算量。
例如，基于第一位置参数确定搜索所述目标对象的搜索范围，而不用在整个当前图像帧上搜索所述目标对象，从而减少搜索过程中的计算量。具体以第一位置参数在当前图像帧对应的边缘位置，向外扩展预设个像素，作为所述搜索区域；然后通过将搜索区域与前一图像帧中目标对象的图像匹配，定位出所述目标对象在当前图像帧中的第二位置参数。采用这种方式，若在当前搜索区域未搜索到所述目标对象之后，则再次扩大所述搜索区域或变更搜索区域，直至搜索到整个当前图像帧为止，这样的话，显然一些位置移动缓慢的目标对象，可以很快的被定为出在当前图像帧中的第二位置参数。
当然以上仅是一种基于第一位置参数确定第二位置参数的方式，但是不局限于上述方法。例如，所述步骤S142可包括：
基于在前一图像帧对所述目标对象的跟踪，确定所述目标对象在当前帧中的基准点，其中，所述基准点为表征所述目标对象在前一图像帧的显示位置的像素点；
确定在当前图像帧的各个特征点相对于所述基准点的偏移向量，其中，所述特征点为第一灰度值的第一像素点；所述第一灰度值与第一像素点邻近的第二像素点的第二灰度值的差异满足预设差异条件；
基于所述偏移向量，确定各个所述特征点相对于所述基准点的均值偏移向量，其中，所述均值偏移向量包括：均值偏移方向和均值偏移量；
基于所述基准点及所述均值偏移向量，定位所述目标对象对应的目标点；其中，所述目标点为下一图像帧的基准点，且与所述第二位置参数相对应。
在本实施例中所述基准点可为所述目标对象在前一图像帧中的中心位置，但是不局限于中心等。
在本实施例中首先会提取当前图像帧中各个特征点，这里的特征点同样是为与其周围的像素点的灰度差满足预设条件的像素点。构建以所述基准点为向量起始位置的指向各个特征点的偏移向量；获取各个偏移向量的偏移量，然后求取这个偏移量的均值，得到所述均值偏移向量的均值偏移量；再结合各个偏移向量，进行方向的矢量运算从而可确定出所述均值偏移向量对应的均值偏移方向。通常情况下，所述均值平移向量的偏移方向会指向特征点密度高的位置。在本实施例中所述目标点的位置，则可为以所述基准点为起点的所述均值偏移向量的终点。所述目标点可作为所述第二位置参数的组成部分；在对下一图像帧进行目标对象的跟踪时，可以以当前图像帧的目标点，作为下一个图像帧的基准点；从而进行反复迭代跟 踪，这种定位目标对象在当前图像帧的显示位置的方式，具有计算量小及实现简便的特点。
如图2所示，图2中每一个黑心实点表示一个特征点，空心原点表示基准点；单线箭头表示的偏移向量；空心箭头表征的为均值偏移向量，显然均值偏移向量是从当前的基准点出发，指向特征点分布密度较高的区域；均值偏移向量的均值偏移量等于所有偏移向量的偏移量的均值。
在一些实施例中，所述目标对象为显示在待识别图像的焦点区域内的图形对象。
每一帧图像的采集都可以是由摄像头基于特定的焦点位置进行，每一图像帧都有其对应的焦点区域；通常位于焦点区域的图像对象是最清晰的图形对象，也同时是用户重点关注的图像对象，在本实施例中为了减少识别工作量，所述目标对象为至少部分位于所述焦点区域内的图形对象。
在一些实施例中，所述方法还包括:
在获取所述AR信息的过程中，在所述视频的画面上显示获取提示，其中，所述获取提示，用于提示当前正在获取所述AR信息。
在一些情况下，若待识别图像的识别需要消耗一些时间，为了避免用户当前还未开始识别或识别出现故障，通过所述获取提示的显示，提示用户当前处于AR信息的获取过程中。所述获取提示可为文本信息，也可以为图像信息。例如，可为显示在所述当前图像帧上的半透明蒙层等，以进一步提升用户体验。
如图9所示，本发明实施例还提供另一种信息显示方法，用于在显示设备上实时显示目标对象的增强现实信息，该方法包括：
步骤S210：获取目标对象的视频流；该视频流包括多个连续采集单位视频帧；
步骤S220：接收视频流中目标对象的增强现实信息；视频帧中显示有 目标对象的图形，该增强现实信息可为各种与该目标对象相关的信息，例如，指示目标对象的属性的属性信息及指示目标对象的状态的状态信息；
步骤S230：根据特定算法获取所述目标对象在所述视频流的当前图像帧中的最新位置信息；以及
步骤S240：根据所述最新位置信息，将所述增强现实信息添加到所述当前图像帧中。
前述特定算法可为各跟踪目标的跟踪算法。在本实施例中，利用特定算法将获取当前图像帧中的目标对象的位置信息(即所述最新位置信息)，然后在步骤S240中基于最新位置信息，在当前图像中显示增强现实信息。
可选地，所述S230，可包括：
根据均值漂移算法跟踪所述目标对象在所述视频中各图像帧的位置信息，获得所述目标对象在当前显示的当前图像帧的位置参数。
所述步骤S240可包括：定位所述视频流中的前一图像帧所述目标对象的第一位置参数；基于所述第一位置参数，获取所述目标对象在所述当前图像帧中的第二位置参数。
例如，所述基于所述第一位置参数，获取所述目标对象在所述当前图像帧中的第二位置参数，包括：基于在前一图像帧对所述目标对象的跟踪，确定所述目标对象在当前图像帧中的基准点，其中，所述基准点为表征所述目标对象在前一图像帧的显示位置的像素点；确定在当前图像帧的各个特征点相对于所述基准点的偏移向量，其中，所述特征点为满足第一灰度值的第一像素点，且所述第一灰度值与第一像素点邻近的第二像素点的第二灰度值的差异满足预设差异条件；基于所述偏移向量，确定各个所述特征点相对于所述基准点的均值偏移向量，其中，所述均值偏移向量至少包括：均值偏移方向和均值偏移量；基于所述基准点及所述均值偏移向量，定位所述目标对象对应的目标点；其中，所述目标点为下一图像帧的基准 点，且与所述第二位置参数相对应。
以下结合上述任意一个实施例提供一个应用示例：
如图3所示，本示例提供一种AR处理方法，可以应用手机、智能眼镜等各种智能终端中，智能终端在采集视频时，需要进行AR信息添加(例如叠加)，以提升显示效果，具体可包括：
步骤S110：基于视频流，显示视频；
步骤S121：提取所述视频流中满足预设清晰条件的一帧或多帧待识别图像；其中，所述待识别图像中包括目标对象；
步骤S122：将所实话待识别图像发送给服务平台；
步骤S123：接收所述服务平台基于对所述待识别图像的识别结果返回的AR信息；
步骤S131：跟踪所述目标对象在各图像帧的显示位置，从而获得所述目标对象在当前图像帧的位置参数；
步骤S141：根据所述位置参数，将所述AR信息叠加显示在所述当前图像帧中。
图4中通过多个嵌套的虚线圆圈作为所述获取提示。
如图6所示，本实施例提供一种增强现实AR处理装置，应用于显示终端中，包括：
显示单元110，配置为基于视频流，显示视频；
获取单元120，配置为获取所述视频中目标对象的AR信息；
跟踪单元130，配置为跟踪所述目标对象在当前显示的所述视频的当前图像帧中的显示位置；
所述显示单元110，还配置为根据所述显示位置，将所述AR信息添加到所述当前图像帧中。
本实施例提供的显示终端可为各种包括显示屏的终端，这里的显示屏 可为液晶显示屏、电子墨水显示屏或投影显示屏等各种类型的显示屏。
所述获取单元120及跟踪单元130对应于终端中的处理器或处理电路。所述处理器可为中央处理器(CPU)、微处理器(MCU)、数字信号处理器(DSP)、应用处理器(AP)或可编程阵列(PLC)等。所述处理器电路可为专用集成电路(ASIC)。所述处理器或处理电路，可通过可执行代码的执行，实现上述操作。
总之，本实施例提供的装置，在显示终端进行AR显示时，会跟踪所述目标对象在视频中各帧图像中的显示位置，从而确保AR信息叠加显示在对应的目标对象附件，从而减少目标对象A的AR信息叠加到目标对象B的周围的现象，减少AR信息偏离目标对象的现象，提升用户体验。
可选地，所述获取单元120，配置为提取所述视频流中满足预设清晰条件的一帧或多帧待识别图像；获取基于一帧或多帧所述待识别图像中所述目标对象的识别结果对应的AR信息。
所述获取单元120，配置为将所述待识别图像发送给服务平台，其中，所述待识别图像用于在所述服务平台进行图像识别，以获得识别结果；接收所述服务平台基于所述识别结果返回的AR信息。
在本实施例中所述AR信息来自于服务平台，所述服务平台可以通过信息搜索，可向所述客户端提供尽可能多的信息，从而减少因为终端自身信息存储不够的导致的AR信息的不够丰富或信息量小的问题。
可选地，所述获取单元120，配置为提取所述视频流中至少部分图像的图像特征；依据所述图像特征，确定所述待识别图像是否满足所述预设清晰条件。
在本实施例中所述获取单元120，主要用于通过图像特征的提取，选择一帧或多帧足够清晰的图像，发送给服务平台或自身进行识别，提升识别精确度和识别成功的概率。
可选地，所述获取单元120，配置为提取所述视频流中至少部分图像的特征点，其中，所述特征点为第一灰度值的第一像素点；所述第一灰度值与第一像素点邻近的第二像素点的第二灰度值的差异满足预设差异条件；根据所述特征点的数目，判断满足所述预设清晰条件的所述待识别图像。
在本所实施例中通过特征点的提取，来确定出满足预设清晰条件的待识别图像，例如，采用FAST特征点的检测。所述FAST可为Features from Accelerated Segment Test的缩写。
在一些实施例中，所述跟踪单元130，配置为定位所述视频流中的前一图像帧所述目标对象的第一位置参数；基于所述第一位置参数，搜索所述目标对象在所述当前图像帧中的第二位置参数。
在本实施例中基于相邻两个图像帧位置参数的关联性，获取目标对象在当前图像帧中的第二位置参数，减少定位第二位置参数的计算量。
可选地，所述跟踪单元130，配置为基于在前一图像帧对所述目标对象的跟踪，确定所述目标对象在当前帧中的基准点，其中，所述基准点为表征所述目标对象在前一图像帧的显示位置的像素点；确定在当前图像帧的各个特征点相对于所述基准点的偏移向量，其中，所述特征点为第一灰度值的第一像素点；所述第一灰度值与第一像素点邻近的第二像素点的第二灰度值的差异满足预设差异条件；基于所述偏移向量，确定各个所述特征点相对于所述基准点的均值偏移向量，其中，所述均值偏移向量包括：均值偏移方向和均值偏移量；基于所述基准点及所述均值偏移向量，定位所述目标对象对应的目标点；其中，所述目标点为下一图像帧的基准点，且与所述第二位置参数相对应。
在本实施例中所述第一位置参数可包括前一图像帧的目标点的坐标；第二位置参数可为当前图像帧的目标点的坐标。通过均值偏移向量的确定，基于前一图像帧的目标点，快速定位出当前图像帧的目标点。
可选地，所述目标对象为显示在待识别图像的焦点区域内的图形对象。这样可以减少不必要图形对象的识别和AR信息的返回，减少不必要图形信息的显示，减少对用户的信息干扰。
可选地，所述显示单元110，还配置为在获取所述AR信息的过程中，在所述视频的画面上显示获取提示，其中，所述获取提示，用于提示当前正在获取所述AR信息。
在本实施例中通过所述获取提示的显示，可以提示用户当前正在获取AR信息，减少用户在等待过程中的焦虑状态，再次提升了用户体验。
如图7所示，本实施例提供一种显示终端，包括：
显示器210，配置为信息显示；
存储器220，配置为存储计算机程序；
处理器230，与所述显示器及所述存储器连接，配置为通过执行所述计算机程序，控制所述显示终端执行前述任意一个实施例提供的AR处理方法，例如，如图1提供的所述AR处理方法等。
在本实施例中所述显示器210可为各种类型的显示器、液晶显示器、投影显示器或电子墨水显示器等。
所述存储器220可为各种类型的存储介质，例如，随机存储介质、只读存储介质、闪存或光碟等。在本实施例中所述存储器220至少包括部分非瞬间存储介质，这里的非瞬间存储介质，可用于存储所述计算机程序。
所述处理器230可为CPU、MCU、DSP、AP或PLC或ASIC等各种处理器或处理电路，可通过计算机程序的执行，执行显示器210在视频的当前图像帧上叠加显示AR信息。
如图7所示，所述终端显示器210、存储器220、处理器230均通过总线250连接，所述总线250可包括可如集成电路总线(IIC)总线或外设部件互连标准(PCI)总线等。
在一些实施例中所述客户端还可包括网络接口240，该网络接口240可用于连接到网络侧，与所述服务平台连接。
本实施例还提供一种计算机存储介质，所述计算机存储介质中存储计算机程序；所述计算机程序，用于被处理器执行后能够实现前述任意一个实施例提供的AR处理方法，例如，如图1所示的AR处理方法等。
所述计算机存储介质可为各种类型的存储介质，可选为非瞬间存储介质。所述计算机存储介质，可选为移动存储设备、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码等介质。
以下结合上述任意实施例提供一个具体示例：
如图8所示，本示例提供一种AR处理系统，包括：
客户端及服务器端；
所述客户端，可为显示AR信息的所述终端；
所述服务器端，可为所述客户端提供AR处理支持的网络侧的服务平台。
所述客户端，包括：
内核模块，对应于操作系统的内核，可以用于进行后台的信息处理，例如，可通过与服务平台的交互，获取AR信息；
AR引擎(SDK)，用于获取目标对象的位置信息，基于位置信息将对应的位置参数发送给显示屏；
显示屏，对应于显示用户界面，可以用于视频显示，基于ARSDK提供的位置参数，将从内核模块转发的AR信息，将AR信息正确叠加到对应的目标对象附近。
所述服务器端可包括：代理服务器、识别服务器及搜索服务器；
所述代理服务器，用于与客户端进行信息交互，例如，接收客户端发 送的待识别图像；
所述识别服务器与所述代理服务器连接，用于接收代理服务器转发的待识别图像，然后将识别结果给到搜索服务器；
所述搜索服务器与识别服务器相连，用于基于搜索结果，查询AR信息，并将所述AR信息通过代理服务器发送给客户端。
以下具体提供一种AR信息在上述系统中的应用方法，包括：
物体实景扫描跟踪，是终端长传实时图片云端识别后再有终端展示的过程。本示例中终端的后台与云端的识别服务器、搜索服务器和进行信息整合的代理服务器连接。终端包含ARSDK、终端与网络平台数据传输的数据传输单元和与用户进行显示交互的UI单元。具体流程如下
终端通过应用打开摄像头，此时视频流会由UI单元导入终端网络传输单元，网络传输单元通过FAST特征点检测，FAST特征点的数量可以代表图片中的物体是否具备足够的识别条件，满足特征点要求的图像是足够预设清晰条件的图像，那么我们把此帧图像传入后台代理服务器。
后台代理服务器，收到终端上传图片，将此图片发送给云端识别服务器，云端识别服务器会进行图像泛化识别，出别出图像中物体类别、位置、个数信息，然后传给后台代理服务器
后台代理服务器拿到图像类别信息之后，将信息发送给咨询搜索中心服务器，去捞取此物体类别相关咨询信息，如果没有相关资讯信息那么直接返回空。此时代理服务器在将图像的信息和咨询信息发送给终端。这里的相关资讯信息为前述的AR信息的一种。
终端有模块接收，如果次数将信息直接传给UI绘制，那么由于网络传输和识别耗时，此时当前的物体运动可能发生变化，如果还按照上传那帧画面的位置绘制，很可能出现绘制偏移。所以，此时数据传输模块并不是将信息塞给UI，而是发给ARSDK去获取位置的更新。
ARSDK将传输模块的图像帧数据传给本地跟随模块，本地跟随模块使用均值漂移算法，先算出第一帧画面特征点点的偏移均值，然后以此为新的起始点，再次查处找下一帧图像对应特征点移动位置，可以一直跟随图像中物体，可以实时获取物体的位置信息，此时收到内核传来的物体，便可以将此物体的最新位置传给内核模块。
内核更新到了物体最新位置信息，将咨询信息、类别信息和位置信息一同传给UI单元，UI单元收到相关信息之后，边可以在屏幕上绘制标识了。而且位置会准确无误。
具体如图8所示，所述AR信息处理方法可包括：
步骤1：显示屏向内核模块提供视频流；
步骤2：内核模块向代理服务器提供待识别图像；
步骤3：代理服务器向识别服务器提供待识别图像；
步骤4：识别服务器将识别结果给到代理服务器，在一些实施例中所述识别服务器可以直接向搜索服务器反馈所述识别结果；
步骤5：代理服务器将识别结果发送给搜索服务器；
步骤6：搜索服务器将基于识别结果搜索到的AR信息返回给代理服务器；
步骤7：代理服务器将AR信息转发给客户端的内核模块；
步骤8：内核模块向ARSDK获取新的位置参数；这里的位置参数为目标对象在当前图像帧中的位置参数；
步骤9：ARSDK将更新的位置参数发送给内核模块；
步骤10：内核模块将AR信息和更新后的位置参数返回给显示屏，可显示屏在显示视频的同时，在目标对象的附近叠加显示AR信息。
在本申请所提供的几个实施例中，应该理解到，所揭露的设备和方法，可以通过其它的方式实现。以上所描述的设备实施例仅仅是示意性 的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，如：多个单元或组件可以结合，或可以集成到另一个系统，或一些特征可以忽略，或不执行。另外，所显示或讨论的各组成部分相互之间的耦合、或直接耦合、或通信连接可以是通过一些接口，设备或单元的间接耦合或通信连接，可以是电性的、机械的或其它形式的。
上述作为分离部件说明的单元可以是、或也可以不是物理上分开的，作为单元显示的部件可以是、或也可以不是物理单元，即可以位于一个地方，也可以分布到多个网络单元上；可以根据实际的需要选择其中的部分或全部单元来实现本实施例方案的目的。
另外，在本申请各实施例中的各功能单元可以全部集成在一个处理模块中，也可以是各单元分别单独作为一个单元，也可以两个或两个以上单元集成在一个单元中；上述集成的单元既可以采用硬件的形式实现，也可以采用硬件加软件功能单元的形式实现。
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。

Claims (23)

  1. 一种增强现实AR处理方法，应用于显示终端中，包括：
    基于视频流，显示视频；
    获取所述视频中目标对象的AR信息；
    跟踪所述目标对象在当前显示的所述视频的当前图像帧中的显示位置；
    根据所述显示位置，将所述AR信息添加到所述当前图像帧中。
  2. 根据权利要求1所述的方法，其中，
    所述获取所述视频中目标对象的AR信息，包括：
    提取所述视频流中满足预设清晰条件的一帧或多帧待识别图像；
    获取基于一帧或多帧所述待识别图像中所述目标对象的识别结果对应的AR信息。
  3. 根据权利要求2所述的方法，其中，
    所述获取基于一帧或多帧所述待识别图像中目标对象的识别结果对应的AR信息，包括：
    将所述待识别图像发送给服务平台，其中，所述待识别图像用于在所述服务平台进行图像识别，以获得识别结果；
    接收所述服务平台基于所述识别结果返回的AR信息。
  4. 根据权利要求2所述的方法，其中，
    所述提取所述视频流中满足预设清晰条件的一帧或多帧待识别图像，包括：
    提取所述视频流中至少部分图像的特征点，其中，所述特征点为第一灰度值的第一像素点；所述第一灰度值与第一像素点邻近的第二像素点的第二灰度值的差异满足预设差异条件；
    根据所述特征点的数目，判断满足所述预设清晰条件的所述待识别图 像。
  5. 根据权利要求1至3任一项所述的方法，其中，
    所述跟踪所述目标对象在当前显示的所述视频的当前图像帧中的显示位置，包括：
    跟踪所述目标对象在所述视频中各图像帧的显示位置，从而获得所述目标对象在当前显示的当前图像帧的位置参数；
    所述根据所述显示位置，将所述AR信息叠加到视频的图像帧中，包括：
    根据所述位置参数，将所述AR信息叠加在所述当前图像帧中与所述位置参数对应的位置。
  6. 根据权利要求5所述的方法，其中，
    所述跟踪所述目标对象在所述视频中各图像帧的显示位置，从而获得所述目标对象在当前显示的当前图像帧的位置参数，包括：
    定位所述视频流中的前一图像帧所述目标对象的第一位置参数；
    基于所述第一位置参数，搜索所述目标对象在所述当前图像帧中的第二位置参数。
  7. 根据权利要求6所述的方法，其中，
    所述基于所述第一位置参数，搜索所述目标对象在所述当前图像帧中的第二位置参数，包括：
    基于在前一图像帧对所述目标对象的跟踪，确定所述目标对象在当前帧中的基准点，其中，所述基准点为表征所述目标对象在前一图像帧的显示位置的像素点；
    确定在当前图像帧的各个特征点相对于所述基准点的偏移向量，其中，所述特征点为第一灰度值的第一像素点；所述第一灰度值与第一像素点邻近的第二像素点的第二灰度值的差异满足预设差异条件；
    基于所述偏移向量，确定各个所述特征点相对于所述基准点的均值偏 移向量，其中，所述均值偏移向量包括：均值偏移方向和均值偏移量；
    基于所述基准点及所述均值偏移向量，定位所述目标对象对应的目标点；其中，所述目标点为下一图像帧的基准点，且与所述第二位置参数相对应。
  8. 根据权利要求1至3任一项所述的方法，其中，
    所述目标对象为显示在待识别图像的焦点区域内的图形对象。
  9. 根据权利要求1至3任一项所述的方法，还包括:
    在获取所述AR信息的过程中，在所述视频的画面上提示当前正在获取所述AR信息。
  10. 一种增强现实AR处理装置，其中，应用于显示终端中，包括：
    显示单元，配置为基于视频流，显示视频；
    获取单元，配置为获取所述视频中目标对象的AR信息；
    跟踪单元，配置为跟踪所述目标对象在当前显示的所述视频的当前图像帧中的显示位置；
    所述显示单元，还配置为根据所述显示位置，将所述AR信息添加到所述当前图像帧中。
  11. 根据权利要求10所述的装置，其中，
    所述获取单元，配置为于提取所述视频流中满足预设清晰条件的一帧或多帧待识别图像；获取基于一帧或多帧所述待识别图像中所述目标对象的识别结果对应的AR信息。
  12. 根据权利要求11所述的装置，其中，
    所述获取单元，配置为将所述待识别图像发送给服务平台，其中，所述待识别图像用于在所述服务平台进行图像识别，以获得识别结果；接收所述服务平台基于所述识别结果返回的AR信息。
  13. 根据权利要求11所述的装置，其中，
    所述获取单元，配置为提取所述视频流中至少部分图像的特征点，其中，所述特征点为第一灰度值的第一像素点；所述第一灰度值与第一像素点邻近的第二像素点的第二灰度值的差异满足预设差异条件；及根据所述特征点的数目，判断满足所述预设清晰条件的所述待识别图像。
  14. 根据权利要求10至13任一项所述的装置，其中，
    所述跟踪单元，配置为跟踪所述目标对象在所述视频中各图像帧的显示位置，从而获得所述目标对象在当前显示的当前图像帧的位置参数；
    所述显示单元，具体用于根据所述位置参数，将所述AR信息叠加显示在所述当前图像帧中。
  15. 根据权利要求14所述的装置，其中，
    所述跟踪单元，配置为基于在前一图像帧对所述目标对象的跟踪，确定所述目标对象在当前帧中的基准点，其中，所述基准点为表征所述目标对象在前一图像帧的显示位置的像素点；确定在当前图像帧的各个特征点相对于所述基准点的偏移向量，其中，所述特征点为第一灰度值的第一像素点；所述第一灰度值与第一像素点邻近的第二像素点的第二灰度值的差异满足预设差异条件；基于所述偏移向量，确定各个所述特征点相对于所述基准点的均值偏移向量，其中，所述均值偏移向量包括：均值偏移方向和均值偏移量；基于所述基准点及所述均值偏移向量，定位所述目标对象对应的目标点；其中，所述目标点为下一图像帧的基准点，且与第二位置参数相对应。
  16. 根据权利要求10至13任一项所述的装置，其中，
    所述目标对象为显示在待识别图像的焦点区域内的图形对象。
  17. 根据权利要求10至13任一项所述的装置，其中，
    所述显示单元，还配置为在获取所述AR信息的过程中，在所述视频的画面上显示获取提示提示当前正在获取所述AR信息。
  18. 一种显示终端，包括：
    显示器，配置为信息显示；
    存储器，配置为存储计算机程序；
    处理器，与所述显示器及所述存储器连接，配置为通过执行所述计算机程序，控制所述显示终端执行权利要求1至9任一项所述的AR处理方法。
  19. 一种信息显示方法，用于在显示设备上实时显示目标对象的增强现实信息，该方法包括：
    获取目标对象的视频流；
    接收视频流中目标对象的增强现实信息；
    根据特定算法获取所述目标对象在所述视频流的当前图像帧中的最新位置信息；以及
    根据所述最新位置信息，将所述增强现实信息添加到所述当前图像帧中。
  20. 根据权利要求19所述的方法，其中，所述根据特定算法获取所述目标对象在所述视频流的当前图像帧中的最新位置信息，包括：
    根据均值漂移算法跟踪所述目标对象在所述视频中各图像帧的位置信息，获得所述目标对象在当前显示的当前图像帧的位置参数。
  21. 根据权利要求20所述的方法，其中，所述跟踪所述目标对象在所述视频中各图像帧的显示位置，包括：
    定位所述视频流中的前一图像帧所述目标对象的第一位置参数；
    基于所述第一位置参数，获取所述目标对象在所述当前图像帧中的第二位置参数。
  22. 根据权利要求21所述的方法，其中，所述基于所述第一位置参数，获取所述目标对象在所述当前图像帧中的第二位置参数，包括：
    基于在前一图像帧对所述目标对象的跟踪，确定所述目标对象在当前 图像帧中的基准点，其中，所述基准点为表征所述目标对象在前一图像帧的显示位置的像素点；
    确定在当前图像帧的各个特征点相对于所述基准点的偏移向量，其中，所述特征点为满足第一灰度值的第一像素点，且所述第一灰度值与第一像素点邻近的第二像素点的第二灰度值的差异满足预设差异条件；
    基于所述偏移向量，确定各个所述特征点相对于所述基准点的均值偏移向量，其中，所述均值偏移向量至少包括：均值偏移方向和均值偏移量；
    基于所述基准点及所述均值偏移向量，定位所述目标对象对应的目标点；其中，所述目标点为下一图像帧的基准点，且与所述第二位置参数相对应。
  23. 一种计算机存储介质，所述计算机存储介质中存储计算机程序；所述计算机程序，用于被处理器执行后能够实现权利要求1至9或19至22任一项提供的AR处理方法。
PCT/CN2018/080094 2017-05-15 2018-03-22 增强现实处理方法及装置、显示终端及计算机存储介质 WO2018210055A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710340898.0 2017-05-15
CN201710340898.0A CN108875460B (zh) 2017-05-15 2017-05-15 增强现实处理方法及装置、显示终端及计算机存储介质

Publications (1)

Publication Number Publication Date
WO2018210055A1 true WO2018210055A1 (zh) 2018-11-22

Family

ID=64273268

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/080094 WO2018210055A1 (zh) 2017-05-15 2018-03-22 增强现实处理方法及装置、显示终端及计算机存储介质

Country Status (3)

Country Link
CN (1) CN108875460B (zh)
TW (1) TWI669956B (zh)
WO (1) WO2018210055A1 (zh)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110619266A (zh) * 2019-08-02 2019-12-27 青岛海尔智能技术研发有限公司 目标物识别方法、装置和冰箱
CN110784733A (zh) * 2019-11-07 2020-02-11 广州虎牙科技有限公司 直播数据处理方法、装置、电子设备及可读存储介质
CN110856005A (zh) * 2019-11-07 2020-02-28 广州虎牙科技有限公司 直播流显示方法、装置、电子设备及可读存储介质
CN112734938A (zh) * 2021-01-12 2021-04-30 北京爱笔科技有限公司 行人位置预测方法、装置、计算机设备和存储介质
CN113596350A (zh) * 2021-07-27 2021-11-02 深圳传音控股股份有限公司 图像处理方法、移动终端及可读存储介质
CN115361576A (zh) * 2022-07-20 2022-11-18 中国电信股份有限公司 视频数据处理方法、装置，以及，电子设备

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113386785B (zh) * 2019-07-03 2022-11-22 北京百度网讯科技有限公司 用于显示增强现实警示信息的方法和装置
CN112445318A (zh) * 2019-08-30 2021-03-05 龙芯中科技术股份有限公司 一种对象显示方法、装置、电子设备及存储介质
CN111583329B (zh) * 2020-04-09 2023-08-04 深圳奇迹智慧网络有限公司 增强现实眼镜显示方法、装置、电子设备和存储介质
CN112017300A (zh) * 2020-07-22 2020-12-01 青岛小鸟看看科技有限公司 混合现实图像的处理方法、装置及设备
CN112328628A (zh) * 2020-11-10 2021-02-05 山东爱城市网信息技术有限公司 基于ar技术的公交实时查询方法及系统
CN112583976B (zh) * 2020-12-29 2022-02-18 咪咕文化科技有限公司 图形码显示方法、设备及可读存储介质
CN113038001A (zh) * 2021-02-26 2021-06-25 维沃移动通信有限公司 显示方法、装置及电子设备
CN114419699A (zh) * 2021-12-28 2022-04-29 青岛畅索科技有限公司 图像信息处理方法、系统、设备及存储介质
CN114415839A (zh) * 2022-01-27 2022-04-29 歌尔科技有限公司 一种信息显示方法、装置、设备及存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102479251A (zh) * 2010-11-29 2012-05-30 株式会社泛泰 使用增强现实数据库提供增强现实的移动终端和方法
CN103119627A (zh) * 2010-09-20 2013-05-22 高通股份有限公司 用于云辅助式扩增现实的可适应性框架
CN104541300A (zh) * 2012-09-28 2015-04-22 英特尔公司 增强现实信息的确定
CN104936034A (zh) * 2015-06-11 2015-09-23 三星电子（中国）研发中心 基于视频的信息录入方法及装置
CN105103198A (zh) * 2013-04-04 2015-11-25 索尼公司 显示控制装置、显示控制方法以及程序
CN105760849A (zh) * 2016-03-09 2016-07-13 北京工业大学 基于视频的目标对象行为数据获取方法及装置

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101750017A (zh) * 2010-01-18 2010-06-23 战强 大视场中多移动目标位姿的视觉检测方法
CN101807300B (zh) * 2010-03-05 2012-07-25 北京智安邦科技有限公司 一种目标碎片区域融合的方法及装置
US9996150B2 (en) * 2012-12-19 2018-06-12 Qualcomm Incorporated Enabling augmented reality using eye gaze tracking
CN103426184B (zh) * 2013-08-01 2016-08-10 华为技术有限公司 一种光流跟踪方法和装置
KR102114618B1 (ko) * 2014-01-16 2020-05-25 엘지전자 주식회사 포터블 디바이스 및 그 제어 방법
CN105654512B (zh) * 2015-12-29 2018-12-07 深圳微服机器人科技有限公司 一种目标跟踪方法和装置
CN105635712B (zh) * 2015-12-30 2018-01-19 视辰信息科技(上海)有限公司 基于增强现实的视频实时录制方法及录制设备
CN105760826B (zh) * 2016-02-03 2020-11-13 歌尔股份有限公司 一种人脸跟踪方法、装置和智能终端
CN106056046B (zh) * 2016-05-20 2019-01-18 北京集创北方科技股份有限公司 从图像中提取特征的方法和装置
CN106454350A (zh) * 2016-06-28 2017-02-22 中国人民解放军陆军军官学院 一种用于红外图像的无参考评价方法
CN106250938B (zh) * 2016-07-19 2021-09-10 易视腾科技股份有限公司 目标跟踪方法、增强现实方法及其装置
CN106371585B (zh) * 2016-08-23 2024-05-28 孙文 扩增实境系统及其方法
CN106650965B (zh) * 2016-12-30 2020-11-06 触景无限科技（北京）有限公司 一种远程视频处理方法及装置

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103119627A (zh) * 2010-09-20 2013-05-22 高通股份有限公司 用于云辅助式扩增现实的可适应性框架
CN102479251A (zh) * 2010-11-29 2012-05-30 株式会社泛泰 使用增强现实数据库提供增强现实的移动终端和方法
CN104541300A (zh) * 2012-09-28 2015-04-22 英特尔公司 增强现实信息的确定
CN105103198A (zh) * 2013-04-04 2015-11-25 索尼公司 显示控制装置、显示控制方法以及程序
CN104936034A (zh) * 2015-06-11 2015-09-23 三星电子（中国）研发中心 基于视频的信息录入方法及装置
CN105760849A (zh) * 2016-03-09 2016-07-13 北京工业大学 基于视频的目标对象行为数据获取方法及装置

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110619266A (zh) * 2019-08-02 2019-12-27 青岛海尔智能技术研发有限公司 目标物识别方法、装置和冰箱
CN110619266B (zh) * 2019-08-02 2024-01-19 青岛海尔智能技术研发有限公司 目标物识别方法、装置和冰箱
CN110784733A (zh) * 2019-11-07 2020-02-11 广州虎牙科技有限公司 直播数据处理方法、装置、电子设备及可读存储介质
CN110856005A (zh) * 2019-11-07 2020-02-28 广州虎牙科技有限公司 直播流显示方法、装置、电子设备及可读存储介质
CN112734938A (zh) * 2021-01-12 2021-04-30 北京爱笔科技有限公司 行人位置预测方法、装置、计算机设备和存储介质
CN113596350A (zh) * 2021-07-27 2021-11-02 深圳传音控股股份有限公司 图像处理方法、移动终端及可读存储介质
CN113596350B (zh) * 2021-07-27 2023-11-17 深圳传音控股股份有限公司 图像处理方法、移动终端及可读存储介质
CN115361576A (zh) * 2022-07-20 2022-11-18 中国电信股份有限公司 视频数据处理方法、装置，以及，电子设备

Also Published As

Publication number Publication date
TWI669956B (zh) 2019-08-21
CN108875460B (zh) 2023-06-20
CN108875460A (zh) 2018-11-23
TW201902225A (zh) 2019-01-01

Similar Documents

Publication Publication Date Title
WO2018210055A1 (zh) 增强现实处理方法及装置、显示终端及计算机存储介质
KR102166861B1 (ko) 시선 추적을 이용한 증강 현실의 인에이블링
US9424255B2 (en) Server-assisted object recognition and tracking for mobile devices
US9924102B2 (en) Image-based application launcher
EP2852933B1 (en) Image-driven view management for annotations
CN108932051B (zh) 增强现实图像处理方法、装置及存储介质
KR101259957B1 (ko) 이미지 매칭을 이용한 부가 정보 제공 시스템 및 방법
US9418292B2 (en) Methods, apparatuses, and computer program products for restricting overlay of an augmentation
CN106197445B (zh) 一种路线规划的方法及装置
CN106982240B (zh) 信息的显示方法和装置
JP2012103789A (ja) オブジェクト表示装置及びオブジェクト表示方法
EP2791883A2 (en) Information processing device, information processing method and program
EP3159829A1 (en) Methods of detecting and managing a fiducial marker displayed on a display device
CN110689014B (zh) 感兴趣区域的检测方法、装置、电子设备及可读存储介质
US10496694B2 (en) Rating and advising for selection of augmented reality markers
JP2012185591A (ja) 画像処理装置、情報提供システム及びプログラム
CN115619904A (zh) 图像处理方法、装置及设备
US10108882B1 (en) Method to post and access information onto a map through pictures
US20130208990A1 (en) Image comparison device, image comparison method, image comparison system, server, terminal, terminal control method, and terminal control program
JP2018151829A (ja) プログラム、情報端末、情報表示方法及び情報表示システム
WO2017161476A1 (en) Rating and advising for selection of augmented reality markers
JP5650060B2 (ja) 画像追跡装置及び画像追跡方法
CN113744172A (zh) 文档图像的处理方法及装置、训练样本的生成方法及装置
JP6566652B2 (ja) サーバ、端末特定方法および端末特定プログラム
JP2019101934A (ja) 速度推定装置、速度推定方法、及び速度推定プログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18801898

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18801898

Country of ref document: EP

Kind code of ref document: A1