WO2019128742A1 - 图像处理方法、装置、终端及存储介质 - Google Patents
图像处理方法、装置、终端及存储介质 Download PDFInfo
- Publication number
- WO2019128742A1 WO2019128742A1 PCT/CN2018/121268 CN2018121268W WO2019128742A1 WO 2019128742 A1 WO2019128742 A1 WO 2019128742A1 CN 2018121268 W CN2018121268 W CN 2018121268W WO 2019128742 A1 WO2019128742 A1 WO 2019128742A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- feature point
- target
- frame image
- point set
- feature
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/04845—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/0485—Scrolling or panning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
- G06F2203/048—Indexing scheme relating to G06F3/048
- G06F2203/04806—Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
Definitions
- the present application relates to image processing technologies, and in particular, to an image processing method, apparatus, terminal, and storage medium.
- the annotation function in the existing screen sharing only supports the static image. For example, in the static state, that is, the screen does not perform the scrolling or zooming operation, the annotation information can be shared, but if the screen is operated For example, if the scrolling or zooming operation is performed, the previous annotation information will disappear, which greatly reduces the ease of use of the annotation function in the screen sharing scene and limits the usage scenario of the annotation function.
- An image processing method, apparatus, terminal, and storage medium are provided according to various embodiments of the present application.
- An image processing method is performed by a terminal, the method comprising:
- an annotation area in the first frame image displayed by the display content is determined, and a first feature point set characterizing the annotation area is determined, wherein the annotation area corresponds to annotation information ;
- An image processing apparatus comprising:
- a first determining unit in a state for displaying content sharing, determining an annotation area in the first frame image displayed by the display content, and determining a first feature point set characterizing the annotation area, where The annotation area corresponds to annotation information; and is further configured to determine a second frame image to obtain a second feature point set characterizing the second frame image, wherein the second frame image is associated with the first frame image Image;
- a feature point matching unit configured to match the second feature point set with the first feature point set, and select, according to the matching result, the feature in the first feature point set from the second feature point set Point matching target feature points to obtain a target feature point set;
- a second determining unit configured to determine, according to the target feature point set, a target annotation area that matches an annotation area of the first frame image in the second frame image, where the target annotation area corresponds to The annotation information matching the annotation information of the annotation area in the first frame image.
- a terminal comprising a memory and a processor, the memory storing computer readable instructions, the computer readable instructions being executed by the processor such that the processor performs the following steps:
- an annotation area in the first frame image displayed by the display content is determined, and a first feature point set characterizing the annotation area is determined, wherein the annotation area corresponds to annotation information ;
- a non-transitory computer readable storage medium storing computer readable instructions, when executed by one or more processors, causes the one or more processors to perform the following steps:
- an annotation area in the first frame image displayed by the display content is determined, and a first feature point set characterizing the annotation area is determined, wherein the annotation area corresponds to annotation information ;
- An image processing apparatus comprising a display component and a processor:
- the display component is configured to display display content on the display interface
- the processor is configured to send the display content displayed by the display interface to other electronic devices to share the display content displayed by the display interface with other electronic devices;
- the processor is further configured to determine, in a state in which content sharing is displayed, an annotation area in a first frame image displayed by the display content, and determine a first feature point set that represents the annotation area, where The annotation area corresponds to annotation information; determining a second frame image to obtain a second feature point set characterizing the second frame image, wherein the second frame image is associated with the first frame image And matching the second feature point set with the first feature point set, and selecting, from the second feature point set, a target that matches the feature point in the first feature point set based on the matching result; a feature point, a target feature point set is obtained; and a target annotation area matching the annotation area of the first frame image in the second frame image is determined based on the target feature point set, wherein the target annotation area corresponds to There is annotation information that matches the annotation information of the annotation area in the first frame image.
- An image processing apparatus includes a processor and a display component:
- the processor is configured to acquire display content shared by other electronic devices
- the display component is configured to display, on the display interface, display content shared by other acquired electronic devices;
- the processor is further configured to determine, in a state in which content sharing is displayed, an annotation area in a first frame image displayed by the display content, and determine a first feature point set that represents the annotation area, where The annotation area corresponds to annotation information; determining a second frame image to obtain a second feature point set characterizing the second frame image, wherein the second frame image is associated with the first frame image And matching the second feature point set with the first feature point set, and selecting, from the second feature point set, a target that matches the feature point in the first feature point set based on the matching result; a feature point, a target feature point set is obtained; and a target annotation area matching the annotation area of the first frame image in the second frame image is determined based on the target feature point set, wherein the target annotation area corresponds to There is annotation information that matches the annotation information of the annotation area in the first frame image.
- FIG. 1 is a schematic flowchart of an implementation process of an image processing method according to an embodiment of the present invention
- FIG. 2 is a schematic diagram of a display interface after annotating in a state in which content sharing is displayed according to an embodiment of the present invention
- FIG. 3 is a schematic diagram of a display interface scrolling operation after an annotation is performed in a state in which content is shared and an annotation is performed;
- FIG. 4 is a schematic diagram of a selection rule of a feature point of a target center according to an embodiment of the present invention
- FIG. 5 is a schematic flowchart of an implementation process of an image processing method according to an embodiment of the present invention.
- FIG. 6 is a schematic diagram of an application flow of an annotation performed by a sender terminal in a display content sharing scenario according to an embodiment of the present invention
- FIG. 7 is a schematic diagram of an application flow of an annotation performed by a receiver terminal in a display content sharing scenario according to an embodiment of the present invention.
- FIG. 8 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present invention.
- the previous annotation information will also disappear, but in the actual application, the annotation information is strongly pointed to a shared content point, which is required for communication and discussion in the off-site.
- the reliable information recorded has the need to look back and summarize the deposit at any time. Therefore, the disappearance of the annotation information will make the annotation function a temporary writing function, which limits the usage scenario.
- the embodiment provides an image processing method. Specifically, the solution is proposed in the prior art for sharing the annotation information, not adapting to the dynamic position change, and not adapting to the scaling. On the basis of solving the above problems, the following functions can be realized, namely:
- the user can perform screen operations through the mouse, and at the same time, support the annotation function, for example, creating a comment area requiring a key annotation by mouse click and drag, and generating and displaying an annotation. information.
- the existing annotation information dynamically changes according to the movement and/or scaling of the current screen content, and ensures that the annotation information accurately corresponds (such as box selection) to the annotation area of the original key annotation. .
- the content of the annotation area is incomplete due to occlusion display, or after scrolling out of the screen with the display content, for example, it is detected that there is no annotation area in the currently displayed content, or the existing annotation information does not correspond to the currently displayed content. After that, stop displaying the comment information.
- the partial annotation information may be displayed in a proportional manner, or the annotation information may be stopped, which is not limited in this embodiment, and may be Arbitrarily set for actual needs.
- the annotation area is returned to the screen range. For example, after detecting that the annotation area reappears, the annotation information is displayed at the corresponding position, and thus the annotation information is reproduced, and the annotation information is returned.
- the screen sharing is divided into a screen sharing sender terminal and a receiver terminal, and the method described in this embodiment supports the two ends (eg, the sender terminal or the receiver terminal) to annotate the shared display content in the screen sharing scenario. That is, whether the annotation information marked by the sender terminal for the shared display content or the annotation information of the recipient terminal for annotating the shared display content can be shared.
- the method described in this embodiment is not limited by the sender terminal or the receiver terminal, and can be implemented at both ends.
- FIG. 1 is a schematic flowchart of an implementation of an image processing method according to an embodiment of the present invention. As shown in FIG. 1 , the method includes:
- Step 101 In the state where the content sharing is displayed, the annotation area in the first frame image displayed by the display content is determined, and the first feature point set representing the annotation area is determined, wherein the annotation area corresponds to the annotation information.
- the first frame image may be an annotation area selected in the annotation state, and an image corresponding to one frame after the completion of the annotation information is edited.
- the first frame image may also be a frame image selected after the annotation area is selected in the annotation state, and after the annotation information is completed, and before the display content is scrolled and/or scaled.
- the annotation area represents an area for highlighting at least part of the content of the shared display content.
- the annotation area may be used to frame part of the content of the display content that needs to be highlighted.
- the annotation information may be text information, a comment box, and the like for explaining at least part of the content corresponding to the annotation area.
- the annotation information includes but is not limited to at least one of the following: a wireframe for displaying the content in the frame selection part, and text information. , comment boxes, etc. That is to say, the annotation information of the embodiment includes, but is not limited to, any information that can be obtained by editing under the existing annotation function.
- the existing annotation function includes five types: a straight line, an arrow, a brush, a box, and a text.
- the annotation information includes, but is not limited to, at least one type of information that can be obtained by the five types.
- FIG. 2 is a schematic diagram of a display interface after an annotation is performed in a state in which content sharing is displayed according to an embodiment of the present invention.
- the annotation information includes a wire frame of the frame selection annotation area, and text information displayed around the wire frame. .
- Step 102 Determine a second frame image, and obtain a second feature point set that represents a second frame image, where the second frame image is an image associated with the first frame image;
- the second frame image is a frame image that appears after the first frame image, for example, the second frame image is an image obtained after a scroll operation for the first frame image.
- FIG. 3 is a schematic diagram of the display interface scrolling operation after the annotation is performed in the state of displaying the content sharing according to the embodiment of the present invention. As shown in FIG. 3, after the display content is scrolled, the annotation information changes according to the position change of the original annotation area. The resulting image, that is, the second frame image. The annotation information matching the annotation information of the first frame image is displayed in the second frame image.
- the feature point set includes a plurality of feature points, each of which can represent a local feature of the corresponding image.
- the first feature point set includes at least two first feature points, and the first feature point can represent local feature information of the annotation area.
- the second feature point set includes at least two second feature points, and the second feature point can represent local feature information of the second frame image.
- the image scaling problem may exist in the actual application. Therefore, in order to avoid the accurate tracking of the annotation information after the image scaling processing, the feature points determined in this embodiment cannot change with the scaling of the image, but only in the image scaling. Thereafter, the position of the feature points, and/or the distance between the feature points changes.
- a feature-invariant correlation feature algorithm can be used to determine the feature points of the image, for example, using the SIFT (Scale Invariant Feature Transform) algorithm, the BRISK (Binary Robust Invariant Scalable Keypoints) algorithm, or the FAST (Features From Accelerated Segment Test).
- An algorithm or the like extracts feature points of the annotation area of the first frame image, and extracts a specific point of the second frame image.
- the feature points extracted by the above algorithm can ensure that they do not change with the image scaling, but the position of the feature points and/or the distance between the feature points will change after the image is scaled.
- Step 103 Matching the second feature point set with the first feature point set, and selecting a target feature point that matches the feature point in the first feature point set from the second feature point set based on the matching result, to obtain the target feature point. set;
- the process of performing matching is equivalent to the process of similarity judgment, that is, determining the similarity between the second feature point in the second feature point set and the first feature point in the first feature point set, and further In the second set of feature points, the point with the highest similarity of the first feature point in the first feature point set, that is, the target feature point, is selected to finally obtain the target feature point set that matches the first feature point set.
- the matching process ie the process of determining similarity
- step 103 may be specifically: determining a distance feature between the second feature point in the second feature point set and the first feature point in the first feature point set, and selecting from the second feature point set. The distance feature satisfies the target feature point of the preset distance rule.
- the feature points may be identified by a feature vector.
- the vector A (x1, x2, ..., xn) is used to represent a specific first feature point in the first feature point set, and the vector B (y1, y2 is utilized).
- n is a positive integer greater than or equal to 2; at this time, the Euclidean distance between the feature point A and the feature point B is:
- the Euclidean distance between the specific first feature point A and all the second feature points in the second frame image is determined by using the Euclidean distance, and then the second feature having the smallest Euclidean distance from the specific first feature point A is selected.
- the second feature point having the smallest Euclidean distance from the specific first feature point A is the target feature point that best matches the specific first feature point A.
- the method of the embodiment may also determine an image movement feature transformed from the first frame image to the second frame image, and estimate from the second frame image based on the image movement feature.
- a target feature point matching the feature points in the first feature point set is obtained, and a first estimated target feature point set is obtained.
- the optical flow method is used to determine the optical flow characteristics of the first frame image to the second frame image, and the optical flow characteristics are predicted from the second frame image to match the feature points in the first feature point set.
- the target feature point is obtained, and the first estimated target feature point set is obtained.
- step 103 is specifically: selecting, according to the matching result, a target feature point that matches a feature point in the first feature point set from the second feature point set, and obtaining a second estimated target feature point set, and further An estimated target feature point set and a second predicted target feature point set are obtained to obtain a target feature point set, for example, taking a union of the first predicted target feature point set and the second predicted target feature point set as the target feature point set.
- Step 104 Determine, according to the target feature point set, a target annotation area in the second frame image that matches the annotation area of the first frame image, where the target annotation area corresponds to the annotation information matching the annotation area in the first frame image. Annotation information.
- the target annotation area may be determined from the second frame image based on the target feature point set, and the target annotation area is the second frame image and the first frame image. Match the area corresponding to the area.
- the method of the embodiment of the present invention may further obtain an image scaling feature according to the first frame image and the second frame image, and further target the target based on the image scaling feature.
- the annotation information of the annotation area is subjected to scaling processing, and the annotation information after the scaling processing is displayed in the target annotation area of the second frame image.
- the real replay annotation information moves with the movement of the display content, and the scene scaled with the zoom of the display content increases the usage scenario of the annotation function, and also improves the user experience.
- the local feature information represented by the two target feature points is similar, but only one of the two target feature points is a feature point corresponding to the annotation area of the first frame image, and the other one is not.
- the target annotation area is determined based on the target feature point set, the accuracy of the target annotation area is lowered. Therefore, in order to reduce the interference of the similar feature points, and to further improve the accuracy of the determined target annotation area, in an embodiment, the annotation area of the second frame image and the first frame image is determined based on the target feature point set.
- the matching target annotation area may be specifically: obtaining, according to the first feature point set and the target feature point set, a target central feature point in the second frame image that matches the annotation area in the first frame image; based on the first feature
- the point set and the target center feature point determine a target annotation area in the second frame image, wherein the target center feature point is located in a central area of the target annotation area. That is to say, in this example, the middle target feature point is determined first, and then the target comment area is determined around the target center feature point.
- the specific manner of determining the target central feature point is obtained in the second frame image that matches the annotation area in the first frame image.
- the target center feature point may be specifically: determining a center feature point based on the first feature point in the first feature point set and the target feature point corresponding to the first feature point in the target feature point set, and obtaining the central feature point set Selecting a target center feature point that satisfies a preset rule from the central feature point set.
- a voting (clustering) mechanism may be selected to select the number of votes from the central feature point set.
- the highest target center feature point As shown in FIG. 4, for example, based on the first feature point set and the target feature point set, three central feature points shown in the left figure of FIG. 4 are determined, wherein five points point to the center feature point A, and two point-to-center features Point C, one points to the central feature point B. Therefore, based on the voting (clustering) mechanism, the central feature point A with the highest number of votes is selected as the target center feature point.
- the feature point matching the edge region of the annotation area of the first frame image is selected from the second frame image in a similar manner, and the target annotation region is obtained, and
- the target annotation area obtained by this method reduces the interference of similar feature points, improves the accuracy of the annotation area tracking, and lays a foundation for improving the user experience.
- the annotation area in the first frame image displayed by the display content is determined in the state in which the content sharing is displayed, and the first feature point set representing the annotation area is determined, wherein the annotation area corresponds to An annotation information; determining a second frame image to obtain a second feature point set of the second frame image, wherein the second frame image is an image associated with the first frame image; and the second feature point set and the first feature point set Performing matching, selecting, according to the matching result, the target feature points matching the feature points in the first feature point set from the second feature point set to obtain the target feature point set; determining the second frame image based on the target feature point set The target annotation area matched by the annotation area of the first frame image, wherein the target annotation area corresponds to the annotation information matching the annotation information of the annotation area in the first frame image, so that on the basis of implementing the annotation information sharing, the implementation is implemented.
- the purpose of the annotation information changes according to the change of the display content, for example, after the content scrolling or zooming
- the method of the embodiment can still ensure that the annotation information changes correspondingly with operations such as scrolling or zooming, thus enriching the usage scenario of the annotation function, increasing the ease of use of the annotation function in the screen sharing scenario, and simultaneously improving the user.
- operations such as scrolling or zooming
- the method of the embodiment of the present invention is not limited by the annotation state, that is, the annotation information can be changed correspondingly with the operation of scrolling or zooming, whether or not it is in the annotation state, thereby avoiding the screen content operation and the annotation state switching back and forth.
- the problem of user operating costs has improved the user experience.
- the method of the embodiment of the invention can satisfy the requirement for the user to review and summarize the existing annotation information, further improve the ease of use of the annotation function, and enrich the use scenario of the annotation function.
- the annotation area definition is stored as an interest area, and the entire interest area is decomposed into a plurality of small areas, such as a plurality of feature points, and the interest area is represented by the expression of the feature points.
- the feature point itself does not change, but the position and/or distance of the feature point changes, so based on the above principle, the example adopts the feature.
- Point static adaptive clustering method to accurately describe the initial interest area by using feature points, so as to achieve the purpose of dynamically changing the annotation information following the display content.
- an initial annotation area also referred to as an initial interest area
- the initial annotation is calculated.
- the feature points of the area are used to quickly recapture the feature points and calculate a new annotation follow position after operations such as sliding or zooming.
- the feature point corresponding to the initial annotation area in the previous frame is tracked by using the optical flow method to estimate the feature points corresponding to the initial annotation area in the current frame, and thus, the first estimated target feature point set is obtained.
- the feature descriptor is used to globally match the feature points corresponding to the current frame with the feature points corresponding to the initial annotation area to obtain a second estimated target feature point set.
- the target center feature point is determined, and then the target annotation area is determined based on the target center feature point, for example, the feature points of the sliding or scaling are re-consensus, and the feature points of the non-initial interest area are removed, centering on the target center feature point
- the target annotation area is determined in the form of a bounding box.
- FIG. 5 is a schematic flowchart of an implementation process of an image processing method according to an embodiment of the present invention. As shown in FIG. 5, the flow of the annotation information following algorithm is as follows:
- Step 1 Select the image frame of the annotation process in the user frame as the first frame, and perform key point detection on the first frame (for example, using the FAST algorithm) to obtain an annotation area of the first frame (hereinafter referred to as an initial annotation area).
- the feature points of the BRISK algorithm are used to describe the detected key points, that is, the feature points of the initial annotation area are determined as foreground feature points; here, each feature point in the initial annotation area is used relative to the initial The relative coordinate representation of the center of the annotation area.
- Step 2 Starting from the second frame, each frame extracts feature points of the image frame by using a feature descriptor corresponding to the BRISK algorithm as a background feature point.
- the background feature point In order to continuously track the initial approval area, the background feature point needs to be the first frame.
- the feature points of the initial annotation area are globally matched, and the position of the foreground feature points in the current frame is found, that is, the above target annotation area.
- the Euclidean distance of each foreground feature point in the first frame is determined, and the nearest and next most recent ratio is used as a scale to determine the background feature point and the first The predicted target feature point that best matches the foreground feature points in the frame.
- Step 3 Using forward and backward tracking methods, such as the LK optical flow method, to predict the position of the foreground feature point in the current frame to select an estimated target feature point that matches the foreground feature point in the current frame.
- forward and backward tracking methods such as the LK optical flow method
- Step 4 Perform preliminary fusion, that is, combine the estimated target feature points obtained in steps 2 and 3 to obtain the target feature points, and record the absolute coordinate values of the target feature points in the image after fusion.
- Step 5 subtracting the relative coordinate value of the foreground feature point corresponding to the target feature point in the first frame by subtracting the absolute coordinate value of the target feature point in the current frame, and obtaining the center in the current frame corresponding to the target feature point. Feature points.
- the rotation angle and the scale factor may be evaluated by using the first frame and the current frame to obtain a scaling factor, thereby realizing that the target annotation area is scaled according to the scaling of the display content. Specifically, before the difference is made, the relative feature of the foreground feature point in the first frame is multiplied by the scaling factor to make a difference.
- Step 6 The positions of the central feature points obtained by each target key point may be inconsistent. Therefore, the voting (clustering) mechanism is used to perform the consistency constraint, and the central feature point corresponding to the target feature point with the highest number of votes is the target center feature point.
- Figure 4 shows.
- Step 7 After obtaining the target center feature points, perform local matching and secondary fusion to obtain the target annotation area. Traverse to find the specific position of the edge region in the initial annotation area in the first frame, such as the position of the four corners, after determining the four corner positions of the initial annotation area, the absolute coordinate value of the target central feature point + plus the first frame The relative coordinate values of the foreground feature points corresponding to each corner of the corner, the four corner positions for the current frame are obtained, the target annotation area is obtained, and the current frame containing the target annotation area is obtained, and the target annotation area is displayed. Current frame.
- the relative coordinate value of the foreground feature point corresponding to each corner is multiplied by the scaling factor before the addition operation, and then the absolute coordinate value of the target central feature point is added to obtain the scaling process. After the target annotation area, this achieves the goal of dynamic follow-up.
- the shared screen content in the annotation state, can also be scrolled, zoomed, etc., that is, the operation limit is not performed in this embodiment; and, after the screen content is scrolled, zoomed, etc.
- the annotation information is also moved and scaled to achieve the purpose of dynamic follow-up. Further, after the annotation area is moved out of the screen and moved back to the screen, the annotation information can appear again in the response position.
- FIG. 6 is a display content sharing of the sender terminal according to the embodiment of the present invention.
- the application flow diagram of the annotation in the scenario is as shown in Figure 6.
- the sender terminal has the following application scenarios:
- Scene 1 The process of annotating. Specifically, the display content sharing is started, the annotation button is clicked, the annotation state is entered, and the annotation processing is performed in the annotation state, such as creating, modifying, or deleting the annotation information. Taking the creation of an annotation as an example, after creation, the annotation information is generated, and the generated annotation information is added to the annotation information manager.
- Scenario 2 The sharing process of annotation information in the non-annotation state. Specifically, in the non-annotation state, the audio and video SDK performs video frame collection, tracks the generated annotation information, adjusts the display position of the annotation information, and correspondingly modifies the annotation information manager, and displays the adjusted annotation information to implement the annotation information. Dynamically follow the purpose. Further, the adjusted annotation information is sent to the receiver terminal to realize synchronous display of the receiver terminal and the sender terminal.
- the annotation information in the annotation information manager is synthesized into a picture, and then the synthesized picture is synthesized with the current frame collected by the audio and video SDK, and synthesized.
- the synthesized frame is transmitted to the audio and video SDK.
- the video is capable of recording annotation information and recording the process of annotation information dynamically following.
- Scenario 3 In the non-annotation state, the annotation information is received, for example, the annotation information sent by the receiver is received. The received annotation information is added to the annotation information manager to display the received annotation information at the corresponding location.
- FIG. 7 is a schematic diagram of an application flow of an annotation performed by a receiver terminal in a display content sharing scenario according to an embodiment of the present invention.
- the receiver terminal has the following application scenarios, namely:
- Scene 1 Enter the display content sharing state.
- the annotation state the annotation information is received, and the annotation manager is updated to display the received annotation information at the corresponding location.
- Scene 2 Enter the display content sharing state, click the annotation button, enter the annotation state, display the annotation information in the annotation manager; add, delete and modify the annotation information, and update the local annotation manager after the processing, the change will be changed.
- the post comment information is sent to the sender terminal.
- a message is sent to the sender terminal to inform the sender terminal that the receiver terminal has entered the comment state.
- the sender terminal deletes the annotation information corresponding to the receiver terminal in the annotation manager, and performs corresponding deletion processing in the video stream, that is, deletes the annotation information corresponding to the receiver terminal in the video stream.
- the receiving terminal adds, deletes and changes the annotation information of the user, and after processing, updates the local annotation manager, and sends all the updated annotation information to the sender terminal to achieve the synchronization purpose of the content displayed at both ends.
- the receiver terminal and the sender terminal can only be modified according to actual needs, and the annotation information corresponding to the receiver can be modified, or the receiver terminal and the sender terminal can modify their corresponding All annotation information in the annotation manager, including the annotation information edited by itself, and the annotation information edited by the other party.
- the method in the embodiment of the present invention improves the annotation experience in the screen sharing process, expands the usage scenario of the annotation function, provides better marking and recording capabilities, and reduces online communication costs.
- the embodiment further provides an image processing apparatus. As shown in FIG. 8, the apparatus includes:
- the first determining unit 81 is configured to determine, in a state of content sharing, an annotation area in the first frame image displayed by the display content, and determine a first feature point set that represents the annotation area, where the annotation area corresponds to Comment information.
- the first determining unit 81 is further configured to determine the second frame image to obtain a second feature point set characterizing the second frame image, wherein the second frame image is an image associated with the first frame image.
- the feature point matching unit 82 is configured to match the second feature point set with the first feature point set, and select, according to the matching result, the target feature point that matches the feature point in the first feature point set from the second feature point set. , get the target feature point set.
- a second determining unit 83 configured to determine, according to the target feature point set, a target annotation area that matches an annotation area of the first frame image in the second frame image, where the target annotation area corresponds to an annotation area in the first frame image
- the annotation information matches the annotation information.
- the first determining unit 81 is further configured to determine an image moving feature that is transformed from the first frame image to the second frame image; and estimate and the first image from the second frame image based on the image moving feature. A target feature point in which the feature points are matched in the feature point set, and a first estimated target feature point set is obtained.
- the feature point matching unit 82 is further configured to: select, according to the matching result, the target feature points that match the feature points in the first feature point set from the second feature point set to obtain the second estimated target feature point set; and An estimated target feature point set and a second predicted target feature point set are obtained to obtain a target feature point set.
- the feature point matching unit 82 is further configured to determine a distance feature between the second feature point in the second feature point set and the first feature point in the first feature point set; and the second feature The target feature points whose distance features satisfy the preset distance rule are selected in the point set.
- the second determining unit 83 is further configured to obtain, according to the first feature point set and the target feature point set, a target central feature point in the second frame image that matches the annotation area in the first frame image; And determining a target annotation area in the second frame image based on the first feature point set and the target center feature point, wherein the target center feature point is located in a central area of the target annotation area.
- the second determining unit 83 is further configured to determine the center feature point based on the first feature point in the first feature point set and the target feature point corresponding to the first feature point in the target feature point set.
- the central feature point set is obtained; that is, the target central feature point that satisfies the preset rule is selected from the central feature point set.
- the image processing apparatus further includes: an image scaling unit; wherein the image scaling unit is configured to obtain an image scaling feature according to the first frame image and the second frame image; and the target annotation region based on the image scaling feature
- the annotation information is subjected to scaling processing, and the annotation information after the scaling processing is displayed in the target annotation area of the second frame image.
- the embodiment further provides a terminal, including a memory and a processor.
- the memory stores computer readable instructions.
- the processor executes the following steps: displaying the content sharing status, determining And displaying a comment area in the first frame image displayed by the content, and determining a first feature point set characterizing the annotation area, wherein the annotation area corresponds to annotation information; determining the second frame image to obtain the second frame image is obtained.
- the second frame image is an image associated with the first frame image; the second feature point set is matched with the first feature point set, and the second feature point set is selected from the second feature point set based on the matching result a target feature point matching the feature points in the first feature point set to obtain a target feature point set; and determining, according to the target feature point set, a target annotation area in the second frame image that matches the annotation area of the first frame image,
- the target annotation area corresponds to annotation information that matches the annotation information of the annotation area in the first frame image.
- the computer readable instructions are executed by the processor such that the processor further performs the steps of: determining an image shifting feature that is transformed from the first frame image to the second frame image; based on the image moving feature, from the second frame A target feature point matching the feature points in the first feature point set is estimated in the image, and a first estimated target feature point set is obtained.
- the processor When the computer readable instructions are executed by the processor, the processor causes the processor to perform the step of selecting the target feature points that match the feature points in the first feature point set from the second set of feature points based on the matching result, and obtaining the target feature point set And performing the following steps: selecting, according to the matching result, the target feature points matching the feature points in the first feature point set from the second feature point set to obtain the second estimated target feature point set; and based on the first prediction A target feature point set and a second estimated target feature point set are obtained to obtain a target feature point set.
- the computer readable instructions when executed by the processor, causing the processor to perform matching of the second set of feature points with the first set of feature points, and selecting from the second set of feature points based on the matching result
- the following steps are performed: determining a distance feature between the second feature point in the second feature point set and the first feature point in the first feature point set And selecting, from the second set of feature points, the target feature points whose distance features satisfy the preset distance rule.
- the processor when the computer readable instructions are executed by the processor, causing the processor to perform the step of determining a target annotation region in the second frame image that matches the annotation region of the first frame image based on the set of target feature points Performing the following steps: obtaining a target central feature point in the second frame image that matches the annotation area in the first frame image based on the first feature point set and the target feature point set; based on the first feature point set and the target center feature The point determines a target annotation area in the second frame image, wherein the target center feature point is located in a central area of the target annotation area.
- the computer readable instructions are executed by the processor such that the processor is configured to match the annotation regions in the first frame image in the second frame image based on the first set of feature points and the set of target feature points.
- the step of the target center feature point is performed, the following steps are performed: determining the center feature point based on the first feature point in the first feature point set and the target feature point corresponding to the first feature point in the target feature point set, The central feature point set; the target center feature points satisfying the preset rule are selected from the central feature point set.
- the computer readable instructions are executed by the processor such that the processor further performs the steps of: obtaining an image scaling feature based on at least the first frame image and the second frame image; annotating the target annotation region based on the image scaling feature
- the information is subjected to scaling processing, and the annotation information after the scaling processing is displayed in the target annotation area of the second frame image.
- the embodiment further provides a computer readable storage medium, a non-transitory computer readable storage medium storing computer readable instructions, when the computer readable instructions are executed by one or more processors, such that Or the plurality of processors perform the following steps: in the state of displaying content sharing, determining an annotation area in the first frame image displayed by the display content, and determining a first feature point set representing the annotation area, wherein the annotation area corresponds to An annotation information; determining a second frame image to obtain a second feature point set of the second frame image, wherein the second frame image is an image associated with the first frame image; and the second feature point set and the first feature point are The set is matched, and the target feature points matching the feature points in the first feature point set are selected from the second feature point set based on the matching result to obtain the target feature point set; and the second frame image is determined based on the target feature point set.
- a target annotation area matching the annotation area of the first frame image wherein the target annotation area corresponds to an annotation area in the image of the first frame
- the computer readable instructions are executed by the processor such that the processor further performs the steps of: determining an image shifting feature that is transformed from the first frame image to the second frame image; based on the image moving feature, from the second frame A target feature point matching the feature points in the first feature point set is estimated in the image, and a first estimated target feature point set is obtained.
- the processor When the computer readable instructions are executed by the processor, the processor causes the processor to perform the step of selecting the target feature points that match the feature points in the first feature point set from the second set of feature points based on the matching result, and obtaining the target feature point set And performing the following steps: selecting, according to the matching result, the target feature points that match the feature points in the first feature point set from the second feature point set, to obtain the second estimated target feature point set; and based on the first predicted target The feature point set and the second estimated target feature point set obtain a target feature point set.
- the computer readable instructions when executed by the processor, causing the processor to perform matching of the second set of feature points with the first set of feature points, and selecting from the second set of feature points based on the matching result
- the following steps are performed: determining a distance feature between the second feature point in the second feature point set and the first feature point in the first feature point set And selecting, from the second set of feature points, the target feature points whose distance features satisfy the preset distance rule.
- the processor when the computer readable instructions are executed by the processor, causing the processor to perform the step of determining a target annotation region in the second frame image that matches the annotation region of the first frame image based on the set of target feature points Performing the following steps: obtaining a target central feature point in the second frame image that matches the annotation area in the first frame image based on the first feature point set and the target feature point set; based on the first feature point set and the target center feature The point determines a target annotation area in the second frame image, wherein the target center feature point is located in a central area of the target annotation area.
- the computer readable instructions are executed by the processor such that the processor is configured to match the annotation regions in the first frame image in the second frame image based on the first set of feature points and the set of target feature points.
- the step of the target center feature point is performed, the following steps are performed: determining the center feature point based on the first feature point in the first feature point set and the target feature point corresponding to the first feature point in the target feature point set, The central feature point set; the target center feature points satisfying the preset rule are selected from the central feature point set.
- the processor when the computer readable instructions are executed by the processor, the processor further performs the steps of: obtaining an image scaling feature based on at least the first frame image and the second frame image; and annotating the target annotation area based on the image scaling feature The scaling process is performed, and the annotation information after the scaling process is displayed in the target annotation area of the second frame image.
- the embodiment further provides an image processing device.
- the image processing device may be specifically any electronic device having a display component, such as a personal computer, a mobile terminal, or the like.
- the display component can be specifically a display.
- the image processing device specifically corresponds to the above-described sender terminal.
- the image processing apparatus includes at least a display component and a processor: a display component for displaying display content on the display interface, and a processor for sharing display content displayed by the display interface with other electronic devices (such as a receiver terminal); for example, The display content displayed on the display interface is sent to the other electronic device to share the display content displayed on the display interface with the other electronic device; the processor is further configured to determine, in the state of displaying the content sharing, the first displayed by the display content An annotation area in the frame image, and determining a first feature point set representing the annotation area, wherein the annotation area corresponds to annotation information; determining the second frame image to obtain a second feature point set characterizing the second frame image, wherein The second frame image is an image associated with the first frame image; the second feature point set is matched with the first feature point set, and the feature in the first feature point set is selected from the second feature point set based on the matching result.
- Point matching target feature points to obtain a target feature point set determining a second frame map based on the target feature point set In the region of the first frame image annotation matches the target annotation region, wherein the region corresponding to the target annotation information is endorsed annotation region and the first frame image annotation information matches.
- the computer readable instructions are executed by the processor such that the processor further performs the steps of: determining an image shifting feature that is transformed from the first frame image to the second frame image; based on the image moving feature, from the second frame A target feature point matching the feature points in the first feature point set is estimated in the image, and a first estimated target feature point set is obtained.
- the processor When the computer readable instructions are executed by the processor, causing the processor to obtain a target feature point set by matching the feature points matching the feature points in the first feature point set from the second set of feature points based on at least the matching result
- the following steps are performed: selecting a target feature point that matches a feature point in the first feature point set from the second feature point set based on the matching result, and obtaining a second estimated target feature point set; The target feature point set and the second predicted target feature point set are estimated to obtain a target feature point set.
- the computer readable instructions when executed by the processor, causing the processor to perform matching of the second set of feature points with the first set of feature points, and selecting at least from the second set of feature points based on the matching result
- the following steps are performed: determining a distance between the second feature point in the second feature point set and the first feature point in the first feature point set Feature; selecting, from the second set of feature points, a target feature point whose distance feature satisfies a preset distance rule.
- the processor when the computer readable instructions are executed by the processor, causing the processor to perform the step of determining a target annotation region in the second frame image that matches the annotation region of the first frame image based on the set of target feature points Performing the following steps: obtaining a target central feature point in the second frame image that matches the annotation area in the first frame image based on the first feature point set and the target feature point set; based on the first feature point set and the target center feature The point determines a target annotation area in the second frame image, wherein the target center feature point is located in a central area of the target annotation area.
- the computer readable instructions are executed by the processor such that the processor is configured to match the annotation regions in the first frame image in the second frame image based on the first set of feature points and the set of target feature points.
- the step of the target center feature point is performed, the following steps are performed: determining the center feature point based on the first feature point in the first feature point set and the target feature point corresponding to the first feature point in the target feature point set, and obtaining the center A feature point set; a target center feature point that satisfies a preset rule is selected from the central feature point set.
- the computer readable instructions are executed by the processor, such that the processor further performs the steps of: obtaining image scaling features from the first frame image and the second frame image; and performing annotation information on the target annotation region based on the image scaling features
- the scaling process displays the annotation information after the scaling process in the target annotation area of the second frame image.
- the embodiment further provides an image processing device.
- the image processing device may be specifically any electronic device having a display component, such as a personal computer, a mobile terminal, or the like.
- the display component can be specifically a display.
- the image processing device specifically corresponds to the above-described receiver terminal.
- the image processing apparatus includes at least a display component and a processor: a processor for acquiring display content shared by other electronic devices such as a sender terminal.
- the display component is configured to display, on the display interface, the display content shared by other acquired electronic devices.
- the processor is further configured to determine, in a state in which the content is shared, an annotation area in the first frame image displayed by the display content, and determine a first feature point set that represents the annotation area, wherein the annotation area corresponds to the annotation Determining a second frame image to obtain a second feature point set characterizing the second frame image, wherein the second frame image is an image associated with the first frame image; and the second feature point set and the first feature point set are Performing matching, at least selecting a target feature point matching the feature points in the first feature point set from the second feature point set based on the matching result to obtain a target feature point set; and determining a second frame image based on at least the target feature point set a target annotation area matching the annotation area of the first frame image, wherein the target annotation
- the computer readable instructions are executed by the processor such that the processor further performs the steps of: determining an image shifting feature that is transformed from the first frame image to the second frame image; based on the image moving feature, from the second frame A target feature point matching the feature points in the first feature point set is estimated in the image, and a first estimated target feature point set is obtained.
- the processor When the computer readable instructions are executed by the processor, causing the processor to obtain a target feature point set by matching the feature points matching the feature points in the first feature point set from the second set of feature points based on at least the matching result
- the following steps are performed: selecting a target feature point that matches a feature point in the first feature point set from the second feature point set based on the matching result, and obtaining a second estimated target feature point set; The target feature point set and the second predicted target feature point set are estimated to obtain a target feature point set.
- the computer readable instructions when executed by the processor, causing the processor to perform matching of the second set of feature points with the first set of feature points, and selecting at least from the second set of feature points based on the matching result
- the following steps are performed: determining a distance between the second feature point in the second feature point set and the first feature point in the first feature point set Feature; selecting, from the second set of feature points, a target feature point whose distance feature satisfies a preset distance rule.
- the processor when the computer readable instructions are executed by the processor, causing the processor to perform the step of determining a target annotation region in the second frame image that matches the annotation region of the first frame image based on the set of target feature points Performing the following steps: obtaining a target central feature point in the second frame image that matches the annotation area in the first frame image based on the first feature point set and the target feature point set; based on the first feature point set and the target center feature The point determines a target annotation area in the second frame image, wherein the target center feature point is located in a central area of the target annotation area.
- the computer readable instructions are executed by the processor such that the processor is configured to match the annotation regions in the first frame image in the second frame image based on the first set of feature points and the set of target feature points.
- the step of the target center feature point is performed, the following steps are performed: determining the center feature point based on the first feature point in the first feature point set and the target feature point corresponding to the first feature point in the target feature point set, and obtaining the center A feature point set; a target center feature point that satisfies a preset rule is selected from the central feature point set.
- the computer readable instructions are executed by the processor, such that the processor further performs the steps of: obtaining image scaling features from the first frame image and the second frame image; and performing annotation information on the target annotation region based on the image scaling features
- the scaling process displays the annotation information after the scaling process in the target annotation area of the second frame image.
- the disclosed apparatus and method may be implemented in other manners.
- the device embodiments described above are merely illustrative.
- the division of the unit is only a logical function division.
- there may be another division manner such as: multiple units or components may be combined, or Can be integrated into another system, or some features can be ignored or not executed.
- the coupling, or direct coupling, or communication connection of the components shown or discussed may be indirect coupling or communication connection through some interfaces, devices or units, and may be electrical, mechanical or other forms. of.
- the units described above as separate components may or may not be physically separated, and the components displayed as the unit may or may not be physical units, that is, may be located in one place or distributed to multiple network units; Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
- each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may be separately used as one unit, or two or more units may be integrated into one unit;
- the unit can be implemented in the form of hardware or in the form of hardware plus software functional units.
- the foregoing program may be stored in a computer readable storage medium, and the program is executed when executed.
- the foregoing storage medium includes: a mobile storage device, a read only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk. The medium in which the program code is stored.
- the above-described integrated unit of the present invention may be stored in a computer readable storage medium if it is implemented in the form of a software function module and sold or used as a standalone product.
- the technical solution of the embodiments of the present invention may be embodied in the form of a software product in essence or in the form of a software product stored in a storage medium, including a plurality of instructions.
- a computer device (which may be a personal computer, server, or network device, etc.) is caused to perform all or part of the methods described in various embodiments of the present invention.
- the foregoing storage medium includes: a removable storage device, a read only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like, which can store program codes.
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Processing Or Creating Images (AREA)
- User Interface Of Digital Computer (AREA)
- Image Processing (AREA)
Abstract
一种图像处理方法,包括:显示内容共享的状态下,确定出所述显示内容所显示的第一帧图像中的批注区域,并确定出表征所述批注区域的第一特征点集合;确定第二帧图像,得到表征所述第二帧图像的第二特征点集合;将所述第二特征点集合与所述第一特征点集合进行匹配,基于匹配结果从所述第二特征点集合中选取出与所述第一特征点集合中特征点相匹配的目标特征点,得到目标特征点集合;及基于所述目标特征点集合确定出所述第二帧图像中与所述第一帧图像的批注区域相匹配的目标批注区域,其中,所述目标批注区域对应有与所述第一帧图像中批注区域的批注信息相匹配的批注信息。
Description
相关申请的交叉引用
本申请要求于2017年12月26日提交中国专利局,申请号为201711428095.7、发明名称为“图像处理方法及其装置、存储介质”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。
本申请涉及图像处理技术,尤其涉及一种图像处理方法、装置、终端及存储介质。
在异地沟通和讨论场景中,人们经常会使用屏幕共享(也即显示内容共享)功能来演示一份文档,并基于该演示的文档展开讨论;在讨论和沟通过程中,通常会使用批注功能来标记或记录讨论过程,以此降低线上沟通成本。但是,现有屏幕共享中的批注功能,仅支持针对静态映像来进行,比如,在静态下,即屏幕不进行滚动或缩放操作的状态下,能够实现批注信息的共享,但若对屏幕进行操作,如滚动或缩放操作,则之前的批注信息就会消失,这样,大大降低了屏幕共享场景中批注功能的易用性,限制了批注功能的使用场景。
发明内容
根据本申请的各种实施例提供一种图像处理方法、装置、终端及存储介质。
一种图像处理方法,由终端执行,所述方法包括:
显示内容共享的状态下,确定出所述显示内容所显示的第一帧图像中的批注区域,并确定出表征所述批注区域的第一特征点集合,其中,所述批注区域对应有批注信息;
确定第二帧图像,得到表征所述第二帧图像的第二特征点集合,其中, 所述第二帧图像为与所述第一帧图像相关联的图像;
将所述第二特征点集合与所述第一特征点集合进行匹配,基于匹配结果从所述第二特征点集合中选取出与所述第一特征点集合中特征点相匹配的目标特征点,得到目标特征点集合;及
基于所述目标特征点集合确定出所述第二帧图像中与所述第一帧图像的批注区域相匹配的目标批注区域,其中,所述目标批注区域对应有与所述第一帧图像中批注区域的批注信息相匹配的批注信息。
一种图像处理装置,所述装置包括:
第一确定单元,用于显示内容共享的状态下,确定出所述显示内容所显示的第一帧图像中的批注区域,并确定出表征所述批注区域的第一特征点集合,其中,所述批注区域对应有批注信息;还用于确定第二帧图像,得到表征所述第二帧图像的第二特征点集合,其中,所述第二帧图像为与所述第一帧图像相关联的图像;
特征点匹配单元,用于将所述第二特征点集合与所述第一特征点集合进行匹配,基于匹配结果从所述第二特征点集合中选取出与所述第一特征点集合中特征点相匹配的目标特征点,得到目标特征点集合;
第二确定单元,用于基于所述目标特征点集合确定出所述第二帧图像中与所述第一帧图像的批注区域相匹配的目标批注区域,其中,所述目标批注区域对应有与所述第一帧图像中批注区域的批注信息相匹配的批注信息。
一种终端,包括存储器和处理器,所述存储器中存储有计算机可读指令,所述计算机可读指令被所述处理器执行时,使得所述处理器执行以下步骤:
显示内容共享的状态下,确定出所述显示内容所显示的第一帧图像中的批注区域,并确定出表征所述批注区域的第一特征点集合,其中,所述批注区域对应有批注信息;
确定第二帧图像,得到表征所述第二帧图像的第二特征点集合,其中,所述第二帧图像为与所述第一帧图像相关联的图像;
将所述第二特征点集合与所述第一特征点集合进行匹配,基于匹配结果从所述第二特征点集合中选取出与所述第一特征点集合中特征点相匹配的目标特征点,得到目标特征点集合;及
基于所述目标特征点集合确定出所述第二帧图像中与所述第一帧图像的批注区域相匹配的目标批注区域,其中,所述目标批注区域对应有与所述第一帧图像中批注区域的批注信息相匹配的批注信息。
一种非易失性的计算机可读存储介质,存储有计算机可读指令,所述计算机可读指令被一个或多个处理器执行时,使得所述一个或多个处理器执行以下步骤:
显示内容共享的状态下,确定出所述显示内容所显示的第一帧图像中的批注区域,并确定出表征所述批注区域的第一特征点集合,其中,所述批注区域对应有批注信息;
确定第二帧图像,得到表征所述第二帧图像的第二特征点集合,其中,所述第二帧图像为与所述第一帧图像相关联的图像;
将所述第二特征点集合与所述第一特征点集合进行匹配,基于匹配结果从所述第二特征点集合中选取出与所述第一特征点集合中特征点相匹配的目标特征点,得到目标特征点集合;及
基于所述目标特征点集合确定出所述第二帧图像中与所述第一帧图像的批注区域相匹配的目标批注区域,其中,所述目标批注区域对应有与所述第一帧图像中批注区域的批注信息相匹配的批注信息。
一种图像处理装置,所述装置包括显示组件和处理器:
所述显示组件,用于在显示界面展示显示内容;
所述处理器,用于将显示界面展示的显示内容发送至其他电子设备,以与其他电子设备共享所述显示界面所展示的显示内容;
所述处理器,还用于在显示内容共享的状态下,确定出所述显示内容所显示的第一帧图像中的批注区域,并确定出表征所述批注区域的第一特征点集合,其中,所述批注区域对应有批注信息;确定第二帧图像,得到表征所述第二帧图像的第二特征点集合,其中,所述第二帧图像为与所述第一帧图像相关联的图像;将所述第二特征点集合与所述第一特征点集合进行匹配,基于匹配结果从所述第二特征点集合中选取出与所述第一特征点集合中特征点相匹配的目标特征点,得到目标特征点集合;基于所述目标特征点集合确定出所述第二帧图像中与所述第一帧图像的批注区域相匹配的目标批注区 域,其中,所述目标批注区域对应有与所述第一帧图像中批注区域的批注信息相匹配的批注信息。
一种图像处理装置,所述装置包括处理器和显示组件:
所述处理器,用于获取其他电子设备所共享的显示内容;
所述显示组件,用于在显示界面展示获取到的其他电子设备所共享的显示内容;
所述处理器,还用于在显示内容共享的状态下,确定出所述显示内容所显示的第一帧图像中的批注区域,并确定出表征所述批注区域的第一特征点集合,其中,所述批注区域对应有批注信息;确定第二帧图像,得到表征所述第二帧图像的第二特征点集合,其中,所述第二帧图像为与所述第一帧图像相关联的图像;将所述第二特征点集合与所述第一特征点集合进行匹配,基于匹配结果从所述第二特征点集合中选取出与所述第一特征点集合中特征点相匹配的目标特征点,得到目标特征点集合;基于所述目标特征点集合确定出所述第二帧图像中与所述第一帧图像的批注区域相匹配的目标批注区域,其中,所述目标批注区域对应有与所述第一帧图像中批注区域的批注信息相匹配的批注信息。
本申请的一个或多个实施例的细节在下面的附图和描述中提出。本申请的其它特征、目的和优点将从说明书、附图以及权利要求书变得明显。
为了更清楚地说明本申请实施例中的技术方案,下面将对实施例中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本申请的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其它的附图。
图1为本发明实施例图像处理方法的实现流程示意图;
图2为本发明实施例在显示内容共享的状态下进行批注后的显示界面示意图;
图3为本发明实施例在显示内容共享的状态下且进行批注后的显示界面滚动操作后的示意图;
图4为本发明实施例目标中心特征点的选取规则示意图;
图5为本发明实施例图像处理方法在一具体示例中的实现流程示意图;
图6为本发明实施例发送方终端在显示内容共享场景中进行批注的应用流程示意图;
图7为本发明实施例接收方终端在显示内容共享场景中进行批注的应用流程示意图;
图8为本发明实施例图像处理装置的组成结构示意图。
在屏幕共享场景下使用批注功能,需要先进入批注状态,然后,在该批注状态下进行标记和/或批注,此时,现有方案中屏幕内容无法进行滚动、缩放等操作;而若要进行滚动、缩放等操作,则需先退出批注状态,而现有方案中,在退出批注状态后之前的批注信息则随之消失,综上,可以看出现有技术存在如下缺点:
第一,批注状态下,共享的屏幕内容无法进行滚动、缩放等操作,若想进行滚动、缩放等操作,则需要退出批注状态,所以,在实际应用过程中,会出现屏幕内容操作和批注状态来回切换的情况,因此,增加了用户的操作成本,影响了使用体验。
第二,在取消批注状态,也即退出批注状态后,之前的批注信息也会随之消失,但实际应用中,批注信息是强指向某个共享的内容点,是异地沟通和讨论场景中需要记录的可靠信息,存在随时回看和汇总沉淀的需求,因此,批注信息消失会让批注功能变成一种临时的写画功能,限制了使用场景。
综上,亟需提供一种在屏幕共享场景下,批注信息可以跟随屏幕内容移动和/或缩放的方法。具体地,为了能够更加详尽地了解本发明的特点与技术内容,下面结合附图对本发明的实现进行详细阐述,所附附图仅供参考说明之用,并非用来限定本发明。
本实施例提供了一种图像处理方法,具体地,在本方案是针对现有屏幕共享技术中批注信息无法回顾、不适应动态位置变化以及不适应缩放的问题所提出的,而且,本实施例在解决上述问题的基础上,能够实现如下功能, 即:
第一,本发明实施例在屏幕共享场景中,用户可以通过鼠标进行屏幕操作,同时,支持批注功能,比如,以鼠标点击拖拽的方式创建需要重点批注的批注区域,并能够生成并显示批注信息。
第二,进行移动、缩放等操作时,已存在的批注信息会动态跟随当前屏幕内容的移动和/或缩放而相应变化,并能确保批注信息准确对应(如框选)原重点批注的批注区域。
第三,批注区域的内容由于遮挡显示不全,亦或是随显示内容滚动移出画面后,比如检测到当前显示的内容中不存在批注区域后,或者,已有批注信息与当前显示的内容不对应后,停止显示批注信息。这里,实际应用中,若出现由于滚动批注区域的内容被部分遮挡的情况,此时,可以等比例的显示部分批注信息,亦或是停止显示批注信息,本实施例对此不作限制,可以根据实际需求而任意设置。
第四,批注区域重新回到屏幕范围内,比如检测到批注区域重新出现后,在对应位置显示批注信息,如此,重现批注信息,实现批注信息的回看。
第五,屏幕共享分为屏幕共享发送方终端和接收方终端,而本实施例所述的方法支持两端(如,发送方终端或接收方终端)在屏幕共享场景中对共享显示内容进行批注,即无论是发送方终端对共享显示内容标注的批注信息,还是接收方终端对共享显示内容进行批注的批注信息,均能够实现共享。换言之,本实施例所述的方法不受发送方终端或接收方终端的限制,在两端均能实现。
具体地,图1为本发明实施例图像处理方法的实现流程示意图,如图1所示,方法包括:
步骤101:显示内容共享的状态下,确定出显示内容所显示的第一帧图像中的批注区域,并确定出表征批注区域的第一特征点集合,其中,批注区域对应有批注信息。
这里,第一帧图像可以是在批注状态下选中批注区域,并编辑完成批注信息后所对应的一帧图像。当然,实际应用中,第一帧图像还可以是在批注状态下选中批注区域,并编辑完成批注信息之后,且在显示内容进行滚动和/ 或缩放等操作之前的一帧图像。
实际应用中,批注区域表征用于对共享的显示内容中的至少部分内容进行重点说明的区域,比如,批注区域可以用来框选显示内容中需要重点进行说明的部分内容。批注信息可以是用于对批注区域对应的至少部分内容进行说明的文本信息、批注框等,举例来说,批注信息包括但不限于以下至少一种:框选部分显示内容的线框,文本信息、批注框等。也就是说,本实施例的批注信息包括但不限于现有批注功能下编辑所能得到的任意信息,比如,现有批注功能包括:直线、箭头、画笔、方框和文字五种类型,此时,批注信息则包括但不限于该五种类型所能得到的至少一种信息。比如,图2为本发明实施例在显示内容共享的状态下进行批注后的显示界面示意图,如图2所示,批注信息包括框选批注区域的线框,以及线框周边所显示的文本信息。
步骤102:确定第二帧图像,得到表征第二帧图像的第二特征点集合,其中,第二帧图像为与第一帧图像相关联的图像;
在一个实施例中,第二帧图像为第一帧图像之后所出现的一帧图像,比如,第二帧图像为针对第一帧图像进行滚动操作后所得到的图像。图3为本发明实施例在显示内容共享的状态下且进行批注后的显示界面滚动操作后的示意图,如图3所示,显示内容滚动后,批注信息跟随原始批注区域的位置变化而变化后得到的图像,也即第二帧图像。其中,第二帧图像中显示有与第一帧图像的批注信息相匹配的批注信息。
在一个实施例中,特征点集合中包含有若干个特征点,该特征点均能够表征出对应图像的局部特征。具体地,第一特征点集合中包含有至少两个第一特征点,该第一特征点能够表征出批注区域的局部特征信息。相应地,第二特征点集合中包含有至少两个第二特征点,该第二特征点能够表征出第二帧图像的局部特征信息。
这里,由于实际应用中,会存在图像缩放问题,所以,为避免图像缩放处理后不能准确追踪到批注信息,本实施例中确定出的特征点本身不能随图像的缩放而变化,只是在图像缩放后,特征点的位置,和/或特征点之间的距离发生变化。基于此,可以采用尺寸不变的相关特征算法来确定图像的特征点,比如,利用SIFT(Scale Invariant Feature Transform)算法、BRISK(Binary Robust Invariant Scalable Keypoints)算法、或FAST(Features From Accelerated Segment Test)算法等来提取第一帧图像的批注区域的特征点,以及提取第二帧图像的特定点。如此,利用上述算法提取出的特征点能够确保本身不会随图像缩放而发生变化,只是在图像缩放后,特征点的位置和/或特征点之间的距离会发生变化而已。
步骤103:将第二特征点集合与第一特征点集合进行匹配,基于匹配结果从第二特征点集合中选取出与第一特征点集合中特征点相匹配的目标特征点,得到目标特征点集合;
在一个实施例中,进行匹配的过程就是相当于是相似度判断的过程,即判断第二特征点集合中的第二特征点与第一特征点集合中的第一特征点的相似度,进而从第二特征点集合中,选取出于第一特征点集合中的第一特征点相似度最高的点,也即目标特征点,以最终得到与第一特征点集合相匹配的目标特征点集合。
在一个实施例中,匹配过程,也即相似度的判断过程可以用距离来度量。在一个实施例中,步骤103可以具体为:确定第二特征点集合中的第二特征点与第一特征点集合中的第一特征点之间的距离特征,从第二特征点集合中选取出距离特征满足预设距离规则的目标特征点。
比如,对于第二特征点集合中每一个第二特征点而言,求其与第一特征点集合中每一个第一特征点的欧氏距离,并用距离最近与次最近的比值作为标度,来从第二特征点集合中选取出与第一特征点集合中第一特征点最匹配的目标特征点。实际应用中,特征点可以通过特征向量来标识,举例来说,利用向量A(x1,x2,…,xn)来表示第一特征点集合中特定第一特征点,利用向量B(y1,y2,…,yn)来表示第二帧图像中的第二特征点,其中,n为大于等于2的正整数;此时,特征点A和特征点B的欧式距离为:
进一步地,利用上述欧式距离确定出特定第一特征点A与第二帧图像中所有第二特征点之间的欧式距离,进而选取出与特定第一特征点A的欧式距离最小的第二特征点,该与特定第一特征点A的欧式距离最小的第二特征点 即为与该特定第一特征点A最匹配的目标特征点。
这里,为提高批注信息的显示位置准确度,本实施例的方法还可以确定从第一帧图像变换到第二帧图像的图像移动特征,并基于图像移动特征,从第二帧图像中预估出与第一特征点集合中特征点相匹配的目标特征点,得到第一预估目标特征点集合。比如,利用光流法,确定出从第一帧图像变换到第二帧图像的光流特征,进而基于光流特征从第二帧图像中预估出与第一特征点集合中特征点相匹配的目标特征点,得到第一预估目标特征点集合。对应地,步骤103则具体为:基于匹配结果从第二特征点集合中选取出与第一特征点集合中特征点相匹配的目标特征点,得到第二预估目标特征点集合,进而基于第一预估目标特征点集合和第二预估目标特征点集合,得到目标特征点集合,比如,取第一预估目标特征点集合和第二预估目标特征点集合的并集作为目标特征点集合。
步骤104:基于目标特征点集合确定出第二帧图像中与第一帧图像的批注区域相匹配的目标批注区域,其中,目标批注区域对应有与第一帧图像中批注区域的批注信息相匹配的批注信息。
实际应用中,在确定出目标特征点集合后,即可基于目标特征点集合从第二帧图像中确定出目标批注区域,而该目标批注区域即为第二帧图像中与第一帧图像的匹配区域相对应的区域。
这里,考虑到实际应用中可以还会存在针对显示内容的缩放操作,所以,本发明实施例的方法还可以根据第一帧图像和第二帧图像得到图像缩放特征,进而基于图像缩放特征对目标批注区域的批注信息进行缩放处理,在第二帧图像的目标批注区域中显示缩放处理后的批注信息。如此,真实复现批注信息随显示内容的移动而移动,随显示内容的缩放而缩放的场景,增加了批注功能的使用场景,也提升了用户体验。
这里,实际应用中,会存在相似特征点,即目标特征点集合中存在两个目标特征点的情况。比如,该两个目标特征点表征的局部特征信息相似,但该两个目标特征点中仅有一个是与第一帧图像的批注区域相对应的特征点,另外一个不是。此时,若基于目标特征点集合确定出目标批注区域,则会降低目标批注区域的准确度。因此,为降低相似特征点的干扰,以及为进一步 提升确定出的目标批注区域的准确性,在一实施例中,基于目标特征点集合确定出第二帧图像中与第一帧图像的批注区域相匹配的目标批注区域,可以具体为:基于第一特征点集合和目标特征点集合,得到第二帧图像中与第一帧图像中的批注区域相匹配的目标中心特征点;基于第一特征点集合以及目标中心特征点确定出第二帧图像中的目标批注区域,其中,目标中心特征点位于目标批注区域的中心区域。也就是说,本示例中,先确定中目标中心特征点,然后,围绕目标中心特征点确定出目标批注区域。
进一步地,在一个实施例中,确定目标中心特征点的具体方式,也即基于第一特征点集合和目标特征点集合,得到第二帧图像中与第一帧图像中的批注区域相匹配的目标中心特征点,可以具体为:基于第一特征点集合中第一特征点、以及目标特征点集合中与第一特征点相对应的目标特征点,确定出中心特征点,得到中心特征点集合;从中心特征点集合中选取出满足预设规则的目标中心特征点。
也就是说,不同特征点可能确定出的中心特征点不同,所以,为进一步提升确定出的目标中心特征点的精确度,可以选用投票(聚类)机制从中心特征点集合中选取出投票数最高的目标中心特征点。如图4所示,比如基于第一特征点集合和目标特征点集合,确定出图4左图中所示的三个中心特征点,其中,五个指向中心特征点A,两个指向中心特征点C,一个指向中心特征点B,因此,基于投票(聚类)机制选取出投票数最多的中心特征点A作为目标中心特征点。
进一步地,在确定出目标中心特征点之后,再利用相似的方式,从第二帧图像中选取与第一帧图像的批注区域的边缘区域相匹配的特征点,即可得到目标批注区域,且利用该方式得到的目标批注区域降低了相似特征点的干扰,提升了批注区域追踪的准确性,进而为提升用户体验奠定了基础。
这样,上述图像处理方法,通过显示内容共享的状态下,确定出显示内容所显示的第一帧图像中的批注区域,并确定出表征批注区域的第一特征点集合,其中,批注区域对应有批注信息;确定第二帧图像,得到第二帧图像的第二特征点集合,其中,第二帧图像为与第一帧图像相关联的图像;将第二特征点集合与第一特征点集合进行匹配,基于匹配结果从第二特征点集合 中选取出与第一特征点集合中特征点相匹配的目标特征点,得到目标特征点集合;基于目标特征点集合确定出第二帧图像中与第一帧图像的批注区域相匹配的目标批注区域,其中,目标批注区域对应有与第一帧图像中批注区域的批注信息相匹配的批注信息,如此,在实现批注信息共享的基础上,实现了批注信息随显示内容的变化而相应变化的目的,比如,显示内容滚动或缩放等操作后,本实施例的方法依然能够确保批注信息随滚动或缩放等操作而相应变化,如此,丰富了批注功能的使用场景,增加了屏幕共享场景中批注功能的易用性,同时,也提升了用户体验。
而且,本发明实施例的方法不受批注状态的限制,即无论是否处于批注状态,批注信息均能随滚动或缩放等操作而相应变化,因此,避免了屏幕内容操作和批注状态来回切换而增加用户操作成本的问题,提升了用户的使用体验。进一步地,本发明实施例的方法能够满足用户回看和汇总沉淀已有批注信息的需求,进一步提升了批注功能的易用性,丰富了批注功能的使用场景。
以下结合具体示例,对本发明实施例做进一步详细说明。具体地,本示例中把批注区域定义存储为兴趣区域,并将整个兴趣区域分解成为许多小区域,比如分解为若干特征点,以特征点的表达方式来表征该兴趣区域。这里,实际应用中,批注区域对应的显示内容移动或缩放后,其特征点本身并不会发生改变,但特征点的位置和/或距离会产生变化,所以,基于上述原理,本示例采用特征点静态自适应的聚类方式,来利用特征点准确描述初始兴趣区域,以达到批注信息追随显示内容而动态变化的目的。
这里,在屏幕共享的过程中,存在一帧图像且该帧图像中存在用户已批注的批注区域,可称为初始批注区域(也可称为初始兴趣区域),此时,计算得到该初始批注区域的特征点,进而采用如下方式来实现在滑动或缩放等操作后快速重新捕获特征点并计算出新的批注跟随位置。具体地,首先,采用光流法,追踪上一帧中初始批注区域对应的特征点,来预估当前帧中与初始批注区域对应的特征点,如此,得到第一预估目标特征点集合。其次,利用特征描述符来将当前帧对应的特征点与初始批注区域对应的特征点进行全局 匹配,得到第二预估目标特征点集合。最后,取第一预估目标特征点集合和第二预估目标特征点集合的并集,得到目标特征点集合,基于目标特征点集合中每一个特征点对中心特征点进行投票的方式,选取出目标中心特征点,进而基于目标中心特征点确定出目标批注区域,比如,使发生滑动或缩放的特征点重新达成共识,同时,除去非初始兴趣区域的特征点,以目标中心特征点为中心,以包围盒的形态确定出目标批注区域。
进一步地,图5为本发明实施例图像处理方法在一具体示例中的实现流程示意图,如图5所示,批注信息跟随算法的流程如下:
步骤1:将用户框选批注区域完成批注过程的影像帧作为第一帧,对第一帧进行关键点检测(如利用FAST算法)得到第一帧的批注区域(以下称为初始批注区域),使用BRISK算法对应的特征描述子对检测到的关键点进行特征描述,即确定出该初始批注区域的特征点,作为前景特征点;这里,初始批注区域中的每个特征点都用相对于初始批注区域中心的相对坐标表示。
步骤2:从第二帧开始,每一帧用BRISK算法对应的特征描述子提取该影像帧的特征点,作为背景特征点,为了持续追踪初始批准区域,需要将背景特征点与第一帧的初始批注区域的特征点进行全局匹配,找到当前帧中前景特征点的位置,即以上的目标批注区域。具体地,对每一个背景特征点而言,求其和第一帧中每一个前景特征点的欧氏距离,并用最近与次最近的比值作为标度来确定出该背景特征点中与第一帧中前景特征点最匹配的预估目标特征点。
步骤3:采用前向和后向跟踪法,如LK光流法,来预测前景特征点在当前帧的位置,以在当前帧中选出与前景特征点相匹配的预估目标特征点。
步骤4:进行初步融合,即将步骤2和3中得到的预估目标特征点取并集,得到目标特征点,融合后记录这些目标特征点在图像中的绝对坐标值。
步骤5:将当前帧中目标特征点的绝对坐标值减去第一帧中与该目标特征点对应的前景特征点的相对坐标值,即可得到与该目标特征点对应的当前帧中的中心特征点。
这里,为了匹配目标批注区域的缩放处理,可以利用第一帧和当前帧来 评估旋转角度和尺度因子,得到缩放因子,以此实现目标批注区域随显示内容的缩放而缩放。具体地,在上述做差前,将前景特征点在第一帧中的相对坐标乘上缩放因子后做差。
步骤6:各目标关键点得到的中心特征点的位置可能不一致,所以,使用投票(聚类)机制进行一致性约束,票数最高的目标特征点对应的中心特征点即为目标中心特征点,参见图4所示。
步骤7:得到目标中心特征点后,进行局部匹配和二次融合,即可得到目标批注区域。遍历寻找第一帧中初始批注区域中边缘区域的具体位置,如四个边角的位置,确定初始批注区域四个边角位置后,将目标中心特征点的绝对坐标值+加上第一帧中每一边角对应的前景特征点的相对坐标值,即可得到针对当前帧的四个边角位置,得到目标批注区域,进而得到包含有目标批注区域的当前帧,并显示包含有目标批注区域的当前帧。
这里,若存在缩放处理,则在进行加法运算前,将每一边角对应的前景特征点的相对坐标值乘以缩放因子,然后,加上目标中心特征点的绝对坐标值,即可得到缩放处理后的目标批注区域,如此,实现了动态跟随的目标。
综上,本发明实施例的方法,在批注状态下,共享的屏幕内容也可以进行滚动、缩放等操作,即本实施例不做操作限制;而且,在屏幕内容进行滚动、缩放等改变操作后,批注信息也随之移动、缩放,实现了动态跟随的目的。进一步地,在批注区域移出屏幕又移回屏幕后,批注信息可以在响应位置再次出现。
结合具体示例,本发明实施例还给出了如下具体应用场景,如此,来实现接收方终端和发送方终端批注信息的交互,具体地,图6为本发明实施例发送方终端在显示内容共享场景中进行批注的应用流程示意图,如图6所示,发送方终端存在如下应用场景,即:
场景一:进行批注的流程。具体地,开启显示内容共享,点击批注按键,进入批注状态,在批注状态下进行批注处理,如创建、修改或删除批注信息等。以创建批注为例,创建后,生成批注信息,将生成的批注信息添加到批注信息管理器中。
场景二:在非批注状态下,批注信息的共享流程。具体地,非批注状态 下,音视频SDK进行视频帧的采集,跟踪生成的批注信息,调整批注信息的显示位置,以及相应修改批注信息管理器,显示调整后的批注信息,以实现批注信息的动态跟随目的。进一步地,将调整后的批注信息发送至接收方终端,实现接收方终端与发送方终端的同步显示。这里,在调整批注信息的显示位置,以及相应修改批注信息管理器后,将批注信息管理器中的批注信息合成为图片,进而将合成的图片与音视频SDK采集到的当前帧合成,在合成后,将合成好的帧传输给音视频SDK。实际应用中,还可能存在录屏需求,此时,,确定是否处于录屏状态,即判定录屏是否已开启,确定开启后,将合成后的帧传输给录屏接口,以确保录制的音视频能够记录批注信息,以及记录批注信息动态跟随的过程。
场景三:在非批注状态下,接收到批注信息,比如,接收到接收方发送的批注信息。将接收到的批注信息添加到批注信息管理器中,以在对应位置展示接收到的批注信息。
图7为本发明实施例接收方终端在显示内容共享场景中进行批注的应用流程示意图,如图7所示,接收方终端存在如下应用场景,即:
场景一:进入显示内容共享状态,在批注状态下,接收到批注信息,更新批注管理器,以在对应位置展示接收到的批注信息。
场景二:进入显示内容共享状态,点击批注按键,进入批注状态,展示批注管理器中自己的批注信息;对自己的批注信息进行增删改查处理,处理后,更新本地的批注管理器,将变化后的批注信息发送给发送方终端。
或者,该场景二中,在进入批注状态后,向发送方终端发送一个消息,以告知发送方终端该接收方终端进入了批注状态。随后,发送方终端在批注管理器中删除该接收方终端对应的批注信息,并在视频流中进行对应删除处理,即在视频流中删除该接收方终端对应的批注信息。接收方终端对自己的批注信息进行增删改查处理,处理后,更新本地的批注管理器,并将自己更新后的所有批注信息发送至发送方终端,以实现两端显示内容的同步目的。
这里,值得注意的是,实际应用中,可以根据实际需求,设置成接收方终端和发送方终端只能修改自己对应的批注信息,或者,设置成接收方终端和发送方终端能够修改自身对应的批注管理器中的所有批注信息,如包括自 己编辑的批注信息,也包括对方编辑的批注信息。
如此,本发明实施例的方法提升了屏幕共享过程中的批注体验,扩展了批注功能的使用场景,提供了更好的标记和记录能力,同时,降低了线上沟通成本。
本实施例还提供了一种图像处理装置,如图8所示,装置包括:
第一确定单元81,用于显示内容共享的状态下,确定出显示内容所显示的第一帧图像中的批注区域,并确定出表征批注区域的第一特征点集合,其中,批注区域对应有批注信息。
第一确定单元81还用于确定第二帧图像,得到表征第二帧图像的第二特征点集合,其中,第二帧图像为与第一帧图像相关联的图像。
特征点匹配单元82,用于将第二特征点集合与第一特征点集合进行匹配,基于匹配结果从第二特征点集合中选取出与第一特征点集合中特征点相匹配的目标特征点,得到目标特征点集合。
第二确定单元83,用于基于目标特征点集合确定出第二帧图像中与第一帧图像的批注区域相匹配的目标批注区域,其中,目标批注区域对应有与第一帧图像中批注区域的批注信息相匹配的批注信息。
在一个实施例中,第一确定单元81,还用于确定从第一帧图像变换到第二帧图像的图像移动特征;及基于图像移动特征,从第二帧图像中预估出与第一特征点集合中特征点相匹配的目标特征点,得到第一预估目标特征点集合。
特征点匹配单元82,还用于基于匹配结果从第二特征点集合中选取出与第一特征点集合中特征点相匹配的目标特征点,得到第二预估目标特征点集合;及基于第一预估目标特征点集合和第二预估目标特征点集合,得到目标特征点集合。
在一个实施例中,特征点匹配单元82,还用于确定第二特征点集合中的第二特征点与第一特征点集合中的第一特征点之间的距离特征;及从第二特征点集合中选取出距离特征满足预设距离规则的目标特征点。
在一个实施例中,第二确定单元83,还用于基于第一特征点集合和目标 特征点集合,得到第二帧图像中与第一帧图像中的批注区域相匹配的目标中心特征点;及基于第一特征点集合以及目标中心特征点确定出第二帧图像中的目标批注区域,其中,目标中心特征点位于目标批注区域的中心区域。
在一个实施例中,第二确定单元83,还用于基于第一特征点集合中第一特征点、以及目标特征点集合中与第一特征点相对应的目标特征点,确定出中心特征点,得到中心特征点集合;即从中心特征点集合中选取出满足预设规则的目标中心特征点。
在一个实施例中,该图像处理装置还包括:图像缩放单元;其中,图像缩放单元,用于根据第一帧图像和第二帧图像得到图像缩放特征;及基于图像缩放特征对目标批注区域的批注信息进行缩放处理,在第二帧图像的目标批注区域中显示缩放处理后的批注信息。
这里需要指出的是:以上装置实施例的描述,与上述方法实施例的描述是类似的,具有同方法实施例相似的有益效果,因此不做赘述。对于本发明装置实施例中未披露的技术细节,请参照本发明方法实施例的描述而理解,为节约篇幅,因此不再赘述。
本实施例还提供了一种终端,包括存储器和处理器,存储器中存储有计算机可读指令,计算机可读指令被处理器执行时,使得处理器执行以下步骤:显示内容共享的状态下,确定出显示内容所显示的第一帧图像中的批注区域,并确定出表征批注区域的第一特征点集合,其中,批注区域对应有批注信息;确定第二帧图像,得到表征第二帧图像的第二特征点集合,其中,第二帧图像为与第一帧图像相关联的图像;将第二特征点集合与第一特征点集合进行匹配,基于匹配结果从第二特征点集合中选取出与第一特征点集合中特征点相匹配的目标特征点,得到目标特征点集合;及基于目标特征点集合确定出第二帧图像中与第一帧图像的批注区域相匹配的目标批注区域,其中,目标批注区域对应有与第一帧图像中批注区域的批注信息相匹配的批注信息。
在一个实施例中,计算机可读指令被处理器执行时,使得处理器还执行以下步骤:确定从第一帧图像变换到第二帧图像的图像移动特征;基于图像移动特征,从第二帧图像中预估出与第一特征点集合中特征点相匹配的目标 特征点,得到第一预估目标特征点集合。计算机可读指令被处理器执行时,使得处理器在执行基于匹配结果从第二特征点集合中选取出与第一特征点集合中特征点相匹配的目标特征点,得到目标特征点集合的步骤时,执行以下步骤:基于匹配结果从第二特征点集合中选取出与第一特征点集合中特征点相匹配的目标特征点,得到第二预估目标特征点集合;及基于第一预估目标特征点集合和第二预估目标特征点集合,得到目标特征点集合。
在一个实施例中,计算机可读指令被处理器执行时,使得处理器在执行将第二特征点集合与第一特征点集合进行匹配,基于匹配结果从第二特征点集合中选取出与第一特征点集合中特征点相匹配的目标特征点的步骤时,执行以下步骤:确定第二特征点集合中的第二特征点与第一特征点集合中的第一特征点之间的距离特征;从第二特征点集合中选取出距离特征满足预设距离规则的目标特征点。
在一个实施例中,计算机可读指令被处理器执行时,使得处理器在执行基于目标特征点集合确定出第二帧图像中与第一帧图像的批注区域相匹配的目标批注区域的步骤时,执行以下步骤:基于第一特征点集合和目标特征点集合,得到第二帧图像中与第一帧图像中的批注区域相匹配的目标中心特征点;基于第一特征点集合以及目标中心特征点确定出第二帧图像中的目标批注区域,其中,目标中心特征点位于目标批注区域的中心区域。
在一个实施例中,计算机可读指令被处理器执行时,使得处理器在执行基于第一特征点集合和目标特征点集合,得到第二帧图像中与第一帧图像中的批注区域相匹配的目标中心特征点的步骤时,执行以下步骤:基于第一特征点集合中第一特征点、以及目标特征点集合中与第一特征点相对应的目标特征点,确定出中心特征点,得到中心特征点集合;从中心特征点集合中选取出满足预设规则的目标中心特征点。
在一个实施例中,计算机可读指令被处理器执行时,使得处理器还执行以下步骤:至少根据第一帧图像和第二帧图像得到图像缩放特征;基于图像缩放特征对目标批注区域的批注信息进行缩放处理,在第二帧图像的目标批注区域中显示缩放处理后的批注信息。
本实施例还提供了一种计算机可读存储介质,一种非易失性的计算机可读存储介质,存储有计算机可读指令,计算机可读指令被一个或多个处理器执行时,使得一个或多个处理器执行以下步骤:显示内容共享的状态下,确定出显示内容所显示的第一帧图像中的批注区域,并确定出表征批注区域的第一特征点集合,其中,批注区域对应有批注信息;确定第二帧图像,得到第二帧图像的第二特征点集合,其中,第二帧图像为与第一帧图像相关联的图像;将第二特征点集合与第一特征点集合进行匹配,基于匹配结果从第二特征点集合中选取出与第一特征点集合中特征点相匹配的目标特征点,得到目标特征点集合;及基于目标特征点集合确定出第二帧图像中与第一帧图像的批注区域相匹配的目标批注区域,其中,目标批注区域对应有与第一帧图像中批注区域的批注信息相匹配的批注信息。
在一个实施例中,计算机可读指令被处理器执行时,使得处理器还执行以下步骤:确定从第一帧图像变换到第二帧图像的图像移动特征;基于图像移动特征,从第二帧图像中预估出与第一特征点集合中特征点相匹配的目标特征点,得到第一预估目标特征点集合。计算机可读指令被处理器执行时,使得处理器在执行基于匹配结果从第二特征点集合中选取出与第一特征点集合中特征点相匹配的目标特征点,得到目标特征点集合的步骤时,执行以下步骤:基于匹配结果从第二特征点集合中选取出与第一特征点集合中特征点相匹配的目标特征点,得到第二预估目标特征点集合;基于第一预估目标特征点集合和第二预估目标特征点集合,得到目标特征点集合。
在一个实施例中,计算机可读指令被处理器执行时,使得处理器在执行将第二特征点集合与第一特征点集合进行匹配,基于匹配结果从第二特征点集合中选取出与第一特征点集合中特征点相匹配的目标特征点的步骤时,执行以下步骤:确定第二特征点集合中的第二特征点与第一特征点集合中的第一特征点之间的距离特征;从第二特征点集合中选取出距离特征满足预设距离规则的目标特征点。
在一个实施例中,计算机可读指令被处理器执行时,使得处理器在执行基于目标特征点集合确定出第二帧图像中与第一帧图像的批注区域相匹配的目标批注区域的步骤时,执行以下步骤:基于第一特征点集合和目标特征点 集合,得到第二帧图像中与第一帧图像中的批注区域相匹配的目标中心特征点;基于第一特征点集合以及目标中心特征点确定出第二帧图像中的目标批注区域,其中,目标中心特征点位于目标批注区域的中心区域。
在一个实施例中,计算机可读指令被处理器执行时,使得处理器在执行基于第一特征点集合和目标特征点集合,得到第二帧图像中与第一帧图像中的批注区域相匹配的目标中心特征点的步骤时,执行以下步骤:基于第一特征点集合中第一特征点、以及目标特征点集合中与第一特征点相对应的目标特征点,确定出中心特征点,得到中心特征点集合;从中心特征点集合中选取出满足预设规则的目标中心特征点。
在一个实施例中,计算机可读指令被处理器执行时,使得处理器还执行以下步骤至少根据第一帧图像和第二帧图像得到图像缩放特征;基于图像缩放特征对目标批注区域的批注信息进行缩放处理,在第二帧图像的目标批注区域中显示缩放处理后的批注信息。
本实施例还提供了一种图像处理装置,实际应用中,图像处理装置可以具体为具有显示组件的任意电子设备,如个人电脑、移动终端等。其中,显示组件可以具体为显示器。图像处理装置具体对应上述的发送方终端。
该图像处理装置至少包括显示组件和处理器:显示组件,用于在显示界面展示显示内容;处理器,用于与其他电子设备(如接收方终端)共享显示界面所展示的显示内容;比如,将显示界面展示的显示内容发送至其他电子设备,以与其他电子设备共享显示界面所展示的显示内容;处理器,还用于在显示内容共享的状态下,确定出显示内容所显示的第一帧图像中的批注区域,并确定出表征批注区域的第一特征点集合,其中,批注区域对应有批注信息;确定第二帧图像,得到表征第二帧图像的第二特征点集合,其中,第二帧图像为与第一帧图像相关联的图像;将第二特征点集合与第一特征点集合进行匹配,基于匹配结果从第二特征点集合中选取出与第一特征点集合中特征点相匹配的目标特征点,得到目标特征点集合;基于目标特征点集合确定出第二帧图像中与第一帧图像的批注区域相匹配的目标批注区域,其中,目标批注区域对应有与第一帧图像中批注区域的批注信息相匹配的批注信 息。
在一个实施例中,计算机可读指令被处理器执行时,使得处理器还执行以下步骤:确定从第一帧图像变换到第二帧图像的图像移动特征;基于图像移动特征,从第二帧图像中预估出与第一特征点集合中特征点相匹配的目标特征点,得到第一预估目标特征点集合。计算机可读指令被处理器执行时,使得处理器在执行将至少基于匹配结果从第二特征点集合中选取出与第一特征点集合中特征点相匹配的目标特征点,得到目标特征点集合的步骤时,执行以下步骤:基于匹配结果从第二特征点集合中选取出与第一特征点集合中特征点相匹配的目标特征点,得到第二预估目标特征点集合;基于第一预估目标特征点集合和第二预估目标特征点集合,得到目标特征点集合。
在一个实施例中,计算机可读指令被处理器执行时,使得处理器在执行将第二特征点集合与第一特征点集合进行匹配,至少基于匹配结果从第二特征点集合中选取出与第一特征点集合中特征点相匹配的目标特征点的步骤时,执行以下步骤:确定第二特征点集合中的第二特征点与第一特征点集合中的第一特征点之间的距离特征;从第二特征点集合中选取出距离特征满足预设距离规则的目标特征点。
在一个实施例中,计算机可读指令被处理器执行时,使得处理器在执行基于目标特征点集合确定出第二帧图像中与第一帧图像的批注区域相匹配的目标批注区域的步骤时,执行以下步骤:基于第一特征点集合和目标特征点集合,得到第二帧图像中与第一帧图像中的批注区域相匹配的目标中心特征点;基于第一特征点集合以及目标中心特征点确定出第二帧图像中的目标批注区域,其中,目标中心特征点位于目标批注区域的中心区域。
在一个实施例中,计算机可读指令被处理器执行时,使得处理器在执行基于第一特征点集合和目标特征点集合,得到第二帧图像中与第一帧图像中的批注区域相匹配的目标中心特征点的步骤时,执行以下步骤:基于第一特征点集合中第一特征点与目标特征点集合中与第一特征点相对应的目标特征点,确定出中心特征点,得到中心特征点集合;从中心特征点集合中选取出满足预设规则的目标中心特征点。
在一个实施例中,计算机可读指令被处理器执行时,使得处理器还执行 以下步骤根据第一帧图像和第二帧图像得到图像缩放特征;基于图像缩放特征对目标批注区域的批注信息进行缩放处理,在第二帧图像的目标批注区域中显示缩放处理后的批注信息。
这里需要指出的是:以上装置实施例的描述,与上述方法实施例的描述是类似的,具有同方法实施例相似的有益效果,因此不做赘述。对于本发明装置实施例中未披露的技术细节,请参照本发明方法实施例的描述而理解,为节约篇幅,因此不再赘述。
本实施例还提供了一种图像处理装置,实际应用中,图像处理装置可以具体为具有显示组件的任意电子设备,如个人电脑、移动终端等。其中,显示组件可以具体为显示器。图像处理装置具体对应上述的接收方终端。
该图像处理装置至少包括显示组件和处理器:处理器,用于获取其他电子设备(如发送方终端)所共享的显示内容。显示组件,用于在显示界面展示获取到的其他电子设备所共享的显示内容。处理器,还用于在显示内容共享的状态下,确定出显示内容所显示的第一帧图像中的批注区域,并确定出表征批注区域的第一特征点集合,其中,批注区域对应有批注信息;确定第二帧图像,得到表征第二帧图像的第二特征点集合,其中,第二帧图像为与第一帧图像相关联的图像;将第二特征点集合与第一特征点集合进行匹配,至少基于匹配结果从第二特征点集合中选取出与第一特征点集合中特征点相匹配的目标特征点,得到目标特征点集合;至少基于目标特征点集合确定出第二帧图像中与第一帧图像的批注区域相匹配的目标批注区域,其中,目标批注区域对应有与第一帧图像中批注区域的批注信息相匹配的批注信息。
在一个实施例中,计算机可读指令被处理器执行时,使得处理器还执行以下步骤:确定从第一帧图像变换到第二帧图像的图像移动特征;基于图像移动特征,从第二帧图像中预估出与第一特征点集合中特征点相匹配的目标特征点,得到第一预估目标特征点集合。计算机可读指令被处理器执行时,使得处理器在执行将至少基于匹配结果从第二特征点集合中选取出与第一特征点集合中特征点相匹配的目标特征点,得到目标特征点集合的步骤时,执行以下步骤:基于匹配结果从第二特征点集合中选取出与第一特征点集合中 特征点相匹配的目标特征点,得到第二预估目标特征点集合;基于第一预估目标特征点集合和第二预估目标特征点集合,得到目标特征点集合。
在一个实施例中,计算机可读指令被处理器执行时,使得处理器在执行将第二特征点集合与第一特征点集合进行匹配,至少基于匹配结果从第二特征点集合中选取出与第一特征点集合中特征点相匹配的目标特征点的步骤时,执行以下步骤:确定第二特征点集合中的第二特征点与第一特征点集合中的第一特征点之间的距离特征;从第二特征点集合中选取出距离特征满足预设距离规则的目标特征点。
在一个实施例中,计算机可读指令被处理器执行时,使得处理器在执行基于目标特征点集合确定出第二帧图像中与第一帧图像的批注区域相匹配的目标批注区域的步骤时,执行以下步骤:基于第一特征点集合和目标特征点集合,得到第二帧图像中与第一帧图像中的批注区域相匹配的目标中心特征点;基于第一特征点集合以及目标中心特征点确定出第二帧图像中的目标批注区域,其中,目标中心特征点位于目标批注区域的中心区域。
在一个实施例中,计算机可读指令被处理器执行时,使得处理器在执行基于第一特征点集合和目标特征点集合,得到第二帧图像中与第一帧图像中的批注区域相匹配的目标中心特征点的步骤时,执行以下步骤:基于第一特征点集合中第一特征点与目标特征点集合中与第一特征点相对应的目标特征点,确定出中心特征点,得到中心特征点集合;从中心特征点集合中选取出满足预设规则的目标中心特征点。
在一个实施例中,计算机可读指令被处理器执行时,使得处理器还执行以下步骤根据第一帧图像和第二帧图像得到图像缩放特征;基于图像缩放特征对目标批注区域的批注信息进行缩放处理,在第二帧图像的目标批注区域中显示缩放处理后的批注信息。
这里需要指出的是:以上装置实施例的描述,与上述方法实施例的描述是类似的,具有同方法实施例相似的有益效果,因此不做赘述。对于本发明装置实施例中未披露的技术细节,请参照本发明方法实施例的描述而理解,为节约篇幅,因此不再赘述。
在本申请所提供的几个实施例中,应该理解到,所揭露的设备和方法,可以通过其它的方式实现。以上所描述的设备实施例仅仅是示意性的,例如,所述单元的划分,仅仅为一种逻辑功能划分,实际实现时可以有另外的划分方式,如:多个单元或组件可以结合,或可以集成到另一个系统,或一些特征可以忽略,或不执行。另外,所显示或讨论的各组成部分相互之间的耦合、或直接耦合、或通信连接可以是通过一些接口,设备或单元的间接耦合或通信连接,可以是电性的、机械的或其它形式的。
上述作为分离部件说明的单元可以是、或也可以不是物理上分开的,作为单元显示的部件可以是、或也可以不是物理单元,即可以位于一个地方,也可以分布到多个网络单元上;可以根据实际的需要选择其中的部分或全部单元来实现本实施例方案的目的。
另外,在本发明各实施例中的各功能单元可以全部集成在一个处理单元中,也可以是各单元分别单独作为一个单元,也可以两个或两个以上单元集成在一个单元中;上述集成的单元既可以采用硬件的形式实现,也可以采用硬件加软件功能单元的形式实现。
本领域普通技术人员可以理解:实现上述方法实施例的全部或部分步骤可以通过程序指令相关的硬件来完成,前述的程序可以存储于一计算机可读取存储介质中,该程序在执行时,执行包括上述方法实施例的步骤;而前述的存储介质包括:移动存储设备、只读存储器(ROM,Read Only Memory)、随机存取存储器(RAM,Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。
或者,本发明上述集成的单元如果以软件功能模块的形式实现并作为独立的产品销售或使用时,也可以存储在一个计算机可读取存储介质中。基于这样的理解,本发明实施例的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来,该计算机软件产品存储在一个存储介质中,包括若干指令用以使得一台计算机设备(可以是个人计算机、服务器、或者网络设备等)执行本发明各个实施例所述方法的全部或部分。而前述的存储介质包括:移动存储设备、只读存储器(ROM,Read Only Memory)、随机存取存储器(RAM,Random Access Memory)、磁碟或者光盘等各种可 以存储程序代码的介质。
以上所述,仅为本发明的具体实施方式,但本发明的保护范围并不局限于此,任何熟悉本技术领域的技术人员在本发明揭露的技术范围内,可轻易想到变化或替换,都应涵盖在本发明的保护范围之内。因此,本发明的保护范围应以所述权利要求的保护范围为准。
Claims (26)
- 一种图像处理方法,由终端执行,所述方法包括:显示内容共享的状态下,确定出所述显示内容所显示的第一帧图像中的批注区域,并确定出表征所述批注区域的第一特征点集合,其中,所述批注区域对应有批注信息;确定第二帧图像,得到表征所述第二帧图像的第二特征点集合,其中,所述第二帧图像为与所述第一帧图像相关联的图像;将所述第二特征点集合与所述第一特征点集合进行匹配,基于匹配结果从所述第二特征点集合中选取出与所述第一特征点集合中特征点相匹配的目标特征点,得到目标特征点集合;及基于所述目标特征点集合确定出所述第二帧图像中与所述第一帧图像的批注区域相匹配的目标批注区域,其中,所述目标批注区域对应有与所述第一帧图像中批注区域的批注信息相匹配的批注信息。
- 根据权利要求1所述的方法,其特征在于,所述方法还包括:确定从所述第一帧图像变换到所述第二帧图像的图像移动特征;基于所述图像移动特征,从所述第二帧图像中预估出与所述第一特征点集合中特征点相匹配的目标特征点,得到第一预估目标特征点集合;所述基于匹配结果从所述第二特征点集合中选取出与所述第一特征点集合中特征点相匹配的目标特征点,得到目标特征点集合,包括:基于匹配结果从所述第二特征点集合中选取出与所述第一特征点集合中特征点相匹配的目标特征点,得到第二预估目标特征点集合;及基于所述第一预估目标特征点集合和所述第二预估目标特征点集合,得到目标特征点集合。
- 根据权利要求1所述的方法,其特征在于,所述将所述第二特征点集合与所述第一特征点集合进行匹配,基于匹配结果从所述第二特征点集合中选取出与所述第一特征点集合中特征点相匹配的目标特征点,包括:确定所述第二特征点集合中的第二特征点与所述第一特征点集合中的第一特征点之间的距离特征;及从所述第二特征点集合中选取出距离特征满足预设距离规则的目标特征 点。
- 根据权利要求1所述的方法,其特征在于,所述基于所述目标特征点集合确定出所述第二帧图像中与所述第一帧图像的批注区域相匹配的目标批注区域,包括:基于所述第一特征点集合和所述目标特征点集合,得到所述第二帧图像中与第一帧图像中的批注区域相匹配的目标中心特征点;及基于所述第一特征点集合以及所述目标中心特征点确定出所述第二帧图像中的目标批注区域,其中,所述目标中心特征点位于所述目标批注区域的中心区域。
- 根据权利要求4所述的方法,其特征在于,所述基于所述第一特征点集合和所述目标特征点集合,得到所述第二帧图像中与第一帧图像中的批注区域相匹配的目标中心特征点,包括:基于所述第一特征点集合中第一特征点、以及目标特征点集合中与所述第一特征点相对应的目标特征点,确定出中心特征点,得到中心特征点集合;及从所述中心特征点集合中选取出满足预设规则的目标中心特征点。
- 根据权利要求1所述的方法,其特征在于,所述方法还包括:根据所述第一帧图像和所述第二帧图像得到图像缩放特征;及基于所述图像缩放特征对所述目标批注区域的批注信息进行缩放处理,在所述第二帧图像的目标批注区域中显示缩放处理后的批注信息。
- 一种图像处理装置,其特征在于,所述装置包括:第一确定单元,用于显示内容共享的状态下,确定出所述显示内容所显示的第一帧图像中的批注区域,并确定出表征所述批注区域的第一特征点集合,其中,所述批注区域对应有批注信息;还用于确定第二帧图像,得到表征所述第二帧图像的第二特征点集合,其中,所述第二帧图像为与所述第一帧图像相关联的图像;特征点匹配单元,用于将所述第二特征点集合与所述第一特征点集合进行匹配,基于匹配结果从所述第二特征点集合中选取出与所述第一特征点集合中特征点相匹配的目标特征点,得到目标特征点集合;第二确定单元,用于基于所述目标特征点集合确定出所述第二帧图像中与所述第一帧图像的批注区域相匹配的目标批注区域,其中,所述目标批注区域对应有与所述第一帧图像中批注区域的批注信息相匹配的批注信息。
- 根据权利要求7所述的装置,其特征在于,所述第一确定单元,还用于确定从所述第一帧图像变换到所述第二帧图像的图像移动特征;基于所述图像移动特征,从所述第二帧图像中预估出与所述第一特征点集合中特征点相匹配的目标特征点,得到第一预估目标特征点集合;所述特征点匹配单元,还用于基于匹配结果从所述第二特征点集合中选取出与所述第一特征点集合中特征点相匹配的目标特征点,得到第二预估目标特征点集合;及基于所述第一预估目标特征点集合和所述第二预估目标特征点集合,得到目标特征点集合。
- 根据权利要求7所述的装置,其特征在于,所述特征点匹配单元,还用于确定所述第二特征点集合中的第二特征点与所述第一特征点集合中的第一特征点之间的距离特征;及从所述第二特征点集合中选取出距离特征满足预设距离规则的目标特征点。
- 根据权利要求7所述的装置,其特征在于,所述第二确定单元,还用于基于所述第一特征点集合和所述目标特征点集合,得到所述第二帧图像中与第一帧图像中的批注区域相匹配的目标中心特征点;及基于所述第一特征点集合以及所述目标中心特征点确定出所述第二帧图像中的目标批注区域,其中,所述目标中心特征点位于所述目标批注区域的中心区域。
- 根据权利要求10所述的装置,其特征在于,所述第二确定单元,还用于基于所述第一特征点集合中第一特征点、以及目标特征点集合中与所述第一特征点相对应的目标特征点,确定出中心特征点,得到中心特征点集合;及从所述中心特征点集合中选取出满足预设规则的目标中心特征点。
- 根据权利要求7所述的装置,其特征在于,所述装置还包括:图像缩放单元;其中,所述图像缩放单元,用于根据所述第一帧图像和所述第二帧图像得到图像缩放特征;及基于所述图像缩放特征对所述目标批注区域的批注信息进行缩放处理,在所述第二帧图像的目标批注区域中显示缩放处理后的批注信息。
- 一种终端,包括存储器和处理器,所述存储器中存储有计算机可读指令,所述计算机可读指令被所述处理器执行时,使得所述处理器执行以下步骤:显示内容共享的状态下,确定出所述显示内容所显示的第一帧图像中的批注区域,并确定出表征所述批注区域的第一特征点集合,其中,所述批注区域对应有批注信息;确定第二帧图像,得到表征所述第二帧图像的第二特征点集合,其中,所述第二帧图像为与所述第一帧图像相关联的图像;将所述第二特征点集合与所述第一特征点集合进行匹配,基于匹配结果从所述第二特征点集合中选取出与所述第一特征点集合中特征点相匹配的目标特征点,得到目标特征点集合;及基于所述目标特征点集合确定出所述第二帧图像中与所述第一帧图像的批注区域相匹配的目标批注区域,其中,所述目标批注区域对应有与所述第一帧图像中批注区域的批注信息相匹配的批注信息。
- 根据权利要求13所述的终端,其特征在于,所述计算机可读指令被所述处理器执行时,使得所述处理器还执行以下步骤:确定从所述第一帧图像变换到所述第二帧图像的图像移动特征;基于所述图像移动特征,从所述第二帧图像中预估出与所述第一特征点集合中特征点相匹配的目标特征点,得到第一预估目标特征点集合;所述计算机可读指令被所述处理器执行时,使得所述处理器在执行基于匹配结果从所述第二特征点集合中选取出与所述第一特征点集合中特征点相匹配的目标特征点,得到目标特征点集合的步骤时,执行以下步骤:基于匹配结果从所述第二特征点集合中选取出与所述第一特征点集合中特征点相匹配的目标特征点,得到第二预估目标特征点集合;及基于所述第一预估目标特征点集合和所述第二预估目标特征点集合,得到目标特征点集合。
- 根据权利要求13所述的终端,其特征在于,所述计算机可读指令被所述处理器执行时,使得所述处理器在执行将所述第二特征点集合与所述第一特征点集合进行匹配,基于匹配结果从所述第二特征点集合中选取出与所 述第一特征点集合中特征点相匹配的目标特征点的步骤时,执行以下步骤:确定所述第二特征点集合中的第二特征点与所述第一特征点集合中的第一特征点之间的距离特征;及从所述第二特征点集合中选取出距离特征满足预设距离规则的目标特征点。
- 根据权利要求13所述的终端,其特征在于,所述计算机可读指令被所述处理器执行时,使得所述处理器在执行基于所述目标特征点集合确定出所述第二帧图像中与所述第一帧图像的批注区域相匹配的目标批注区域的步骤时,执行以下步骤:基于所述第一特征点集合和所述目标特征点集合,得到所述第二帧图像中与第一帧图像中的批注区域相匹配的目标中心特征点;及基于所述第一特征点集合以及所述目标中心特征点确定出所述第二帧图像中的目标批注区域,其中,所述目标中心特征点位于所述目标批注区域的中心区域。
- 根据权利要求16所述的终端,其特征在于,所述计算机可读指令被所述处理器执行时,使得所述处理器在执行基于所述第一特征点集合和所述目标特征点集合,得到所述第二帧图像中与第一帧图像中的批注区域相匹配的目标中心特征点的步骤时,执行以下步骤:基于所述第一特征点集合中第一特征点、以及目标特征点集合中与所述第一特征点相对应的目标特征点,确定出中心特征点,得到中心特征点集合;及从所述中心特征点集合中选取出满足预设规则的目标中心特征点。
- 根据权利要求13所述的终端,其特征在于,所述计算机可读指令被所述处理器执行时,使得所述处理器还执行以下步骤:根据所述第一帧图像和所述第二帧图像得到图像缩放特征;及基于所述图像缩放特征对所述目标批注区域的批注信息进行缩放处理,在所述第二帧图像的目标批注区域中显示缩放处理后的批注信息。
- 一种非易失性的计算机可读存储介质,存储有计算机可读指令,所述计算机可读指令被一个或多个处理器执行时,使得所述一个或多个处理器 执行以下步骤:显示内容共享的状态下,确定出所述显示内容所显示的第一帧图像中的批注区域,并确定出表征所述批注区域的第一特征点集合,其中,所述批注区域对应有批注信息;确定第二帧图像,得到表征所述第二帧图像的第二特征点集合,其中,所述第二帧图像为与所述第一帧图像相关联的图像;将所述第二特征点集合与所述第一特征点集合进行匹配,基于匹配结果从所述第二特征点集合中选取出与所述第一特征点集合中特征点相匹配的目标特征点,得到目标特征点集合;及基于所述目标特征点集合确定出所述第二帧图像中与所述第一帧图像的批注区域相匹配的目标批注区域,其中,所述目标批注区域对应有与所述第一帧图像中批注区域的批注信息相匹配的批注信息。
- 根据权利要求19所述的计算机可读存储介质,其特征在于,所述计算机可读指令被所述处理器执行时,使得所述处理器还执行以下步骤:确定从所述第一帧图像变换到所述第二帧图像的图像移动特征;基于所述图像移动特征,从所述第二帧图像中预估出与所述第一特征点集合中特征点相匹配的目标特征点,得到第一预估目标特征点集合;所述计算机可读指令被所述处理器执行时,使得所述处理器在执行基于匹配结果从所述第二特征点集合中选取出与所述第一特征点集合中特征点相匹配的目标特征点,得到目标特征点集合的步骤时,执行以下步骤:基于匹配结果从所述第二特征点集合中选取出与所述第一特征点集合中特征点相匹配的目标特征点,得到第二预估目标特征点集合;及基于所述第一预估目标特征点集合和所述第二预估目标特征点集合,得到目标特征点集合。
- 根据权利要求19所述的计算机可读存储介质,其特征在于,所述计算机可读指令被所述处理器执行时,使得所述处理器在执行将所述第二特征点集合与所述第一特征点集合进行匹配,基于匹配结果从所述第二特征点集合中选取出与所述第一特征点集合中特征点相匹配的目标特征点的步骤时,执行以下步骤:确定所述第二特征点集合中的第二特征点与所述第一特征点集合中的第一特征点之间的距离特征;及从所述第二特征点集合中选取出距离特征满足预设距离规则的目标特征点。
- 根据权利要求19所述的计算机可读存储介质,其特征在于,所述计算机可读指令被所述处理器执行时,使得所述处理器在执行基于所述目标特征点集合确定出所述第二帧图像中与所述第一帧图像的批注区域相匹配的目标批注区域的步骤时,执行以下步骤:基于所述第一特征点集合和所述目标特征点集合,得到所述第二帧图像中与第一帧图像中的批注区域相匹配的目标中心特征点;及基于所述第一特征点集合以及所述目标中心特征点确定出所述第二帧图像中的目标批注区域,其中,所述目标中心特征点位于所述目标批注区域的中心区域。
- 根据权利要求22所述的计算机可读存储介质,其特征在于,所述计算机可读指令被所述处理器执行时,使得所述处理器在执行基于所述第一特征点集合和所述目标特征点集合,得到所述第二帧图像中与第一帧图像中的批注区域相匹配的目标中心特征点的步骤时,执行以下步骤:基于所述第一特征点集合中第一特征点、以及目标特征点集合中与所述第一特征点相对应的目标特征点,确定出中心特征点,得到中心特征点集合;及从所述中心特征点集合中选取出满足预设规则的目标中心特征点。
- 根据权利要求19所述的计算机可读存储介质,其特征在于,所述计算机可读指令被所述处理器执行时,使得所述处理器还执行以下步骤:根据所述第一帧图像和所述第二帧图像得到图像缩放特征;及基于所述图像缩放特征对所述目标批注区域的批注信息进行缩放处理,在所述第二帧图像的目标批注区域中显示缩放处理后的批注信息。
- 一种图像处理装置,其特征在于,所述装置包括显示组件和处理器:所述显示组件,用于在显示界面展示显示内容;所述处理器,用于将显示界面展示的显示内容发送至其他电子设备,以 与其他电子设备共享所述显示界面所展示的显示内容;所述处理器,还用于在显示内容共享的状态下,确定出所述显示内容所显示的第一帧图像中的批注区域,并确定出表征所述批注区域的第一特征点集合,其中,所述批注区域对应有批注信息;确定第二帧图像,得到表征所述第二帧图像的第二特征点集合,其中,所述第二帧图像为与所述第一帧图像相关联的图像;将所述第二特征点集合与所述第一特征点集合进行匹配,基于匹配结果从所述第二特征点集合中选取出与所述第一特征点集合中特征点相匹配的目标特征点,得到目标特征点集合;基于所述目标特征点集合确定出所述第二帧图像中与所述第一帧图像的批注区域相匹配的目标批注区域,其中,所述目标批注区域对应有与所述第一帧图像中批注区域的批注信息相匹配的批注信息。
- 一种图像处理装置,其特征在于,所述装置包括处理器和显示组件:所述处理器,用于获取其他电子设备所共享的显示内容;所述显示组件,用于在显示界面展示获取到的其他电子设备所共享的显示内容;所述处理器,还用于在显示内容共享的状态下,确定出所述显示内容所显示的第一帧图像中的批注区域,并确定出表征所述批注区域的第一特征点集合,其中,所述批注区域对应有批注信息;确定第二帧图像,得到表征所述第二帧图像的第二特征点集合,其中,所述第二帧图像为与所述第一帧图像相关联的图像;将所述第二特征点集合与所述第一特征点集合进行匹配,基于匹配结果从所述第二特征点集合中选取出与所述第一特征点集合中特征点相匹配的目标特征点,得到目标特征点集合;基于所述目标特征点集合确定出所述第二帧图像中与所述第一帧图像的批注区域相匹配的目标批注区域,其中,所述目标批注区域对应有与所述第一帧图像中批注区域的批注信息相匹配的批注信息。
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201711428095.7A CN109960452B (zh) | 2017-12-26 | 2017-12-26 | 图像处理方法及其装置、存储介质 |
CN201711428095.7 | 2017-12-26 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2019128742A1 true WO2019128742A1 (zh) | 2019-07-04 |
Family
ID=67021605
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2018/121268 WO2019128742A1 (zh) | 2017-12-26 | 2018-12-14 | 图像处理方法、装置、终端及存储介质 |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN109960452B (zh) |
WO (1) | WO2019128742A1 (zh) |
Families Citing this family (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110035329B (zh) * | 2018-01-11 | 2022-08-30 | 腾讯科技(北京)有限公司 | 图像处理方法、装置及存储介质 |
CN110737417B (zh) * | 2019-09-30 | 2024-01-23 | 深圳市格上视点科技有限公司 | 一种演示设备及其标注线的显示控制方法和装置 |
CN111291768B (zh) * | 2020-02-17 | 2023-05-30 | Oppo广东移动通信有限公司 | 图像特征匹配方法及装置、设备、存储介质 |
CN111627041B (zh) * | 2020-04-15 | 2023-10-10 | 北京迈格威科技有限公司 | 多帧数据的处理方法、装置及电子设备 |
CN111882582B (zh) * | 2020-07-24 | 2021-10-08 | 广州云从博衍智能科技有限公司 | 一种图像跟踪关联方法、系统、设备及介质 |
CN112995467A (zh) * | 2021-02-05 | 2021-06-18 | 深圳传音控股股份有限公司 | 图像处理方法、移动终端及存储介质 |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20100257188A1 (en) * | 2007-12-14 | 2010-10-07 | Electronics And Telecommunications Research Institute | Method and apparatus for providing/receiving stereoscopic image data download service in digital broadcasting system |
CN104363407A (zh) * | 2014-10-31 | 2015-02-18 | 华为技术有限公司 | 一种视频会议系统通讯方法及相应装置 |
CN106650965A (zh) * | 2016-12-30 | 2017-05-10 | 触景无限科技(北京)有限公司 | 一种远程视频处理方法及装置 |
CN107168674A (zh) * | 2017-06-19 | 2017-09-15 | 浙江工商大学 | 投屏批注方法和系统 |
CN107308646A (zh) * | 2017-06-23 | 2017-11-03 | 腾讯科技(深圳)有限公司 | 确定匹配对象的方法、装置及存储介质 |
Family Cites Families (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101206640B (zh) * | 2006-12-22 | 2011-01-26 | 深圳市学之友教学仪器有限公司 | 用于对便携式电子设备中的电子资料进行批注的方法 |
US9654727B2 (en) * | 2015-06-01 | 2017-05-16 | Apple Inc. | Techniques to overcome communication lag between terminals performing video mirroring and annotation operations |
CN105573702A (zh) * | 2015-12-16 | 2016-05-11 | 广州视睿电子科技有限公司 | 远程批注移动、缩放的同步方法与系统 |
CN106940632A (zh) * | 2017-03-06 | 2017-07-11 | 锐达互动科技股份有限公司 | 一种屏幕批注的方法 |
CN107274431A (zh) * | 2017-03-07 | 2017-10-20 | 阿里巴巴集团控股有限公司 | 视频内容增强方法及装置 |
CN106843797A (zh) * | 2017-03-13 | 2017-06-13 | 广州视源电子科技股份有限公司 | 一种图像文件的编辑方法及装置 |
-
2017
- 2017-12-26 CN CN201711428095.7A patent/CN109960452B/zh active Active
-
2018
- 2018-12-14 WO PCT/CN2018/121268 patent/WO2019128742A1/zh active Application Filing
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20100257188A1 (en) * | 2007-12-14 | 2010-10-07 | Electronics And Telecommunications Research Institute | Method and apparatus for providing/receiving stereoscopic image data download service in digital broadcasting system |
CN104363407A (zh) * | 2014-10-31 | 2015-02-18 | 华为技术有限公司 | 一种视频会议系统通讯方法及相应装置 |
CN106650965A (zh) * | 2016-12-30 | 2017-05-10 | 触景无限科技(北京)有限公司 | 一种远程视频处理方法及装置 |
CN107168674A (zh) * | 2017-06-19 | 2017-09-15 | 浙江工商大学 | 投屏批注方法和系统 |
CN107308646A (zh) * | 2017-06-23 | 2017-11-03 | 腾讯科技(深圳)有限公司 | 确定匹配对象的方法、装置及存储介质 |
Also Published As
Publication number | Publication date |
---|---|
CN109960452B (zh) | 2022-11-04 |
CN109960452A (zh) | 2019-07-02 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2019128742A1 (zh) | 图像处理方法、装置、终端及存储介质 | |
CN110035329B (zh) | 图像处理方法、装置及存储介质 | |
JP6179889B2 (ja) | コメント情報生成装置およびコメント表示装置 | |
CN107633241B (zh) | 一种全景视频自动标注和追踪物体的方法和装置 | |
EP3195601B1 (en) | Method of providing visual sound image and electronic device implementing the same | |
WO2019140997A1 (zh) | 批注显示方法、装置、设备及存储介质 | |
US9179096B2 (en) | Systems and methods for real-time efficient navigation of video streams | |
JP5659307B2 (ja) | コメント情報生成装置およびコメント情報生成方法 | |
US7995074B2 (en) | Information presentation method and information presentation apparatus | |
KR20140139859A (ko) | 멀티미디어 콘텐츠 검색을 위한 사용자 인터페이스 방법 및 장치 | |
JP2005108225A (ja) | オーディオビジュアルプレゼンテーションのコンテンツの要約及び索引付けするための方法及び装置 | |
JP2017049968A (ja) | ユーザインタラクションを検出、分類及び可視化する方法、システム及びプログラム | |
WO2023202570A1 (zh) | 图像处理方法和处理装置、电子设备和可读存储介质 | |
JP6203188B2 (ja) | 類似画像検索装置 | |
CN103219028B (zh) | 信息处理装置和信息处理方法 | |
JP2018005011A (ja) | プレゼンテーション支援装置、プレゼンテーション支援システム、プレゼンテーション支援方法及びプレゼンテーション支援プログラム | |
US20160210101A1 (en) | Document display support device, terminal, document display method, and computer-readable storage medium for computer program | |
US20210089783A1 (en) | Method for fast visual data annotation | |
JP2009294984A (ja) | 資料データ編集システム及び資料データ編集方法 | |
US20230043683A1 (en) | Determining a change in position of displayed digital content in subsequent frames via graphics processing circuitry | |
US20120233281A1 (en) | Picture processing method and apparatus for instant communication tool | |
US11557065B2 (en) | Automatic segmentation for screen-based tutorials using AR image anchors | |
JP2008269421A (ja) | 記録装置と記録装置のためのプログラム | |
WO2023029924A1 (zh) | 评论信息的显示方法、装置、设备、存储介质及程序产品 | |
JP7513019B2 (ja) | 画像処理装置および方法、並びに、プログラム |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 18895125 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 18895125 Country of ref document: EP Kind code of ref document: A1 |