CN110111347B - Image sign extraction method, device and storage medium - Google Patents
Image sign extraction method, device and storage medium Download PDFInfo
- Publication number
- CN110111347B CN110111347B CN201910316438.3A CN201910316438A CN110111347B CN 110111347 B CN110111347 B CN 110111347B CN 201910316438 A CN201910316438 A CN 201910316438A CN 110111347 B CN110111347 B CN 110111347B
- Authority
- CN
- China
- Prior art keywords
- image
- value
- gray
- frame
- preset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000605 extraction Methods 0.000 title claims abstract description 55
- 238000000034 method Methods 0.000 claims abstract description 42
- 230000008569 process Effects 0.000 claims abstract description 21
- 239000003550 marker Substances 0.000 claims description 23
- 238000006243 chemical reaction Methods 0.000 claims description 17
- 230000001186 cumulative effect Effects 0.000 claims description 11
- 238000012545 processing Methods 0.000 claims description 8
- 238000004364 calculation method Methods 0.000 claims description 7
- 230000000694 effects Effects 0.000 abstract description 8
- 238000001914 filtration Methods 0.000 description 14
- 230000006870 function Effects 0.000 description 11
- 238000012546 transfer Methods 0.000 description 10
- 238000009825 accumulation Methods 0.000 description 6
- 238000003708 edge detection Methods 0.000 description 4
- 238000013459 approach Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000010191 image analysis Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/12—Edge-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The application provides an image sign extraction method, an image sign extraction device and a storage medium, wherein the method comprises the following steps: in the video playing process, acquiring a video image with continuous preset frames; determining a gray image corresponding to each frame of the video image to obtain a preset frame gray image; determining edge feature point information from the gray level image by using a preset algorithm; the image mark is extracted from the video according to the edge feature point information of the preset frame gray level image, so that the accuracy of extracting the semitransparent image mark in the video is improved, the method is simple, and the extraction effect is good.
Description
Technical Field
The present application relates to the field of display technologies, and in particular, to a method and an apparatus for extracting an image mark, and a storage medium.
Background
In image recognition and image analysis, the edge information can well describe the outline shape of an object. The shape characteristics of the object can be extracted through edge detection, and the data volume required to be processed in subsequent image analysis can be greatly reduced, so that the image edge detection is an important technology in image processing and is widely applied to the fields of target identification, target tracking, fingerprint identification and the like.
However, in the image mark extraction method in the prior art, because the edge of the semitransparent image feature region has information partially changed with the background content, the traditional extraction method based on color information is not accurate and has weak extraction effect, which brings limitation to the use of the method.
In summary, the image marker of the prior art has a problem of weak extraction effect of the semitransparent mark in the image area.
Disclosure of Invention
The application provides an image sign extraction method, an image sign extraction device and a storage medium, which can improve the accuracy of extracting semitransparent image signs in a video, and have the advantages of simple method and good extraction effect.
The application provides an image sign extraction method, which is applied to electronic equipment and comprises the following steps:
in the video playing process, acquiring a video image with continuous preset frames;
determining a gray image corresponding to each frame of the video image to obtain a preset frame gray image;
determining edge feature point information from the gray level image by using a preset algorithm;
and extracting an image mark from the video according to the edge characteristic point information of the preset frame gray level image.
The application provides an image sign extraction element, is applied to electronic equipment, includes:
the acquisition module is used for acquiring a video image with continuous preset frames in the video playing process;
the determining module is used for determining a gray image corresponding to each frame of video image to obtain a preset frame gray image;
the calculation module is used for determining edge feature point information from the gray level image by using a preset algorithm;
and the extraction module is used for extracting an image mark from the video according to the edge characteristic point information of the preset frame gray level image.
Embodiments of the present application further provide a computer-readable storage medium, where the storage medium has a plurality of instructions, and the instructions are adapted to be executed by a processor to perform any one of the above image marker extraction methods.
The image mark extraction method, the image mark extraction device and the storage medium are applied to electronic equipment, and video images with continuous preset frames are obtained in the video playing process; determining a gray image corresponding to each frame of the video image to obtain a preset frame gray image; determining edge feature point information from the gray level image by using a preset algorithm; and extracting image marks from the video according to the edge characteristic point information of the preset frame gray level image, so that the accuracy of extracting the semitransparent image marks in the video is improved, and the method is simple and has a good extraction effect.
Drawings
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Fig. 1 is a schematic flow chart of an image marker extraction method according to an embodiment of the present application.
Fig. 2 is a schematic flowchart of step S101 provided in the embodiment of the present application.
Fig. 3 is a schematic flowchart of step S103 according to an embodiment of the present application.
Fig. 4 is a scene schematic diagram of an image marker extraction method provided in an embodiment of the present application
Fig. 5 is a schematic structural diagram of an image marker extraction device according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
An image mark extraction method is applied to electronic equipment and comprises the steps of acquiring a video image with continuous preset frames in the video playing process; determining a gray image corresponding to each frame of the video image to obtain a preset frame gray image; determining edge feature point information from the gray level image by using a preset algorithm; and extracting an image mark from the video according to the edge characteristic point information of the preset frame gray level image.
As shown in fig. 1, fig. 1 is a schematic flowchart of an image marker extracting method provided in an embodiment of the present application, where the image marker extracting method is applied to an electronic device, and a specific flow may be as follows:
s101, in the video playing process, video images with continuous preset frames are obtained.
In this embodiment, the video images with consecutive preset frames are obtained as arbitrary selections in the video. A frame is a single image frame of the smallest unit in a video image, and corresponds to each shot in a movie. Each frame is a still image and displaying frames in rapid succession creates the illusion of a moving video image, the video images of which are preset in succession, see fig. 4.
For example, referring to fig. 2, the step S101 may specifically include the following steps:
s1011, acquiring the RGB value of the pixel point in each frame of video image.
In this embodiment, each frame of the video image is obtained as a color image, and a pixel point of the video image is composed of three components of R/G/B, where R is a red component of the color image, G is a green component of the color image, and B is a blue component of the color image; each point is represented by three bytes to be R/G/B respectively; R/G/B is typically divided into 256 levels from 0 to 255, with 0 being darkest (all black) and 255 being brightest (all white).
S1012, converting the RGB value from the RGB channel model into an HSV channel model to obtain a corresponding HSV value.
In this embodiment, the RGB channel model is an RGB color model divided into three color channels, red (R), green (G), and blue (B), and the RGB channel model is used to assign an intensity value in the range of 0 to 255 to the RGB component of each pixel in the image; the HSV channel model is divided into three HSV modes with channel hue (H), saturation (S) and lightness (V) with the visual color characteristics, and the HSV channel model is more in line with the habit of human eyes.
Among them, the HSV channel model includes HSV values (Hue, Saturation, value (brightness) lightness, also called HSB).
Further, the above step S1012 can be implemented by using the following formula (1-1):
V＝max
wherein H is the hue of the pixel point and ranges from 0 to 1; s is the saturation of the pixel point and ranges from 0 to 1; v is the lightness of the pixel point and ranges from 0 to 1; max is the maximum value of R, G and B, and the range is 0-255; min is the minimum value of the R, G pixel point and the B value, and the range is 0-255.
And S1013, determining the gray level image of the corresponding frame video image according to the HSV value of the pixel point.
In this embodiment, the gray image determined according to the HSV value of the pixel point is an image that only contains brightness information and does not contain color information, that is, a black-and-white image in a general sense: the brightness of the gray scale image is changed from dark to light, and the change is continuous.
It should be noted that in this embodiment, H is the saturation of the pixel, and S is the saturation, and is related to the shade of the corresponding image of the preset frame, when S is equal to 0, the pixel has only gray scale, and V is brightness, and represents the brightness of the color, but is not directly related to the light intensity. On the basis of the theory, the H value of the pixel point in the preset frame video image is removed, and the purpose of determining the gray level image of the corresponding frame video image is achieved.
And S102, determining a gray level image corresponding to each frame of video image to obtain a preset frame gray level image.
In this embodiment, obtaining the preset frame gray image requires calculating a gray image value corresponding to the pixel point according to the HSV value corresponding to the pixel point, and further obtaining the preset frame gray image, where the step S102 may specifically include the following two steps:
and S1021, calculating a corresponding target value according to the saturation value of the pixel point.
In this embodiment, the formula (1-2) of the target value of the pixel point is
T＝1-S (1-2)
And T is a target value corresponding to the saturation value S of the pixel point.
And S1022, multiplying the target value by the brightness value of the corresponding pixel point to obtain the gray level image value of the pixel point.
In this embodiment, the gray image value formula (1-3) of the pixel point is;
SV＝T×V (1-3)
wherein, V is the lightness of the pixel point, SV is the gray image value of the pixel point, and the ranges of T, V, S and SV are both 0-1. Further, the S value and the V value of the pixel point are calculated in the above formula (1-1). The calculation method eliminates the H value of the pixel point in the preset frame video image, and further can determine the gray level image of the corresponding frame video image.
And S1023, determining the gray level image of the corresponding frame video image according to the gray level image value to obtain a preset frame gray level image.
In this embodiment, it should be noted that the gray scale image value SV of the pixel is an SV value of the pixel, and the gray scale image determined in step S1022 corresponding to the frame video image is the whole preset frame video image, and the corresponding frame video image, and similarly, the preset frame gray scale image includes the gray scale image values of all pixels on the image. Therefore, in this embodiment, the position of each pixel point needs to be traversed, and then the grayscale image of the preset frame video image is determined.
For example, assume that the preset frame video image is an image composed of X rows and Y columns of pixels, where X and Y are natural numbers, the image can be approximately considered as a matrix in X rows and Y columns, the coordinate values of the individual pixels can be represented as (i, j), (i, j) to identify the coordinate value of the pixel in the ith row and jth column, and 0< i ≦ X, and 0< j ≦ Y. The step of traversing includes: scanning from the line 1 to the line X, traversing the scanning line from the line 1 to the line Y, assuming that the traversing process reaches the line i (0< i ≦ X), only traversing the line i, for example (i, 1), (i, 2), (i, 3) … … (i, Y), when the pixel point (i, Y) is traversed, turning to traverse the line i +1, traversing the line i +1, for example (i +1, 1), (i +1, 2), (i +1, 3) … … (i +1, Y), and repeating the operation until the last pixel point traversed to the line X, namely the pixel point with the coordinate value of (X, Y), completes the traversing operation of the whole preset frame video image, thereby determining the gray level image of the preset frame video image.
Similarly, the step of traversing may further include: scanning from the 1 st column to the Y th column, traversing the 1 st row to the Y th column of the scanning column, assuming that the traversing process reaches the jth column (0< j is less than or equal to Y), only performing row traversal on the jth column, such as (1, j), (2, j), (3, j) … … (X, j), when traversing to a pixel point (X, j), turning to traverse the jth +1 column, performing row traversal on the jth +1 column, such as (1, j +1), (2, j +1), (3, j +1) … … (X, j +1), and repeating the operation until traversing to the last pixel point of the jth column, namely the pixel point with the coordinate value of (X, Y), so as to complete the traversal operation of the whole preset frame video image, thereby determining the gray level image of the preset frame video image.
And S103, determining edge feature point information from the gray level image by using a preset algorithm.
In this embodiment, the preset algorithm includes a classical edge extraction method for image processing, and specifically includes a first differential operator and a second differential operator, where the first differential operator includes Roberts (Roberts) and Sobel (Sobel), and the Sobel is a template that convolves an image with a vertical direction template and a horizontal direction template to perform edge detection.
For example, referring to fig. 3, the step S103 may specifically include the following steps:
and S1031, processing the gray level image of the preset frame video image by using a high-pass filter to obtain a processed image.
In this embodiment, a fourier transform is first calculated for the grayscale image of the preset frame video image, and the fourier transform formula (1-4) is:
k＝1，2，...，X b＝1,2，...，Y
wherein, X and Y are respectively the total pixel number of the preset frame video image in the horizontal direction and the longitudinal direction, and SV (k, b) is the gray image value of the preset frame video image at the position where the coordinate value is traversed to (k, b), wherein k is a positive integer from 1 to X, and b is a positive integer from 1 to Y; SV (i, j) is a gray image value of fourier transform whose coordinate value is a pixel point in the ith row and the jth column, and the following steps are performed with this coordinate point being a pixel point (i, j) in the embodiment of the present application.
And then carrying out high-pass filtering on the gray level image value subjected to Fourier transform, wherein the filtering formula (1-5) is as follows:
the transfer function equations (1-6) of the filter satisfy:
wherein d0 is a distance from the preset cutoff frequency to the origin, d (i, j) is a distance from the point (i, j) to the origin, H (i, j) is a transfer function of the filter traversed to the pixel point (i, j), and in the embodiment of the present application, the transfer functions of the filters corresponding to each pixel point are the same.
The Fourier transformed gray level image value is subjected to a transfer function of filtering by a filter to obtain a Fourier filtered gray level image value, and the formula (1-7) of the gray level image value is as follows:
G(i，j)＝SV(i，j)×H(i，j) (1-7)
wherein, SV (i, j) is the gray image value of Fourier transform with the coordinate value of the pixel point (i, j), and is calculated by the formula (1-4); h (i, j) is the transfer function of the filter from the filter to the pixel point (i, j), and the formula is (1-6); g (i, j) is the gray scale image value of the filtered Fourier of the pixel (i, j).
And finally, performing inverse Fourier transform on the filtered Fourier gray image value G (i, j) to obtain an image obtained by high-pass filtering, wherein the inverse Fourier transform (1-8) is as follows:
k＝1,2，...，X
b＝1，2，...，Y
in the filtering process, a gray image value G (i, j) of a filtering Fourier needs to go through a traversal process, the value of i is from 1 to X, the value of j is from 1 to Y, the coordinate point (i, j) can traverse to the last pixel point of the filtering image, namely the pixel point with the coordinate value of (X, Y), the gray value G (i, j) of the filtered image is obtained, and then the image obtained through high-pass filtering, namely the processed image is obtained.
S1032, sharpening the processed image by using a Sobel operator, and calculating the conversion gray values of all pixel points in the gray image of the preset frame video image.
In this embodiment, the gray value conversion is used for the following step of determining whether the pixel point is an edge point, and the sobel operator formula (1-9) is:
sharpening the processed image with the sobel operator, the gradient value for point (i, j) being calculated by the following equation (1-10):
wherein, g (i, j) coordinate value is the filtered image gray value of the pixel point of (i, j), Si and Sj are the gradient values of the image in the horizontal and vertical directions, respectively, and the conversion gray value S (i, j) of the pixel point (i, j) is calculated by the following formula (1-11):
it should be noted that, the final result of steps S1031 to S1032 is that the coordinate point (i, j) is the converted gray scale value of the pixel, and this step S1031 to S1032 completes this step, and it is necessary to traverse the gray scale image value of each pixel in the preset frame image to obtain the converted gray scale value of each pixel, and this traversal method is explained in step S1022 in this embodiment, and is not described again here.
S1033, judging whether the conversion gray scale value of the pixel point is not smaller than a first preset threshold, if so, executing the following step S1034, and if not, executing the step S1032.
In this embodiment, the first preset threshold is an artificially set optimal conversion gray scale value, which is mainly determined according to the brightness value of the edge of the pixel point in the image edge processing. And when the conversion gray value of the pixel point is not less than the first preset threshold value, the pixel point can be set as an edge point, so that the edge characteristic point information of the whole preset frame image can be determined.
S1034, determining the image information of the pixel points as edge feature point information.
In this embodiment, the converted gray value of the pixel point is obtained by sharpening in step S1032, and the value has a sharp contrast due to sharpening, so that whether the pixel point is the edge feature point information of the preset frame image can be determined according to the gray value of the pixel point after conversion.
And S104, extracting an image mark from the video according to the edge characteristic point information of the preset frame gray level image.
In this embodiment, the edge feature point information of the preset frame gray image is already calculated in step S103, and the step of extracting the image mark in this step may specifically include the following two steps:
s1041, accumulating and multiplying the edge characteristic point information of the same pixel point in the preset frame gray image.
In this embodiment, the edge feature point information of the same pixel point in the preset frame gray image is multiplied, and when the preset frame gray image is an nth frame, where the nth frame is any one of the video images in the video playing process, the nth frame includes M frames of gray images before the nth frame, where M is not greater than N-1, where the M frames of gray images include: a 1 st frame gray image, a 2 nd frame gray image … … M frame gray image; for example, when M is equal to N-1, the M-frame gray image includes: the 1 st frame gray scale image and the 2 nd frame gray scale image … … are obtained. In this embodiment, a corresponding position of a gray scale image value SV of a pixel of an nth frame and a previous M frames of images is multiplied to obtain a multiplication value Q, and the formula (1-12) is:
wherein, SVSnAnd when the gray image is multiplied to the nth frame gray image, corresponding to the value SV of the gray image value of the same pixel point, wherein the value range of N is a positive integer from N-M to N. For example, when M ═ N-1, the previous multiplication value can be expressed by the following formula (1-13):
Q＝SVS(N-M)×SVS(N-M+1)×SVS(N-M+2)……SVSN (1-13)
specifically, assuming that N is 20, that is, the N represents a preset frame gray image, and is the 20 th frame gray image in the video playing; m takes 16, and the M frames of gray images comprise: the 1 st frame, the 2 nd frame, and the 3 rd frame … … th frame are the 15 th frames, the multiplication value is obtained by multiplying the gray scale image value SV of the image between the 15 th frame and the 20 th frame, that is, the gray scale image value SV of the same pixel point of the 17 th frame, the 18 th frame, the 19 th frame to the 20 th frame is multiplied, and the multiplication value is the product of the SV of the same pixel point of the 4 th frame image.
It should be noted that, in this embodiment, the cumulative multiplication value is a cumulative multiplication value of the same pixel point in the preset frame gray image, and this step S1041 is completed, and the gray image value SV of each pixel point in the preset frame image needs to be traversed to obtain the cumulative multiplication value of the gray image value of each pixel point, and this traversal method is explained in step S1022 in this embodiment, and is not described here again.
And S1042, extracting the graph formed by the pixel points with the cumulative multiplication value larger than a second preset threshold value as an image mark.
In this embodiment, the previous step is described, the ranges of the saturation S value and the brightness V are both 0 to 1, and since the gray image value SV is subjected to the accumulation step, the information of the edge information points is the same, and the gray image value of the pixel point has a small variation range of the accumulation product when the accumulation step is performed; when the edge point information is different, the gray image value SV of the pixel point approaches to a value approximate to 0; therefore, the second predetermined threshold in the embodiment of the present application may be a value close to 0, for example, 0.05. Further, the user may set the preset threshold according to the accuracy of extracting the image marker, for example, the original preset threshold is 0.05, when the user subjectively or objectively considers that the extraction effect of the marker extraction method is weak in the image marker extraction process, the preset threshold may be manually adjusted to a value smaller than the original preset threshold, for example, 0.005, so as to enhance the accuracy of the image marker extraction method.
As can be seen from the above, the image mark extraction method provided by this embodiment is applied to an electronic device, and obtains a video image with consecutive preset frames in a video playing process; determining a gray image corresponding to each frame of the video image to obtain a preset frame gray image; determining edge feature point information from the gray level image by using a preset algorithm; and extracting image marks from the video according to the edge characteristic point information of the preset frame gray level image, so that the accuracy of extracting the semitransparent image marks in the video is improved, and the method is simple and has a good extraction effect.
The present embodiment will be further described from the perspective of an image marker extraction device that may be embodied as a separate entity, in particular, in accordance with the methods described in the above embodiments.
The embodiment provides an image sign extraction device and system.
Referring to fig. 4, the system may include any one of the image tag extraction apparatuses provided in the embodiments of the present invention, and the image tag extraction apparatus may be specifically integrated in an electronic device such as a server or a terminal.
The method comprises the steps that the electronic equipment obtains video images with continuous preset frames in the video playing process; determining a gray image corresponding to each frame of the video image to obtain a preset frame gray image; determining edge feature point information from the gray level image by using a preset algorithm; and extracting an image mark from the video according to the edge characteristic point information of the preset frame gray level image.
The video images with continuous preset frames are images randomly selected from the video. The edge feature point information may include texture, shape, spatial relationship, and the like. Specifically, for the video images with continuous preset frames, the gray level image corresponding to each frame of video image is determined through the depth model, the gray level image of the preset frame is obtained, the edge feature point information of the gray level image of the preset frame is determined, and then the image mark is extracted from the edge feature point information. For example, when the user obtains the video images with continuous preset frames, the video images with continuous preset frames all include image marks, and the marks in the video images with continuous preset frames can be extracted through the determination, analysis and extraction of the image mark device.
Referring to fig. 5, fig. 5 specifically describes that the image marker extracting apparatus provided in the embodiment of the present application is applied to an electronic device, and the electronic device may include a mobile phone, a tablet computer, a personal PC, and other devices with an image display function. The image flag extraction device may include: an obtaining module 10, a determining module 20, a calculating module 30 and an extracting module 40, wherein:
(1) acquisition module 10
The acquiring module 10 is configured to acquire a video image with consecutive preset frames in a video playing process.
In this embodiment, the video images with consecutive preset frames are obtained as arbitrary selections in the video. A frame is a single image frame of the smallest unit in a video image, and corresponds to each shot in a movie. Each frame is a still image and displaying frames in rapid succession creates the illusion of a moving video image, the video images of which are preset in succession, see fig. 4.
For example, when acquiring a video image with a preset frame succession, the acquiring module 10 is specifically configured to:
(11) acquiring the RGB value of the pixel point in each frame of the video image.
In this embodiment, each frame of the video image is obtained as a color image, and a pixel point of the video image is composed of three components of R/G/B, where R is a red component of the color image, G is a green component of the color image, and B is a blue component of the color image; each point is represented by three bytes to be R/G/B respectively; R/G/B is typically divided into 256 levels from 0 to 255, with 0 being darkest (all black) and 255 being brightest (all white).
(12) And converting the RGB value from the RGB channel model into an HSV channel model to obtain a corresponding HSV value.
In this embodiment, the RGB channel model is an RGB color model divided into three color channels, red (R), green (G), and blue (B), and the RGB channel model is used to assign an intensity value in the range of 0 to 255 to the RGB component of each pixel in the image; the HSV channel model is divided into three HSV modes with channel hue (H), saturation (S) and lightness (V) with the visual color characteristics, and the HSV channel model is more in line with the habit of human eyes.
Among them, the HSV channel model includes HSV values (Hue, Saturation, value (brightness) lightness, also called HSB).
Further, the RGB channel model is converted into a corresponding HSV channel model, which is specifically implemented by using the following formula (2-1):
V＝max
wherein H is the hue of the pixel point and ranges from 0 to 1; s is the saturation of the pixel point and ranges from 0 to 1; v is the lightness of the pixel point and ranges from 0 to 1; max is the maximum value of R, G and B, and the range is 0-255; min is the minimum value of the R, G pixel point and the B value, and the range is 0-255.
(13) Determining the gray level image of the corresponding frame video image according to the HSV value of the pixel point.
In this embodiment, the gray image determined according to the HSV value of the pixel point is an image that only contains brightness information and does not contain color information, that is, a black-and-white image in a general sense: the brightness of the gray scale image is changed from dark to light, and the change is continuous.
It should be noted that in this embodiment, H is the saturation of the pixel, and S is the saturation, and is related to the shade of the corresponding image of the preset frame, when S is equal to 0, the pixel has only gray scale, and V is brightness, and represents the brightness of the color, but is not directly related to the light intensity. On the basis of the theory, the H value of the pixel point in the preset frame video image is removed, and the purpose of determining the gray level image of the corresponding frame video image is achieved.
(2) Determination module 20
And the determining module 20 is configured to determine a grayscale image corresponding to each frame of the video image, so as to obtain a preset frame grayscale image.
In this embodiment, obtaining the preset frame gray image requires calculating a gray image value corresponding to the pixel point according to the HSV value corresponding to the pixel point, so as to obtain the preset frame gray image, and the determining module 20 is specifically configured to:
(21) calculating a corresponding target value according to the saturation value of the pixel point.
In this embodiment, the formula (2-2) of the target value of the pixel point is:
T＝1-S (2-2)
and T is a target value corresponding to the saturation value S of the pixel point.
(22) Multiplying the target value by the brightness value of the corresponding pixel point to obtain the gray image value of the pixel point.
In this embodiment, the formula (2-3) of the gray image value of the pixel point is:
SV＝T×V (2-3)
wherein, V is the lightness of the pixel point, SV is the gray image value of the pixel point, and the ranges of T, V, S and SV are both 0-1. Further, the S value and the V value of the pixel point are calculated in the above formula (2-1). The calculation method eliminates the H value of the pixel point in the preset frame video image, and further can determine the gray level image of the corresponding frame video image.
(23) And determining the gray level image of the corresponding frame video image according to the gray level image value to obtain a preset frame gray level image.
In this embodiment, it should be noted that the gray scale image value SV of the pixel is an SV value of the pixel, and the gray scale image determined by the determining module 20 corresponding to the frame video image is the whole preset frame video image, and the corresponding frame video image, and similarly, the preset frame gray scale image includes the gray scale image values of all pixels on the image. Therefore, in this embodiment, the position of each pixel point needs to be traversed, and then the grayscale image of the preset frame video image is determined.
For example, assume that the preset frame video image is an image composed of X rows and Y columns of pixels, where X and Y are natural numbers, the image can be approximately considered as a matrix in X rows and Y columns, the coordinate values of the individual pixels can be represented as (i, j), (i, j) to identify the coordinate value of the pixel in the ith row and jth column, and 0< i ≦ X, and 0< j ≦ Y. The step of traversing includes: scanning from the line 1 to the line X, traversing the scanning line from the line 1 to the line Y, assuming that the traversing process reaches the line i (0< i ≦ X), only traversing the line i, for example (i, 1), (i, 2), (i, 3) … … (i, Y), when the pixel point (i, Y) is traversed, turning to traverse the line i +1, traversing the line i +1, for example (i +1, 1), (i +1, 2), (i +1, 3) … … (i +1, Y), and repeating the operation until the last pixel point traversed to the line X, namely the pixel point with the coordinate value of (X, Y), completes the traversing operation of the whole preset frame video image, thereby determining the gray level image of the preset frame video image.
Similarly, the step of traversing may further include: scanning from the 1 st column to the Y th column, traversing the 1 st row to the Y th column of the scanning column, assuming that the traversing process reaches the jth column (0< j is less than or equal to Y), only performing row traversal on the jth column, such as (1, j), (2, j), (3, j) … … (X, j), when traversing to a pixel point (X, j), turning to traverse the jth +1 column, performing row traversal on the jth +1 column, such as (1, j +1), (2, j +1), (3, j +1) … … (X, j +1), and repeating the operation until traversing to the last pixel point of the jth column, namely the pixel point with the coordinate value of (X, Y), so as to complete the traversal operation of the whole preset frame video image, thereby determining the gray level image of the preset frame video image.
(3) Calculation module 30
And the calculating module 30 is used for determining the edge feature point information from the gray-scale image by using a preset algorithm.
In this embodiment, the preset algorithm includes a classical edge extraction method for image processing, and specifically includes a first differential operator and a second differential operator, where the first differential operator includes Roberts (Roberts) and Sobel (Sobel), and the Sobel is a template that convolves an image with a vertical direction template and a horizontal direction template to perform edge detection.
For example, the calculating module 30 may be specifically configured to:
(31) processing the gray level image of the preset frame video image by using a high-pass filter to obtain a processed image.
In this embodiment, a fourier transform is first calculated for the grayscale image of the preset frame video image, and the fourier transform formula (2-4) is:
k＝1，2，...，X b＝1,2，...，Y
wherein, X and Y are respectively the total pixel number of the preset frame video image in the horizontal direction and the longitudinal direction, and SV (k, b) is the gray image value of the preset frame video image at the position where the coordinate value is traversed to (k, b), wherein k is a positive integer from 1 to X, and b is a positive integer from 1 to Y; SV (i, j) is a gray image value of fourier transform whose coordinate value is a pixel point in the ith row and the jth column, and this coordinate point is a pixel point in (i, j) in the embodiment of the present application.
And then carrying out high-pass filtering on the gray level image value subjected to Fourier transform, wherein the filtering formula (1-5) is as follows:
the transfer function equation (2-6) of the filter satisfies:
wherein d0 is a distance from the preset cutoff frequency to the origin, d (i, j) is a distance from the point (i, j) to the origin, H (i, j) is a transfer function of the filter traversed to the pixel point (i, j), and in the embodiment of the present application, the transfer functions of the filters corresponding to each pixel point are the same.
And (3) passing the Fourier transformed gray level image value through a filter transfer function to obtain a Fourier filtered gray level image value, wherein the formula (2-7) of the gray level image value is as follows:
G(i，j)＝SV(i，j)×H(i，j) (2-7)
wherein, SV (i, j) is the gray image value of Fourier transform with the coordinate value of the pixel point (i, j), and is calculated by the formula (2-4); h (i, j) is the transfer function of the filter from the filter to the pixel point (i, j), and the formula is (2-6); g (i, j) is the gray scale image value of the filtered Fourier of the pixel (i, j).
And finally, carrying out inverse Fourier transform on the filtered Fourier gray level image value G (i, j) to obtain an image obtained by high-pass filtering, wherein the inverse Fourier transform (2-8) is as follows:
k＝1,2，...，X b＝1，2，...，Y
g (i, j) is the gray value of the filtered image of the pixel point with the coordinate value (i, j), in the filtering process, the gray image value G (i, j) of the filtering Fourier needs to go through the traversal process, the value of i is from 1 to X, the value of j is from 1 to Y, the coordinate point (i, j) can traverse to the last pixel point of the filtered image, namely the pixel point with the coordinate value (X, Y), the gray value G (i, j) of the filtered image is obtained, and then the image obtained by high-pass filtering, namely the processed image, is obtained.
(32) Sharpening the processed image by using a Sobel operator, and calculating the conversion gray values of all pixel points in the gray image of the preset frame video image.
In this embodiment, the gray value conversion is used in the following step of determining whether the pixel point is an edge point, where the sobel operator (2-9) is:
sharpening the processed image with the sobel operator, the gradient value for point (i, j) being calculated by the following equation (2-10):
wherein, g (i, j) coordinate value is the filtered image gray value of the pixel point of (i, j), Si and Sj are the gradient values of the image in the horizontal and vertical directions, respectively, and the conversion gray value S (i, j) of the pixel point (i, j) is calculated by the following formula (2-11):
it should be noted that, the calculation module 30 finally obtains the converted gray value with the coordinate point (i, j) as the pixel point, and the calculation module 30 needs to traverse the gray image value of each pixel point in the preset frame image to obtain the converted gray value of each pixel point, and this traversal method is described in the acquisition module 10 in this embodiment and is not described herein again.
(33) Judging whether the conversion gray value of the pixel point is not less than a first preset threshold value.
In this embodiment, the first preset threshold is an artificially set optimal conversion gray scale value, which is mainly determined according to the brightness value of the edge of the pixel point in the image edge processing. And when the conversion gray value of the pixel point is not less than the first preset threshold value, the pixel point can be set as an edge point, so that the edge characteristic point information of the whole preset frame image can be determined.
(34) Determining the image information of the pixel point as edge feature point information when the conversion gray value of the pixel point is not less than the first preset threshold value.
In this embodiment, the converted gray value of the pixel point is a value obtained by sharpening through the calculating module 30, and the value has a sharp contrast due to sharpening, so that whether the pixel point is the edge feature point information of the preset frame image can be determined according to the gray value of the pixel point after conversion.
(4) Extraction module 40
And the extraction module 40 is configured to extract an image marker from the video according to the edge feature point information of the preset frame grayscale image.
In this embodiment, the edge feature point information of the preset frame gray image is already calculated by the obtaining module 10, and the extracting module 40 is specifically configured to:
(41) and multiplying the edge characteristic point information of the same pixel point in the preset frame gray level image.
In this embodiment, the edge feature point information of the same pixel point in the preset frame gray image is multiplied, and when the preset frame gray image is an nth frame, where the nth frame is any one of the video images in the video playing process, the nth frame includes M frames of gray images before the nth frame, where M is not greater than N-1, where the M frames of gray images include: a 1 st frame gray image, a 2 nd frame gray image … … M frame gray image; for example, when M is equal to N-1, the M-frame gray image includes: the 1 st frame gray scale image and the 2 nd frame gray scale image … … are obtained. In this embodiment, a corresponding position of a gray scale image value SV of a pixel of an nth frame and a previous M frames of images is multiplied to obtain a multiplication value Q, and a formula (2-12) is:
wherein, SVSnAnd when the gray image is multiplied to the nth frame gray image, corresponding to the value SV of the gray image value of the same pixel point, wherein the value range of N is a positive integer from N-M to N. For example, when M ═ N-1, the previous multiplication value can be expressed by the following formula (1-13):
Q＝SVS(N-M)×SVS(N-M+1)×SVS(N-M+2)……SVSN (2-13)
specifically, assuming that N is 20, that is, the N represents a preset frame gray image, and is the 20 th frame gray image in the video playing; m takes 16, and the M frames of gray images comprise: the 1 st frame, the 2 nd frame, and the 3 rd frame … … th frame are the 15 th frames, the multiplication value is obtained by multiplying the gray scale image value SV of the image between the 15 th frame and the 20 th frame, that is, the gray scale image value SV of the same pixel point of the 17 th frame, the 18 th frame, the 19 th frame to the 20 th frame is multiplied, and the multiplication value is the product of the SV of the same pixel point of the 4 th frame image.
It should be noted that, in this embodiment, the cumulative value is a cumulative value of the same pixel point in the preset frame gray image, the extracting module 40 needs to traverse the gray image value SV of each pixel point in the preset frame image to obtain the cumulative value of the gray image value of each pixel point, and this traversal method is described in the obtaining module 10 in this embodiment, and is not described herein again.
(42) And taking the graph formed by the pixel points with the cumulative multiplication value larger than a second preset threshold value as an image mark for extraction.
In this embodiment, the previous step is described, the ranges of the saturation S value and the brightness V are both 0 to 1, and since the gray image value SV is subjected to the accumulation step, the information of the edge information points is the same, and the gray image value of the pixel point has a small variation range of the accumulation product when the accumulation step is performed; when the edge point information is different, the gray image value SV of the pixel point approaches to a value approximate to 0; therefore, the second predetermined threshold in the embodiment of the present application may be a value close to 0, for example, 0.05. Further, the user may set the preset threshold according to the accuracy of extracting the image marker, for example, the original preset threshold is 0.05, when the user subjectively or objectively considers that the extraction effect of the marker extraction method is weak in the image marker extraction process, the preset threshold may be manually adjusted to a value smaller than the original preset threshold, for example, 0.005, so as to enhance the accuracy of the image marker extraction method.
In a specific implementation, the above units may be implemented as independent entities, or may be combined arbitrarily to be implemented as the same or several entities, and the specific implementation of the above units may refer to the foregoing method embodiments, which are not described herein again.
It will be understood by those skilled in the art that all or part of the steps of the methods of the above embodiments may be performed by instructions or by associated hardware controlled by the instructions, which may be stored in a computer readable storage medium and loaded and executed by a processor. To this end, the present application provides a storage medium, in which a plurality of instructions are stored, where the instructions can be loaded by a processor to execute the steps in any one of the image flag extraction methods provided in the present application.
Wherein the storage medium may include: read Only Memory (ROM), Random Access Memory (RAM), magnetic or optical disk, and the like.
Since the instructions stored in the storage medium can execute the steps in any image flag extraction method provided in the embodiments of the present application, beneficial effects that can be achieved by any image flag extraction method provided in the embodiments of the present application can be achieved, which are detailed in the foregoing embodiments and will not be described herein again.
The above operations can be implemented in the foregoing embodiments, and are not described in detail herein.
In summary, although the present application has been described with reference to the preferred embodiments, the above-described preferred embodiments are not intended to limit the present application, and those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present application, so that the scope of the present application shall be determined by the appended claims.
Claims (9)
1. An image marker extraction method, characterized in that the method comprises:
in the video playing process, acquiring a video image with continuous preset frames;
determining a gray image corresponding to each frame of the video image to obtain a preset frame gray image;
determining edge feature point information from the gray level image by using a preset algorithm;
extracting an image mark from the video according to the edge characteristic point information of the preset frame gray level image;
the extracting an image mark from the video according to the edge feature point information of the preset frame gray image specifically includes:
accumulating and multiplying the edge characteristic point information of the same pixel point in the preset frame gray level image;
and extracting the graph formed by the pixel points with the cumulative multiplication value larger than a second preset threshold value as an image mark.
2. The method for extracting image markers according to claim 1, wherein the obtaining of the video images of consecutive preset frames specifically comprises:
acquiring the RGB value of a pixel point in each frame of the video image;
converting the RGB value from the RGB channel model into an HSV channel model to obtain a corresponding HSV value;
and determining the gray level image of the corresponding frame video image according to the HSV value of the pixel point.
3. The method for extracting image markers according to claim 2, wherein the HSV values include a hue value, a saturation value and a brightness value, and the determining a gray level image of a corresponding frame of video image according to the HSV values of the pixel points specifically includes:
calculating a corresponding target value according to the saturation value of the pixel point;
multiplying the target value by the brightness value of the corresponding pixel point to obtain a gray image value of the pixel point;
and determining the gray level image of the corresponding frame video image according to the gray level image value to obtain a preset frame gray level image.
4. The method for extracting image markers according to claim 1, wherein the determining edge feature point information from the grayscale image by using a preset algorithm specifically comprises:
processing the gray level image of the preset frame video image by using a high-pass filter to obtain a processed image;
sharpening the processed image by using a Sobel operator, and calculating conversion gray values of all pixel points in a gray image of the preset frame video image;
and judging whether the conversion gray value of the pixel point is not less than a first preset threshold value, if so, determining the image information of the pixel point as edge feature point information.
5. An image marker extraction device, characterized in that the device comprises:
the acquisition module is used for acquiring a video image with continuous preset frames in the video playing process;
the determining module is used for determining a gray image corresponding to each frame of video image to obtain a preset frame gray image;
the calculation module is used for determining edge feature point information from the gray level image by using a preset algorithm;
the extraction module is used for extracting an image mark from the video according to the edge characteristic point information of the preset frame gray level image;
the extracting an image mark from the video according to the edge feature point information of the preset frame gray image specifically includes:
accumulating and multiplying the edge characteristic point information of the same pixel point in the preset frame gray level image;
and extracting the graph formed by the pixel points with the cumulative multiplication value larger than a second preset threshold value as an image mark.
6. The image marker extraction device according to claim 5, wherein the acquisition module is specifically configured to:
acquiring the RGB value of a pixel point in each frame of the video image;
converting the RGB value from the RGB channel model into an HSV channel model to obtain a corresponding HSV value;
and determining the gray level image of the corresponding frame video image according to the HSV value of the pixel point.
7. The image marker extraction device of claim 6, wherein the HSV values include hue values, saturation values, and lightness values, and the determination module is specifically configured to:
calculating a corresponding target value according to the saturation value of the pixel point;
multiplying the target value by the brightness value of the corresponding pixel point to obtain a gray image value of the pixel point;
and determining the gray level image of the corresponding frame video image according to the gray level image value to obtain a preset frame gray level image.
8. The image marker extraction device according to claim 5, wherein the extraction module is specifically configured to:
accumulating and multiplying the edge characteristic point information of the same pixel point in the preset frame gray level image;
and extracting the graph formed by the pixel points with the cumulative multiplication value larger than the preset threshold value as an image mark.
9. A computer-readable storage medium having stored thereon a plurality of instructions adapted to be loaded by a processor to perform the image signature extraction method of any of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910316438.3A CN110111347B (en) | 2019-04-19 | 2019-04-19 | Image sign extraction method, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910316438.3A CN110111347B (en) | 2019-04-19 | 2019-04-19 | Image sign extraction method, device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110111347A CN110111347A (en) | 2019-08-09 |
| CN110111347B true CN110111347B (en) | 2021-04-27 |
Family
ID=67485901
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910316438.3A Active CN110111347B (en) | 2019-04-19 | 2019-04-19 | Image sign extraction method, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110111347B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021087773A1 (en) * | 2019-11-05 | 2021-05-14 | 深圳市欢太科技有限公司 | Recognition method and apparatus, electronic device, and storage medium |
| CN111160215B (en) * | 2019-12-25 | 2024-01-12 | Tcl华星光电技术有限公司 | Brightness regulator and method for image mark |
| CN113688849B (en) * | 2021-08-30 | 2023-10-24 | 中国空空导弹研究院 | Gray image sequence feature extraction method for convolutional neural network |
| CN113923513B (en) * | 2021-09-08 | 2024-05-28 | 浙江大华技术股份有限公司 | Video processing method and device |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101408942A (en) * | 2008-04-17 | 2009-04-15 | 浙江师范大学 | Method for locating license plate under a complicated background |
| KR20120058851A (en) * | 2010-11-30 | 2012-06-08 | 엘지디스플레이 주식회사 | Image processing unit and display device using the same, and image processing method |
| CN108961299A (en) * | 2017-05-18 | 2018-12-07 | 北京金山云网络技术有限公司 | A kind of foreground image preparation method and device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR100975221B1 (en) * | 2008-11-05 | 2010-08-10 | 매그나칩 반도체 유한회사 | Apparatus and method for improving sharpness |
-
2019
- 2019-04-19 CN CN201910316438.3A patent/CN110111347B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101408942A (en) * | 2008-04-17 | 2009-04-15 | 浙江师范大学 | Method for locating license plate under a complicated background |
| KR20120058851A (en) * | 2010-11-30 | 2012-06-08 | 엘지디스플레이 주식회사 | Image processing unit and display device using the same, and image processing method |
| CN108961299A (en) * | 2017-05-18 | 2018-12-07 | 北京金山云网络技术有限公司 | A kind of foreground image preparation method and device |
Non-Patent Citations (2)
| Title |
|---|
| 《Multi-feature histogram intersection for Efficient Content Based Image Retrieval》;Manoj D. Chaudhary等;《2014 International Conference on Circuit, Power and Computing Technologies》;20141231;第1366-1371页 * |
| 《融合边缘检测与HSV颜色特征的车牌定位技术》;王枚等;《计算机应用研究》;20070831;第24卷(第8期);第209-211页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110111347A (en) | 2019-08-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110111347B (en) | Image sign extraction method, device and storage medium | |
| EP3916627A1 (en) | Living body detection method based on facial recognition, and electronic device and storage medium | |
| Zhang et al. | A no-reference evaluation metric for low-light image enhancement | |
| CN109978854B (en) | Screen content image quality evaluation method based on edge and structural features | |
| US20140079319A1 (en) | Methods for enhancing images and apparatuses using the same | |
| US10062195B2 (en) | Method and device for processing a picture | |
| WO2023160312A1 (en) | Person re-identification method and apparatus based on self-supervised learning, and device and storage medium | |
| CN103841410B (en) | Based on half reference video QoE objective evaluation method of image feature information | |
| JP2021531571A (en) | Certificate image extraction method and terminal equipment | |
| CN109190617B (en) | Image rectangle detection method and device and storage medium | |
| CN107767358B (en) | Method and device for determining ambiguity of object in image | |
| CN111612741A (en) | Accurate non-reference image quality evaluation method based on distortion recognition | |
| CN113592776A (en) | Image processing method and device, electronic device and storage medium | |
| CN109829924A (en) | A kind of image quality evaluating method based on body feature analysis | |
| CN111444555A (en) | Temperature measurement information display method and device and terminal equipment | |
| CN106529549B (en) | Vision significance detection method based on self-adaptive features and discrete cosine transform | |
| CN103366390A (en) | Terminal, image processing method and device thereof | |
| CN110766153A (en) | Neural network model training method and device and terminal equipment | |
| CN111784658B (en) | Quality analysis method and system for face image | |
| CN105979283A (en) | Video transcoding method and device | |
| CN111027564A (en) | Low-illumination imaging license plate recognition method and device based on deep learning integration | |
| CN111047618A (en) | Multi-scale-based non-reference screen content image quality evaluation method | |
| CN111126250A (en) | Pedestrian re-identification method and device based on PTGAN | |
| CN112839167B (en) | Image processing method, device, electronic equipment and computer readable medium | |
| CN108682021A (en) | Rapid hand tracking, device, terminal and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 9-2 Tangming Avenue, Guangming New District, Shenzhen City, Guangdong Province Applicant after: TCL China Star Optoelectronics Technology Co.,Ltd. Address before: 9-2 Tangming Avenue, Guangming New District, Shenzhen City, Guangdong Province Applicant before: Shenzhen China Star Optoelectronics Technology Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |