CN106780334B - Image classification method and system - Google Patents
Image classification method and system Download PDFInfo
- Publication number
- CN106780334B CN106780334B CN201611162822.5A CN201611162822A CN106780334B CN 106780334 B CN106780334 B CN 106780334B CN 201611162822 A CN201611162822 A CN 201611162822A CN 106780334 B CN106780334 B CN 106780334B
- Authority
- CN
- China
- Prior art keywords
- image
- spliced
- color
- edge intensity
- characteristic value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4038—Scaling the whole image or part thereof for image mosaicing, i.e. plane images composed of plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/56—Extraction of image or video features relating to colour
Abstract
The invention discloses an image classification method and system, wherein the method comprises the following steps: acquiring a spliced image obtained by splicing the images to be processed again, and marking the splicing position of the spliced image; extracting characteristic values of the spliced image, wherein the characteristic values comprise: color feature values and/or texture feature values; and classifying the spliced images according to the characteristic values of the splicing positions. According to the method, the spliced image obtained after the image to be processed is spliced again is obtained, the characteristic value of the spliced image obtained after splicing again is extracted, the image to be processed is classified according to the characteristic value of the splicing position, the image to be processed in the video frame is automatically classified, the video is labeled, manual labeling and classification of the video uploaded by a user are not needed, whether the manually labeled video category label is correct or not can be checked, the playing experience of the user on the panoramic video due to mistaken image classification is avoided, and the playing experience of the user on the panoramic video is enhanced.
Description
Technical Field
The invention belongs to the technical field of image classification, and particularly relates to an image classification method and system.
Background
In recent years, VR (Virtual Reality) has become the technology that receives the most attention. VR refers to a computer simulation system that can create and experience a virtual world, which simulates a virtual environment using various means, such that a user can be immersed in the virtual environment and generate an interactive system simulation of three-dimensional dynamic views and physical behaviors. As one of the key technologies, the playing of panoramic video has been the direction in which various large video content providers are most focused.
At present, a video Content provider has opened a panoramic video uploading service of a PGC (professional-generated Content) User, and supports immersive playing of a panoramic video in a playing process, and some manufacturers have or will open a panoramic video uploading and playing service for a UGC (User-generated Content) User. Due to the specialty of the PGC user, when uploading the panoramic video, the website can be assisted to label the panoramic video, and the video website can select to play the video in an immersive manner or in a common manner according to the label of the PGC user; however, for UGC users, the UGC users do not have professional knowledge, part of the users cannot actively mark video types when uploading videos, or errors exist in the marking process, and even some of the users intentionally mark some common videos as panoramic videos or the panoramic videos as the common videos, so that destructive attack is performed on playing services.
Due to the fact that the uploading amount of UGC users is huge, full-time manual monitoring is difficult to achieve, and playing experience of the users on panoramic videos can be influenced if the video type labels are incorrect. Therefore, the videos can be classified by classifying images corresponding to video frames in panoramic videos and common videos, then labeling the video types according to image classification results, classifying the images uploaded by each UGC user manually at present, and then labeling the video types according to the classification results, but the work is tedious work, the work process is complicated, and mistakes are easily made, so that the playing experience of the panoramic videos by users is influenced.
Disclosure of Invention
In view of the above, the present invention provides an image classification method and system, so as to solve the problems in the prior art that since images in video frames uploaded by each UGC user are classified manually, and then video types are labeled according to classification results, the working process is complicated, and misclassification is easily caused, which affects the playing experience of a user on a panoramic video.
In order to achieve the purpose, the invention provides the following technical scheme:
an image classification method, comprising:
acquiring a spliced image obtained by splicing the images to be processed again, and marking the splicing seam position of the spliced image;
extracting feature values of the spliced image, wherein the feature values comprise: color feature values and/or texture feature values;
and classifying the spliced images according to the characteristic values of the splicing positions.
The method for obtaining the spliced image obtained by splicing the images to be processed again and marking the splicing seam position of the spliced image comprises the following steps:
acquiring an image to be processed, and extracting image areas which are respectively positioned at the rightmost side and the leftmost side of the image to be processed and have the same area;
and splicing the obtained image areas to form a spliced image, and marking the splicing position of the spliced image, wherein the rightmost image area of the image to be processed is positioned on the left side of the spliced image, and the leftmost image area of the image to be processed is positioned on the right side of the spliced image.
Wherein, the extracting the effective characteristic value of the spliced image, the effective characteristic being a color characteristic value and/or a texture characteristic value, comprises:
carrying out space conversion on the color space of the color mode RGB of the spliced image through a conversion formula to obtain a converted space value;
calculating the cumulative sum of the color average differences of all the left side pixels and all the right side pixels at the splicing position according to the color space value of the RGB color space or the converted space value, and taking the calculation result as a color characteristic value;
and/or
Extracting an edge intensity map of an image area of the spliced image by adopting an edge intensity map extraction method;
extracting a first edge intensity image of an image area at the rightmost side of the image to be processed by adopting the edge intensity image extraction method;
extracting a second edge intensity image of an image area at the leftmost side of the image to be processed by adopting the edge intensity image extraction method;
stitching the first edge intensity map and the second edge intensity map into a third edge intensity map of the same size as the edge intensity map of the stitched image;
subtracting the edge intensity image of the spliced image and the third edge intensity image pixel by pixel to obtain a difference edge intensity image;
and adding the edge intensities of all pixels at the joint positions of the difference edge intensity graph to obtain the edge intensity accumulation sum of all pixels as a texture characteristic value.
The edge intensity map extraction method specifically comprises the following steps:
carrying out space conversion on the color space of the color mode RGB of the image area of the edge intensity image to be extracted through a conversion formula to obtain a brightness color separation image;
performing convolution on the brightness and color separation image by utilizing a horizontal direction edge gradient operator to obtain a horizontal edge image;
performing convolution on the brightness and color separation image by using a vertical edge gradient operator to obtain a vertical edge image;
and calculating an edge strength map according to the horizontal edge map and the vertical edge map.
Wherein, the classifying the spliced image according to the effective characteristic value comprises:
presetting a threshold value of a preset color characteristic value and a threshold value of a preset texture characteristic value;
when the extracted characteristic value is a color characteristic value, judging whether the color characteristic value is smaller than a threshold value of the preset color characteristic value, if so, determining that the spliced image is a panoramic image, and if not, determining that the spliced image is a common image;
when the extracted characteristic value is a texture characteristic value, judging whether the texture characteristic value is smaller than a threshold value of the preset texture characteristic value, if so, determining that the spliced image is a panoramic image, and if not, determining that the spliced image is a common image;
and when the extracted characteristic values are the color characteristic value and the texture characteristic value, judging whether the color characteristic value is smaller than a threshold value of a preset color characteristic value or not, and whether the texture characteristic value is smaller than the threshold value of the preset texture characteristic value or not, if so, the spliced image is a panoramic image, and if not, the spliced image is a common image.
An image classification system comprising:
the acquisition module is used for acquiring a spliced image obtained by splicing the images to be processed again and marking the splicing position of the spliced image;
an extraction module, configured to extract feature values of the stitched image, where the feature values include: color feature values and/or texture feature values;
and the classification module is used for classifying the spliced images according to the characteristic values of the splicing positions.
Wherein the acquisition module comprises:
the device comprises a first acquisition unit, a second acquisition unit and a third acquisition unit, wherein the first acquisition unit is used for acquiring an image to be processed and extracting image areas which are respectively positioned at the rightmost side and the leftmost side of the image to be processed and have the same area;
and the splicing unit is used for splicing the obtained image areas to form a spliced image and marking the splicing position of the spliced image, the rightmost image area of the image to be processed is positioned on the left side of the spliced image, and the leftmost image area of the image to be processed is positioned on the right side of the spliced image.
Wherein the extraction module comprises:
the first space conversion unit is used for carrying out space conversion on the color space of the color mode RGB of the spliced image through a conversion formula to obtain a converted space value;
a color characteristic value extracting unit, configured to calculate, according to the color space value of the RGB color space or the converted space value, a cumulative sum of color average differences of all left-side pixels and all right-side pixels at the seam position, and use a calculation result as a color characteristic value;
and/or
The first extraction unit is used for extracting an edge intensity map of an image area of the spliced image by adopting an edge intensity map extraction method;
the second extraction unit is used for extracting a first edge intensity map of an image area on the rightmost side of the image to be processed by adopting the edge intensity map extraction method;
a third extraction unit, configured to extract a second edge intensity map of a leftmost image region of the image of the to-be-processed image by using the edge intensity map extraction method;
a first stitching unit for stitching the first edge intensity map and the second edge intensity map into a third edge intensity map of the same size as the edge intensity map of the stitched image;
the pixel subtraction unit is used for subtracting the edge intensity image of the spliced image from the third edge intensity image pixel by pixel to obtain a difference edge intensity image;
and the texture characteristic value extracting unit is used for adding the edge intensities of all pixels at the joint positions of the difference edge intensity graph to obtain the edge intensity accumulated sum of all pixels as a texture characteristic value.
The edge intensity map extraction method specifically comprises the following steps:
carrying out space conversion on the color space of the color mode RGB of the image area of the edge intensity image to be extracted through a conversion formula to obtain a brightness color separation image;
performing convolution on the brightness and color separation image by utilizing a horizontal direction edge gradient operator to obtain a horizontal edge image;
performing convolution on the brightness and color separation image by using a vertical edge gradient operator to obtain a vertical edge image;
and calculating an edge strength map according to the horizontal edge map and the vertical edge map.
Wherein the classification module comprises:
the preset unit is used for presetting a threshold value of a preset color characteristic value and a threshold value of a preset texture characteristic value;
a first judging unit, configured to, when the extracted feature value is a color feature value, judge whether the color feature value is smaller than a threshold of the preset color feature value, if yes, the stitched image is a panoramic image, and if not, the stitched image is a normal image;
a second judging unit, configured to, when the extracted feature value is a texture feature value, judge whether the texture feature value is smaller than a threshold of the preset texture feature value, if yes, the stitched image is a panoramic image, and if not, the stitched image is a normal image;
and the third judging unit is used for judging whether the color characteristic value is smaller than the threshold value of the preset color characteristic value or not and whether the texture characteristic value is smaller than the threshold value of the preset texture characteristic value or not when the extracted characteristic values are the color characteristic value and the texture characteristic value, if so, the spliced image is a panoramic image, and if not, the spliced image is a common image.
Compared with the prior art, the invention discloses an image classification method and system, and the method comprises the following steps: acquiring a spliced image obtained by splicing the images to be processed again, and marking the splicing position of the spliced image; extracting characteristic values of the spliced image, wherein the characteristic values comprise: color feature values and/or texture feature values; and classifying the spliced images according to the characteristic values of the splicing positions. According to the method, the spliced image obtained after the image to be processed is spliced again is obtained, the characteristic value of the spliced image obtained after splicing again is extracted, the image to be processed is classified according to the characteristic value of the splicing position, the image to be processed in the video frame is automatically classified, the video is labeled, manual labeling and classification of the video uploaded by a user are not needed, whether the manually labeled video category label is correct or not can be checked, the playing experience of the user on the panoramic video due to mistaken image classification is avoided, and the playing experience of the user on the panoramic video is enhanced.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
FIG. 1 is a schematic diagram of a panoramic image in an embodiment of the present invention;
FIG. 2 is a diagram illustrating a generic image in an embodiment of the present invention;
fig. 3 is a schematic flow chart of an image classification method according to an embodiment of the present invention;
fig. 4 is a schematic flowchart of step S101 in fig. 3 according to an embodiment of the present invention;
fig. 5 is a schematic diagram of a re-stitched image in which an image to be processed is a panoramic image according to an embodiment of the present invention;
fig. 6 is a schematic diagram of a re-stitched image in which an image to be processed is a common image according to an embodiment of the present invention;
fig. 7 is a flowchart illustrating a specific process of step S102 when the feature value is the color feature value in fig. 3 according to an embodiment of the present invention;
fig. 8 is a flowchart illustrating a specific process of step S102 when the feature value is the texture feature value in fig. 3 according to an embodiment of the present invention;
FIG. 9 is a schematic diagram illustrating extraction of texture feature values for a panoramic image according to an embodiment of the present invention;
FIG. 10 is a schematic diagram illustrating the extraction of texture feature values for a normal image according to the present embodiment of the invention;
fig. 11 is a schematic structural diagram of an image classification system according to an embodiment of the present invention;
fig. 12 is a schematic structural diagram of the obtaining module 1101 in fig. 11 according to an embodiment of the present invention;
fig. 13 is a schematic structural diagram of the extraction module 1102 in fig. 11 when the feature value is a color feature value according to an embodiment of the present invention;
fig. 14 is a schematic structural diagram of the extracting module 1102 in fig. 11 when the feature value is a texture feature value according to an embodiment of the present invention;
fig. 15 is a schematic structural diagram of the classification module 1103 in fig. 11 according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Please refer to fig. 1 and 2, which are schematic diagrams of a panoramic image and a general image. The invention provides a method for classifying panoramic images and common images, which can form complete images, namely panoramic images, by splicing images on two sides of the images corresponding to video frames, namely: the color difference and/or texture difference at the annular splicing part of the panoramic image is used as the characteristic, the image to be processed is automatically classified into the panoramic image or the common image, the algorithm is simple, and the accuracy is high.
Referring to fig. 3, fig. 3 is a flowchart illustrating an image classification method according to an embodiment of the present invention. As shown in fig. 3, the present invention discloses an image classification method, which comprises the following specific steps:
s301, obtaining a spliced image obtained by splicing the images to be processed again, and marking the splicing seam position of the spliced image.
Referring to fig. 4, the steps specifically include the following steps:
s401, acquiring an image to be processed, and extracting image regions which are respectively positioned at the rightmost side and the leftmost side of the image to be processed and have the same area;
s402, splicing the obtained image areas to form a spliced image, marking the splicing position of the spliced image, wherein the rightmost image area of the image to be processed is on the left side of the spliced image, and the leftmost image area of the image to be processed is on the right side of the spliced image.
It should be noted that, in this embodiment, as shown in fig. 5, the image to be processed is firstly re-stitched, and the regions of interest ROI1 and ROI2 of the image are specified, the selection method of the ROI1 is a region with a certain area on the leftmost side of the image, and the selection method of the ROI2 is a region with an area equal to that of the ROI1 on the rightmost side of the image, that is: the important criteria to distinguish between panoramic and non-panoramic are that the panoramic head-to-tail image stitching is continuous, while the non-panoramic is discontinuous.
The obtained ROI1 is compared with the ROI2, splicing to form a re-spliced image, wherein the splicing method is that the ROI2 area is on the left side of the re-spliced image, the ROI1 area is on the right side of the image, and the position of the splicing seam where the re-spliced image is recorded is WLThe method realizes the head-to-tail splicing of the panoramic images. For a normal picture, as shown in fig. 6, the spliced image formed by such end-to-end splicing is discontinuous.
S302, extracting characteristic values of the spliced image, wherein the characteristic values comprise: a color feature value and/or a texture feature value.
The step may specifically include the steps of:
referring to fig. 7, when the feature value is a color feature value, the specific method for extracting the feature value of the stitched image includes the following steps:
s701, carrying out space conversion on the color space of the color mode RGB of the spliced image through a conversion formula to obtain a converted space value.
Specifically, it should be noted that, for the extraction of the color features, specifically: the obtained re-spliced image is converted into a YUV space from an RGB color space, and can also be converted into YCbCr, HSV, Lab or space non-conversion.
Here, the present embodiment takes YUV three channels of an image as an example, and the conversion formula is as follows:
Y＝0.299R+0.587G+0.114B
U＝-0.1687R-0.3313G+0.5B+128
V＝0.5R-0.4187G-0.0813B+128
if other color spaces are selected, the YUV three channels can be replaced, the RGB can be directly used without conversion, and the conversion formula can be converted according to the corresponding conversion formula.
S702, calculating the accumulated sum of the color average differences of all the left side pixels and all the right side pixels at the splicing position according to the color space value of the RGB color space or the converted space value, and taking the calculation result as a color characteristic value.
Specifically, it should be noted that the seam position W is calculatedLAt all left pixels P (W)L-1, Y) { Y, U, V } with all right-hand pixels P (W)LSum of color mean differences diff of { Y, U, V }, Y)color. The color mean difference diff for each pixel is defined as:
diff(y)＝DY+DU+DV
where D represents the average difference of the pixels in each channel (Y, U, V), and is defined as the difference between the average sums of the upper and lower adjacent pixels (taking Y channel as an example, U, V are calculated in the same way):
and WLThe cumulative sum of the color mean differences diff for all pixels is:
wherein: h is the image height.
Referring to fig. 8, when the feature value is a texture feature value, the specific method for extracting the feature value of the stitched image includes the following steps:
s801, extracting an edge intensity image of an image area of the spliced image by adopting an edge intensity image extraction method.
It should be noted that, in this embodiment, for the re-spliced image, the color space of the color mode RGB of the re-spliced image is first subjected to space conversion (such as YUV, HSV, HSL, and LAB) through a conversion formula to obtain a luminance-color separation image, and for the luminance-color separation space, the conversion formula of luminance Y is as follows, taking YUV as an example:
Y＝0.299R+0.587G+0.114B
for texture feature values, only the luminance is converted, and UV is not required in this portion and may not be converted.
Performing convolution on the image with brightness and color separation by using the edge gradient operator in the horizontal direction to obtain a horizontal edge image Eh。
Convolving the edge gradient operator in the vertical direction with the brightness color separation image to obtain a vertical edge image Ev。
In this embodiment, the edge gradient operators in the horizontal direction and the vertical direction take Sobel operators as an example, and other operators are also applicable, and the specific Sobel operators:
computing an edge intensity map E from the horizontal and vertical edge mapsall。
Calculate edge intensity map EallI.e. for any point E on the edge mapall(x，y)，Eall(x，y)＝sqrt(Ev(x，y)2+Eh(x，y)2)。
S802, extracting a first edge intensity image of the rightmost image area of the image to be processed by adopting an edge intensity image extraction method.
The edge intensity map E is extracted for the rightmost image region ROI1 of the image to be processed using the method in step S801 described aboveright。
And S803, extracting a second edge intensity image of the leftmost image area of the image to be processed by adopting an edge intensity image extraction method.
The edge intensity map E is extracted for the leftmost image region ROI2 of the image to be processed using the method in step S801 described aboveleft。
And S804, splicing the first edge intensity graph and the second edge intensity graph into a third edge intensity graph with the same size as the edge intensity graph of the spliced image.
Will EleftAnd ErightSplicing to sum EallAnother edge map of the same size, called ES。
And S805, subtracting the edge intensity image of the spliced image and the third edge intensity image pixel by pixel to obtain a difference edge intensity image.
For two edge maps EallAnd ESSubtracting the two pixels one by one to obtain the difference Eresult。
S806, adding the edge intensities of all the pixels at the joint positions of the difference edge intensity graph to obtain the edge intensity accumulation sum of all the pixels as a texture characteristic value.
For EresultThe abutted seam WLAdding the edge intensities of all the pixels to obtain sumedge。
Specifically, please refer to fig. 9 and 10, fig. 9 is a schematic diagram illustrating extraction of texture feature values of a panoramic image according to an embodiment of the present invention; fig. 10 is a schematic diagram illustrating extraction of texture feature values of a normal image according to the present embodiment of the invention.
And S303, classifying the spliced images according to the characteristic values of the splicing positions.
Specifically, the method comprises the following steps:
presetting a threshold value of a preset color characteristic value and a threshold value of a preset texture characteristic value;
when the extracted characteristic value is a color characteristic value, judging whether the color characteristic value is smaller than a threshold value of a preset color characteristic value, if so, determining that the spliced image is a panoramic image, and if not, determining that the spliced image is a common image;
when the extracted characteristic value is a texture characteristic value, judging whether the texture characteristic value is smaller than a threshold value of a preset texture characteristic value, if so, determining that the spliced image is a panoramic image, and if not, determining that the spliced image is a common image;
and when the extracted characteristic values are the color characteristic value and the texture characteristic value, judging whether the color characteristic value is smaller than a threshold value of a preset color characteristic value or not, and whether the texture characteristic value is smaller than a threshold value of a preset texture characteristic value or not, if so, determining that the spliced image is a panoramic image, and if not, determining that the spliced image is a common image.
The image classification method can adopt the simplest method of segmenting by using a threshold value, or can train a classifier by using a machine learning method, collect common images and panoramic images, obtain features according to the previous method, obtain corresponding classification models on the features by using corresponding machine learning algorithms, and classify the images to be processed by using the obtained classification models.
The invention discloses an image classification method, which comprises the following steps: acquiring a spliced image obtained by splicing the images to be processed again, and marking the splicing position of the spliced image; extracting characteristic values of the spliced image, wherein the characteristic values comprise: color feature values and/or texture feature values; and classifying the spliced images according to the characteristic values of the splicing positions. According to the method, the spliced image obtained after the image to be processed is spliced again is obtained, the characteristic value of the spliced image obtained after splicing again is extracted, the image to be processed is classified according to the characteristic value of the splicing position, the image to be processed in the video frame is automatically classified, the video is labeled, manual labeling and classification of the video uploaded by a user are not needed, whether the manually labeled video category label is correct or not can be checked, the playing experience of the user on the panoramic video due to mistaken image classification is avoided, and the playing experience of the user on the panoramic video is enhanced.
The present invention also discloses a system based on the above-mentioned method, please refer to fig. 11, and fig. 11 is a schematic structural diagram of an image classification system according to an embodiment of the present invention. As shown in fig. 11, the present invention discloses an image classification system, which may specifically include the following structure:
an obtaining module 1101, configured to obtain a stitched image obtained by re-stitching an image to be processed, and mark a seam position of the stitched image;
an extracting module 1103, configured to extract feature values of the stitched image, where the feature values include: color feature values and/or texture feature values;
and the classifying module 1103 is configured to classify the stitched images according to the feature values of the stitch positions.
Preferably, referring to fig. 12, the obtaining module 1101 in fig. 11 may include:
a first obtaining unit 1201, configured to obtain an image to be processed, and extract image regions with equal areas that are located at the rightmost side and the leftmost side of the image to be processed, respectively;
and the splicing unit 1202 is configured to splice the obtained image areas to form a spliced image, mark a splicing position of the spliced image, and place an image area on the rightmost side of the image to be processed on the left side of the spliced image and place an image area on the leftmost side of the image to be processed on the right side of the spliced image.
Preferably, referring to fig. 13 and 14, the extraction module 1102 in fig. 11 may include:
the first space conversion unit 1301 is configured to perform space conversion on a color space of a color mode RGB of the stitched image through a conversion formula to obtain a converted space value;
the color feature value extracting unit 1302 is configured to calculate, according to the color space value of the RGB color space or the converted space value, the cumulative sum of the color average differences of all the left-side pixels and all the right-side pixels at the seam position, and use the calculation result as the color feature value.
And/or
A first extraction unit 1401, configured to extract an edge intensity map of an image region of a stitched image by using an edge intensity map extraction method;
a second extraction unit 1402, configured to extract a first edge intensity map of an image area on the rightmost side of the image to be processed by using an edge intensity map extraction method;
a third extraction unit 1403, configured to extract a second edge intensity map of the leftmost image area of the image to be processed by using an edge intensity map extraction method;
a first stitching unit 1404 for stitching the first edge intensity map and the second edge intensity map into a third edge intensity map of the same size as the edge intensity map of the stitched image;
a pixel subtraction unit 1405, configured to subtract the edge intensity map of the stitched image and the third edge intensity map pixel by pixel to obtain a difference edge intensity map;
the texture feature value extracting unit 1406 is configured to add the edge intensities of all pixels at the seam position of the difference edge intensity map to obtain an edge intensity accumulated sum of all pixels as a texture feature value.
Referring to fig. 15, the classification module 1103 in fig. 11 may include:
a preset unit 1501, configured to preset a threshold of a preset color feature value and a threshold of a preset texture feature value;
a first judging unit 1502, configured to, when the extracted feature value is a color feature value, judge whether the color feature value is smaller than a threshold of a preset color feature value, if yes, the stitched image is a panoramic image, and if not, the stitched image is a normal image;
a second determining unit 1503, configured to determine whether the extracted feature value is a texture feature value, if so, the stitched image is a panoramic image, and if not, the stitched image is a normal image;
a third determining unit 1504, configured to determine whether the color feature value is smaller than a threshold of a preset color feature value and whether the texture feature value is smaller than the threshold of a preset texture feature value when the extracted feature values are the color feature value and the texture feature value, if yes, the stitched image is a panoramic image, and if not, the stitched image is a normal image.
The embodiment of the invention discloses an image classification system, which extracts a characteristic value of a spliced image obtained after splicing a to-be-processed image again by obtaining the spliced image obtained after splicing the to-be-processed image, classifies the to-be-processed image according to the characteristic value of a splicing position, realizes automatic classification of the to-be-processed image in a video frame, marks the video, does not need to manually mark and classify the uploaded video of a user, can detect whether a manually marked video category label is correct or not, avoids playing experience of the user on panoramic video due to mistaken image classification, and further enhances playing experience of the user on viewing the panoramic video.
In summary, the present invention discloses an image classification method and system, the method includes: acquiring a spliced image obtained by splicing the images to be processed again, and marking the splicing position of the spliced image; extracting characteristic values of the spliced image, wherein the characteristic values comprise: color feature values and/or texture feature values; and classifying the spliced images according to the characteristic values of the splicing positions. According to the method, the spliced image obtained after the image to be processed is spliced again is obtained, the characteristic value of the spliced image obtained after splicing again is extracted, the image to be processed is classified according to the characteristic value of the splicing position, the image to be processed in the video frame is automatically classified, the video is labeled, manual labeling and classification of the video uploaded by a user are not needed, whether the manually labeled video category label is correct or not can be checked, the playing experience of the user on the panoramic video due to mistaken image classification is avoided, and the playing experience of the user on the panoramic video is enhanced.
It should be noted that, in the present specification, the embodiments are all described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments may be referred to each other.
The method proposed by the present invention is described above by way of example with reference to the accompanying drawings, and the above description of the embodiments is only intended to help the understanding of the core ideas of the present invention. For those skilled in the art, variations can be made in the specific embodiments and applications without departing from the spirit of the invention. In view of the above, the present disclosure should not be construed as limiting the invention.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (8)
1. An image classification method, comprising:
acquiring a spliced image obtained by splicing the images to be processed again, and marking the splicing seam position of the spliced image;
extracting feature values of the spliced image, wherein the feature values comprise: color feature values and/or texture feature values;
classifying the spliced images into panoramic images and common images according to the characteristic values of the splicing positions;
the method for obtaining the spliced image obtained by splicing the images to be processed again and marking the splicing seam position of the spliced image comprises the following steps:
acquiring an image to be processed, and extracting image areas which are respectively positioned at the rightmost side and the leftmost side of the image to be processed and have the same area;
and splicing the obtained image areas to form a spliced image, and marking the splicing position of the spliced image, wherein the rightmost image area of the image to be processed is positioned on the left side of the spliced image, and the leftmost image area of the image to be processed is positioned on the right side of the spliced image.
2. The image classification method according to claim 1, wherein the extracting effective feature values of the stitched image, the effective features being color feature values and/or texture feature values, comprises:
carrying out space conversion on the color space of the color mode RGB of the spliced image through a conversion formula to obtain a converted space value;
calculating the cumulative sum of the color average differences of all the left side pixels and all the right side pixels at the splicing position according to the color space value of the RGB color space or the converted space value, and taking the calculation result as a color characteristic value;
and/or
Extracting an edge intensity map of an image area of the spliced image by adopting an edge intensity map extraction method;
extracting a first edge intensity image of an image area at the rightmost side of the image to be processed by adopting the edge intensity image extraction method;
extracting a second edge intensity image of an image area at the leftmost side of the image to be processed by adopting the edge intensity image extraction method;
stitching the first edge intensity map and the second edge intensity map into a third edge intensity map of the same size as the edge intensity map of the stitched image;
subtracting the edge intensity image of the spliced image and the third edge intensity image pixel by pixel to obtain a difference edge intensity image;
and adding the edge intensities of all pixels at the joint positions of the difference edge intensity graph to obtain the edge intensity accumulation sum of all pixels as a texture characteristic value.
3. The image classification method according to claim 2, wherein the edge intensity map extraction method specifically comprises:
carrying out space conversion on the color space of the color mode RGB of the image area of the edge intensity image to be extracted through a conversion formula to obtain a brightness color separation image;
performing convolution on the brightness and color separation image by utilizing a horizontal direction edge gradient operator to obtain a horizontal edge image;
performing convolution on the brightness and color separation image by using a vertical edge gradient operator to obtain a vertical edge image;
and calculating an edge strength map according to the horizontal edge map and the vertical edge map.
4. The image classification method according to claim 1, wherein the classifying the stitched image according to the feature value of the seam position includes:
presetting a threshold value of a preset color characteristic value and a threshold value of a preset texture characteristic value;
when the extracted characteristic value is a color characteristic value, judging whether the color characteristic value is smaller than a threshold value of the preset color characteristic value, if so, determining that the spliced image is a panoramic image, and if not, determining that the spliced image is a common image;
when the extracted characteristic value is a texture characteristic value, judging whether the texture characteristic value is smaller than a threshold value of the preset texture characteristic value, if so, determining that the spliced image is a panoramic image, and if not, determining that the spliced image is a common image;
and when the extracted characteristic values are the color characteristic value and the texture characteristic value, judging whether the color characteristic value is smaller than a threshold value of a preset color characteristic value or not, and whether the texture characteristic value is smaller than the threshold value of the preset texture characteristic value or not, if so, the spliced image is a panoramic image, and if not, the spliced image is a common image.
5. An image classification system, comprising:
the acquisition module is used for acquiring a spliced image obtained by splicing the images to be processed again and marking the splicing position of the spliced image;
an extraction module, configured to extract feature values of the stitched image, where the feature values include: color feature values and/or texture feature values;
the classification module is used for classifying the spliced images into panoramic images and common images according to the characteristic values of the splicing positions;
wherein the acquisition module comprises:
the device comprises a first acquisition unit, a second acquisition unit and a third acquisition unit, wherein the first acquisition unit is used for acquiring an image to be processed and extracting image areas which are respectively positioned at the rightmost side and the leftmost side of the image to be processed and have the same area;
and the splicing unit is used for splicing the obtained image areas to form a spliced image and marking the splicing position of the spliced image, the rightmost image area of the image to be processed is positioned on the left side of the spliced image, and the leftmost image area of the image to be processed is positioned on the right side of the spliced image.
6. The image classification system of claim 5, wherein the extraction module comprises:
the first space conversion unit is used for carrying out space conversion on the color space of the color mode RGB of the spliced image through a conversion formula to obtain a converted space value;
a color characteristic value extracting unit, configured to calculate, according to the color space value of the RGB color space or the converted space value, a cumulative sum of color average differences of all left-side pixels and all right-side pixels at the seam position, and use a calculation result as a color characteristic value;
and/or
The first extraction unit is used for extracting an edge intensity map of an image area of the spliced image by adopting an edge intensity map extraction method;
the second extraction unit is used for extracting a first edge intensity map of an image area on the rightmost side of the image to be processed by adopting the edge intensity map extraction method;
a third extraction unit, configured to extract a second edge intensity map of a leftmost image region of the image of the to-be-processed image by using the edge intensity map extraction method;
a first stitching unit for stitching the first edge intensity map and the second edge intensity map into a third edge intensity map of the same size as the edge intensity map of the stitched image;
the pixel subtraction unit is used for subtracting the edge intensity image of the spliced image from the third edge intensity image pixel by pixel to obtain a difference edge intensity image;
and the texture characteristic value extracting unit is used for adding the edge intensities of all pixels at the joint positions of the difference edge intensity graph to obtain the edge intensity accumulated sum of all pixels as a texture characteristic value.
7. The image classification system according to claim 6, wherein the edge intensity map extraction method specifically comprises:
carrying out space conversion on the color space of the color mode RGB of the image area of the edge intensity image to be extracted through a conversion formula to obtain a brightness color separation image;
performing convolution on the brightness and color separation image by utilizing a horizontal direction edge gradient operator to obtain a horizontal edge image;
performing convolution on the brightness and color separation image by using a vertical edge gradient operator to obtain a vertical edge image;
and calculating an edge strength map according to the horizontal edge map and the vertical edge map.
8. The image classification system of claim 5, wherein the classification module comprises:
the preset unit is used for presetting a threshold value of a preset color characteristic value and a threshold value of a preset texture characteristic value;
a first judging unit, configured to, when the extracted feature value is a color feature value, judge whether the color feature value is smaller than a threshold of the preset color feature value, if yes, the stitched image is a panoramic image, and if not, the stitched image is a normal image;
a second judging unit, configured to, when the extracted feature value is a texture feature value, judge whether the texture feature value is smaller than a threshold of the preset texture feature value, if yes, the stitched image is a panoramic image, and if not, the stitched image is a normal image;
and the third judging unit is used for judging whether the color characteristic value is smaller than the threshold value of the preset color characteristic value or not and whether the texture characteristic value is smaller than the threshold value of the preset texture characteristic value or not when the extracted characteristic values are the color characteristic value and the texture characteristic value, if so, the spliced image is a panoramic image, and if not, the spliced image is a common image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611162822.5A CN106780334B (en) | 2016-12-15 | 2016-12-15 | Image classification method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611162822.5A CN106780334B (en) | 2016-12-15 | 2016-12-15 | Image classification method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106780334A CN106780334A (en) | 2017-05-31 |
| CN106780334B true CN106780334B (en) | 2020-02-07 |
Family
ID=58891624
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611162822.5A Active CN106780334B (en) | 2016-12-15 | 2016-12-15 | Image classification method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106780334B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107948547B (en) * | 2017-12-29 | 2020-11-24 | 北京奇艺世纪科技有限公司 | Processing method and device for panoramic video stitching and electronic equipment |
| CN109191379B (en) * | 2018-07-26 | 2022-03-29 | 北京纵目安驰智能科技有限公司 | Panorama splicing semantic annotation method, system, terminal and storage medium |
| CN110399842B (en) * | 2019-07-26 | 2021-09-28 | 北京奇艺世纪科技有限公司 | Video processing method and device, electronic equipment and computer readable storage medium |
| CN111008985B (en) * | 2019-11-07 | 2021-08-17 | 贝壳找房(北京)科技有限公司 | Panorama picture seam detection method and device, readable storage medium and electronic equipment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102087652A (en) * | 2009-12-08 | 2011-06-08 | 百度在线网络技术（北京）有限公司 | Method for screening images and system thereof |
| CN103020941A (en) * | 2012-12-28 | 2013-04-03 | 昆山市工业技术研究院有限责任公司 | Panoramic stitching based rotary camera background establishment method and panoramic stitching based moving object detection method |
| CN103279939A (en) * | 2013-04-27 | 2013-09-04 | 北京工业大学 | Image stitching processing system |
| CN103871070A (en) * | 2014-04-03 | 2014-06-18 | 深圳市德赛微电子技术有限公司 | Automatic calibration method of vehicle-mounted panoramic imaging system |
| CN104598933A (en) * | 2014-11-13 | 2015-05-06 | 上海交通大学 | Multi-feature fusion based image copying detection method |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TW201005673A (en) * | 2008-07-18 | 2010-02-01 | Ind Tech Res Inst | Example-based two-dimensional to three-dimensional image conversion method, computer readable medium therefor, and system |
-
2016
- 2016-12-15 CN CN201611162822.5A patent/CN106780334B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102087652A (en) * | 2009-12-08 | 2011-06-08 | 百度在线网络技术（北京）有限公司 | Method for screening images and system thereof |
| CN103020941A (en) * | 2012-12-28 | 2013-04-03 | 昆山市工业技术研究院有限责任公司 | Panoramic stitching based rotary camera background establishment method and panoramic stitching based moving object detection method |
| CN103279939A (en) * | 2013-04-27 | 2013-09-04 | 北京工业大学 | Image stitching processing system |
| CN103871070A (en) * | 2014-04-03 | 2014-06-18 | 深圳市德赛微电子技术有限公司 | Automatic calibration method of vehicle-mounted panoramic imaging system |
| CN104598933A (en) * | 2014-11-13 | 2015-05-06 | 上海交通大学 | Multi-feature fusion based image copying detection method |
Non-Patent Citations (1)
| Title |
|---|
| 基于特征配准的全景图像拼接技术研究;孙向南;《中国优秀硕士学位论文全文数据库 信息科技辑》;20090615(第06期);第I138-676页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106780334A (en) | 2017-05-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106777114B (en) | Video classification method and system | |
| CN106780334B (en) | Image classification method and system | |
| CN106254933B (en) | Subtitle extraction method and device | |
| CN102799669B (en) | Automatic grading method for commodity image vision quality | |
| JP4351911B2 (en) | Method and apparatus for evaluating photographic quality of captured images in a digital still camera | |
| CN110517246B (en) | Image processing method and device, electronic equipment and storage medium | |
| TWI435288B (en) | Image processing apparatus and method, and program product | |
| CN107590447A (en) | A kind of caption recognition methods and device | |
| JP2019525515A (en) | Multiview scene segmentation and propagation | |
| US20130258138A1 (en) | Apparatus for generating an image with defocused background and method thereof | |
| CN112862685B (en) | Image stitching processing method, device and electronic system | |
| US8861867B2 (en) | Reuse of image processing information | |
| JP2014235750A (en) | Target detection method and target detection device | |
| US8948533B2 (en) | Increased quality of image objects based on depth in scene | |
| CN109525786B (en) | Video processing method and device, terminal equipment and storage medium | |
| JP2013140428A (en) | Edge detection device, edge detection program, and edge detection method | |
| US8311269B2 (en) | Blocker image identification apparatus and method | |
| US9094617B2 (en) | Methods and systems for real-time image-capture feedback | |
| CN108446603B (en) | News title detection method and device | |
| US20090214134A1 (en) | System and method for image data extraction and assembly in digital cameras | |
| CN109948605B (en) | Picture enhancement method and device for small target | |
| AU2016273984A1 (en) | Modifying a perceptual attribute of an image using an inaccurate depth map | |
| CN110782392A (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| WO2022056875A1 (en) | Method and apparatus for segmenting nameplate image, and computer-readable storage medium | |
| Masia et al. | Content-aware reverse tone mapping |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |