CN110033488B - Self-adaptive light source direction analysis method and system based on compressed information - Google Patents
Self-adaptive light source direction analysis method and system based on compressed information Download PDFInfo
- Publication number
- CN110033488B CN110033488B CN201910291948.XA CN201910291948A CN110033488B CN 110033488 B CN110033488 B CN 110033488B CN 201910291948 A CN201910291948 A CN 201910291948A CN 110033488 B CN110033488 B CN 110033488B
- Authority
- CN
- China
- Prior art keywords
- illumination direction
- illumination
- image
- analysis
- frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a self-adaptive light source direction analysis method and system based on compressed information. The method comprises the steps of aiming at a system supporting multi-method illumination direction analysis, firstly determining whether a current identification video belongs to an established illumination analysis scene by using a reference object matching method, and then enabling the system to start analysis modes of different illumination directions by combining a matching result; then, aiming at different illumination direction analysis methods, the illumination direction analysis area of the current image is rapidly divided by utilizing the information such as block division, prediction mode and the like of the code stream, so that the analysis speed is improved.
Description
Technical Field
The invention relates to the field of image processing, in particular to a self-adaptive light source direction analysis method and system based on compressed information.
Background
In practical application, the illumination condition can have a great influence on the face detection precision, such as backlight, polarized light sources and the like. At this time, if the light source position can be obtained, illumination compensation can be performed in advance, and the recognition accuracy can be improved. The conventional light source direction analysis methods include a classification recognition method based on databases of different illumination directions and a light source direction detection method based on image information such as illumination models and reflection. The first type of method has high recognition accuracy but has scene limitation, and the second type of method has the difficulty that actual image information is complex, so that an algorithm can take a large amount of calculation amount on the step of extracting a region for light source direction analysis from an image. On the other hand, when searching video, if the division and prediction information carried by the video code stream is ignored and can be used as auxiliary information for light source analysis, the calculation amount of frame-by-frame full-frame searching is wasted.
Disclosure of Invention
The embodiment of the invention aims to provide a self-adaptive light source direction analysis method based on compressed information, and aims to solve the problem of large calculation amount of a light source direction analysis method in the prior art.
The embodiment of the invention is realized in such a way that the self-adaptive light source direction analysis method based on the compressed information comprises the following steps:
judging whether the current video to be detected belongs to a preset illumination analysis scene or not by using a reference object matching method;
and starting a corresponding illumination direction analysis mode according to the judgment result.
Further, the determining whether the current video to be detected belongs to a preset scene of illumination analysis by using the reference object matching method includes:
selecting a mode screening diagram corresponding to each scene from preset classification recognition templates according to the rule of one scene and one image, and dividing the recognition templates of the same scene into a subclass;
and performing reference object matching by using the first frame image and the mode screening image of the current video to be detected.
Another object of an embodiment of the present invention is to provide an adaptive light source direction analysis system based on compressed information, the system including:
the illumination analysis scene judging device is used for judging whether the current video to be detected belongs to a preset illumination analysis scene or not by using a reference object matching method;
and the illumination direction analysis device is used for starting a corresponding illumination direction analysis module according to the judgment result.
Further, the illumination analysis scene determination apparatus includes:
the mode screening diagram selecting module is used for selecting a mode screening diagram corresponding to each scene from preset classification recognition templates according to the rule of one scene to one image, and dividing the recognition templates of the same scene into a subclass;
and the reference object matching device is used for matching the reference object by utilizing the first frame image and the mode screening image of the current video to be detected.
The beneficial effects of the invention are that
The invention provides a self-adaptive light source direction analysis method and system based on compressed information. The method comprises the steps of aiming at a system supporting multi-method illumination direction analysis, firstly determining whether a current identification video belongs to an established illumination analysis scene by using a reference object matching method, and then enabling the system to start analysis modes of different illumination directions by combining a matching result; then, aiming at different illumination direction analysis methods, the illumination direction analysis area of the current image is rapidly divided by utilizing the information such as block division, prediction mode and the like of the code stream, so that the analysis speed is improved.
Drawings
FIG. 1 is a flow chart of an adaptive light source direction analysis method based on compressed information in accordance with a preferred embodiment of the present invention;
fig. 2 is a flowchart of a method for determining whether a video to be detected currently belongs to a preset scene of illumination analysis by using a reference object matching method according to an embodiment of the present invention;
FIG. 3 is a flowchart of a method for performing reference object matching by using a first frame image and a pattern screening chart of a video to be detected according to an embodiment of the present invention;
FIG. 4 is a flowchart of another method for performing reference object matching by using a first frame image and a pattern screening chart of a video to be detected according to an embodiment of the present invention;
FIG. 5 is a block diagram of an adaptive light source direction analysis system based on compressed information in accordance with a preferred embodiment of the present invention;
FIG. 6 is a block diagram of a scene determination device for illumination analysis according to an embodiment of the present invention;
FIG. 7 is a diagram of a structure of a reference matching device according to an embodiment of the present invention;
fig. 8 is a block diagram of another reference object matching apparatus according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be further described in detail with reference to the accompanying drawings and examples, and only the portions related to the examples of the present invention are shown for convenience of description. It should be understood that the particular embodiments described herein are illustrative only and are not intended to limit the invention.
The invention provides a self-adaptive light source direction analysis method and system based on compressed information. The method comprises the steps of aiming at a system supporting multi-method illumination direction analysis, firstly determining whether a current identification video belongs to an established illumination analysis scene by using a reference object matching method, and then enabling the system to start analysis modes of different illumination directions by combining a matching result; then, aiming at different illumination direction analysis methods, the illumination direction analysis area of the current image is rapidly divided by utilizing the information such as block division, prediction mode and the like of the code stream, so that the analysis speed is improved.
Example 1
FIG. 1 is a flow chart of an adaptive light source direction analysis method based on compressed information in accordance with a preferred embodiment of the present invention; the method provided by the embodiment of the invention is applied to the encoder supporting multi-method illumination direction analysis;
the method of the embodiment of the invention comprises the following steps:
s1, judging whether a current video to be detected belongs to a preset illumination analysis scene or not by using a reference object matching method;
if the reference object matching is successful, representing a scene belonging to the preset illumination analysis, and if the reference object matching is unsuccessful, representing a scene not belonging to the preset illumination analysis;
fig. 2 is a method for determining whether a current video to be detected belongs to a preset scene of illumination analysis by using a reference object matching method according to an embodiment of the present invention, which specifically includes:
s101: and selecting a mode screening diagram corresponding to each scene from preset classification recognition templates according to the rule of one scene and one image, and simultaneously dividing the recognition templates of the same scene into a subclass.
The classification recognition templates are image recognition templates which are acquired in a pattern recognition training stage and classified according to illumination directions, the image recognition templates are stored in an encoder in advance, and each image corresponds to one pattern screening diagram in the classification recognition templates.
S102: and performing reference object matching by using the first frame image and the mode screening image of the current video to be detected.
Fig. 3 is a method for performing reference object matching by using a first frame image and a pattern screening chart of a current video to be detected according to an embodiment of the present invention:
step A1: and respectively forming a short video group of two frames of images by the first frame image of the current video to be detected and each mode screening image, wherein the first frame image of the current video to be detected is made to be the second frame image of each short video.
Step A2: the respective short video is encoded.
Step A3: firstly, locating the block dividing position of a reference object, if a block in the second frame of the current short video takes the block in which the reference object is located as a reference block, setting a mode screening diagram corresponding to the first frame of the current short video as a mode screening diagram successfully matched, and then entering a step S2, otherwise, entering a step A4.
Step A4: if all the short videos are judged to be finished, the step S2 is carried out; otherwise, returning to the step A3 to judge the next short video.
Fig. 4 is a schematic diagram of another method for performing reference object matching by using a first frame image and a pattern screening chart of a video to be detected according to an embodiment of the present invention:
step B1: and respectively forming a short video group by the first frame image of the current video to be detected and each mode screening image, wherein the first frame image of the current video to be detected is made to be the first frame image of the short video.
Step B2: encoding the short video;
wherein the first frame image is guaranteed to be within the screening range over the span of the reference frame (i.e. the reference frame comprises the first frame image) during the encoding of the short video.
Step B3: the current frame is set to be the short video second frame.
Step B4: if the prediction modes of the blocks corresponding to the reference objects in the current frame are all inter-frame prediction modes, and the blocks in the first frame of the short video are all taken as reference blocks of the blocks corresponding to the reference objects, setting a mode screening diagram corresponding to the current frame as a successfully matched mode screening diagram, and then entering step S2; otherwise, enter step B5;
step B5: if all frames are judged to be finished, the step S2 is entered; otherwise, setting the next frame of the current frame as the current frame, and returning to the step B4.
The reference object matching can be performed by using the first frame image and the mode screening image of the current video to be detected by other methods known in the art.
S2, starting a corresponding illumination direction analysis mode according to a judgment result;
the method comprises the following steps: if the reference object is successfully matched (belongs to a preset illumination analysis scene), entering a first illumination direction analysis mode; if the reference object is not successfully matched (does not belong to a preset illumination analysis scene), entering a second illumination direction analysis mode;
first illumination direction analysis mode: classifying, identifying and template matching is carried out on a first frame image of a current video to be detected by utilizing a sub-class to which a pattern screening diagram successfully matched belongs, and a specific illumination direction is determined; and then carrying out illumination direction detection on the residual frame image of the video to be detected currently.
Second illumination direction analysis mode: firstly, according to the characteristics of images with different illumination directions in a package packaged by a preset light source direction analysis method, dividing an illumination direction analysis area of a current image by combining code stream information of a current video to be detected, and then carrying out illumination direction detection on the illumination direction analysis area to determine a specific illumination direction; then, carrying out illumination direction detection on the residual frame image of the video to be detected currently;
the code stream information includes block division, prediction modes, and the like.
The light source direction analysis method packaging package is a detection method of the encoder preset packaging for different image information illumination directions, and comprises a classification recognition method based on a database of different illumination directions, an illumination direction detection method based on image information such as illumination models, reflection and the like; all are the prior art.
In particular, in application, if the light source direction analysis method package is a reflection-based illumination direction detection analysis method, the image characteristics can be known by the package: compared with the object, the image reflection has the characteristics of simple texture, low brightness and deficient chromaticity information;
dividing the illumination direction analysis area of the current image, then carrying out illumination direction detection on the illumination direction analysis area, and determining that the specific illumination direction is specifically:
step C1: firstly, deleting an image area of a block division non-maximum block in a first frame image of a current video to be detected;
step C2: then, further deleting the blocks with the luminance average value exceeding the luminance threshold value by utilizing the luminance average value of the blocks;
step C3: then, the back image is utilized to be the characteristic without color, and the non-zero blocks of the chrominance components are further deleted, so that the invalid data deletion is completed;
step C4: finally, the reflection is extracted from the current residual image area by using the method disclosed in the industry, and the illumination direction analysis is carried out to determine the specific illumination direction.
In particular, in the application, if the light source direction analysis method package is based on illumination direction detection analysis of an illumination model, the image characteristics can be known by the package: the brightness of the image changes along with the change of the included angle between the object surface direction and the illumination direction, and any illumination direction can generate a specific image brightness distribution (thereby judging the illumination direction specifically, and the object surface effective for illumination direction estimation needs to be screened out during illumination analysis); whether concave or convex, the video code stream has the characteristic of simple texture, and the brightness is not completely black when illumination is needed;
dividing the illumination direction analysis area of the current image, then carrying out illumination direction detection on the illumination direction analysis area, and determining that the specific illumination direction is specifically:
step D1: firstly, deleting an image area of a block division non-maximum block in a first frame image of a current video to be detected;
step D2: then, further deleting the blocks with the luminance average value smaller than the luminance threshold value by utilizing the luminance average value of the blocks;
step D3: then, further deleting the isolated maximum block by using connectivity, and taking the connected block as a candidate concave-convex surface detection area to finish invalid data deletion;
step D4: finally, calculating an included angle to acquire an illumination direction by using a Pentland method disclosed in the industry or an improved Pentland method for each candidate concave-convex surface detection area in the current residual image area:
further, if it is desired to obtain a more accurate result, the "calculating the included angle to obtain the illumination direction by using the Pentland method or the modified Pentland method disclosed in the industry" may be replaced by "extracting the effective data by using the Pentland method and then calculating the illumination direction".
The method for detecting the illumination direction of the residual frame image of the video to be detected comprises the following steps: maintaining the illumination direction analysis result of the current video to be detected until the current video to be detected appears an I frame again (the default coding mode of the first frame in video coding is the I frame), deleting all frames before the I frame of the current video to be detected at the moment, and returning to S1; if the current video to be detected does not appear I frame, the illumination direction analysis result of the current video to be detected is kept until the end.
Example two
FIG. 5 is a block diagram of an adaptive light source direction analysis system based on compressed information in accordance with a preferred embodiment of the present invention; the system comprises:
the illumination analysis scene judging device is used for judging whether the current video to be detected belongs to a preset illumination analysis scene or not by using a reference object matching method;
specifically, if the reference object matching is successful, a scene belonging to the preset illumination analysis is indicated, and if the reference object matching is unsuccessful, a scene not belonging to the preset illumination analysis is indicated;
the illumination direction analysis device is used for starting a corresponding illumination direction analysis module according to the judgment result; the illumination direction analysis module comprises a first illumination direction analysis module and a second illumination direction analysis module
Specifically, if the reference object is successfully matched (belongs to a preset illumination analysis scene), a first illumination direction analysis module is started; if the reference object is not successfully matched (does not belong to a preset illumination analysis scene), a second illumination direction analysis module is started;
further, fig. 6 is a block diagram of an illumination analysis scene determination apparatus according to an embodiment of the present invention; the illumination analysis scene determination device includes:
and the mode screening diagram selecting module is used for selecting the mode screening diagram corresponding to each scene from the preset classification recognition templates according to the rule of one scene to one image, and simultaneously dividing the recognition templates of the same scene into a subclass.
The classification recognition templates are image recognition templates which are acquired in a pattern recognition training stage and classified according to illumination directions, the image recognition templates are stored in an encoder in advance, and each image corresponds to one pattern screening diagram in the classification recognition templates.
And the reference object matching device is used for matching the reference object by utilizing the first frame image and the mode screening image of the current video to be detected.
Further, fig. 7 is a structural diagram of a reference object matching device provided in an embodiment of the present invention;
the first short video construction module is used for respectively constructing a short video group of two frames of images by the first frame image of the current video to be detected and each mode screening image, wherein the first frame image of the current video to be detected is made to be the second frame image of each short video.
And the first short video coding module is used for coding each short video.
The reference object positioning and first reference block judging module is used for positioning the block dividing position of the reference object at first, and entering the first matching success mode screening diagram setting module if the block in the second frame of the current short video takes the block in which the reference object is positioned as a reference block; otherwise, entering a first judging and processing module;
a first successful matching pattern screening diagram setting module, configured to set a pattern screening diagram corresponding to a first frame of a current short video as a successful matching pattern screening diagram, and then enter an illumination direction analysis device
The first judging and processing module is used for judging that if all the short videos are judged to be finished, the illumination direction analysis device is started; otherwise, returning to the reference object positioning and first reference block judging module.
Further, fig. 8 is a block diagram of another reference object matching apparatus according to an embodiment of the present invention.
The second short video construction module is used for respectively constructing a short video group by the first frame image of the current video to be detected and each mode screening image, wherein the first frame image of the current video to be detected is made to be the first frame image of the short video;
the second short video coding module is used for coding the short video;
in the process of encoding the short video, ensuring that the first frame image is within a screening range on a reference frame span (namely, the reference frame comprises the first frame image);
the second short video second frame setting module is used for setting the current frame as a short video second frame;
the reference object prediction mode and the second reference block judging module are used for judging that if the prediction modes of the reference object corresponding blocks in the current frame are all inter prediction modes and the reference object corresponding blocks all take the blocks in the first frame of the short video as the reference blocks, the second matching success mode screening diagram setting module is entered; otherwise, entering a second judging and processing module;
the second successful matching mode screening diagram setting module is used for setting the mode screening diagram corresponding to the current frame as a successful matching mode screening diagram and then entering the illumination direction analysis device;
the second judging and processing module is used for judging that if all the frames are judged to be finished, the illumination direction analyzing device is started; otherwise, setting the next frame of the current frame as the current frame, and then returning to the reference object prediction mode and the second reference block judgment module;
further, the first illumination direction analysis module is configured to screen the subclasses to which the image belongs by using a pattern that is successfully matched, perform classification recognition template matching on the first frame image of the current video to be detected, and determine a specific illumination direction; and then carrying out illumination direction detection on the residual frame image of the video to be detected currently.
The second illumination direction analysis module is used for firstly packaging the characteristics of images with different illumination directions in the package according to a preset light source direction analysis method, dividing an illumination direction analysis area of the current image according to the code stream information of the current video to be detected, and then carrying out illumination direction detection on the illumination direction analysis area to determine a specific illumination direction; then, carrying out illumination direction detection on the residual frame image of the video to be detected currently;
the code stream information includes block division, prediction modes, and the like.
The light source direction analysis method packaging package is a detection method of the encoder preset packaging for different image information illumination directions, and comprises a classification recognition method based on a database of different illumination directions, an illumination direction detection method based on image information such as illumination models, reflection and the like; all are the prior art.
In particular, in application, if the light source direction analysis method package is a reflection-based illumination direction detection analysis method, the image characteristics can be known by the package: compared with the object, the image reflection has the characteristics of simple texture, low brightness and deficient chromaticity information;
dividing the illumination direction analysis area of the current image, then carrying out illumination direction detection on the illumination direction analysis area, and determining that the specific illumination direction is specifically:
step C1: firstly, deleting an image area of a block division non-maximum block in a first frame image of a current video to be detected;
step C2: then, further deleting the blocks with the luminance average value exceeding the luminance threshold value by utilizing the luminance average value of the blocks;
step C3: then, the back image is utilized to be the characteristic without color, and the non-zero blocks of the chrominance components are further deleted, so that the invalid data deletion is completed;
step C4: finally, the reflection is extracted from the current residual image area by using the method disclosed in the industry, and the illumination direction analysis is carried out to determine the specific illumination direction.
In particular, in the application, if the light source direction analysis method package is based on illumination direction detection analysis of an illumination model, the image characteristics can be known by the package: the brightness of the image changes along with the change of the included angle between the object surface direction and the illumination direction, and any illumination direction can generate a specific image brightness distribution (thereby judging the illumination direction specifically, and the object surface effective for illumination direction estimation needs to be screened out during illumination analysis); whether concave or convex, the video code stream has the characteristic of simple texture, and the brightness is not completely black when illumination is needed;
dividing the illumination direction analysis area of the current image, then carrying out illumination direction detection on the illumination direction analysis area, and determining that the specific illumination direction is specifically:
step D1: firstly, deleting an image area of a block division non-maximum block in a first frame image of a current video to be detected;
step D2: then, further deleting the blocks with the luminance average value smaller than the luminance threshold value by utilizing the luminance average value of the blocks;
step D3: then, further deleting the isolated maximum block by using connectivity, and taking the connected block as a candidate concave-convex surface detection area to finish invalid data deletion;
step D4: finally, calculating an included angle to obtain an illumination direction by using a Pentland method disclosed in the industry or an improved Pentland method for each candidate concave-convex surface detection area in the current residual image area;
further, if it is desired to obtain a more accurate result, the "calculating the included angle to obtain the illumination direction by using the Pentland method or the modified Pentland method disclosed in the industry" may be replaced by "extracting the effective data by using the Pentland method and then calculating the illumination direction".
The method for detecting the illumination direction of the residual frame image of the video to be detected comprises the following steps: maintaining the illumination direction analysis result of the current video to be detected until the current video to be detected shows an I frame again (the default coding mode of the first frame in video coding is the I frame), deleting all frames before the I frame of the current video to be detected, and returning to the illumination analysis scene judgment device; if the current video to be detected does not appear I frame, the illumination direction analysis result of the current video to be detected is kept until the end.
It will be appreciated by those of ordinary skill in the art that implementing all or part of the steps of the methods of the embodiments described above may be accomplished by program instruction related hardware, and the program may be stored on a computer readable storage medium, which may be a ROM, a RAM, a magnetic disk, an optical disk, etc.
The foregoing description of the preferred embodiments of the invention is not intended to be limiting, but rather is intended to cover all modifications, equivalents, and alternatives falling within the spirit and principles of the invention.
Claims (13)
1. An adaptive light source direction analysis method based on compressed information, the method comprising:
judging whether the current video to be detected belongs to a preset illumination analysis scene or not by using a reference object matching method;
the determining whether the current video to be detected belongs to a preset illumination analysis scene by using the reference object matching method comprises the following steps:
selecting a mode screening diagram corresponding to each scene from preset classification recognition templates according to the rule of one scene and one image, and dividing the recognition templates of the same scene into a subclass;
performing reference object matching by using a first frame image and a mode screening image of a current video to be detected;
starting a corresponding illumination direction analysis mode according to the judgment result;
the starting of the corresponding illumination direction analysis mode according to the judgment result specifically comprises the following steps: if the reference object is successfully matched, a first illumination direction analysis mode is entered; if the reference object matching is unsuccessful, entering a second illumination direction analysis mode;
first illumination direction analysis mode: classifying, identifying and template matching is carried out on a first frame image of a current video to be detected by utilizing a sub-class to which a pattern screening diagram successfully matched belongs, and a specific illumination direction is determined; then, carrying out illumination direction detection on the residual frame image of the video to be detected currently;
second illumination direction analysis mode: firstly, according to the characteristics of images with different illumination directions in a package packaged by a preset light source direction analysis method, dividing an illumination direction analysis area of a current image by combining code stream information of a current video to be detected, and then carrying out illumination direction detection on the illumination direction analysis area to determine a specific illumination direction; then, carrying out illumination direction detection on the residual frame image of the video to be detected currently; the light source direction analysis method packaging package is used for presetting a detection method of the packaged illumination direction aiming at different image information for the encoder, wherein the detection method of the illumination direction comprises an illumination model-based detection method and a reflection-based detection method of the illumination direction.
2. The adaptive light source direction analysis method based on compressed information according to claim 1, wherein the performing reference object matching using the first frame image of the video to be detected and the pattern filter comprises the steps of:
step A1: respectively forming a short video group of two frames of images by the first frame image of the current video to be detected and each mode screening image, wherein the first frame image of the current video to be detected is made to be a second frame image of each short video;
step A2: encoding each short video;
step A3: firstly, positioning the block dividing position of a reference object, if a block in a second frame of the current short video takes the block in which the reference object is positioned as a reference block, setting a mode screening diagram corresponding to the first frame of the current short video as a mode screening diagram successfully matched, then entering a corresponding illumination direction analysis mode started according to a judging result, otherwise, entering a step A4;
step A4: if all the short videos are judged to be finished, entering a corresponding illumination direction analysis mode started according to a judgment result; otherwise, returning to the step A3 to judge the next short video.
3. The adaptive light source direction analysis method based on compressed information according to claim 1, wherein the performing reference object matching using the first frame image of the video to be detected and the pattern filter comprises the steps of:
step B1: respectively forming a short video group by the first frame image of the current video to be detected and each mode screening image, wherein the first frame image of the current video to be detected is made to be the first frame image of the short video;
step B2: encoding the short video; in the process of encoding the short video, ensuring that a first frame image is in a screening range on a reference frame span;
step B3: setting the current frame as a short video second frame;
step B4, if the prediction modes of the reference object corresponding blocks in the current frame are all inter-frame prediction modes, and the reference object corresponding blocks take the blocks in the first frame of the short video as reference blocks, setting the mode screening diagram corresponding to the current frame as a successfully matched mode screening diagram, and then entering a corresponding illumination direction analysis mode started according to a judging result; otherwise, enter step B5;
step B5: if all frames are judged to be finished, a corresponding illumination direction analysis mode is started according to a judgment result; otherwise, setting the next frame of the current frame as the current frame, and returning to the step B4.
4. The method of claim 1, wherein if the light source direction analysis package is a reflection-based illumination direction detection analysis method,
dividing the illumination direction analysis area of the current image, then carrying out illumination direction detection on the illumination direction analysis area, and determining that the specific illumination direction is specifically:
step C1: firstly, deleting an image area of a block division non-maximum block in a first frame image of a current video to be detected;
step C2: then, further deleting the blocks with the luminance average value exceeding the luminance threshold value by utilizing the luminance average value of the blocks;
step C3: then, the back image is utilized to be the characteristic without color, and the non-zero blocks of the chrominance components are further deleted, so that the invalid data deletion is completed;
step C4: finally, the reflection is extracted from the current residual image area, and the illumination direction analysis is carried out to determine the specific illumination direction.
5. The method of claim 1, wherein if the light source direction analysis package is a light direction detection analysis based on a light model,
dividing the illumination direction analysis area of the current image, then carrying out illumination direction detection on the illumination direction analysis area, and determining that the specific illumination direction is specifically:
step D1: firstly, deleting an image area of a block division non-maximum block in a first frame image of a current video to be detected;
step D2: then, further deleting the blocks with the luminance average value smaller than the luminance threshold value by utilizing the luminance average value of the blocks;
step D3: then, further deleting the isolated maximum block by using connectivity, and taking the connected block as a candidate concave-convex surface detection area to finish invalid data deletion;
step D4: and finally, calculating an included angle to acquire the illumination direction by using a Pentland method or an improved Pentland method for each candidate concave-convex surface detection area in the current residual image area.
6. The method for adaptive light source direction analysis based on compressed information of claim 5, wherein obtaining the direction of illumination further comprises extracting valid data using the Pentland method and then calculating the direction of illumination.
7. An adaptive light source direction analysis system based on compressed information, the system comprising:
the illumination analysis scene judging device is used for judging whether the current video to be detected belongs to a preset illumination analysis scene or not by using a reference object matching method;
the illumination analysis scene determination device includes:
the mode screening diagram selecting module is used for selecting a mode screening diagram corresponding to each scene from preset classification recognition templates according to the rule of one scene to one image, and dividing the recognition templates of the same scene into a subclass;
the reference object matching device is used for matching the reference object by utilizing the first frame image and the mode screening image of the current video to be detected;
the illumination direction analysis device is used for starting a corresponding illumination direction analysis module according to the judgment result;
the illumination direction analysis device comprises a first illumination direction analysis module and a second illumination direction analysis module;
the first illumination direction analysis module is used for screening subclasses of the image by using a mode which is successfully matched, classifying and identifying template matching is carried out on a first frame image of the current video to be detected, and a specific illumination direction is determined; then, carrying out illumination direction detection on the residual frame image of the video to be detected currently;
the second illumination direction analysis module is used for firstly packaging the characteristics of images with different illumination directions in the package according to a preset light source direction analysis method, dividing an illumination direction analysis area of the current image according to the code stream information of the current video to be detected, and then carrying out illumination direction detection on the illumination direction analysis area to determine a specific illumination direction; then, carrying out illumination direction detection on the residual frame image of the video to be detected currently; the light source direction analysis method packaging package is used for presetting a detection method of the packaged illumination direction aiming at different image information for the encoder, wherein the detection method of the illumination direction comprises an illumination model-based detection method and a reflection-based detection method of the illumination direction.
8. The adaptive light source direction analysis system based on compressed information of claim 7, wherein the reference matching means comprises:
the first short video construction module is used for respectively constructing a first frame image of the current video to be detected and each mode screening image into a short video group of two frame images, wherein the first frame image of the current video to be detected is made to be a second frame image of each short video;
the first short video coding module is used for coding each short video;
the reference object positioning and first reference block judging module is used for positioning the block dividing position of the reference object at first, and entering the first matching success mode screening diagram setting module if the block in the second frame of the current short video takes the block in which the reference object is positioned as a reference block; otherwise, entering a first judging and processing module;
the first successful matching mode screening diagram setting module is used for setting a mode screening diagram corresponding to a first frame of the current short video as a successful matching mode screening diagram and then entering the illumination direction analysis device;
the first judging and processing module is used for judging that if all the short videos are judged to be finished, the illumination direction analysis device is started; otherwise, returning to the reference object positioning and first reference block judging module.
9. The adaptive light source direction analysis system based on compressed information of claim 7, wherein the reference matching means comprises:
the second short video construction module is used for respectively constructing a short video group by the first frame image of the current video to be detected and each mode screening image, wherein the first frame image of the current video to be detected is made to be the first frame image of the short video;
the second short video coding module is used for coding the short video; in the process of encoding the short video, ensuring that a first frame image is in a screening range on a reference frame span;
the second short video second frame setting module is used for setting the current frame as a short video second frame;
the reference object prediction mode and the second reference block judging module are used for judging that if the prediction modes of the reference object corresponding blocks in the current frame are all inter prediction modes and the reference object corresponding blocks take the blocks in the first frame of the short video as the reference blocks, the second matching success mode screening diagram setting module is entered; otherwise, entering a second judging and processing module;
the second successful matching mode screening diagram setting module is used for setting the mode screening diagram corresponding to the current frame as a successful matching mode screening diagram and then entering the illumination direction analysis device;
the second judging and processing module is used for judging that if all the frames are judged to be finished, the illumination direction analyzing device is started; otherwise, setting the next frame of the current frame as the current frame, and then returning to the reference object prediction mode and the second reference block judgment module.
10. An adaptive light source direction analysis system based on compressed information according to claim 7,
the light source direction analysis method packaging package is a detection method of the encoder preset packaging for different image information illumination directions, and comprises a classification recognition method based on a database of different illumination directions and an illumination direction detection method based on image information such as illumination models, reflection and the like.
11. The adaptive light source direction analysis system according to claim 10, wherein if the light source direction analysis package is a reflection-based illumination direction detection analysis method,
dividing the illumination direction analysis area of the current image, then carrying out illumination direction detection on the illumination direction analysis area, and determining that the specific illumination direction is specifically:
step C1: firstly, deleting an image area of a block division non-maximum block in a first frame image of a current video to be detected;
step C2: then, further deleting the blocks with the luminance average value exceeding the luminance threshold value by utilizing the luminance average value of the blocks;
step C3: then, the back image is utilized to be the characteristic without color, and the non-zero blocks of the chrominance components are further deleted, so that the invalid data deletion is completed;
step C4: finally, the reflection is extracted from the current residual image area, and the illumination direction analysis is carried out to determine the specific illumination direction.
12. The adaptive light source direction analysis system according to claim 10, wherein if the light source direction analysis package is a light direction detection analysis based on a light model,
dividing the illumination direction analysis area of the current image, then carrying out illumination direction detection on the illumination direction analysis area, and determining that the specific illumination direction is specifically:
step D1: firstly, deleting an image area of a block division non-maximum block in a first frame image of a current video to be detected;
step D2: then, further deleting the blocks with the luminance average value smaller than the luminance threshold value by utilizing the luminance average value of the blocks;
step D3: then, further deleting the isolated maximum block by using connectivity, and taking the connected block as a candidate concave-convex surface detection area to finish invalid data deletion;
step D4: and finally, calculating an included angle to acquire the illumination direction by using a Pentland method or an improved Pentland method for each candidate concave-convex surface detection area in the current residual image area.
13. The adaptive light source direction analysis system of claim 12, wherein obtaining the direction of illumination further comprises extracting valid data using a Pentland method and then calculating the direction of illumination.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910291948.XA CN110033488B (en) | 2019-04-09 | 2019-04-09 | Self-adaptive light source direction analysis method and system based on compressed information |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910291948.XA CN110033488B (en) | 2019-04-09 | 2019-04-09 | Self-adaptive light source direction analysis method and system based on compressed information |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110033488A CN110033488A (en) | 2019-07-19 |
| CN110033488B true CN110033488B (en) | 2023-09-15 |
Family
ID=67238094
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910291948.XA Active CN110033488B (en) | 2019-04-09 | 2019-04-09 | Self-adaptive light source direction analysis method and system based on compressed information |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110033488B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116071807B (en) * | 2023-03-06 | 2023-06-16 | 深圳市网联天下科技有限公司 | Campus card intelligent early warning method and system based on video monitoring |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102761737A (en) * | 2011-04-28 | 2012-10-31 | 财团法人工业技术研究院 | Systems and methods for processing shadows in compressed video images |
| CN108765463A (en) * | 2018-05-30 | 2018-11-06 | 河海大学常州校区 | A kind of moving target detecting method calmodulin binding domain CaM extraction and improve textural characteristics |
| CN109446967A (en) * | 2018-10-22 | 2019-03-08 | 深圳市梦网百科信息技术有限公司 | A kind of method for detecting human face and system based on compression information |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10887597B2 (en) * | 2015-06-09 | 2021-01-05 | Qualcomm Incorporated | Systems and methods of determining illumination compensation parameters for video coding |
-
2019
- 2019-04-09 CN CN201910291948.XA patent/CN110033488B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102761737A (en) * | 2011-04-28 | 2012-10-31 | 财团法人工业技术研究院 | Systems and methods for processing shadows in compressed video images |
| CN108765463A (en) * | 2018-05-30 | 2018-11-06 | 河海大学常州校区 | A kind of moving target detecting method calmodulin binding domain CaM extraction and improve textural characteristics |
| CN109446967A (en) * | 2018-10-22 | 2019-03-08 | 深圳市梦网百科信息技术有限公司 | A kind of method for detecting human face and system based on compression information |
Non-Patent Citations (1)
| Title |
|---|
| 基于光度立体算法的图像建模系统的设计及实现;张屹凌;《中国优秀博硕士学位论文全文数据库（硕士）》;20060915;原文第7-10页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110033488A (en) | 2019-07-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20050200722A1 (en) | Image capturing apparatus, image capturing method, and machine readable medium storing thereon image capturing program | |
| US20080240555A1 (en) | Two Stage Detection for Photographic Eye Artifacts | |
| CN110189294B (en) | RGB-D image significance detection method based on depth reliability analysis | |
| KR20100073189A (en) | Apparatus and method for detecting face image | |
| CN101029823B (en) | Method for tracking vehicle based on state and classification | |
| JPH0944670A (en) | Method and device for extracting specified image area | |
| KR20090122275A (en) | Two stage detection for photographic eye artifacts | |
| US20070047824A1 (en) | Method, apparatus, and program for detecting faces | |
| CN103186887A (en) | Image demisting device and image demisting method | |
| CN110596120A (en) | Glass boundary defect detection method, device, terminal and storage medium | |
| EP1300804A2 (en) | Face detecting method by skin color recognition | |
| WO2018053952A1 (en) | Video image depth extraction method based on scene sample library | |
| CN111008608B (en) | Night vehicle detection method based on deep learning | |
| CN101771825B (en) | Method and device for compensating chroma of images | |
| CN114359235A (en) | Wood surface defect detection method based on improved YOLOv5l network | |
| CN110033488B (en) | Self-adaptive light source direction analysis method and system based on compressed information | |
| JP3774495B2 (en) | Image information extracting apparatus and method | |
| CN116596903A (en) | Defect identification method, device, electronic equipment and readable storage medium | |
| CN116052090A (en) | Image quality evaluation method, model training method, device, equipment and medium | |
| JP2501076B2 (en) | Fastener-Automatic inspection method and inspection device | |
| CN115239551A (en) | Video enhancement method and device | |
| CN114463242A (en) | Image detection method, device, storage medium and device | |
| CN111797691A (en) | Method for improving face recognition accuracy and processing subsystem | |
| CN113706401B (en) | Slide automatic shooting and intelligent editing method based on mobile phone camera | |
| Rahman et al. | Real-time face-based auto-focus for digital still and cell-phone cameras |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 518000 Guangdong city of Shenzhen province Nanshan District Guangdong streets high in the four Longtaili Technology Building Room 325 No. 30 Applicant after: Shenzhen mengwang video Co.,Ltd. Address before: 518000 Guangdong city of Shenzhen province Nanshan District Guangdong streets high in the four Longtaili Technology Building Room 325 No. 30 Applicant before: SHENZHEN MONTNETS ENCYCLOPEDIA INFORMATION TECHNOLOGY Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |