CN112183373B - Light source identification method, device, terminal equipment and computer readable storage medium - Google Patents
Light source identification method, device, terminal equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN112183373B CN112183373B CN202011052635.8A CN202011052635A CN112183373B CN 112183373 B CN112183373 B CN 112183373B CN 202011052635 A CN202011052635 A CN 202011052635A CN 112183373 B CN112183373 B CN 112183373B
- Authority
- CN
- China
- Prior art keywords
- differential image
- light source
- phase
- phase point
- frequency
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 28
- 238000012795 verification Methods 0.000 claims abstract description 38
- 238000002310 reflectometry Methods 0.000 claims description 15
- 101150032064 VTS1 gene Proteins 0.000 claims description 12
- 238000003384 imaging method Methods 0.000 claims description 9
- 238000011156 evaluation Methods 0.000 claims description 5
- 238000004590 computer program Methods 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 11
- 230000006870 function Effects 0.000 description 4
- 238000001514 detection method Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000012806 monitoring device Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G06V20/584—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of vehicle lights or traffic lights
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/70—Circuitry for compensating brightness variation in the scene
- H04N23/73—Circuitry for compensating brightness variation in the scene by influencing the exposure time
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Signal Processing (AREA)
- Studio Devices (AREA)
Abstract
The embodiment of the application discloses a light source identification method, a device, a terminal device and a computer readable storage medium, wherein the light source identification method comprises the following steps: acquiring two first images of a first interval output by a long exposure channel, analyzing the two first images to obtain a first differential image, acquiring two second images of a second interval output by a short exposure channel, analyzing the two second images to obtain a second differential image, carrying out phase identification on the first differential image and the second differential image to obtain a first phase point and a second phase point which are respectively included in the first differential image and the second differential image, and carrying out phase verification on the first phase point and the second phase point to identify whether a light source with target frequency exists in the current environment or not, wherein the target frequency is the frequency of the light source contained in the first differential image and/or the second differential image. By implementing the method and the device, whether the light source with the target frequency exists in the current environment can be identified efficiently and accurately.
Description
Technical Field
The present disclosure relates to the field of communications technologies, and in particular, to a light source identification method, a device, a terminal device, and a computer readable storage medium.
Background
Along with the popularization of road condition monitoring equipment, the vehicle-mounted equipment recognizes that the function of the road condition monitoring equipment is improved to be an important functional point. In general, the frequency of the light source used by the road condition monitoring device is different from the frequency of the street lamp (50 Hz or 60 Hz), so the detection of the artificial light frequency by the vehicle-mounted device brings new demands: how to detect a light source of a specific frequency without affecting the detection of conventional frequencies is a problem to be solved.
Disclosure of Invention
The embodiment of the application provides a light source identification method, a light source identification device and terminal equipment, which can effectively identify a light source with target frequency.
In a first aspect, a light source identification method is provided, including: acquiring two frames of first images output by a long exposure channel, and analyzing the two frames of first images to obtain a first differential image, wherein the long exposure channel refers to a channel with exposure time being an integer multiple of light source cycle time (band step); acquiring two frames of second images output by a short exposure channel, and analyzing the two frames of second images to obtain a second differential image, wherein the short exposure channel refers to a channel with exposure time being less than one bridging step of a preset high-frequency light source; performing phase identification on the first differential image and the second differential image to obtain a first phase point and a second phase point respectively included in the first differential image and the second differential image; and carrying out phase verification on a first phase point and a second phase point which are respectively included in the first differential image and the second differential image so as to identify whether a light source with a target frequency exists in the current environment, wherein the target frequency is the frequency of the light source included in the first differential image and/or the second differential image.
In some embodiments, the phase verifying the first phase point and the second phase point included in each of the first differential image and the second differential image to identify whether the light source includes the target frequency in the current environment includes:
if the first phase point and the second phase point meet a preset first phase verification condition, identifying a light source comprising target frequency in the current environment;
the first phase verification condition includes at least one of: the number of the first phase points and the second phase points exceeds a corresponding threshold value, the distances between two adjacent first phase points and the second phase points in the first differential image and the next-frame differential image behind the first differential image are the same, and the corresponding signal strength change trend of the first phase points and the second phase points in the first differential image and the next-frame differential image behind the first differential image accords with the expected setting; the adjacent two first phase points and the second phase point have the same distance in the second differential image and the next differential image behind the second differential image respectively; the first phase point and the second phase point are in accordance with expected setting in the second differential image and the corresponding signal intensity change trend of the next differential image behind the second differential image.
In some embodiments, the first differential image includes at least one region, the second differential image includes at least one region, the at least one region is obtained by dividing two corresponding frames of images according to output characteristics of the image sensor, and the phase identifying the first differential image and the second differential image to obtain a first phase point and a second phase point included in each of the first differential image and the second differential image includes:
carrying out phase identification on each region in the first differential image and the second differential image to obtain a first phase point and a second phase point respectively included in each region in the first differential image and the second differential image;
the phase verifying the first phase point and the second phase point included in the first differential image and the second differential image respectively to identify whether the light source with the target frequency exists in the current environment comprises:
if the first phase point and the second phase point of at least one region in each region meet a preset second phase verification condition, identifying that a light source with target frequency exists in the current environment;
The second phase verification condition includes at least one of: the number of the first phase points and the second phase points of the at least one region exceeds a corresponding threshold, the distances between the adjacent two first phase points and the second phase points in the first differential image and the next-frame differential image behind the first differential image are the same, the distances between the adjacent two first phase points and the second phase points in the at least one region in the second differential image and the next-frame differential image behind the second differential image are the same, the corresponding signal intensity change trend of the first phase points and the second phase points of the at least one region in the first differential image and the next-frame differential image behind the first differential image respectively accords with an expected setting, and the corresponding signal intensity change trend of the first phase points and the second phase points of the at least one region in the second differential image and the next-frame differential image of the second differential image respectively accords with an expected setting.
In some embodiments, the identifying the presence of a light source of a target frequency in the current environment comprises:
performing confidence evaluation on the at least one region to obtain the confidence of the at least one region;
And if the target confidence coefficient exceeding the preset threshold exists in the confidence coefficient of the at least one region, identifying that the light source with the target frequency exists in the current environment.
In some embodiments, the first phase site is a 0 phase site and the second phase site is a pi phase site; alternatively, the first phase point is a pi phase point and the second phase point is a 0 phase point.
In some embodiments, if the exposure time of the first image is an integer multiple of the exposure time of the light source having the first frequency, the first differential image is represented as:
wherein DeltaLum VTS1 As a first differential image E peak1 For the energy peak value of the first frequency light source, ζ is reflectivity, w 1 For angular velocity with the first frequency light source, expo is exposure time, i is the i-th exposure line of the image sensor, and VTS is the frame interval.
In some embodiments, if the exposure time of the first image is an integer multiple of the exposure time of the light source having the second frequency, the first differential image is represented as:
wherein DeltaLum VTS1 As a first differential image E peak0 For the energy peak value, ζ, of the second frequency light sourceIs of reflectivity, w 0 For angular velocity with the second frequency light source, expo is exposure time, i is the i-th exposure line of the image sensor, and VTS is the frame interval.
In some embodiments, the second differential image is represented as:
wherein DeltaLum VTS2 As a second differential image E peak0 For peak energy of first-frequency light source, E peak1 Is the energy peak value of the second frequency light source, and xi is the reflectivity, w 0 For angular velocity of light source with second frequency, w 1 For angular velocity with the first frequency light source, expo is exposure time, i is the i-th exposure line of the image sensor, and VTS is the frame interval.
In a second aspect, a light source identification device is provided, which is capable of performing the method of the first aspect or any of the optional embodiments of the first aspect. The functions can be realized by hardware, and can also be realized by executing corresponding software by hardware. The hardware or software includes one or more units corresponding to the functions described above. The unit may be software and/or hardware.
In a third aspect, there is provided a terminal device comprising: a processor and a memory coupled to the processor; wherein the memory includes computer readable instructions; the processor is configured to execute the computer readable instructions in the memory to cause the vehicle to perform the arrangement of the first aspect or any of the alternative embodiments of the first aspect.
In a fourth aspect, there is provided a computer program product which, when run on a computer, causes the computer to perform the method of the first aspect or any of the alternative embodiments of the first aspect.
In a fifth aspect, a chip product is provided, performing the method of the first aspect or any of the alternative embodiments of the first aspect.
In a sixth aspect, a computer readable storage medium is provided, having instructions stored therein, which when run on a computer, cause the computer to perform the method of the first aspect or any of the alternative embodiments of the first aspect.
Drawings
Fig. 1 is a schematic diagram of a system framework according to an embodiment of the present application.
Fig. 2 is an exposure timing diagram of an imaging device according to an embodiment of the present application.
Fig. 3 is a schematic view of image area division according to an embodiment of the present application.
Fig. 4 is a flowchart of a light source identification method according to an embodiment of the present application.
Fig. 5 is a schematic waveform diagram of a differential image according to an embodiment of the present application.
Fig. 6 is a schematic diagram of a phase determination according to an embodiment of the present application.
Fig. 7 is a schematic structural diagram of a light source identification device according to an embodiment of the present application.
Fig. 8 is a schematic structural diagram of a terminal device according to an embodiment of the present application.
Detailed Description
Specific embodiments of the present application are described in further detail below with reference to the accompanying drawings.
Fig. 1 is a schematic diagram of a system structure according to an embodiment of the present application. The structure shown in fig. 1 includes a light source 100, an imaging element 101, and a terminal device 102. The light source 100 may provide photons, and the imaging element 101 (may also be referred to as an image sensor) captures an image frame in a line exposure manner, and an exposure timing diagram of the imaging element is exemplarily shown in fig. 2. As in fig. 2, the imaging element 101 acquires several image frames at a preset exposure time. The imaging element 101 further transmits the acquired image frames to the terminal device 102 (also referred to as an image signal processor), and the terminal device 102 includes a flicker statistics module 1021, a flicker recognition module 1022, and an exposure control module 1023. Wherein the flicker statistics module 1021 is configured to identify flicker in an image frame, specifically, the image frame may be divided into a number of image areas, such as 64 image areas in the example of fig. 3, and then calculate a luminance value of each image area to identify whether flicker (flicker) exists in each image area. After the flicker statistics module 1021 detects whether flicker is present in each region, each image frame may be sent to the flicker recognition module 1022 to identify the frequency of the light source contained in each image frame. How the flicker recognition module 1022 recognizes whether the light source frequency is included in the image frame is described in detail below, and is not described in detail herein.
Further, the flicker recognition module 1022 transmits the recognized light source frequency (i.e., the ambient light source frequency) to the exposure control module 1023, so that the exposure control module 1023 controls the exposure time of the imaging element 101 to acquire image frames in accordance with the exposure time.
Fig. 4 is a flowchart of a light source identification method according to an embodiment of the present application. The method shown in fig. 1 comprises the following steps:
s401, the terminal equipment acquires two frames of first images of a first interval output by a long exposure channel, and analyzes the two frames of first images to obtain a first differential image.
S402, the terminal equipment acquires two frames of second images at a second interval output by the short exposure channel, and analyzes the two frames of second images to obtain a second differential image.
The first interval and the second interval in the present application are frame intervals set by system user definition, and the first interval and the second interval may be the same or different, which is not limited in the present application. A long exposure channel refers to a channel whose exposure time is an integer multiple of the light source cycle time (banding step). The short exposure channel refers to a channel in which the exposure time is less than one of the biasing steps of the preset high frequency light source.
The terminal equipment acquires two frames of first images of a long exposure channel and two frames of second images of a short exposure channel through the camera system, analyzes the two frames of first images and the two frames of second images, and correspondingly generates a first differential image and a second differential image.
In one example, the image capturing system takes a picture in an electronic shutter exposure mode, and as known by an electronic shutter operating mechanism, under a mixed frequency light source, a differential signal image (i.e., a first differential image) of two adjacent frames can be expressed as the following formula (1):
wherein DeltaLum VTS1 As a first differential image E peak0 And E is peak1 Peak value of the first images of two frames respectively, and xi is reflectivity and w 0 To have a second frequency f 0 Angular velocity of light source, w 1 To have a first frequency f 1 The angular velocity of the light source, expo, is the exposure time, i is the i-th row of the image sensor, and VTS is the frame interval. Wherein f 0 And f 1 For customizing the system, e.g. f 0 50Hz, f 1 Is 36Hz.
In the mixed frequency light source environment, if the exposure time of the first image is according to f 0 The exposure time of the light source is adjusted, the first differential image is expressed as the following formula (2):
wherein DeltaLum VTS1 As a first differential image E peak1 For the energy peak value of the first frequency light source, ζ is reflectivity, w 1 For angular velocity with the first frequency light source, expo is exposure time, i is the i-th row of the image sensor, and VTS is the frame interval (i.e., first interval).
Optionally in a mixed frequency light source environment, if the exposure time of the first image is according to f 1 The exposure time of the light source is adjusted, the first differential image is expressed as the following formula (3):
wherein DeltaLum VTS1 As a first differential image E peak0 For the energy peak value of the second frequency light source, ζ is reflectivity, w 0 For angular velocity with the second frequency light source, expo is exposure time, i is the i-th row of the image sensor, and VTS is the frame interval (i.e., first interval).
Optionally at a single frequency (e.g. f 0 ) In the light source environment, the exposure time of the first image is according to f 0 The exposure time of the light source is adjusted, and then the first differential image thereof is expressed as the following formula (4):
ΔLum VTS1 =0 formula (4)
Optionally at a single frequency (e.g. f 1 ) In the light source environment, the exposure time of the first image is according to f 1 The exposure time of the light source is adjusted, and then the first differential image thereof is expressed as the following formula (5):
accordingly, in a short exposure scene, a differential signal image using short exposure (i.e., a second differential image) in a mixed frequency light source environment can be expressed as the following formula (6):
optionally at a single frequency (e.g. f 0 ) In the light source environment, the second differential image of the short exposure is expressed as the following formula (7):
s403, the terminal equipment carries out phase identification on the first differential image and the second differential image to obtain a first phase point and a second phase point respectively included in the first differential image and the second differential image.
The terminal equipment can conduct phase identification on the first differential signal under the long exposure and the second differential image under the short exposure so as to identify the first phase point and the second phase point respectively included in the first differential image and the second differential image. Fig. 5 shows a waveform diagram of a first differential image. As shown in fig. 5, when the signal intensity of the first differential image is at the critical point from negative to positive, the phase is 0; in contrast, when the signal intensity of the first differential image is at the critical point of negative from positive rotation, the phase is pi. The first phase site and the second phase site in the present application are both special phase sites, which may be any one of a 0 phase site and a pi phase site, and the first phase site is different from the second phase site.
And carrying out phase analysis and identification on the second differential image under short exposure by adopting the same principle to obtain a first phase point and a second phase point included in the second differential image.
It will be appreciated that under mixed frequency light sources, the differential signal image (i.e. the second differential image) under short exposure is more prone to determine if the current light source is a single frequency light source, since the two frequency light sources interfere with each other such that an ideal differential signal image is not obtained, whereas under a single frequency light source an ideal differential signal image is obtained.
S404, the terminal equipment performs phase verification on the first phase point and the second phase point which are respectively included in the first differential image and the second differential image so as to identify whether the current environment comprises the light source with the target frequency. The target frequency is the light source frequency (f) included in each of the first differential image and/or the second differential image 0 And/or f 1 )。
When the terminal device performs phase verification, since the differential signal image (the first differential image or the second differential image) has a cycle repetition characteristic, and when the interval at which the differential signal image is generated is unchanged, the two previous and subsequent frames of differential signal images are changed with a certain phase, so that the correctness of the phase point identified in S403 can be verified by using the differential signal image of the next frame. Specifically, when the first phase point and the second phase point in S403 satisfy the preset first phase verification condition, the first phase verification condition may be specifically set by a user or may be set by a system in a custom manner, and may include, but is not limited to, any one or more of the following combinations: the number of the first phase points and the second phase points exceeds a corresponding threshold, the distances between the adjacent two first phase points and the second phase points in the first differential image and the next differential image behind the first differential image are the same, the distances between the adjacent two first phase points and the second phase points in the second differential image and the next differential image behind the second differential image are the same, the change trend of the signal intensity (intensity size) of the phase points and the second phase points in the first differential image and the next differential image of the first differential image accords with the expected setting, and the change trend of the signal intensity of the phase points and the second phase points in the second differential image and the next differential image of the second differential image accords with the expected setting. In other words, the change trend of the special phase point (0 and pi phase point) selected in S103 in the next frame of differential image is utilized to determine whether the special phase point selected in S403 is correct, if so, S404 is continuously executed; otherwise, S403 is repeatedly executed.
Whether the signal strength change trend accords with the expected setting or not can be verified specifically through the following formula:
the differential signal expression of the X frame and the Y frame is:
When 2i+expo＝Tn(n∈Z + ),VTS＝Tm+ΔPhase,y＞x≥0(x,y∈Z)Then
from the above, it can be seen that: at the position ofKnowing the frame interval VTS, Δlum can be calculated from the difference in light source frequency VTS . For example, a period of 20ms is known for 50Hz and 16.7ms for 60Hz. Let VTS be 35ms and expo be 5ms. Δphase=2vts%t; how to decide whether the current light source is 50Hz or 60Hz.
Specifically, the definition of Δphase can be found as follows:
from this, it can be seen that the phase point of 0 in the current frame is still 0 in the case of 50Hz after the next frame; and is not 0 in the case of 60Hz. The current light source is thus a 50Hz light source, see in particular fig. 6.
The characteristic that the differential signal image has cycle repetition is proved by the following formula:
When 2i+expo+VTS＝Tn(n∈Z + ),i＝i+VTS，Then
When 2i+expo+VTS＝Tn(n∈Z + ),i＝i+VTS，Then
in still another example, after the terminal device collects the two first images or the two second images in S401 and S402, the two first images and the two second images may be divided into a plurality of areas according to the output characteristics (mainly, the image height and the exposure time corresponding to the target frequency) of the image sensor, so as to facilitate the subsequent independent detection of flicker for each area. Two frames of first images including at least one region and two frames of second images including at least one region are correspondingly obtained. Further, the terminal device analyzes the two frames of first images containing at least one region, and can generate a first differential image containing at least one region. Similarly, the terminal device analyzes the two frames of second images containing at least one region, and can generate a second differential image containing at least one region.
In the phase identification in S403, the terminal device performs phase identification on each region in the first differential image and the second differential image, so as to obtain a first phase point and a second phase point respectively included in each region. Further in S104, the terminal device performs phase verification on the first phase point and the second phase point included in each area, so as to determine whether the current environment includes the light source with the target frequency. Specifically, if the first phase point and the second phase point of at least one region exist in each region and meet a preset second phase verification condition, a light source including a target frequency in the current environment can be identified. The second phase verification condition is system-custom set, which may include, but is not limited to, at least one of: the number of the first phase points and the second phase points of the at least one region exceeds a corresponding threshold, the signal intensities of the first phase points and the second phase points of the at least one region in the first differential image and the next differential image of the first differential image are the same, and the signal intensities of the first phase points and the second phase points of the at least one region in the second differential image and the next differential image of the second differential image are the same.
Optionally, after counting the first phase point and the second phase point included in each region, performing confidence evaluation on at least one region, for example, if the signal intensities corresponding to the first phase point and the second phase point of a certain region in the first differential image and the next frame differential image of the first differential image are the same, setting the confidence of the region to be 1; otherwise set to 0. Further, if the target confidence coefficient exceeding the preset threshold exists in the at least one region confidence coefficient, determining that the current environment comprises the light source with the target frequency; otherwise, determining that the light source with the target frequency does not exist in the current environment.
In still another example, if the current environment includes a light source with a target frequency, the terminal device performs exposure photographing according to an exposure mechanism corresponding to the target frequency. The terminal device related to the application includes, but is not limited to, a vehicle-mounted device, an automobile, a mobile phone, a personal palm computer or other devices with communication functions.
In yet another example, if there are two frequency light sources in the current environment, namely a first frequency light source and a second frequency light source, and the first frequency light source has a higher priority than the second frequency light source, the terminal device may control the exposure time to clear the flicker of the higher priority first frequency light source.
The following details an example for a better understanding of the present application. Assuming that the vehicle takes a traffic light crossroad with a passing street lamp frequency of 50Hz as an example, the road condition monitoring device uses a traffic light crossroad with a frequency of 36Hz, please refer to the following table 1 for showing exposure time selections under 4 driving conditions of the vehicle.
As shown in table 1 above, under the condition of the serial number 1, when the automobile runs from a distance to a traffic light intersection, it is assumed that the vehicle-mounted device currently adopts an exposure mechanism corresponding to 50Hz to perform photographing analysis, the differential signal under long exposure is 0, and the frequency of the differential signal under short exposure is 50Hz, and then the exposure mechanism corresponding to 50Hz is still selected in the next exposure period to perform exposure photographing.
In the scene of sequence number 2, when the automobile runs near a traffic light intersection, the vehicle-mounted equipment currently adopts an exposure mechanism corresponding to 50Hz to carry out photographing analysis, the frequency of a differential signal under long exposure is 36Hz, the frequency of the differential signal under short exposure is not purely 50Hz, and then the exposure mechanism corresponding to 36Hz is selected in the next exposure period to carry out exposure photographing.
In the scene of sequence number 3, when the automobile runs at a traffic light intersection, the vehicle-mounted equipment adopts an exposure mechanism corresponding to 36Hz to carry out photographing analysis, the frequency of a differential signal under long exposure is 50Hz, the frequency of the differential signal under short exposure is not purely 50Hz, and then the exposure mechanism corresponding to 36Hz is selected in the next exposure period to carry out exposure photographing.
In the scene of sequence number 4, when the automobile leaves the traffic light intersection, the vehicle-mounted equipment adopts an exposure mechanism corresponding to 36Hz to carry out photographing analysis, the frequency of the differential signal under long exposure is 50Hz, and the frequency of the differential signal under short exposure is 50Hz, and then the exposure mechanism corresponding to 50Hz is selected in the next exposure period to carry out exposure photographing.
By implementing the embodiment of the application, the terminal equipment can acquire two frames of first images at a first interval output by the long exposure channel, and analyze the two frames of first images to obtain a first differential image; acquiring two frames of second images at a second interval output by a short exposure channel, analyzing the two frames of second images to obtain a second differential image, and carrying out phase identification on the first differential image and the second differential image to obtain a first phase point and a second phase point respectively included in the first differential image and the second differential image; and finally, carrying out phase verification on a first phase point and a second phase point which are respectively included in the first differential image and the second differential image so as to identify whether the current environment comprises the light source with the target frequency. Thus, whether the current environment contains the light source with the target frequency can be identified efficiently and accurately.
Fig. 7 is a schematic structural diagram of a light source identification device according to an embodiment of the present application. The apparatus of fig. 7 includes: an acquisition unit 701, an identification unit 702, and a verification unit 703. Wherein:
An obtaining unit 701, configured to obtain two frames of first images output by a long exposure channel, and analyze the two frames of first images to obtain a first differential image, where the long exposure channel refers to a channel whose exposure time is an integer multiple of a light source period time;
the acquiring unit 701 is further configured to acquire two frames of second images output by a short-exposure channel, and analyze the two frames of second images to obtain a second differential image, where the short-exposure channel refers to a channel with an exposure time that is less than a light source cycle time of a preset high-frequency light source;
the identifying unit 702 is configured to perform phase identification on the first differential image and the second differential image, so as to obtain a first phase point and a second phase point included in each of the first differential image and the second differential image;
the verification unit 703 is configured to perform phase verification on a first phase point and a second phase point included in each of the first differential image and the second differential image, so as to identify whether a light source with a target frequency exists in the current environment, where the target frequency is a light source frequency included in the first differential image and/or the second differential image.
In some embodiments, the verification unit 603 is specifically configured to identify that a light source with a target frequency exists in the current environment if the first phase point and the second phase point meet a preset first phase verification condition;
The first phase verification condition includes at least one of: the number of the first phase points and the second phase points exceeds a corresponding threshold value, the corresponding signal intensity change trend of the first phase points and the second phase points in the first differential image and the next frame differential image of the first differential image accords with an expected setting, the corresponding signal intensity change trend of the first phase points and the second phase points in the second differential image and the next frame differential image of the second differential image accords with an expected setting, the distances between the adjacent two first phase points and the second phase points in the first differential image and the next frame differential image behind the first differential image are the same, and the distances between the adjacent two first phase points and the second phase points in the second differential image and the second differential image are the same.
In some embodiments, the signal strength trend is consistent with an expected setting comprising: verifying the change in signal strength of the first phase point and the second phase point in a next frame differential image of the first differential image or the second differential image using the following formula:
If the verification of the formula is satisfied, the signal strength change trend is in accordance with the expected setting. Where Δphase=2vts%t.
In some embodiments, the first differential image includes at least one region, the second differential image includes at least one region, the at least one region is obtained by dividing corresponding two frames of images according to output characteristics of the image sensor,
the identifying unit 702 is specifically configured to identify a phase of each region in the first differential image and the second differential image, so as to obtain a first phase point and a second phase point included in each region in the first differential image and the second differential image;
the verification unit 703 is specifically configured to identify that a light source with a target frequency exists in the current environment if the first phase point and the second phase point of at least one area in each area meet a preset second phase verification condition;
the second phase verification condition includes at least one of: the number of the first phase points and the second phase points of the at least one region exceeds a corresponding threshold value, the corresponding signal intensity change trend of the first phase points and the second phase points of the at least one region in the first differential image and the next frame differential image of the first differential image respectively accords with an expected setting, the corresponding signal intensity change trend of the first phase points and the second phase points of the at least one region in the second differential image and the next frame differential image of the second differential image respectively accords with an expected setting, the distances between the adjacent two first phase points and the adjacent two second phase points in the at least one region in the first differential image and the next frame differential image of the first differential image respectively are the same, and the distances between the adjacent two first phase points and the adjacent two second phase points in the first region in the second differential image and the next frame differential image of the second differential image respectively are the same.
In some embodiments, the verification unit 703 is further configured to perform a confidence evaluation on the at least one region, to obtain a confidence of the at least one region; and if the target confidence coefficient exceeding the preset threshold exists in the confidence coefficient of the at least one region, identifying that the light source with the target frequency exists in the current environment.
In some embodiments, the first phase site is a 0 phase site and the second phase site is a pi phase site; alternatively, the first phase point is a pi phase point and the second phase point is a 0 phase point.
In some embodiments, if the exposure time of the first image is an integer multiple of the exposure time of the light source having the first frequency, the first differential image is represented as:
wherein DeltaLum VTS1 As a first differential image E peak1 For the energy peak value of the first frequency light source, ζ is reflectivity, w 1 For angular velocity with the first frequency light source, expo is exposure time, i is the i-th row of the image sensor, and VTS is the frame interval.
In some embodiments, if the exposure time of the first image is an integer multiple of the exposure time of the light source having the second frequency, the first differential image is represented as:
wherein DeltaLum VTS1 As a first differential image E peak0 For the energy peak value of the second frequency light source, ζ is reflectivity, w 0 For angular velocity with the second frequency light source, expo is exposure time, i is the i-th row of the image sensor, and VTS is the frame interval.
In some embodiments, the second differential image is represented as:
wherein DeltaLum VTS2 As a second differential image E peak0 For the energy peak value of the second frequency light source E peak1 Is the energy peak value of the first frequency light source, and xi is the reflectivity, w 0 For angular velocity of light source with second frequency, w 1 For angular velocity with the first frequency light source, expo is exposure time, i is the i-th row of the image sensor, and VTS is the frame interval.
Fig. 7 is a schematic structural diagram of a terminal device according to an embodiment of the present application. The terminal device 700 as shown in fig. 7 includes: at least one input device 701; at least one output device 702; at least one processor 703, such as a CPU; and a memory 704, the input device 701, the output device 702, the processor 703, and the memory 704 being connected by a bus 705.
The input device 701 may specifically be a touch panel of a mobile terminal, including a touch screen and a touch screen, and is configured to detect an operation instruction on the touch panel of the terminal.
The output device 702 may be a display screen of a mobile terminal, and is used for outputting and displaying information.
The memory 704 may be a high-speed RAM memory or a non-volatile memory (non-volatile memory), such as a disk memory. The memory 704 is used for storing a set of program codes, and the input device 701, the output device 702 and the processor 703 are used for calling the program codes stored in the memory 704 to perform the following operations:
the processor 703 is configured to obtain two first images of a first interval output by a long exposure channel, and analyze the two first images to obtain a first differential image, where the long exposure channel is a channel with an exposure time that is an integer multiple of a light source cycle time (banding step);
acquiring two frames of second images at a second interval output by a short exposure channel, and analyzing the two frames of second images to obtain a second differential image, wherein the short exposure channel refers to a channel with exposure time being less than one light source period time of a preset high-frequency light source;
performing phase identification on the first differential image and the second differential image to obtain a first phase point and a second phase point respectively included in the first differential image and the second differential image;
And carrying out phase verification on a first phase point and a second phase point which are respectively included in the first differential image and the second differential image so as to identify whether a light source with a target frequency exists in the current environment, wherein the target frequency is the frequency of the light source included in the first differential image and/or the second differential image.
In some embodiments, the processor 703 is specifically configured to identify a light source that includes a target frequency in the current environment if the first phase point and the second phase point meet a preset first phase verification condition;
the first phase verification condition includes at least one of: the number of the first phase points and the second phase points exceeds a corresponding threshold value, the corresponding signal strength change trends of the first phase points and the second phase points in the first differential image and the next frame differential image of the first differential image accord with expected setting, the corresponding signal strength change trends of the first phase points and the second phase points in the second differential image and the next frame differential image of the second differential image accord with expected setting, the distances between the adjacent two first phase points and the second phase points in the first differential image and the next frame differential image behind the first differential image are the same, and the distances between the adjacent two first phase points and the second phase points in the second differential image and the next frame differential image behind the second differential image are the same.
In some embodiments, the first differential image includes at least one region, the second differential image includes at least one region, the at least one region is obtained by dividing two corresponding frames of images according to output characteristics of the image sensor, and the processor 703 is specifically configured to perform phase identification on each region in the first differential image and the second differential image, so as to obtain a first phase point and a second phase point included in each region in the first differential image and the second differential image, respectively; if the first phase point and the second phase point of at least one region in each region meet the preset second phase verification condition, identifying a light source comprising target frequency in the current environment;
the second phase verification condition includes at least one of: the number of the first phase points and the second phase points of the at least one region exceeds a corresponding threshold, the corresponding signal intensity variation trend of the first phase points and the second phase points of the at least one region in the first differential image and the next frame differential image of the first differential image respectively accords with an expected setting, the corresponding signal intensity variation of the first phase points and the second phase points of the at least one region in the second differential image and the next frame differential image of the second differential image respectively accords with an expected setting, the distances between the adjacent two first phase points and the second phase points in the at least one region in the next frame differential image behind the first differential image and the first differential image are the same, and the distances between the adjacent two first phase points and the second phase points in the at least one region in the next frame differential image behind the second differential image and the second differential image are the same.
In some embodiments, the processor 703 is specifically configured to perform a confidence evaluation on the at least one region, to obtain a confidence of the at least one region;
and if the target confidence coefficient exceeding the preset threshold exists in the confidence coefficient of the at least one region, identifying the light source comprising the target frequency in the current environment.
In some embodiments, the first phase site is a 0 phase site and the second phase site is a pi phase site; alternatively, the first phase point is a pi phase point and the second phase point is a 0 phase point.
In some embodiments, if the exposure time of the first image is an integer multiple of the exposure time of the light source having the first frequency, the first differential image is represented as:
wherein DeltaLum VTS1 As a first differential image E peak1 For the energy peak value of the first frequency light source, ζ is reflectivity, w 1 For angular velocity with the first frequency light source, expo is exposure time, i is the i-th row of the image sensor, and VTS is the frame interval.
In some embodiments, if the exposure time of the first image is an integer multiple of the exposure time of the light source having the second frequency, the first differential image is represented as:
wherein DeltaLum VTS1 As a first differential image E peak0 For the energy peak value of the second frequency light source, ζ is reflectivity, w 0 For angular velocity with the second frequency light source, expo is exposure time, i is the i-th row of the image sensor, and VTS is the frame interval.
In some embodiments, the second differential image is represented as:
wherein DeltaLum VTS2 As a second differential image E peak0 For the energy peak value of the second frequency light source E peak1 Is the energy peak value of the first frequency light source, and xi is the reflectivity, w 0 For angular velocity of light source with second frequency, w 1 For angular velocity with the first frequency light source, expo is exposure time, i is the i-th row of the image sensor, and VTS is the frame interval.
Based on the same inventive concept, the principle of solving the problem of the terminal device provided in the embodiments of the present application is similar to that of solving the problem of the terminal in the embodiments of the method of the present application, so that the implementation of each device may refer to the implementation of the method, and for brevity, the description is not repeated here.
In the foregoing embodiments, the descriptions of the embodiments are focused on, and for those portions of one embodiment that are not described in detail, reference may be made to the related descriptions of other embodiments.
The steps in the method of the embodiment of the invention can be sequentially adjusted, combined and deleted according to actual needs.
The modules in the terminal equipment of the embodiment of the invention can be combined, divided and deleted according to actual needs.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.
Claims (10)
1. A light source identification method, characterized in that the light source identification method comprises:
acquiring two frames of first images output by a long exposure channel, and analyzing the two frames of first images to obtain a first differential image, wherein the long exposure channel refers to a channel with exposure time being an integer multiple of the period time of a light source;
acquiring two frames of second images output by a short exposure channel, and analyzing the two frames of second images to obtain a second differential image, wherein the short exposure channel refers to a channel with exposure time being less than one light source period time of a preset high-frequency light source;
Performing phase identification on the first differential image and the second differential image to obtain a first phase point and a second phase point respectively included in the first differential image and the second differential image;
performing phase verification on a first phase point and a second phase point which are respectively included in the first differential image and the second differential image to identify whether a light source with a target frequency exists in the current environment, wherein the target frequency is the frequency of the light source included in the first differential image and/or the second differential image;
wherein the first phase point and the second phase point are any one of a 0 phase point and a pi phase point, and the first phase point is different from the second phase point.
2. The method of claim 1, wherein the phase verifying the first phase point and the second phase point included in the first differential image and the second differential image respectively to identify whether the light source with the target frequency exists in the current environment comprises:
if the first phase point and the second phase point meet a preset first phase verification condition, recognizing that a light source with target frequency exists in the current environment;
The first phase verification condition includes at least one of: the number of the first phase points and the second phase points exceeds a corresponding threshold; the adjacent two first phase points and the second phase point have the same distance in the first differential image and the next differential image behind the first differential image; and the adjacent two first phase points and the second phase point have the same distance in the second differential image and the next frame differential image behind the second differential image.
3. The light source identification method according to claim 1, wherein the first differential image includes at least one region, the second differential image includes at least one region, the at least one region is obtained by dividing corresponding two frames of images according to output characteristics of an image pickup sensor, the phase identification is performed on the first differential image and the second differential image, and obtaining a first phase point and a second phase point included in each of the first differential image and the second differential image includes:
carrying out phase identification on each region in the first differential image and the second differential image to obtain a first phase point and a second phase point respectively included in each region in the first differential image and the second differential image;
The phase verifying the first phase point and the second phase point included in the first differential image and the second differential image respectively to identify whether the light source with the target frequency exists in the current environment comprises:
if the first phase point and the second phase point of at least one region in each region meet a preset second phase verification condition, identifying that a light source with target frequency exists in the current environment;
the second phase verification condition includes at least one of: the number of the first phase points and the second phase points of the at least one region exceeds a corresponding threshold, the distances between the adjacent two first phase points and the second phase points in the at least one region in the first differential image and the next frame differential image of the first differential image are the same, and the distances between the adjacent two first phase points and the second phase points in the at least one region in the second differential image and the next frame differential image of the second differential image are the same.
4. A light source identification method according to claim 3, wherein the identifying a light source for which a target frequency is present in the current environment comprises:
Performing confidence evaluation on the at least one region to obtain the confidence of the at least one region;
and if the target confidence coefficient exceeding the preset threshold exists in the confidence coefficient of the at least one region, identifying that the light source with the target frequency exists in the current environment.
5. The light source identification method according to claim 1, wherein if the exposure time of the first image is an integer multiple of the exposure time of the light source having the first frequency, the first differential image is expressed as:
wherein DeltaLum VTS1 As a first differential image E peak1 For the energy peak value of the first frequency light source, ζ is reflectivity, w 1 For angular velocity with the first frequency light source, expo is exposure time, i is the i-th row of the imaging sensor, and VTS is the frame interval.
6. The light source identification method according to claim 5, wherein if the exposure time of the first image is an integer multiple of the exposure time of the light source having the second frequency, the first differential image is expressed as:
wherein DeltaLum VTS1 As a first differential image E peak0 For the energy peak value of the second frequency light source, ζ is reflectivity, w 0 For angular velocity with the second frequency light source, expo is exposure time, i is the i-th row of the imaging sensor, and VTS is the frame interval.
7. The light source identification method according to claim 1, wherein the second differential image is represented as:
wherein DeltaLum VTS2 As a second differential image E peak0 For the energy peak value of the second frequency light source E peak1 Is the energy peak value of the first frequency light source, and xi is the reflectivity, w 0 For angular velocity of light source with second frequency, w 1 For angular velocity of light source with first frequency, expo is exposureLight time, i is the i-th row of the image sensor, and VTS is the frame interval.
8. The light source identification device is characterized by comprising an acquisition unit, an identification unit and a verification unit, wherein:
the acquisition unit is used for acquiring two frames of first images output by a long exposure channel, analyzing the two frames of first images to obtain a first differential image, wherein the long exposure channel refers to a channel with exposure time being an integer multiple of the period time of the light source;
the acquisition unit is further used for acquiring two frames of second images output by a short exposure channel, and analyzing the two frames of second images to obtain a second differential image, wherein the short exposure channel refers to a channel with exposure time being less than one light source period time of a preset high-frequency light source;
the identification unit is used for carrying out phase identification on the first differential image and the second differential image to obtain a first phase point and a second phase point which are respectively included in the first differential image and the second differential image;
The verification unit is configured to perform phase verification on a first phase point and a second phase point included in each of the first differential image and the second differential image, so as to identify whether a light source with a target frequency exists in a current environment, where the target frequency is a light source frequency included in the first differential image and/or the second differential image;
wherein the first phase point and the second phase point are any one of a 0 phase point and a pi phase point, and the first phase point is different from the second phase point.
9. A terminal device comprising a processor and a memory coupled to the processor, the memory comprising computer readable instructions, the processor configured to execute the computer readable instructions in the memory to implement the method of any of claims 1-7.
10. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program comprising program instructions which, when executed by a processor, cause the processor to perform the method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011052635.8A CN112183373B (en) | 2020-09-29 | 2020-09-29 | Light source identification method, device, terminal equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011052635.8A CN112183373B (en) | 2020-09-29 | 2020-09-29 | Light source identification method, device, terminal equipment and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112183373A CN112183373A (en) | 2021-01-05 |
| CN112183373B true CN112183373B (en) | 2024-03-08 |
Family
ID=73946580
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011052635.8A Active CN112183373B (en) | 2020-09-29 | 2020-09-29 | Light source identification method, device, terminal equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112183373B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113596344B (en) * | 2021-07-26 | 2023-10-24 | 艾酷软件技术(上海)有限公司 | Shooting processing method, shooting processing device, electronic equipment and readable storage medium |
| CN115499597B (en) * | 2022-09-13 | 2024-06-04 | 豪威集成电路(成都)有限公司 | Identification method and device for target frequency light source of imaging system and terminal equipment |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101057489A (en) * | 2004-11-15 | 2007-10-17 | 索尼株式会社 | Flicker correcting method, flicker correcting circuit, and imaging device using them |
| JP2012094956A (en) * | 2010-10-25 | 2012-05-17 | Nikon Corp | Imaging device |
| DE102013115000A1 (en) * | 2013-01-07 | 2014-07-10 | GM Global Technology Operations LLC (n. d. Gesetzen des Staates Delaware) | Method for generating glare-reduced image from images captured by camera device of subject vehicle, involves generating glare-reduced image based on modified identified glaring region upon resulting high dynamic range image |
| CN105637783A (en) * | 2013-12-27 | 2016-06-01 | 松下电器（美国）知识产权公司 | Information processing program, receiving program and information processing device |
| CN109274984A (en) * | 2018-10-16 | 2019-01-25 | 深圳开阳电子股份有限公司 | Detect the method, apparatus and image processing equipment of light source scintillation in video sequence |
| CN109361866A (en) * | 2018-11-21 | 2019-02-19 | 豪威科技(上海)有限公司 | A kind of automatic exposure control method and system |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4354449B2 (en) * | 2005-10-26 | 2009-10-28 | オリンパス株式会社 | Display image imaging method and apparatus |
| US9432590B2 (en) * | 2014-01-22 | 2016-08-30 | Nvidia Corporation | DCT based flicker detection |
-
2020
- 2020-09-29 CN CN202011052635.8A patent/CN112183373B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101057489A (en) * | 2004-11-15 | 2007-10-17 | 索尼株式会社 | Flicker correcting method, flicker correcting circuit, and imaging device using them |
| JP2012094956A (en) * | 2010-10-25 | 2012-05-17 | Nikon Corp | Imaging device |
| DE102013115000A1 (en) * | 2013-01-07 | 2014-07-10 | GM Global Technology Operations LLC (n. d. Gesetzen des Staates Delaware) | Method for generating glare-reduced image from images captured by camera device of subject vehicle, involves generating glare-reduced image based on modified identified glaring region upon resulting high dynamic range image |
| CN105637783A (en) * | 2013-12-27 | 2016-06-01 | 松下电器（美国）知识产权公司 | Information processing program, receiving program and information processing device |
| CN109274984A (en) * | 2018-10-16 | 2019-01-25 | 深圳开阳电子股份有限公司 | Detect the method, apparatus and image processing equipment of light source scintillation in video sequence |
| CN109361866A (en) * | 2018-11-21 | 2019-02-19 | 豪威科技(上海)有限公司 | A kind of automatic exposure control method and system |
Non-Patent Citations (1)
| Title |
|---|
| Flicker-free spatial-PSK modulation scheme for vehicular image sensor communications;Trang Nguyen等;《2016 Eighth International Conference on Ubiquitous and Future Networks (ICUFN)》;128-133 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112183373A (en) | 2021-01-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112183373B (en) | Light source identification method, device, terminal equipment and computer readable storage medium | |
| CN106254682B (en) | A kind of photographic method and mobile terminal | |
| US8593522B2 (en) | Digital camera, image processing apparatus, and image processing method | |
| CN110475063B (en) | Image acquisition method and device, and storage medium | |
| CN106331510A (en) | Backlight photographing method and mobile terminal | |
| US20110090345A1 (en) | Digital camera, image processing apparatus, and image processing method | |
| JP2022545899A (en) | Categorization method and apparatus, electronic device, storage medium, and computer program | |
| EP2198394A1 (en) | Face tracking in a camera processor | |
| TWI519156B (en) | Image capture method and image capture system | |
| JP2017055231A (en) | Image processing apparatus, solid state image sensor, and electronic apparatus | |
| CN101377811A (en) | Method and system for recognizing license plate | |
| CN113596344B (en) | Shooting processing method, shooting processing device, electronic equipment and readable storage medium | |
| CN110414445B (en) | Light source adjusting method and device for face recognition and electronic equipment | |
| CN104935698A (en) | Photographing method of smart terminal, photographing device and smart phone | |
| US11546524B2 (en) | Reducing a flicker effect of multiple light sources in an image | |
| KR20220062107A (en) | Light intensity control method, apparatus, electronic device and storage medium | |
| CN105635568A (en) | Image processing method in mobile terminal and mobile terminal | |
| CN113163128A (en) | Image exposure method and electronic device | |
| CN114070993B (en) | Image pickup method, image pickup apparatus, and readable storage medium | |
| JP2022524672A (en) | Information recognition methods and devices, systems, electronic devices, storage media and computer programs | |
| CN112819714A (en) | Target object exposure method, device, storage medium and equipment | |
| CN113225550A (en) | Offset detection method and device, camera module, terminal equipment and storage medium | |
| WO2021136978A1 (en) | Image processing method and apparatus, electronic device, and storage medium | |
| CN102012602B (en) | Camera-module-based ambient light monitoring method | |
| CN115290299B (en) | Method and device for determining falling depth of screen light leakage and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |